Picture composition method and device, electronic equipment and storage medium

Abstract

The invention discloses a composition method and device, electronic equipment and a storage medium. The composition method is applied to the electronic equipment. The method comprises the following steps: acquiring a preview picture collected by the electronic equipment; obtaining an object detection frame meeting a specified condition in the preview picture; obtaining a coordinate of a touch operation acting on the preview picture, and constructing a region of interest based on the coordinate and the object detection frame meeting the specified condition; obtaining an aesthetic composition frame corresponding to the region of interest; and adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview picture to obtain a target composition. According to the method, the interested area is determined based on the interested point which is input by the user and wants to compose the composition, then zooming composition is automatically completed based on the interested area, the preference of the user for composition is met, composition modes in the shooting process are enriched, and then the shooting experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a composition method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of smart terminals in recent years, users use smart terminals for more and more times and in more and more scenarios. In order to facilitate user use, more and more smart terminals have AI (Artificial Intelligence) photography functions. However, the current AI photography function's photo-taking experience needs to be improved. Summary of the invention

[0003] The present application proposes a patterning method, device, electronic device and storage medium to improve the above-mentioned problems.

[0004] In a first aspect, an embodiment of the present application provides a composition method, which is applied to an electronic device, and the method includes: obtaining a preview screen captured by the electronic device; obtaining an object detection frame that meets specified conditions in the preview screen; obtaining the coordinates of a touch operation applied to the preview screen, and constructing a region of interest based on the coordinates and the object detection frame that meets the specified conditions; obtaining an aesthetic composition frame corresponding to the region of interest; adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen, to obtain a target composition.

[0005] In a second aspect, an embodiment of the present application provides a composition device that runs on an electronic device, the device comprising: a preview screen acquisition module, used to acquire a preview screen captured by the electronic device; an object detection frame acquisition module, used to acquire an object detection frame that meets specified conditions in the preview screen; an area of ​​interest acquisition module, used to acquire the coordinates of a touch operation acting on the preview screen, and construct an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; an aesthetic composition frame acquisition module, used to acquire an aesthetic composition frame corresponding to the area of ​​interest; and a composition module, used to adjust the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain a target composition.

[0006] In a third aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method of the first aspect above.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium having program code stored therein, wherein the method of the first aspect is executed when the program code is run.

[0008] The present application provides a composition method, device, electronic device and storage medium. The method obtains a preview screen captured by an electronic device; obtains an object detection frame that meets specified conditions in the preview screen; obtains the coordinates of the touch operation acting on the preview screen, and constructs an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; obtains an aesthetic composition frame corresponding to the area of ​​interest; adjusts the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen, and obtains the target composition. Thus, through the above method, it is possible to determine the area of ​​interest based on the points of interest that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, which satisfies the user's preference for composition, enriches the composition method during the shooting process, and thus improves the user's shooting experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a patterning method provided by an embodiment of the present application is shown.

[0011] Figure 2 An example diagram of a preview screen captured by an electronic device provided in an embodiment of the present application is shown.

[0012] Figure 3 A flow chart of a patterning method provided by another embodiment of the present application is shown.

[0013] Figure 4 A diagram showing an example of the principle of obtaining an object detection frame that meets specified conditions provided in an embodiment of the present application is shown.

[0014] Figure 5 A diagram showing an example of the principle of merging two intersecting or overlapping object detection frames provided in an embodiment of the present application is shown.

[0015] Figure 6 A flowchart of a patterning method provided in yet another embodiment of the present application is shown.

[0016] Figure 7 A flowchart of a patterning method provided in yet another embodiment of the present application is shown.

[0017] Figure 8 An example diagram of a process of mapping reference composition frame coordinates to thumbnails provided by an embodiment of the present application is shown.

[0018] Fig. 9 A structural block diagram of a patterning device provided in an embodiment of the present application is shown.

[0019] Fig.10 A structural block diagram of an electronic device provided in an embodiment of the present application is shown.

[0020] Fig.11 A storage unit for storing or carrying program codes for implementing the patterning method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0022] With the rapid development of smart terminals in recent years, users use smart terminals more and more often and in more and more scenarios. To facilitate user use, more and more smart terminals have AI photography functions. However, current AI photography focuses more on "parameter adjustment and shooting" and "post-processing", while the equally important pre-composition and secondary composition are rarely involved, resulting in the current AI photography function's photo-taking experience still needs to be improved.

[0023] After long-term research, the inventor found that the target composition can be obtained by obtaining a preview screen captured by an electronic device; obtaining an object detection frame that meets specified conditions in the preview screen; obtaining the coordinates of the touch operation acting on the preview screen, and constructing an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; obtaining an aesthetic composition frame corresponding to the area of ​​interest; adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen. Thus, the above method can achieve the determination of the area of ​​interest based on the points of interest that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, satisfying the user's preference for composition, and thus improving the user's shooting experience.

[0024] Therefore, in order to improve the above-mentioned problems, the inventors proposed the composition method, electronic device and computer-readable storage medium provided in the present application, which can determine the area of ​​interest based on the interest points that the user inputs to compose, and then automatically complete the zoom composition based on the area of ​​interest, thereby satisfying the user's preference for composition and improving the user's shooting experience.

[0025] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0026] See also Figure 1, shows a flow chart of a composition method provided by an embodiment of the present application, and this embodiment provides a composition method, which can be applied to an electronic device, and the electronic device can be a mobile phone, a computer, a tablet or a wearable electronic device, and the type of the specific electronic device may not be limited. The method includes:

[0027] Step S110: Acquire a preview image captured by the electronic device.

[0028] The electronic device in the embodiment of the present application is equipped with a camera. When the camera is in the start-up state, the camera is pointed at any shooting picture, and a preview picture corresponding to the shooting picture collected by the electronic device can be obtained, wherein the shooting picture refers to the picture formed by the object in the real scene, and the preview picture refers to the picture displayed on the screen of the electronic device.

[0029] In the implementation manner of the present application, the camera of the electronic device is configured with an AI targeted composition function control, which is used to control the camera to capture images in the AI ​​targeted composition shooting mode. For the specific working principle of the AI ​​targeted composition shooting mode, please refer to the subsequent description.

[0030] As a way, the AI ​​targeted composition function control can be turned on after the camera is started (it can be configured to be automatically triggered when the camera is started or manually turned on). In this way, the electronic device can automatically obtain a preview picture corresponding to the current shooting picture.

[0031] As another way, you can configure the AI ​​targeted composition shooting mode as the camera's default shooting mode, that is, configure the camera's shooting mode to the AI ​​targeted composition shooting mode in advance. In this case, when the camera is started, it can automatically trigger the acquisition of the preview picture corresponding to the current shooting picture.

[0032] In a specific application scenario, see Figure 2 , shows an example diagram of a preview screen collected by an electronic device provided in an embodiment of the present application, such as Figure 2 As shown, when the electronic device 100 faces ( Figure 2 When the image 101 is captured (not shown in the facing posture), the screen 105 of the electronic device 100 may display the following Figure 2 The preview screen shown on the left of the figure, if the camera shutter 103 is clicked, the shooting screen 101 can be photographed to obtain the corresponding image. Among them, the position and size of the screen 105 of the electronic device 100 displaying the preview screen may not be limited, for example, it may be displayed in full screen, or in the following manner: Figure 2 It can be displayed at the position shown on the left of the figure, or it can be displayed in a rectangular frame in the center of the screen.

[0033] Step S120: obtaining an object detection frame in the preview image that meets a specified condition.

[0034] When a preview screen is obtained, the preview screen can be input into a target detection model (each object in the preview screen can correspond to a target), and an image output by the target detection model containing at least one object detection frame is obtained, that is, the preview screen can include one or more object detection frames, and the one or more objects here can actually be understood as one or more objects, that is, each object can be understood as an object. This embodiment is described by taking the preview screen including multiple (two or more) object detection frames as an example. The detection objects of the target detection model can be various types such as people, objects, animals, etc., and the specific detectable types can be not limited.

[0035] After obtaining multiple object detection frames output by the target detection model, the area of ​​each object detection frame can be obtained, and the object detection frames with an area smaller than a first threshold are discarded. The specific value of the first threshold can be set according to actual needs.

[0036] As an implementation method, after discarding object detection frames with an area smaller than a first threshold, the remaining multiple object detection frames can be traversed to determine whether there are intersecting or overlapping object detection frames. If there are intersecting or overlapping object detection frames, the intersecting or overlapping object detection frames can be merged in pairs until there are no intersecting or overlapping object detection frames in the picture. At this time, the obtained object detection frames can be used as object detection frames that meet specified conditions, that is, object detection frames with an area greater than or equal to the first threshold and no intersecting or overlapping object detection frames in the picture are used as object detection frames that meet the specified conditions.

[0037] It should be noted that, in this embodiment, the remaining object detection frames after discarding the object detection frames with an area smaller than the first threshold value may be directly used as the object detection frames that meet the specified conditions; or after obtaining multiple object detection frames output by the target detection model, the multiple object detection frames may be directly traversed to determine whether there are intersecting or overlapping object detection frames. If there are intersecting or overlapping object detection frames, the intersecting or overlapping object detection frames may be merged in pairs until there are no intersecting or overlapping object detection frames in the picture, and then the object detection frames without intersecting or overlapping in the picture may be used as the object detection frames that meet the specified conditions.

[0038] As an implementation, the specific method for obtaining the object detection frame that meets the specified conditions can be determined according to the shooting position. For example, the method for obtaining the object detection frame that meets the specified conditions can be determined according to the distance between the camera and the shooting screen.

[0039] Optionally, if the distance between the camera and the captured screen is less than a distance threshold (the specific value may not be limited), the object detection frames remaining after discarding only the object detection frames with an area smaller than the first threshold (i.e., the object detection frames with an area larger than the first threshold) may be used as object detection frames that meet specified conditions, to ensure that suitable object detection frames can still be selected when the camera is too close to the captured screen; if the distance between the camera and the captured screen is greater than or equal to the distance threshold, the object detection frames that have been discarded with an area smaller than the first threshold and no cross-overlapping in the screen may be used as object detection frames that meet the specified conditions, to improve the accuracy of object detection frame selection.

[0040] Optionally, or if the distance between the camera and the captured picture is less than a first distance threshold (the specific value may not be limited), the object detection frame with an area greater than the first threshold may be directly used as the object detection frame that meets the specified conditions; if the distance between the camera and the captured picture is greater than the first distance threshold but less than the second distance threshold, the object detection frame with an area greater than the first threshold and no cross-overlap in the picture may be used as the object detection frame that meets the specified conditions, wherein the value of the second distance threshold is greater than the value of the first distance threshold; and if the distance between the camera and the captured picture is greater than the second distance threshold, the object detection frame without cross-overlap in the picture may be used as the object detection frame that meets the specified conditions.

[0041] As an implementation mode, for multiple object detection frames output by the target detection model, the ratio of the size of the object detection frame to the size of the thumbnail may be compared with a preset value, and if the ratio is greater than or equal to the preset value, the object detection frame that meets the condition may be used as the object detection frame that meets the specified condition. The specific value of the preset value may not be limited.

[0042] In an embodiment of the present application, an object detection frame that meets specified conditions may be displayed in the preview screen to help users quickly identify various objects; or an object detection frame that meets specified conditions may not be displayed in the preview screen, so that users can flexibly select the objects they want to shoot according to their preferences.

[0043] Step S130: Acquire the coordinates of the touch operation on the preview image, and construct a region of interest based on the coordinates and the object detection frame that meets the specified conditions.

[0044] As an implementation method, the user can click on the object displayed in the preview screen to select the user's preferred shooting object in the current shooting screen. In this way, the coordinates of the touch operation acting on the preview screen can be obtained, and the region of interest is constructed based on the coordinates and the object detection frame that meets the specified conditions. The region of interest refers to the location area where the object that the user prefers to shoot is located.

[0045] Among them, when the user clicks on an object displayed in the preview screen, if the object detection box corresponding to the object is displayed, when the user clicks on the object, it is regarded as clicking on the object detection box corresponding to the object; if the object detection box corresponding to the object is not displayed, when the user clicks on the object, the user may click on any position of the object. For the convenience of calculation, it is regarded as clicking on the center coordinates of the object's position.

[0046] Step S140: obtaining an aesthetic composition frame corresponding to the region of interest.

[0047] When the region of interest is constructed, an aesthetic composition frame corresponding to the region of interest may be obtained to beautify the composition of the image to be captured.

[0048] As an implementation method, the image of the region of interest may be input into the aesthetic composition model, and then the aesthetic composition frame may be obtained.

[0049] Step S150: adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview image to obtain a target composition.

[0050] In this embodiment, the aesthetic composition frame is an object detection frame obtained after multiple size compressions. In order to avoid the appearance of black edges in the subsequent display of the image content corresponding to the aesthetic composition frame, which affects the display effect, and in order to show the composition effect of automatically completing the zoom composition based on the area of ​​interest, the display size of the image content corresponding to the aesthetic composition frame can be adjusted to the display size of the preview screen to obtain the target composition, so that you only need to turn on the function switch (that is, the aforementioned AI targeted composition function control) to zoom in with one key. The zoomed picture presents a good composition, which can bring an automatic zoom composition experience. For the specific process of how to adjust the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen, please refer to the relevant description in the subsequent embodiments.

[0051] In this implementation, the workflow of step S110 to step S150 may be used as the specific working principle of the aforementioned AI targeted composition shooting mode.

[0052] In the present embodiment, when the target composition frame is obtained, when the user clicks the shutter of the camera, it can trigger the generation of a shooting instruction, and then the electronic device can respond to the shooting instruction to obtain a target image corresponding to the target composition, that is, the size of the image content corresponding to the target composition can be enlarged with a specified zoom ratio to obtain the corresponding target image. At this time, the captured target image can be displayed on the screen of the electronic device.

[0053] The composition method provided in this embodiment obtains a preview screen captured by an electronic device; obtains an object detection frame that meets specified conditions in the preview screen; obtains the coordinates of the touch operation acting on the preview screen, and constructs an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; obtains an aesthetic composition frame corresponding to the area of ​​interest; and adjusts the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain a target composition. Thus, through the above method, it is possible to determine the area of ​​interest based on the points of interest that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, satisfying the user's preference for composition, enriching the composition method during the shooting process, and thus improving the user's shooting experience.

[0054] At the same time, by clicking on the preview screen, the camera is triggered to automatically zoom and then capture the corresponding image, which increases the fun of image capture.

[0055] See also Figure 3 , shows a flow chart of a patterning method provided by another embodiment of the present application. This embodiment provides a patterning method that can be applied to an electronic device. The method includes:

[0056] Step S210: Acquire a preview image captured by the electronic device.

[0057] The specific implementation of step S210 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0058] Step S220: obtaining a thumbnail of the preview image.

[0059] In this embodiment, when a preview screen is obtained, in order to reduce the amount of data processing of related models in the composition process, a thumbnail of the preview screen can be obtained, that is, the preview screen is compressed in size to obtain a thumbnail. The size compression here can also be understood as reducing the pixel value of a unit size of the image by a certain multiple (that is, a thumbnail of the preview screen can be obtained after pixel compression of the preview screen).

[0060] Among them, the size of the thumbnail of the preview screen can be 1080P, or a size of other numerical values. The specific numerical values ​​may not be limited. The implementation method of the present application takes the size of the thumbnail of the preview screen as 1080P as an example to illustrate, and the content of the thumbnail of the preview screen is the same as the content of the preview screen.

[0061] Step S230: input the thumbnail into a target detection model, and obtain a target detection result map output by the target detection model, wherein the target detection result map includes at least one object detection box.

[0062] In this implementation, in order to adapt to the data processing type of the model, the thumbnail of the preview screen can be converted into RGB format, and the RGB value normalization processing can be performed on the thumbnail after the format conversion. The normalized image only uses two bytes to represent a pixel value, which can remove the influence of lighting and shadows.

[0063] Furthermore, the normalized thumbnail may be input into a target detection model to obtain a target detection result image output by the target detection model, wherein the target detection result image includes at least one object detection box.

[0064] In this embodiment, before the normalized thumbnail is input into the target detection model, the normalized thumbnail may be preprocessed, for example, the normalized thumbnail is scaled to a first size (the first size here refers to the standard size of the input image of the target detection model, and the specific value may not be limited). Optionally, after the thumbnail is converted into RGB format, it may be scaled to the first size first, and then the RGB value normalization may be performed.

[0065] As an implementation method, a thumbnail that has undergone RGB value normalization and scaled to a first size can be input into a target detection model, and then a target detection result map output by the target detection model is obtained. The target detection result map includes at least one object detection box, and each object detection box has corresponding coordinates.

[0066] In this embodiment, for multiple object detection frames, the types of objects framed by each object detection frame may be the same or different.

[0067] Step S240: If the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, the object detection frame is used as an object detection frame that meets a specified condition.

[0068] As an implementation method, in order to improve data processing efficiency, for multiple object detection frames, the ratio of the size of each object detection frame to the size of the thumbnail can be compared with a preset value to screen out object detection frames that are too small (assuming that object detection frames that are too small are not convenient for providing optimal imaging sizes), and the specific numerical value of the preset value may not be limited.

[0069] If the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, the object detection frame may be used as an object detection frame that meets the specified condition.

[0070] In the process of comparing the ratio of the size of each object detection frame to the size of the thumbnail with the preset value, as a method, the ratio of the short side of the object detection frame to the long side of the thumbnail can be compared with the preset value. In this way, if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to the preset value, the object detection frame whose ratio of the corresponding short side to the long side of the thumbnail is greater than or equal to the preset value can be used as the object detection frame that meets the specified condition, so as to more accurately screen out the object detection frame that is not convenient for providing the optimal imaging size.

[0071] This embodiment is described by taking the target detection result image including multiple object detection frames as an example. When the target detection result image includes multiple object detection frames, for each object detection frame, the ratio of the short side of the object detection frame to the long side of the thumbnail and the size relationship of the preset value can be determined, and then the object detection frame that meets the specified conditions is obtained according to the determination result. The specific value of the preset value may not be limited.

[0072] As a method, if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, the object detection frame whose corresponding short side to the long side of the thumbnail is greater than or equal to the preset value among multiple object detection frames can be used as the object detection frame that meets the specified condition. If the ratio of the short side of the object detection frame to the long side of the thumbnail is less than the preset value, the object detection frame can be directly discarded to avoid calculation errors caused by too small object detection frames.

[0073] In a specific application scenario, see Figure 4 , shows a schematic diagram of the principle of obtaining an object detection frame that meets specified conditions provided by an embodiment of the present application. Figure 4 As shown in FIG, it is assumed that the target detection result graph output by the target detection model (the content of the graph is still a thumbnail, which is equivalent to labeling the thumbnail in the form of an object detection frame) includes 5 object detection frames, namely Figure 4 The object detection frames numbered 1, 2, 3, 4, and 5 shown in the figure can obtain the ratio of the short side of the object detection frames numbered 1, 2, 3, 4, and 5 to the long side of the thumbnail when obtaining the object detection frames that meet the specified conditions. If the obtained ratio is less than 3%, the object detection frame can be ignored; if the obtained ratio is greater than or equal to 3%, the object detection frame can be used as the object detection frame that meets the specified conditions. Figure 4 , assuming that after comparison, the ratio of the short side of the object detection frame numbered 2 to the long side of the thumbnail is less than 3%, and the ratios of the short sides of the object detection frames numbered 1, 3, 4, and 5 to the long sides of the thumbnail are all greater than or equal to 3%, then the object detection frame numbered 2 can be discarded, and the object detection frames numbered 1, 3, 4, and 5 are used as the object detection frames that meet the specified conditions.

[0074] In the present embodiment, when it is determined that the object detection frame has a ratio of the corresponding short side to the long side of the thumbnail that is greater than or equal to the preset value, it is taken into consideration that when the user clicks on the screen of the electronic device to determine the preferred photographic object, the user may not be able to accurately click on the location area where the aforementioned object detection frame (i.e., the object detection frame has a ratio of the corresponding short side to the long side of the thumbnail that is greater than or equal to the preset value) is located due to hand shaking or click deviation, and thus the user's preferred photographic object cannot be accurately obtained.

[0075] In order to optimize the above problem, when it is determined that the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, the boundaries of these object detection frames can be expanded to enlarge the object detection frames to increase the possibility of accurately acquiring the user's preferred shooting object, and then the object detection frames with expanded boundaries are used as object detection frames that meet the specified conditions. There are multiple implementation methods for how to expand the boundaries of these object detection frames.

[0076] As an implementation method, the length of the long side and the length of the short side of the object detection box can be adaptively increased according to the ratio of the long side to the short side of the thumbnail, so that the ratio of the long side to the short side of the object detection box is as close to or the same as the ratio of the long side to the short side of the thumbnail as possible.

[0077] For example, in a specific application scenario, Figure 4 Taking the thumbnail and object detection frame 3 shown as an example, assuming that the long side of the thumbnail is 42 (the unit is not limited), the short side is 36, the long side of the object detection frame 3 is 14, and the short side is 8. Since the ratio of the long side to the short side of the thumbnail is 42 / 36=7 / 6, and the ratio of the long side to the short side of the object detection frame 3 is 14 / 8=7 / 4, the long side of the object detection frame 3 can be kept at 14, and the short side of the object detection frame 3 can be increased from 8 to 12, so that the ratio of the long side to the short side of the object detection frame 3 changes to 14 / 12=7 / 6, that is, the ratio of the long side to the short side of the object detection frame 3 is the same as the ratio of the long side to the short side of the thumbnail. It should be noted that the numerical values ​​shown in this embodiment are only used as examples and do not constitute a limitation of this solution.

[0078] As another implementation method, if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, the short side of the object detection frame can be kept unchanged, and the long side of the object detection frame can be extended by a specified value to expand the area of ​​the object detection frame. The size of the object detection frame after the area expansion is smaller than the size of the thumbnail.

[0079] For example, in a specific application scenario, Figure 4Taking the thumbnail and the object detection frame 3 shown as an example, for the object detection frame 3, the short side of the object detection frame 3 can also be kept unchanged at 8, and its long side can be extended by a fixed length. For example, the fixed length can be 5% (i.e. 2.1) of the long side of the thumbnail, then the long side of the object detection frame 3 can be changed from 14 to 16.1. Among them, the fixed length can also be other values, and the specific value can be not limited.

[0080] In this embodiment, no matter which method is used to expand the boundary of the object detection frame, the size of the object detection frame after the boundary is expanded is smaller than the size of the thumbnail.

[0081] In this embodiment, after the boundary of the object detection frame whose ratio of the corresponding short side to the long side of the thumbnail is greater than or equal to a preset value is expanded, some object detection frames may cross or overlap. In this case, when the user clicks on a certain position, it may not be possible to accurately determine the photographed object that the user actually wants to select, resulting in misjudgment.

[0082] In order to optimize the above problem, assuming that the target detection result image includes two object detection frames, this embodiment adopts the method of merging the two object detection frames if there is intersection or overlap between the two object detection frames after boundary expansion to obtain a merged object detection frame, and then the merged object detection frame can be used as an object detection frame that meets the specified conditions. It should be noted that if the number of object detection frames is greater than two, the processing principle is similar to this.

[0083] As a specific implementation method, after expanding the boundaries of the object detection frame whose ratio of the corresponding short side to the long side of the thumbnail is greater than or equal to a preset value, all object detection frames can be traversed and the object detection frames that overlap with each other can be merged two by two until there are no overlapping frames in the picture.

[0084] When merging two intersecting or overlapping object detection frames, the outer edges of the two intersecting object detection frames may be used as boundaries to form a new object detection frame as a merged object detection frame.

[0085] For example, in a specific application scenario, such as Figure 5 As shown, it shows a principle example diagram of merging two intersecting or overlapping object detection frames provided by an embodiment of the present application. Figure 5 As shown in the figure above, assuming that the object detection frame numbered 6 and the object detection frame numbered 7 intersect, then Figure 5 As shown by the dotted line in the middle figure, the boundary of the object detection frame is re-determined to merge the object detection frame numbered 6 and the object detection frame numbered 7. The object detection frame obtained after the merger is as follows Figure 5As shown in the figure below, the number is the new number "8", and the merged object detection frame is one object detection frame. It should be noted that when the number of object detection frames is greater than two, and there are intersections or overlaps between them, the merging principle is the same as Figure 5 The above are similar and will not be described one by one here.

[0086] It should be noted that the above example of the ratio of the short side of the object detection box to the long side of the thumbnail being greater than or equal to the preset value is only used to illustrate the scheme and does not constitute a limitation on this scheme. In actual implementation, it can also be configured that the ratio of the short side of the object detection box to the short side of the thumbnail is greater than or equal to the specified value, and the specific numerical value of the specified value may not be limited.

[0087] Step S250: Acquire the coordinates of the touch operation on the preview image, and construct a region of interest based on the coordinates and the object detection frame that meets the specified conditions.

[0088] The specific implementation of step S250 may refer to the relevant description of step S130 in the aforementioned embodiment, which will not be repeated here.

[0089] Step S260: obtaining an aesthetic composition frame corresponding to the region of interest.

[0090] The specific implementation of step S260 may refer to the relevant description of step S140 in the aforementioned embodiment, which will not be repeated here.

[0091] Step S270: adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview image to obtain a target composition.

[0092] The specific implementation of step S270 may refer to the relevant description of step S150 in the aforementioned embodiment and subsequent embodiments, and will not be repeated here.

[0093] In this implementation, the workflow of step S210 to step S270 may be used as the specific working principle of the aforementioned AI targeted composition shooting mode.

[0094] The composition method provided in this embodiment is as follows: obtaining a preview screen captured by an electronic device; obtaining a thumbnail of the preview screen; inputting the thumbnail into a target detection model, obtaining a target detection result map output by the target detection model, wherein the target detection result map includes at least one object detection frame; if the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, using the object detection frame as an object detection frame that meets specified conditions; obtaining the coordinates of the touch operation acting on the preview screen, and constructing an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; obtaining an aesthetic composition frame corresponding to the area of ​​interest; adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen, and obtaining a target composition. Thus, through the above method, it is possible to determine the area of ​​interest based on the points of interest that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, satisfying the user's preference for composition, enriching the composition method during the shooting process, and thus improving the user's shooting experience.

[0095] At the same time, by combining multiple judgment logics to determine the object detection frame that meets the specified conditions, it is possible to accurately determine the user's preferred shooting object, further improving the user's shooting experience.

[0096] See also Figure 6 , shows a flow chart of a patterning method provided by another embodiment of the present application. This embodiment provides a patterning method that can be applied to an electronic device. The method includes:

[0097] Step S310: Acquire a preview image captured by the electronic device.

[0098] The specific implementation of step S310 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0099] Step S320: Obtain a thumbnail of the preview image.

[0100] The specific implementation of step S320 may refer to the relevant description of step S220 in the aforementioned embodiment, which will not be repeated here.

[0101] Step S330: input the thumbnail into a target detection model, and obtain a target detection result map output by the target detection model, wherein the target detection result map includes at least one object detection box.

[0102] The specific implementation of step S330 may refer to the relevant description of step S230 in the aforementioned embodiment, which will not be repeated here.

[0103] Step S340: If the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, the object detection frame is used as an object detection frame that meets a specified condition.

[0104] The specific implementation of step S340 may refer to the relevant description of step S240 in the aforementioned embodiment, which will not be repeated here.

[0105] Step S350: Acquire the coordinates of the touch operation on the preview image.

[0106] The specific type of touch operation may not be limited, for example, it may be a touch operation of a click (including a single click, a double click, or a triple click, etc.), a tap, etc. If it is a touch operation of a double click or a triple click, the coordinates are obtained based on the next click operation. Optionally, if it is a slide operation, the coordinates are obtained based on the end point of the slide.

[0107] When the preview screen is obtained, a corresponding coordinate system can be constructed so that when a touch operation is performed on the preview screen, the coordinates of the touch operation on the preview screen can be quickly obtained. Alternatively, the present application can also use any method to obtain the coordinates of the touch operation on the preview screen.

[0108] Step S361: If the coordinates are located within the object detection frame that meets the specified conditions, a region including the object detection frame that meets the specified conditions is taken as a region of interest.

[0109] In this embodiment, when the coordinates of the touch operation on the preview screen are obtained, the region of interest can be constructed based on the coordinates and the object detection frame that meets the specified conditions. Specifically, the method for obtaining the region of interest can be determined based on the positional relationship between the coordinates and the object detection frame that meets the specified conditions.

[0110] As an implementation manner, if the coordinates are located within an object detection frame that satisfies a specified condition, the area including the object detection frame that satisfies the specified condition may be directly used as the region of interest. Figure 4 Taking the object detection frame numbered 3 as an example, assuming that the object detection frame numbered 3 is an object detection frame that meets the specified conditions, when the coordinates of the touch operation applied to the preview screen are located inside the object detection frame numbered 3 (any position within the object detection frame), the area where the object detection frame numbered 3 is located can be directly used as the area of ​​interest.

[0111] Optionally, in this way, in order to avoid the object detection frame being too small to accurately determine the user's selected shooting object, the area of ​​the object detection frame that meets the specified conditions can also be enlarged, and the area where the enlarged object detection frame is located is used as the region of interest. It should be noted that in the process of enlarging the area of ​​the object detection frame that meets the specified conditions, the long side of the enlarged region of interest does not exceed 50% of the long side of the thumbnail, because if the region of interest is too large, for example, as large as the thumbnail, the frame of the composition will be larger, and the zoom ratio will be smaller, which will affect the user's shooting experience.

[0112] Step S362: If the coordinates are outside the object detection frame that meets the specified conditions, a rectangular area centered on the coordinates is obtained as a region of interest.

[0113] As another implementation, if the coordinates are outside the object detection frame that meets the specified conditions, a rectangular area (which may be a square or a rectangle) may be obtained with the coordinates as the center as the region of interest. Similarly, when a rectangular area is taken as the region of interest, the long side of the enlarged region of interest does not exceed 50% of the long side of the thumbnail.

[0114] In this implementation manner, one zoom shooting process includes one region of interest.

[0115] Step S370: Acquire an aesthetic composition frame corresponding to the region of interest.

[0116] The specific implementation of step S370 may refer to the relevant description of step S140 in the aforementioned embodiment, which will not be repeated here.

[0117] Step S380: adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview image to obtain a target composition.

[0118] The specific implementation of step S380 may refer to the relevant description of step S150 in the aforementioned embodiment, which will not be repeated here.

[0119] In this implementation, the workflow of step S310 to step S380 can be used as the specific working principle of the aforementioned AI targeted composition shooting mode.

[0120] The composition method provided in the present embodiment comprises the following steps: obtaining a preview screen captured by an electronic device; obtaining a thumbnail of the preview screen; inputting the thumbnail into a target detection model to obtain a target detection result map output by the target detection model, wherein the target detection result map includes at least one object detection frame; if the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, using the object detection frame as an object detection frame that meets specified conditions; obtaining the coordinates of a touch operation applied to the preview screen; if the coordinates are within the object detection frame that meets the specified conditions, using the area including the object detection frame that meets the specified conditions as an area of ​​interest; if the coordinates are outside the object detection frame that meets the specified conditions, obtaining a rectangular area with the coordinates as the center as the area of ​​interest; obtaining an aesthetic composition frame corresponding to the area of ​​interest; and adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain a target composition. Therefore, through the above method, it is possible to determine the area of ​​interest based on the interest points that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, which meets the user's preference for composition, enriches the composition method during the shooting process, and thus improves the user's shooting experience.

[0121] At the same time, by determining the method of obtaining the area of ​​interest based on the coordinates of the touch operation on the preview screen and the positional relationship with the object detection frame that meets the specified conditions, it is possible to flexibly confirm the user's preferred shooting object, thereby enhancing the shooting-friendly experience.

[0122] See also Figure 7 , shows a flow chart of a composition method provided by another embodiment of the present application. This embodiment provides a composition method that can be applied to an electronic device. The method includes:

[0123] Step S410: Acquire a preview image captured by the electronic device.

[0124] The specific implementation of step S410 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0125] Step S420: obtaining a thumbnail of the preview image.

[0126] The specific implementation of step S420 may refer to the relevant description of step S220 in the aforementioned embodiment, which will not be repeated here.

[0127] Step S430: input the thumbnail into a target detection model, and obtain a target detection result map output by the target detection model, wherein the target detection result map includes at least one object detection box.

[0128] The specific implementation of step S430 may refer to the relevant description of step S230 in the aforementioned embodiment, which will not be repeated here.

[0129] Step S440: If the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, the object detection frame is used as an object detection frame that meets a specified condition.

[0130] The specific implementation of step S440 may refer to the relevant description of step S240 in the aforementioned embodiment, which will not be repeated here.

[0131] Step S450: Acquire the coordinates of the touch operation on the preview image.

[0132] The specific implementation of step S450 may refer to the relevant description of step S350 in the aforementioned embodiment, which will not be repeated here.

[0133] Step S461: If the coordinates are located within the object detection frame that meets the specified conditions, a region including the object detection frame that meets the specified conditions is taken as a region of interest.

[0134] The specific implementation of step S461 may refer to the relevant description of step S361 in the aforementioned embodiment, which will not be repeated here.

[0135] Step S462: If the coordinates are outside the object detection frame that meets the specified conditions, a rectangular area centered on the coordinates is obtained as a region of interest.

[0136] The specific implementation of step S462 may refer to the relevant description of step S362 in the aforementioned embodiment, which will not be repeated here.

[0137] Step S470: inputting the image of the region of interest into an aesthetic composition model, and obtaining coordinates of a reference composition frame output by the aesthetic composition model, wherein the size of the reference composition frame is smaller than the size of the thumbnail.

[0138] As described in the above embodiment, in the process of obtaining the aesthetic composition frame corresponding to the region of interest, the image of the region of interest can be input into the aesthetic composition model, and then the reference composition frame coordinates output by the aesthetic composition model are obtained, and the size of the reference composition frame is smaller than the size of the thumbnail.

[0139] Before inputting the image of the region of interest into the aesthetic composition model, the region of interest may be preprocessed first. For example, the region of interest may be scaled to a second size (the second size here refers to a standard size of the input image of the aesthetic composition model, and the specific value may not be limited), and then the image of the region of interest scaled to the second size may be input into the aesthetic composition model.

[0140] Step S480: Mapping the reference composition frame coordinates to the thumbnail to obtain an aesthetic composition frame corresponding to the region of interest.

[0141] In this embodiment, for the image of the region of interest, after it is input into the aesthetic composition model, the size of the reference composition frame output by the aesthetic composition model is smaller than the size of the region of interest, and the size of the region of interest is smaller than the size of the thumbnail. Therefore, when mapping the reference composition frame coordinates to the thumbnail, the proportion of the reference composition frame (in this embodiment, it can be the ratio of the long side to the short side of the reference composition frame, or the ratio of the short side to the short side of the reference composition frame) can be first restored to the proportion of the region of interest to obtain the composition frame to be processed, and then the proportion of the composition frame to be processed is restored to the proportion of the thumbnail.

[0142] As a specific implementation method, the reference composition frame coordinates can be first mapped to the region of interest to obtain the coordinates to be processed (i.e., the coordinates representing the area enclosed by the composition frame to be processed), and then the coordinates to be processed can be mapped to a thumbnail to obtain the target coordinates, and the area enclosed by the target coordinates is used as the aesthetic composition frame corresponding to the region of interest.

[0143] In a specific application scenario, see Figure 8 , shows an example diagram of the process of mapping the reference composition frame coordinates to the thumbnail provided by the embodiment of the present application. Figure 8 As shown, assuming that the coordinate system of the reference composition frame is x-0-y, the coordinate system of the region of interest is x1-0-y1, and the coordinate system of the thumbnail is x2-0-y2, as an implementation method, Figure 8 The coordinates of the reference frame shown in the left figure are mapped to the coordinate system of the region of interest as x1-0-y1, and we get Figure 8 The region of interest shown in the middle figure, and then the coordinates of the region of interest are mapped to the coordinate system of the thumbnail as x2-0-y2, Figure 8 As shown in the thumbnail on the right, in this way, you can Figure 8 The area enclosed by the thumbnails shown in the right figure serves as the aesthetic composition frame corresponding to the region of interest.

[0144] It should be noted that the size of the aesthetic composition frame corresponding to the region of interest may be larger than the size of the region of interest, or smaller than the size of the region of interest but smaller than the size of the thumbnail.

[0145] Step S490: adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview image to obtain a target composition.

[0146] When the aesthetic composition frame is obtained, in order to avoid black edges on the image and affect the display effect, the display size of the image content corresponding to the aesthetic composition frame can be adjusted to the display size of the preview screen to obtain the target composition.

[0147] As an implementation method, the coordinates of the aesthetic composition frame can be first mapped to the coordinates of the coordinate system corresponding to the preview screen to obtain the composition frame to be confirmed, and then the composition frame to be confirmed and other object detection frames can be judged in the manner described in the above embodiment. Whether there is intersection or overlap between the composition frame to be confirmed and other object detection frames (here, it can be the above-mentioned object detection frame that meets the specified conditions) can be determined. Optionally, if it is determined that there is intersection or overlap between the composition frame to be confirmed and other object detection frames (here, it can be the aforementioned object detection frame that meets the specified conditions), the composition frame surrounded by the outer edges of the two intersecting frames can be obtained as a new composition frame to be confirmed, and the image content corresponding to the new composition frame to be confirmed can be used as the target composition; Optionally, if it is determined that there is no intersection or overlap between the composition frame to be confirmed and other object detection frames, the image content corresponding to the composition frame to be confirmed can be directly used as the target composition. By again judging whether there is intersection or overlap between the composition frame to be confirmed and other object detection frames, the accuracy of the composition can be improved.

[0148] In this implementation, when obtaining the target composition, a relevant scoring model can be added to obtain the aesthetic score of the target composition. If the score is lower than a threshold, the obtained target composition is ignored. If the score is higher than (indicating greater than or equal to) the threshold, it indicates that the composition effect is good and the target composition is retained.

[0149] In this implementation, the workflow of step S410 to step S490 may be used as the specific working principle of the aforementioned AI targeted composition shooting mode.

[0150] The composition method provided in this embodiment comprises the following steps: obtaining a preview screen captured by an electronic device; obtaining a thumbnail of the preview screen; inputting the thumbnail into a target detection model to obtain a target detection result map output by the target detection model, wherein the target detection result map includes at least one object detection frame; if the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, taking the object detection frame as an object detection frame that meets a specified condition; obtaining the coordinates of a touch operation applied to the preview screen; if the coordinates are within the object detection frame that meets the specified condition, taking the area including the object detection frame that meets the specified condition as an area of ​​interest; if the coordinates are outside the object detection frame that meets the specified condition, obtaining a rectangular area centered on the coordinates as an area of ​​interest; inputting the image of the area of ​​interest into an aesthetic composition model to obtain coordinates of a reference composition frame output by the aesthetic composition model, wherein the size of the reference composition frame is smaller than the size of the thumbnail; mapping the reference composition frame coordinates to the thumbnail to obtain an aesthetic composition frame corresponding to the area of ​​interest; adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain a target composition. Therefore, through the above method, it is possible to determine the area of ​​interest based on the interest points that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, which meets the user's preference for composition, enriches the composition method during the shooting process, and thus improves the user's shooting experience.

[0151] At the same time, by adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview image, a target composition with high-resolution display can be obtained.

[0152] See also Fig. 9 , is a structural block diagram of a composition device provided in an embodiment of the present application. This embodiment provides a composition device 500, which can be run on an electronic device. The device 500 includes a preview image acquisition module 510, an object detection frame acquisition module 520, an area of ​​interest acquisition module 530, an aesthetic composition frame acquisition module 540 and a composition module 550:

[0153] The preview image acquisition module 510 is used to acquire the preview image captured by the electronic device.

[0154] The object detection frame acquisition module 520 is used to acquire an object detection frame that meets a specified condition in the preview image.

[0155] As a method, the object detection frame acquisition module 520 can be used to obtain a thumbnail of the preview screen; input the thumbnail into the target detection model, and obtain the target detection result image output by the target detection model, wherein the target detection result image includes at least one object detection frame; if the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, the object detection frame is used as an object detection frame that meets the specified conditions.

[0156] As a specific implementation, the object detection frame acquisition module 520 can be used to, if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, use the object detection frame whose corresponding short side to the long side of the thumbnail is greater than or equal to a preset value as the object detection frame that meets the specified conditions.

[0157] Among them, if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, using the object detection frame whose corresponding short side to the long side of the thumbnail is greater than or equal to the preset value as the object detection frame that meets the specified conditions may include: if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to the preset value, expanding the boundary of the object detection frame; and using the object detection frame with the expanded boundary as the object detection frame that meets the specified conditions.

[0158] Furthermore, if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, expanding the boundary of the object detection frame may include: if the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, keeping the short side of the object detection frame unchanged, and extending the long side of the object detection frame by a specified value to expand the area of ​​the object detection frame, the size of the object detection frame after the area expansion is smaller than the size of the thumbnail.

[0159] As an implementation manner, the device 500 may further include a merging module, which is used for, if the target detection result image includes two object detection frames, before using the object detection frame after the boundary expansion as the object detection frame that meets the specified conditions, if the two object detection frames after the boundary expansion intersect or overlap, merging the two object detection frames to obtain a merged object detection frame. In this manner, using the object detection frame after the boundary expansion as the object detection frame that meets the specified conditions includes: using the merged object detection frame as the object detection frame that meets the specified conditions.

[0160] In this implementation, the object detection frame acquisition module 520 may also be configured to discard the object detection frame if the ratio of the short side of the object detection frame to the long side of the thumbnail is less than a preset value.

[0161] The region of interest acquisition module 530 is used to acquire the coordinates of the touch operation acting on the preview image, and construct a region of interest based on the coordinates and the object detection frame that meets the specified conditions.

[0162] As an embodiment, the region of interest acquisition module 530 can be used to, if the coordinates are located within the object detection frame that meets the specified conditions, take the area including the object detection frame that meets the specified conditions as the region of interest; if the coordinates are located outside the object detection frame that meets the specified conditions, acquire a rectangular area with the coordinates as the center as the region of interest.

[0163] The aesthetic composition frame acquisition module 540 is used to acquire the aesthetic composition frame corresponding to the region of interest.

[0164] As a method, the aesthetic composition frame acquisition module 540 can be used to input the image of the region of interest into the aesthetic composition model, obtain the reference composition frame coordinates output by the aesthetic composition model, and the size of the reference composition frame is smaller than the size of the thumbnail; map the reference composition frame coordinates to the thumbnail to obtain the aesthetic composition frame corresponding to the region of interest.

[0165] The composition module 550 is used to adjust the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview image to obtain a target composition.

[0166] Optionally, the device 500 may also include an aesthetic composition frame updating module, which is used to, before adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain the target composition, if the aesthetic composition frame intersects or overlaps with other object detection frames, obtain a composition frame surrounded by the outer edges of the two intersecting frames as a new aesthetic composition frame.

[0167] Optionally, the device 500 may further include an image acquisition module for, after adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain a target composition, responding to a shooting instruction to acquire a target image corresponding to the target composition.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0169] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0170] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0171] See also Fig.10 Based on the above-mentioned patterning method and device, the embodiment of the present application also provides an electronic device 100 that can execute the above-mentioned patterning method. The electronic device 100 includes a memory 102 and one or more (only one is shown in the figure) processors 104 coupled to each other, and the memory 102 and the processor 104 are connected by a communication line. The memory 102 stores a program that can execute the content of the above-mentioned embodiment, and the processor 104 can execute the program stored in the memory 102.

[0172] Among them, the processor 104 may include one or more processing cores. The processor 104 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 102, and calling data stored in the memory 102. Optionally, the processor 104 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 104 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 104, but may be implemented separately through a communication chip.

[0173] The memory 102 may include a random access memory (RAM) or a read-only memory (ROM). The memory 102 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the aforementioned embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0174] Please refer to Fig.11 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 600 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0175] The computer readable storage medium 600 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 600 has storage space for program code 610 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 610 can be compressed, for example, in an appropriate form.

[0176] In summary, the embodiments of the present application provide a composition method, device, electronic device and storage medium. The method is applied to an electronic device. The method obtains a preview screen captured by the electronic device; obtains an object detection frame that meets specified conditions in the preview screen; obtains the coordinates of the touch operation acting on the preview screen, and constructs an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; obtains an aesthetic composition frame corresponding to the area of ​​interest; adjusts the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen, and obtains the target composition. Thus, through the above method, it is possible to determine the area of ​​interest based on the points of interest that the user wants to compose, and then automatically complete the zoom composition based on the area of ​​interest, which satisfies the user's preference for composition, enriches the composition method during the shooting process, and thus improves the user's shooting experience.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A composition method, characterized in that: Applied to electronic equipment, the method comprises: Obtaining a preview image captured by an electronic device; Obtain an object detection frame that meets specified conditions in the preview image; Acquire coordinates of a touch operation applied to the preview image, and construct a region of interest based on the coordinates and the object detection frame satisfying specified conditions; Acquire an aesthetic composition frame corresponding to the region of interest; The display size of the image content corresponding to the aesthetic composition frame is adjusted to the display size of the preview image to obtain a target composition.

2. The method according to claim 1, characterized in that The obtaining of an object detection frame satisfying a specified condition in the preview image includes: Obtaining a thumbnail of the preview image; Inputting the thumbnail into a target detection model, obtaining a target detection result graph output by the target detection model, wherein the target detection result graph includes at least one object detection box; If the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, the object detection frame is used as an object detection frame that meets a specified condition.

3. The method according to claim 2, characterized in that If the ratio of the size of the object detection frame to the size of the thumbnail is greater than or equal to a preset value, taking the object detection frame as an object detection frame that satisfies a specified condition, comprises: If the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, the object detection frame whose corresponding short side to the long side of the thumbnail is greater than or equal to the preset value is used as the object detection frame that meets the specified conditions.

4. The method according to claim 3, characterized in that If the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, the object detection frame whose ratio of the corresponding short side to the long side of the thumbnail is greater than or equal to the preset value is used as the object detection frame that meets the specified condition, including: If the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, expanding the boundary of the object detection frame; The object detection frame after boundary expansion is used as the object detection frame that meets the specified conditions.

5. The method according to claim 4, characterized in that If the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, expanding the boundary of the object detection frame includes: If the ratio of the short side of the object detection frame to the long side of the thumbnail is greater than or equal to a preset value, the short side of the object detection frame is kept unchanged, and the long side of the object detection frame is extended by a specified value to expand the area of ​​the object detection frame. The size of the object detection frame after the area expansion is smaller than the size of the thumbnail.

6. The method according to claim 4, characterized in that If the target detection result image includes two object detection frames, before taking the object detection frame after boundary expansion as the object detection frame that meets the specified condition, the method further includes: If the two object detection frames after boundary expansion intersect or overlap, merge the two object detection frames to obtain a merged object detection frame; The step of using the object detection frame after the boundary expansion as the object detection frame that meets the specified condition includes: The merged object detection frame is used as an object detection frame that meets specified conditions.

7. The method according to claim 3, characterized in that The method further comprises: If the ratio of the short side of the object detection frame to the long side of the thumbnail is smaller than a preset value, the object detection frame is discarded.

8. The method according to any one of claims 1 to 7, characterized in that: The constructing the region of interest based on the coordinates and the object detection frame that meets the specified conditions includes: If the coordinates are located within the object detection frame that meets the specified conditions, a region including the object detection frame that meets the specified conditions is taken as a region of interest; If the coordinates are outside the object detection frame that meets the specified conditions, a rectangular area centered at the coordinates is acquired as the region of interest.

9. The method according to claim 8, characterized in that The step of obtaining an aesthetic composition frame corresponding to the region of interest includes: Inputting the image of the region of interest into an aesthetic composition model, obtaining coordinates of a reference composition frame output by the aesthetic composition model, wherein the size of the reference composition frame is smaller than the size of the thumbnail; The reference composition frame coordinates are mapped to the thumbnail to obtain an aesthetic composition frame corresponding to the region of interest.

10. The method according to claim 1, characterized in that Before adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain the target composition, the method further includes: If the aesthetic composition frame intersects or overlaps with other object detection frames, a composition frame surrounded by outer sides of the two intersecting frames is obtained as a new aesthetic composition frame.

11. The method according to claim 1 or 10, characterized in that: After adjusting the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview screen to obtain the target composition, the method further includes: In response to a shooting instruction, a target image corresponding to the target composition is acquired.

12. A patterning device, characterized in that: Running on an electronic device, the device comprises: A preview image acquisition module, used to acquire a preview image captured by the electronic device; An object detection frame acquisition module is used to acquire an object detection frame that meets specified conditions in the preview image; An area of ​​interest acquisition module, used to acquire the coordinates of the touch operation acting on the preview screen, and construct an area of ​​interest based on the coordinates and the object detection frame that meets the specified conditions; An aesthetic composition frame acquisition module, used to acquire an aesthetic composition frame corresponding to the region of interest; The composition module is used to adjust the display size of the image content corresponding to the aesthetic composition frame to the display size of the preview picture to obtain a target composition.

13. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 11 is executed.

Images