Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an image processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps of displaying a marked area where a target object is located in a preview image; in response to a trigger instruction for the marked area, displaying a marked amplified area obtained by amplifying the marked area; and obtaining a target image based on the image data corresponding to the marked magnified area. By adopting the method, the composition which is more adaptive to the target object can be formed under the condition that the electronic equipment is not changed and moved, so that the information of the target object is more accurately transmitted.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the popularization of shooting functions of electronic devices and the continuous development of image processing technology, more and more users obtain image information by shooting with electronic devices.

[0003] Traditionally, when users struggle to accurately compose their shots, they can use prompts, guide lines, and arrows to guide them. However, this approach requires the photographer to move their electronic device to achieve the desired composition, making it difficult to control the granularity of the operation and ensuring accurate composition, resulting in inaccurate image information. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, which can obtain accurate image information without moving the electronic device.

[0005] In a first aspect, the present application provides an image processing method, comprising:

[0006] Displaying the marked area where the target object is located in the preview image;

[0007] In response to a trigger instruction for the marked area, displaying an enlarged marked area obtained by enlarging the marked area;

[0008] A target image is obtained based on the image data corresponding to the marked magnified area.

[0009] In a second aspect, the present application further provides an image processing device, comprising:

[0010] a marking module, used for displaying a marked area where a target object is located in a preview image;

[0011] an enlarging module, configured to display an enlarged marked area obtained by enlarging the marked area in response to a trigger instruction on the marked area;

[0012] A generating module is used to obtain a target image based on the image data corresponding to the marked magnified area.

[0013] In a third aspect, the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the image processing method of the first aspect is implemented.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the image processing method of the first aspect when the computer program is executed by a processor.

[0015] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the image processing method of the first aspect when executed by a processor.

[0016] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product display the marked area where the target object is located in the preview image, and can form the concept of image composition in the preview stage. In response to the trigger instruction, the marked area is enlarged, and the composition recommendation in the preview stage is realized. Furthermore, based on the image data corresponding to the marked enlarged area, the composition of the marked enlarged area and the content contained therein are used to form a corresponding target image. As a result, without moving the electronic device, the preview image can be converted into image information for the marked area, so that the composition of the marked area replaces the original preview image, and a target image for the target object is obtained, forming a composition that is more suitable for the target object and obtaining more accurate image information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A diagram showing an application environment of an image processing method in one embodiment;

[0019] Figure 2 1 is a flow chart of an image processing method according to an embodiment;

[0020] Figure 3 A schematic diagram of a camera application display interface in one embodiment;

[0021] Figure 4 A schematic diagram of a non-centrally amplified process in one embodiment;

[0022] Figure 5 A schematic diagram of a camera application display interface in one embodiment;

[0023] Figure 6 A schematic diagram of a process for updating a target object in one embodiment;

[0024] Figure 7is a flowchart of an image processing method in a specific embodiment;

[0025] Figure 8 is a flowchart of an image processing method in another specific embodiment;

[0026] Figure 9 is a structural block diagram of an image processing device in one embodiment;

[0027] Figure 10 FIG. 1 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0029] In one embodiment, the image processing method provided by the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the electronic device 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the electronic device 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0030] In some exemplary embodiments, Figure 2 As shown, an image processing method is provided, which is applied to Figure 1 The electronic device 102 in the embodiment is used as an example for explanation, and can also be applied to the server 104. The method includes the following steps 202 to 206.

[0031] Step 202: Display the marked area where the target object is located in the preview image.

[0032] The preview image is the image displayed by the camera app during the preview phase. Image data captured by the camera is cropped to the preview size before being displayed on the camera app's display. The preview size can be the same as the camera app's display size.

[0033] The preview image may be, but is not limited to, an image under a color channel or a gray channel, or an image that has been preprocessed; the preprocessing includes, but is not limited to, beauty enhancement, filtering, fusion of multiple images, etc.; in addition, the preview image may also be an image set according to each scene type.

[0034] The target object is at least one object in the preview image. The target object can be a complete object or a part of a complete object. The categories of the target object may include, but are not limited to, physical categories such as human bodies, animals, and plants, and may also include categories of virtual objects. Virtual objects include, but are not limited to, virtual elements in virtual reality (VR), augmented reality (AR), or mixed reality (MR). Since the preview image at each moment can be an image from real-time acquisition or cache, the target object, the category to which the target object belongs, and the marked area where the target object is located can all change over time.

[0035] Taking the target object as a human body existing in the earth coordinate system as an example, the category to which the target object belongs can be divided according to the range of the human body, which can include a human body in the range of a complete human body, or a half-body human body in the range of a half-body human body, a head image in the range of a human head, and a foot image in the range of a human foot; wherein, the human body in the range of a complete human body can include the head and body of the person, or the head, body and feet of the person; the half-body human body can be the upper body from the waist and above the waist, or the upper body from the legs and above the legs, or the lower body from the waist and below the waist, or the lower body from the legs and below the legs; the head image in the range of a human head can include the face of the person, and can also include the neck, top of the head and other areas around the face.

[0036] A marked area is used to mark the area where the target object is located. A marked area contains marking information, which is used to distinguish the marked area from areas outside the marked area and to determine the boundary between the marked and unmarked areas. For example, marking information for a marked area includes, but is not limited to, a marking box and a marking symbol. The marking box can be a solid or dashed frame, and can be square or circular, for example. The marking symbol can be multiple sides or corners of the marking box.

[0037] In some embodiments, a preview image may be captured based on a camera application launch instruction or determined from a cached image. A target object may be determined based on object recognition results from the preview image or based on a target object selection instruction. A marked area including the target object may be determined based on the target object's position within the camera application display. Alternatively, a boundary within a target distance from the target object may be used as the marked area based on the target object's position within the camera application display.

[0038] The preview image displays a marked area where the target object is located. The position or range of the marked area in the camera app display can be adjusted, thereby adjusting the preview image composition. This allows you to change the original preview image imaging method without moving the electronic device, either using the entire preview image or based on the preview image within the marked area.

[0039] Step 204 : In response to the trigger instruction for the marked area, displaying the enlarged marked area obtained by enlarging the marked area.

[0040] A trigger instruction is an instruction adjusted based on different image regions. The trigger instruction can be generated by a click, a slide, or a multi-touch operation. When the trigger instruction is directed to a marked area, the trigger instruction is used to at least instruct the electronic device to zoom in on the marked area, and may also be used to instruct the electronic device to zoom in on the marked area and adjacent areas outside the marked area.

[0041] The enlarged area of the mark indicates the same content as the marked area, and the size of the enlarged area of the mark is larger than the size of the marked area. The enlarged area of the mark indicates the same content as the marked area, including indicating the same target object as the marked area, and may also include indicating the same target object as the marked area and the background area of the same target object. Due to the larger size of the enlarged area of the mark, the target object information can be represented by a larger area of pixels while the camera application display screen size remains unchanged. Therefore, the target object indicated by the enlarged area of the mark is larger in size, and the information can be displayed in a larger size.

[0042] In some embodiments, when at least one of a click operation, a slide operation, and a multi-touch operation occurs in the mark area, a trigger instruction for the mark area is detected.

[0043] In response to a trigger command, the marked area is magnified, thereby changing the original layout of the preview image. This increases the proportion of the composition and information conveyed by the marked area in the camera application display, forming an image display dominated by the marked area. Moreover, the process of displaying the magnified marked area can provide user operation guidance during the preview phase and display the content of the corresponding composition in detail to the user, realizing composition recommendations during the preview phase without the need for text interference. As a result, the way the marked area information is presented can be gradually adjusted through the process of magnifying the marked area without moving the electronic device.

[0044] Step 206: Obtain a target image based on the image data corresponding to the marked magnified area.

[0045] The image data corresponding to the target object is data within the marked magnified area that can be used to form an image. The image data corresponding to the marked magnified area includes at least the pixel values of each pixel in the marked magnified area and may also include acquisition parameters for the marked magnified area to facilitate subsequent processing of the target image. The acquisition parameters may include, but are not limited to, the timestamp of the image acquisition, aperture, ISO, focal length, and other parameters.

[0046] The target image is an image constructed based on at least the image data corresponding to the marked magnified area. The target image's composition matches that of the marked magnified area, enabling information to be expressed through the corresponding composition. The target image's size can match that of the preview image so that all images output by the camera application are of the same size. Because the target image is generated based on the marked magnified area, the composition of the original preview image is gradually replaced by the composition of the marked magnified area, adjusting the information presented during the preview phase and the composition during the capture phase to ensure accurate information transmission.

[0047] In some embodiments, the target image can be determined from the raw image data (RAW) corresponding to the marked magnified area; the target image can also be determined from the data obtained by ISP processing of the raw image data (RAW) corresponding to the marked magnified area; and the white balance coefficient, aperture, ISO, focal length and other acquisition parameters determined in the preview stage of the marked magnified area can also be used as acquisition parameters of the target image.

[0048] In some embodiments, obtaining the target image based on the image data corresponding to the marked zoomed-in area includes: obtaining the target image based on the image data corresponding to the marked zoomed-in area in response to a shooting instruction.

[0049] In some embodiments, when the marked magnification area is located in the first picture area and the unadjusted marked magnification area is used for imaging, the first picture area is used to obtain the target image; in other embodiments, the adjusted marked magnification area is used for imaging, and in this case, the first picture area is used to obtain the adjusted image.

[0050] In the above-mentioned image processing method, displaying the marked area where the target object is located in the preview image can form a concept of image composition during the preview phase, making the marked area where the target object is located substantially different from other areas of the preview image, and providing triggerable composition guidance. Furthermore, in response to a trigger instruction for the marked area, an enlarged marked area is displayed, resulting from the enlargement of the marked area. In this case, the marked area is not only used to display information but also to receive specific trigger instructions. This allows different areas of the preview image to have different instruction triggering methods, forming a trigger instruction location that adaptively adjusts according to the location of the target object. In response to the trigger instruction, the marked area is enlarged, thereby changing the original layout of the preview image, forming an image display method dominated by the marked area, forming an operational guide for the preview phase, and presenting the content of the corresponding composition in detail, thus realizing composition recommendations during the preview phase. Furthermore, based on the image data corresponding to the enlarged marked area, the composition and content of the enlarged marked area are used to form a corresponding target image. As a result, without moving the electronic device, it is possible to convert the preview image into image capture for the marked area, so that the composition of the marked area replaces the original preview image, forming a target image captured for the target object, and forming a composition that is more suitable for the target object, thereby more accurately conveying the information of the object.

[0051] In some embodiments, a method for determining a marked area includes: identifying a target object from a preview image; obtaining a category to which the target object belongs, and obtaining corresponding composition parameters based on the category; and obtaining a marked area where the target object is located based on the composition parameters and the size of the target object.

[0052] The category of the target object is the category of the target object recognition result. The category of the target object includes, but is not limited to, one or more of a human face, a half- or full-body human, a sculpture, an animal, a plant, a landscape, a building, a flower, and a tree. For example, a human statue can be considered a type of human portrait or a type of statue, and can also be considered a combination of both a human portrait and a statue. The category of the target object can also include devices and equipment.

[0053] The composition parameters are marking area parameters with the target object as a reference, and are used to indicate the range of the area to be expanded by the target object. The composition parameters are used to select the area extended outside the target object from the preview image, so as to combine the selected area with the target object into a marking area. When the composition parameters change, the size of the area extended outside the target object also changes accordingly. The larger the area, the more pixels are used to express information, and the smaller the area, the fewer pixels are used to express information. The composition parameters may include composition position and range parameters; the range parameters include but are not limited to parameters such as blank distance, composition ratio, and composition range; wherein the composition position indicates the position of the target object in the marking area, such as centered, left, golden section line, etc.; the blank distance is the number of pixels obtained by extending the target object outside according to the size of the target object; the composition ratio is the ratio obtained by extending the target object outside according to the size of the target object; and the composition range is the area or number of pixels obtained by extending the target object outside according to the size of the target object.

[0054] The marked area includes the target object and the area outside the target object; in the marked area, the target object and the area outside the target object represent different information respectively. The area outside the target object can leave blank space for the target object, thereby producing a corresponding contrast effect. These different information have a mutual contrast effect, thereby highlighting the subjectivity of the target object in the marked area.

[0055] The size of the target object indicates its size in the preview image. The size of the target object can be determined based on the number of pixels or area. The size of the target object can include, but is not limited to, height, width, and diagonal dimensions.

[0056] The target object identification, category determination, and composition parameter acquisition processes all precede the process of obtaining the marked area. Therefore, the marked area that does not contain the target object is dynamically adjusted based on the target object's area range and the composition parameters corresponding to the target object's category. For example, if the target object belongs to one of the categories of face, head, torso, or whole body, the composition parameters set for that category can be used to determine the area extending outward from the target object, and then the composition is used to form the corresponding marked area.

[0057] In some embodiments, identifying the target object from the preview image includes identifying the target object in cached images used to obtain the preview image. Thus, the target object identification process is performed based on the cached images used to obtain the preview image, thereby accelerating the identification process and improving processing efficiency.

[0058] In some embodiments, identifying the target object from the preview image includes: identifying the object selected by the selection instruction in the preview image to obtain the target object. Thus, the target object is first selected by the instruction and then identified to ensure processing accuracy.

[0059] In some embodiments, obtaining the category to which the target object belongs includes obtaining the category to which the target object belongs during recognition of the preview image, thereby using the category information obtained during the recognition process of the target object to ensure processing efficiency.

[0060] In some embodiments, obtaining the category to which the target object belongs includes: performing classification recognition based on the target object to obtain the category to which the target object belongs. Thus, after the target object is recognized, classification recognition is performed based on the target object to ensure accuracy.

[0061] In some embodiments, obtaining corresponding composition parameters according to a category includes: mapping the category based on the correspondence between the category and the composition parameters to obtain the composition parameters corresponding to the category. Thus, mapping is performed based on the preset correspondence to ensure processing efficiency.

[0062] In some embodiments, obtaining corresponding composition parameters according to categories includes: obtaining composition parameters corresponding to the category to which the target object belongs based on the orientation of the target object. Thus, according to the orientation of the target object, the composition parameters are adaptively adjusted to form a variety of composition effects. For example, the target object is a face. At this time, at least one of the first direction and the second direction can be adaptively selected according to the orientation of the face, and the composition parameters for extending the target object are determined from the candidate composition parameters corresponding to the face. The first direction and the second direction are different directions. The first direction can be the height direction of the preview image, and the second direction can be the width direction of the preview image. The height direction of the face and the height direction of the preview image may be the same or there may be a large difference. At this time, the composition parameters can be adaptively adjusted according to the orientation of the face to ensure accurate expression of information.

[0063] In some embodiments, determining the marked area where the target object is located based on the composition parameters and the size of the target object includes determining the marked area where the target object is located based on the sum or difference between the white space distance and the size of the target object. Thus, by adjusting the range of the marked area based on the distance corresponding to the category to which the target object belongs, the composition of the marked area can be similar to the process of photographing a person using a mobile electronic device.

[0064] In some embodiments, determining the marked area where the target object is located based on the composition parameters and the size of the target object includes determining the marked area where the target object is located based on the product or quotient of the composition ratio and the size of the target object. Thus, adaptively adjusting the range of the marked area based on the ratio corresponding to the category to which the target object belongs can make the composition of the marked area more reasonable, more similar to the shooting process of a person using a mobile electronic device, and thus more accurately represent the information.

[0065] In some embodiments, a marked area where the target object is located is obtained based on the composition parameters and the size of the target object, including: selecting a preview image other than the target object based on the composition parameters that are negatively correlated with the size of the target object; and determining the marked area where the target object is located based on the selected area and the target object contained in the selected area.

[0066] Because the composition parameters are negatively correlated with the size of the target object, the larger the target object's area, the smaller the area outside the target object. The area outside the target object is the marked background area; the smaller the target object's area, the larger the marked background area. This allows the marked background area to be adaptively adjusted based on the target object's area. Since the target object's area is set based on the target object, dynamically adjusting the size of the marked area in the preview image based on the target object's size in the preview image allows the background to be adaptively adjusted, producing different compositional effects.

[0067] In some embodiments, the negative correlation between the composition parameters and the size of the target object means: when the target object belongs to the human head category containing a face, the human head composition parameters corresponding to the human head are determined; when the target object belongs to the half-length portrait category or the animal category, the half-length portrait category or the animal category corresponding to the half-length portrait category or the animal category are determined; wherein the human head composition parameters are greater than the half-length composition parameters, so that a larger area outside the target object can be selected through the human head composition parameters.

[0068] For example: through the head composition parameters, the area outside the target object is selected to be 70% of the marked area, the distance between the top of the target object and the top of the marked area accounts for 37% of the marked area, and the distance between the bottom of the target object and the bottom of the marked area accounts for 43% of the marked area; correspondingly, through the half-body composition parameters, the area outside the target object is selected to be 45% of the marked area, the distance between the top of the target object and the top of the marked area accounts for 33% of the marked area, and the distance between the bottom of the target object and the bottom of the marked area accounts for 12% of the marked area.

[0069] In some embodiments, the negative correlation between the composition parameter and the size of the target object means that when the target object is a full-body portrait, a marked background area range that is positively correlated with the area range of the full-body portrait is determined to obtain the marked area.

[0070] The marked background area is negatively correlated with the area range of the target object. This can replace the situation where the electronic device is far away from or close to the target object, thereby achieving composition adjustment when the electronic device is not moving, thereby improving the adjustment efficiency of information expression and more quickly obtaining the marked area under the corresponding composition. In some embodiments, the category to which the target object belongs includes a first human category and a second human category, and the human range corresponding to the first human category is smaller than the human range corresponding to the second human category; the marked area includes the target object and the area outside the target object, and the area outside the target object is the marked background area; the composition parameters corresponding to the first human category are greater than the composition parameters corresponding to the second human category.

[0071] The first and second human categories are relative categories set according to the range of the human body. Compared with the first human category, the second human category has a wider range and is more inclined towards the overall information of the subject; compared with the second human category, the first human category has a smaller range and is more inclined towards the local information of the subject. For example, if the first human category is a face category, the second human category is one or more of a head category containing a face, a half-length portrait category, and a full-length portrait category; if the first human category is a head category, the second human category is one or more of a half-length portrait category and a full-length portrait category; if the first human category is a half-length portrait category, the second human category is a full-length portrait.

[0072] The human body range is the size of the human body in the preview image relative to the entire human body. Since images of the human body can be captured by parts such as the head, upper body, or entire body, the target object's human body range is adjustable. While the human body range is not necessarily related to the size of the target object, it can reflect the distance between the human body and the camera, allowing for more detailed simulation of camera movement in the corresponding shooting scene, thereby refining the composition changes of the marked area.

[0073] In some embodiments, the human body range corresponding to the human body category is used to determine the composition parameters corresponding to the human body category, so that the area range of the marked background area corresponding to one human body category is larger than the area range of the marked background area corresponding to a second human body category. The composition parameters corresponding to the face category indicate that the marked background area occupies 79% or 80% of the marked area; the composition parameters corresponding to the head category indicate that the marked background area occupies 68%, 70%, or 72% of the marked area; the composition parameters corresponding to the half-length portrait category or the animal category indicate that the marked background area occupies 44%, 45%, or 46% of the marked area; and the composition parameters corresponding to the full-length portrait category indicate that the marked background area occupies 40%, 42%, or 43% of the marked area.

[0074] In one example, if Figure 3 As shown, before responding to a trigger instruction for marking an area, the camera application display interface includes a preview image 300; in preview image 300, the marked area includes a target object, a human face 301, and a marked background area 302 of human face 301; after responding to the trigger instruction for marking an area, the camera application display interface includes a first screen area 310 and a second screen area 320 that do not overlap. First screen area 310 includes a magnified area 311 and may also include content outside of magnified area 310. First screen area 310 is used for imaging, while second screen area 320 is not used for imaging. The shape of the magnified area and the preview image can be the same or different.

[0075] In some embodiments, the ratio of the target object pair is first determined based on the first human category and the second human category; and then the product of the ratio and the size of the target object is used to determine the area range of the marked background area.

[0076] Therefore, according to the human body range of the human body category, the area range of the marked background area is adaptively adjusted, so that the human body range and the area range of the marked background area form an adjustment dimension, thereby refining the composition change of the marked area, so that the situation of the electronic device moving away from or approaching the target object can be adaptively adjusted with the human body range. Moreover, since the human body range has no necessary relationship with the size of the target object, a dimension of composition adjustment can be formed to change the composition mode in a targeted manner, so that the human body categories under different human body ranges highlight the information transmission of the subject in different ways, thereby accurately expressing the information of the marked area through multi-dimensional composition control.

[0077] In some embodiments, the category to which the target object belongs also includes an animal category; the first human category or the second human category includes a half-length portrait category; the first ratio corresponding to the animal category is the same as the first ratio corresponding to the half-length portrait category, and the second ratio corresponding to the animal category is the same as the second ratio corresponding to the half-length portrait category.

[0078] The ratio of the human body range in the half-length portrait category to the full-length human body range reaches a preset ratio. The human body range in the half-length portrait category can be 40%-60% of the full-length human body range, or 50% or another preset half-length category. Because the first and second human body categories are relative, the half-length portrait category may belong to either of the two categories.

[0079] The backgrounds for animals and half-length portraits are determined using a similar method. Since pets are the primary animals photographed in daily life, and pets and half-length portraits are similar in size, the composition used in the animal category is similar to that used in the half-length portrait. The animal category's composition allows for more accurate image information and provides appropriate emphasis.

[0080] Therefore, since the subject areas of the half-length portrait category and the animal category are similar, when the compositions of the animal category and the half-length portrait category match, the information expression of the two images is more accurate.

[0081] In this embodiment, the target object and at least part of the composition result of the marked area, and the composition parameters of the background marked area are related to the category to which the target object belongs, so the background of the target object can be adaptively adjusted according to the category to which the target object belongs, thereby increasing the composition diversity of the marked area and accurately expressing information according to the type adapted to each type. Because the composition parameters and the target object are obtained before the marked area, and the composition parameters and the target object are used to obtain the marked area, the area range of the marked area is related to the size of the target object. This allows the background of the target object in the marked area to be adaptively adjusted according to the size of the target object in the preview image, so that the composition of the marked area selects an information expression method adapted to the corresponding size, accurately expressing information, thereby accurately expressing the information of the target object through the dynamically changing background.

[0082] In some embodiments, the composition parameters include a first ratio and a second ratio. Obtaining a marked area where the target object is located based on the composition parameters and the size of the target object includes: determining a first distance based on the first ratio and the size of the target object, and determining a first boundary based on the position of the target object and the first distance; determining a second distance based on the second ratio and the size of the target object, and determining a second boundary based on the position of the target object and the second distance; the first boundary and the second boundary are located in different directions of the target object; and obtaining the marked area where the target object is located based on the area between the first boundary and the second boundary.

[0083] The position of a target object is used to characterize the position of the target object. The position of the target object may include at least one position within the target object, which may be a pixel position, a region position, or an edge or corner position of the target object. For example, the position of the target object may include the top and bottom positions of the target object, or the left and right positions of the target object.

[0084] The first and second ratios are used to determine the distance that the target object extends in at least two directions. The first and second ratios can be the same or different. When using the first and second ratios, the first and second distances dynamically change with the size of the target object. Therefore, the first and second distances can be used to more finely control the distance extended in at least two directions, thereby finely adjusting the information presentation to the needs of the target object.

[0085] The first distance and the second distance are the distances that the target object extends in at least two directions. The first distance and the second distance can be the same or different. The first distance and the second distance are both set based on the position of the target object. Therefore, the first boundary and the second boundary can filter the boundaries of the target object extending in at least two directions from the preview image, thereby composing the target object.

[0086] The first boundary and the second boundary are located in different directions of the target object, so that the area between the first boundary and the second boundary includes the target object, thereby forming a marked area composed for the target object. The first boundary and the second boundary can be located in opposite directions of the target object or in intersecting directions of the target object.

[0087] Relative directions include, but are not limited to, the relative direction of the target object in the height direction, the relative direction of the target object in the width direction, and the relative direction of the target object in the diagonal direction. The relative direction of the target object in the height direction includes: the top direction of the target object and the top direction of the target object. The relative direction of the target object in the width direction includes: the left direction of the target object and the right direction of the target object.

[0088] The intersection direction is a different direction other than the relative direction. For example, the intersection direction may be the top direction of the target object and the bottom direction of the target object. The corresponding marking area is selected from the preview image with the expansion area of the top direction and the bottom direction.

[0089] The first boundary and the second boundary exist in at least one pair, with each pair of first boundaries and second boundaries being provided in each relative direction. For example, the relative directions along the width of the target object include one pair of first boundaries and second boundaries, and the relative directions along the diagonal direction of the target object include another pair of first boundaries and second boundaries, thereby forming two pairs of boundaries. The area enclosed by the two pairs of boundaries is the marked area where the target object is located.

[0090] In some embodiments, the first ratio includes a first ratio in the height direction, and the second ratio includes a second ratio in the height direction. A first distance in the height direction can be determined based on the product of the first ratio in the height direction and the size of the target object in the height direction, and a first boundary in the height direction can be determined based on the position of the target object and the first distance in the height direction. Correspondingly, a second distance can be determined based on the product of the second ratio in the height direction and the size of the target object in the height direction, and a second boundary in the height direction can be determined based on the position of the target object and the second distance in the height direction.

[0091] In some embodiments, the first ratio includes a first ratio in the width direction, and the second ratio includes a second ratio in the width direction. A first distance in the width direction can be determined based on the product of the first ratio in the width direction and the size of the target object in the width direction, and a first boundary in the width direction can be determined based on the position of the target object and the first distance in the width direction. Correspondingly, a second distance can be determined based on the product of the second ratio in the width direction and the size of the target object in the width direction, and a second boundary in the width direction can be determined based on the position of the target object and the second distance in the width direction.

[0092] In some embodiments, obtaining a marked area where the target object is located based on the area between the first boundary and the second boundary includes: adding marking information to the boundary formed by the first boundary and the second boundary to obtain the marked area where the target object is located. Thus, for at least one set of first boundaries and second boundaries in different directions, a marked area is enclosed in the preview image.

[0093] In some embodiments, obtaining a marked area where the target object is located based on the area between the first boundary and the second boundary includes adding marking information to the area between the first boundary and the second boundary to obtain the marked area where the target object is located. Thus, the marked area in the preview image is displayed for the area formed by at least one set of first boundaries and second boundaries in different directions.

[0094] In some embodiments, the first boundary and the second boundary are both boundaries in the height direction; obtaining the marked area where the target object is located based on the area between the first boundary and the second boundary includes: determining the marked area where the target object is located based on the area between the first boundary and the second boundary in the height direction and a boundary that is greater than or equal to the width of the target object. In this way, the marked area can be obtained without expanding the boundary of the target object in the width direction.

[0095] In some embodiments, the first boundary and the second boundary are both boundaries in the height direction; obtaining the marked area where the target object is located based on the area between the first boundary and the second boundary includes: determining the marked area where the target object is located based on the area between the first boundary and the second boundary in the height direction and the width boundary of the preview image. Therefore, the marked area can also be obtained by using a larger range of expansion of the width boundary of the target object.

[0096] In some embodiments, the composition parameters include a first ratio and a second ratio in a height direction; based on the composition parameters and the size of the target object, a marked area where the target object is located is obtained, including: determining the distance between the top of the marked area and the top of the target object based on the product of the first ratio in the height direction and the height of the target object; determining the distance between the bottom of the marked area and the bottom of the target object based on the product of the second ratio in the height direction and the height of the target object; and determining the marked area containing the target object based on the distance between the top of the marked area and the top of the target object, and the distance between the bottom of the marked area and the bottom of the target object. Thus, the category to which the target object belongs corresponds to the two ratios, and the area range of the marked area is adjusted according to the two ratios, which can make the composition of the marked area more reasonable, more similar to the shooting process of a person using a mobile electronic device, and thus more accurately represent the information.

[0097] In some embodiments, a marked area where the target object is located is obtained based on the composition parameters and the size of the target object, including: determining a selection ratio sum that is negatively correlated with the size of the target object; the selection ratio sum includes the sum of the above-mentioned first ratio and second ratio; selecting the preview image excluding the target object based on the first ratio and the second ratio; and determining the marked area where the target object is located based on the selected area and the target object contained in the selected area.

[0098] In some embodiments, the height of the marked area is the height of the marked area in the preview image. The height of the marked area can be the number of pixels or the length. Since the height of the human body is often greater than the width of the human body, setting a height scale when capturing images of the human body can more accurately control the changes in the composition.

[0099] In this embodiment, the first and second ratios form an adjustment dimension, which refines the positions of the first and second boundaries, thereby changing the composition of the marked area. The use of the first and second ratios introduces multi-directional expansion relative to the target object, allowing for more precise definition of the composition adjustment for the human body category, thereby more accurately conveying the corresponding information.

[0100] In some embodiments, as Figure 4 As shown, the marked enlarged area obtained by enlarging the marked area is displayed, including:

[0101] Step 402 : Enlarge the marked area with the reference position in the marked area as the center to obtain an enlarged marked area.

[0102] The reference position is the position of any pixel in the marked area. Since the reference position is located in the marked area, the magnification process of the marked area is performed based on the reference position of the marked area, rather than the center of the preview image. Therefore, the magnification process of the marked area is implemented for the marked area, and can express information according to the composition of the marked area, so that the proportion of the information conveyed by the composition in the marked area in the camera application display screen gradually increases until the marked magnification area is obtained, and the information change caused by changing the composition during the magnification process is the largest. Exemplarily, the reference position can be the center point position of the marked area or the location of the center area, or one or more edges of the marked area, or one or more corner points of the marked area.

[0103] In some embodiments, the marked area is magnified with a reference position in the marked area as the center to obtain the magnified marked area, including: magnifying the preview image to obtain an enlarged preview image; and selecting, based on the area range of the marked area contained in the unamplified preview image and with the reference position in the marked area as the center, from the enlarged preview image an image area having the same area ratio as the area range of the marked area contained in the unamplified preview image as the magnified marked area. The preview image can be magnified using digital zoom or optical zoom; thereby, the area of the marked area in the center of the camera application screen is magnified, and because the preview image is also magnified, the area outside the magnified marked area is also magnified, thereby ensuring that the magnified marked area has external information of appropriate size, facilitating image processing and other steps.

[0104] In some embodiments, the marked area is enlarged with a reference position in the marked area as the center to obtain the marked enlarged area, including: zooming the preview image with the reference position in the marked area as the center to obtain the enlarged preview image; at the reference position, according to the proportion of the marked area in the preview image, selecting an area range with the same proportion from the enlarged preview image as the marked enlarged area.

[0105] Step 404 : When the center of the marked enlarged area is located at the center of the camera application display screen, the marked enlarged area is displayed.

[0106] The center of the area marked with the magnified region is used to indicate the location of the magnified region. The center of the area is located within the magnified region and the camera application display screen, so that the magnified region is closer to the center of the camera application display screen.

[0107] The camera application display screen center is the center position of the camera application display screen. The camera application display screen center can be an area or a coordinate point. An electronic device can simultaneously have multiple camera application display screens, each with its own camera application display screen center and target object.

[0108] The region center being located at the center of the camera application display screen means that the positional relationship between the region center and the camera application display screen center is matched. The positional relationship is matched, which means that, when the camera application display screen center is a region, the region center may be located within the camera application display screen center, or the region center may have a positional deviation from a certain point within the camera application display screen center; when the camera application display screen center is a coordinate point, the region center may have the same coordinates as the camera application display screen center, or the coordinate difference between the region center and the camera application display screen center may be less than a coordinate difference threshold.

[0109] Since the weight of information transmitted by the center of the camera application display screen is higher than the weight of information transmitted by other areas, when the center of the marked enlarged area is located at the center of the camera application display screen, the dominance of the information transmitted by the marked enlarged area can be increased, thereby making the proportion of information transmitted by the marked area composition in the camera application display screen higher.

[0110] In this embodiment, the magnification process of the marked area is performed based on a reference position within the marked area, which gradually increases the proportion of information conveyed by the marked area's composition in the camera application display. Furthermore, when the center of the magnified marked area is located at the center of the camera application display, the magnified marked area is displayed, increasing the proportion of information conveyed by the marked area's composition in the camera application display. Thus, by enlarging the area of the marked area and adjusting its position, non-center magnification is achieved, changing the dominant information on the preview screen to that conveyed by the marked area's composition.

[0111] In some embodiments, the method further includes: when the zoom ratio corresponding to the marked area is within the zoom focal length of the first camera, and the zoom ratio corresponding to the marked magnified area is within the zoom focal length of the second camera, based on the zoom ratio corresponding to the marked magnified area, enlarging and cropping the image captured by the first camera to obtain the marked magnified area; wherein the field of view angle of the first camera is greater than the field of view angle of the second camera.

[0112] The zoom ratio corresponding to the marked area is the zoom ratio used by the first camera when capturing the preview image containing the marked area. For example, if the zoom ratio corresponding to the marked area is 1.1X, the preview image is captured at 1.1X.

[0113] The zoom factor corresponding to the marked magnified area is the zoom factor used by the first or second camera during image capture. In this case, because the first camera has a larger field of view, the marked magnified area can be obtained by magnifying and cropping the image captured by the first camera. For example, if the zoom factor corresponding to the marked magnified area is 2.2X, and the first camera's image capture focal length includes 1.9X, then the marked magnified area must be acquired at 2.2X. In this case, the image acquired at 1.9X is magnified and cropped to obtain the marked magnified area.

[0114] The zoom focal length of the first camera is the physical focal length of the first camera, which is used to indicate the zoom magnification range of the image captured by the first camera. At this time, the preview image with the marked area is captured by the first camera.

[0115] The zoom focal length of the second camera is the physical focal length of the second camera, which is used to indicate the range of the zoom ratio of the image captured by the second camera. The marked magnified area should be captured by the second camera, and this process should involve a camera switching process. However, before the camera switches, the image captured by the first camera is cropped based on the zoom ratio of the marked magnified area at the time of capture, so that the information displayed in the marked magnified area matches the image captured by the camera with a smaller field of view, thereby making the information change during the magnification process more natural.

[0116] The focal length is the range of zoom factors that match the camera. This zoom factor controls the field of view of the target image. The zoom factor and field of view are inversely correlated; smaller zoom factors increase the field of view, while larger zoom factors decrease the field of view. Because the first camera has a larger field of view, the image captured by the first camera can be cropped to identify the magnified area.

[0117] In one embodiment, the preview image with the marked area is captured by the first camera, while the marked magnified area needs to be captured by the second camera; in this case, when non-center zoom is performed based on the marked area and the field of view of the marked magnified area is within the field of view of the second camera, the image captured by the first camera is first magnified and cropped so that the marked magnified area matches the field of view of the second camera.

[0118] In one embodiment, based on the zoom factor corresponding to the marked magnified area, the image captured by the first camera is magnified and cropped to obtain the marked magnified area, including: capturing an image according to the current zoom factor within the focal length of the image captured by the first camera to obtain image data to be magnified; and magnifying and cropping the image data corresponding to the image data to be magnified according to the zoom factor corresponding to the marked magnified area to obtain the marked magnified area. Thus, by magnifying and cropping according to the zoom factor, the marked magnified area can be efficiently obtained.

[0119] In one embodiment, based on the zoom factor corresponding to the marked magnified area, an image captured by the first camera is magnified and cropped to obtain the marked magnified area, including: in an image captured by the first camera at the current zoom factor, the marked area is magnified and cropped according to the ratio between the zoom factor corresponding to the marked magnified area and the current zoom factor, thereby obtaining the magnified and cropped marked magnified area. Thus, the zoom factor of the first camera can be selected or not, and magnification can be performed directly according to the zoom factor corresponding to the marked magnified area, which improves processing efficiency.

[0120] In one example, the first camera is a telephoto camera and the second camera is a main camera; when the field of view of the zoom magnification includes the field of view of the telephoto camera, the image captured by the telephoto camera is cropped according to the field of view of the main camera so that the field of view of the image captured by the telephoto camera matches the field of view of the image captured by the main camera, thereby making the information changes in the magnification process more natural and able to express information more accurately.

[0121] In this embodiment, when the marked area and the marked magnified area belong to the physical focal lengths corresponding to different cameras, and the first camera has a larger field of view, the switching process of the physical focal length may cause image problems such as image jumps or a brief black screen; these image problems will affect the magnification process of the marked area, so the magnification process of the marked area is decoupled from the actual light change point switching process of the camera. By cropping the image captured by the camera with a larger field of view, the information displayed in the marked magnified area matches the image captured by the camera with a smaller field of view, so that the information changes in the magnification process are more natural and can express information more accurately.

[0122] In some embodiments, in response to a trigger instruction for the marking area, displaying a marking magnified area obtained by enlarging the marking area includes: in response to a trigger instruction for the marking area, enlarging the marking area by zooming to obtain the marking magnified area; and displaying the marking magnified area.

[0123] In some embodiments, zooming in on the marked area to obtain the magnified marked area includes: magnifying the marked area with a reference position in the marked area as the center to obtain the magnified marked area. Correspondingly, displaying the magnified marked area includes: displaying the magnified marked area when the center of the magnified marked area is located at the center of the camera application display screen.

[0124] In some embodiments, zooming the marked area to obtain the enlarged marked area includes: enlarging the preview image based on the zoom factor of the preview image while maintaining the camera application display unchanged. Correspondingly, displaying the enlarged marked area includes: selecting and displaying the enlarged area obtained by enlarging the marked area from the enlarged preview image.

[0125] In this embodiment, the entire preview image is magnified by zooming, and then the marked area is displayed so that the magnified area has adjacent magnified areas, which is convenient for user adjustment and does not cause information loss due to magnification.

[0126] In some embodiments, obtaining the target image based on the image data corresponding to the marked enlarged area includes: using the image data corresponding to the marked enlarged area as the target image.

[0127] In some embodiments, the image data corresponding to the marked enlarged area can be directly output as an image, and the output image is the target image. Alternatively, the image data corresponding to the marked enlarged area can be further processed for image signal to obtain an output image.

[0128] In some embodiments, using the image data corresponding to the marked zoomed-in area as the target image includes: responding to a capture instruction, using the image data corresponding to the marked zoomed-in area as the target image. Thus, the capture instruction and the trigger instruction are separated into two stages to ensure that the user can intervene in the processing.

[0129] In this embodiment, the image data corresponding to the marked magnified area is used as the target image, so that the corresponding target image is directly formed through the composition within the marked magnified area, thereby realizing adaptive modification of the information expression method.

[0130] In some embodiments, the camera application display screen includes a first screen area and a second screen area. The first screen area is used to display the marked magnified area. The content displayed in the first screen area is used for imaging, and the content displayed in the second screen area is not used for imaging.

[0131] The method further includes: when displaying the enlarged marked area obtained by enlarging the marked area, adjusting the content displayed in the first screen area in response to the area adjustment instruction to obtain adjusted content displayed in the first screen area; the adjusted content displayed in the first screen area includes partial or all image data of the enlarged marked area. Correspondingly, obtaining a target image based on the image data corresponding to the enlarged marked area includes obtaining the adjusted image based on the content displayed in the adjusted first screen area.

[0132] The first screen area is the screen area that forms an image during shooting. The first screen area includes at least the marked magnified area, and may also include a portion of the adjacent area to the marked magnified area. The marked area includes the target object and the marked background area outside the target object, and the marked magnified area is obtained by magnifying the marked area. Therefore, the marked magnified area includes the enlarged target object obtained by magnifying the target object, and the enlarged marked background area obtained by magnifying the marked background area. The second screen area is the screen area that does not form an image during shooting. Before obtaining the target image, both the first screen area and the second screen area are displayed on the screen of the electronic device.

[0133] The content displayed in the first image area after adjustment includes part or all of the image data of the marked magnified area. In this case, the area adjustment command is an operation command for adjusting the imaging. The area adjustment command is used to enlarge or reduce the content displayed in the marked magnified area; the area adjustment command can also be used to adjust the position of the content displayed in the marked magnified area.

[0134] In some embodiments, the region adjustment instruction is used to adjust the zoom ratio to change the information expression of the original image through zooming.

[0135] In some embodiments, the first screen area is the normally displayed screen area, while the second screen area is the screen area covered with a mask. The mask can be at least one of a first color and a second color, and the first color and the second color are different colors, for example, the first color is black and the second color is gray. The first color and the second color can be independent colors. If the mask is the first color, the second screen area is invisible during preview; if the mask is the second color, the second screen area is semi-transparent during preview.

[0136] In one example, if Figure 5 As shown, the first screen area 501 is displayed with normal brightness, while the mask of the second screen area 502 is gray. At this time, the area covered by the first screen area 501 can be adaptively adjusted by scaling, etc. The first screen area 501 includes at least part of the content of the marked area.

[0137] In some embodiments, the area ratio of the first screen area in the camera application display screen is the first screen area ratio, the area ratio of the second screen area in the camera application display screen is the second screen area ratio, and the first screen area ratio is greater than the second screen area ratio. For example, the first screen area ratio is 80%, and the second screen area ratio is 70%.

[0138] In some embodiments, the first screen area is located in a central area of the camera application display screen, and the second screen area is located in an area outside the first screen area.

[0139] In some embodiments, the content displayed in the first screen area is adjusted to obtain the adjusted content displayed in the first screen area, including: in response to an instruction for reducing the zoom ratio, reducing the zoom ratio of the electronic device to expand the content displayed in the marked magnified area to obtain the adjusted area; and / or, in response to an instruction for increasing the zoom ratio, increasing the zoom ratio of the electronic device to reduce the content displayed in the marked magnified area to obtain the adjusted area.

[0140] In some embodiments, the adjusted image can be determined from the original image data (RAW) corresponding to the adjusted area; the adjusted image can also be determined from the data obtained by ISP processing of the original image data (RAW) corresponding to the adjusted area; and the white balance coefficient, aperture, ISO, focal length and other acquisition parameters determined in the preview stage of the adjusted area can also be used as acquisition parameters of the adjusted image.

[0141] In some embodiments, obtaining an adjusted image according to the content displayed in the adjusted first screen area includes: obtaining the adjusted image based on image data corresponding to the content displayed in the adjusted first screen area in response to a shooting instruction.

[0142] In this embodiment, the trigger instruction for marking an area is also used to change the area of the camera application display screen for imaging, so that, without moving the electronic device, an imaging result of the composition contained in the first image area and candidate imaging results of the second image area are formed. This allows at least a preview of the marked magnified area before imaging, facilitating adjustment of the area range of the marked magnified area. Furthermore, the marked magnified area is located in the first image area, and the first image area is adjustable, enabling adaptive modification of the displayed content to form an adapted, adjusted image without moving the electronic device, thereby better conveying the desired capture.

[0143] In some embodiments, as Figure 6 As shown, the method further includes:

[0144] Step 602 : In response to the trigger instruction of the non-marked area, the object indicated by the trigger instruction is identified to obtain a new target object.

[0145] A non-marked area is a marked area without any marking information for a target object. In a preview image, a non-marked area can exist independently or simultaneously with a marked area. Because the non-marked area conveys information through its absence of marking information, it is substantially different from the marked area containing the target object. Therefore, the non-marked area can provide a candidate target object for changing triggerable composition guidelines. This allows the target object to be changed without moving the electronic device.

[0146] The trigger instruction can be a click instruction, a slide instruction, or a multi-touch instruction. If the trigger instruction is for an unmarked area, the trigger instruction at least indicates that a new target object may exist in the corresponding interval, so that the corresponding composition can be updated for different target objects. For example, the target object recognition process needs to be performed in the area indicated by the trigger instruction to update the target object.

[0147] The new target object is a newly recognized target object. The new target object may be located at a different location than the original target object and may also belong to a different category. For example, the original target object may be a face; the new target object may be a head or a bust of a person.

[0148] In some embodiments, identifying the object indicated by the trigger instruction to obtain a new target object includes: identifying the object with the largest area in the area indicated by the trigger instruction to obtain the new target object. In this way, the main body of the image is controlled by area to determine the new target object for the corresponding non-marked area.

[0149] In some embodiments, identifying the object indicated by the trigger instruction to obtain a new target object includes: identifying an object belonging to the human body in a preset area indicated by the trigger instruction; and obtaining the new target object. Thus, the new target object is determined based on the category to which the object belongs, forming a shooting mode for the human body.

[0150] Step 604 , compose the image according to the position of the new target object in the preview image to obtain a marked area where the new target object is located, and then perform a step of displaying the marked area where the new target object is located in the preview image.

[0151] The position of the new target object in the preview image is used to represent the position information of the new target object relative to the preview image. The position of the new target object in the preview image can be a coordinate or an area in the preview image. The new target object can belong to the same or different category as the original target object in step 202.

[0152] In one example, the position of the new target object in the preview image may be the center position of the new target object, or may be the position indicated by the trigger instruction.

[0153] In some embodiments, the marked area is the area enclosed by the preview box, and the unmarked area is the area outside the preview box; composing the image according to the position of the new target object in the preview image to obtain the marked area where the new target object is located, including: when the user clicks on the area outside the preview box, identifying the clicked content, and recommending a reasonable composition area as the marked area where the target object is located based on the trained neural network model.

[0154] In some embodiments, composition is performed based on the position of the new target object in the preview image to obtain a marked area where the new target object is located, including: using the boundaries of each part of the target object in the preview image, through a preset template or neural network model, to compose and mark the target object to obtain the marked area where the new target object is located.

[0155] The step of displaying the marked area where the target object is located in the preview image again means replacing the target object in step 202 with the new target object and performing step 202 and subsequent steps of step 202 again.

[0156] In this embodiment, in response to a trigger instruction in a non-marked area, the object indicated by the trigger instruction is identified to obtain a new target object. Based on the position of the new target object in the preview image, a composition is performed to obtain the marked area where the new target object is located. The marked area where the target object is located is then displayed in the preview image. This allows for updating and adjusting the target object.

[0157] In some embodiments, composing the image according to the position of the new target object in the preview image to obtain the marked area where the target object is located includes: marking the new target object with the boundaries of each part containing the new target object to obtain the marked area where the new target object is located.

[0158] The boundary containing the parts of the new target object means that every part of the new target object is within the range of the boundary. The boundary can be the edges of a marking box, thereby marking. The boundary can be a square boundary or a circular boundary.

[0159] In some embodiments, the marked area includes the new target object and a background marked area corresponding to the new target object. The extent of the new background marked area is related to at least one of the category of the new target object and the size of the new target object. In this case, the meaning of each part of the target object can be adaptively adjusted based on the category of the target object, thereby increasing the compositional diversity of the marked area and accurately expressing information according to the type of information adapted to each category.

[0160] Marking the new target object using the boundaries of each part containing it ensures the integrity of the recognition subject and more comprehensively represents the information of the new target object. If the new target object is a person, sculpture, animal, device, or equipment, the integrity of each part of the sculpture, animal, device, or equipment must also be ensured. Each part of the target object is related to the category to which the target object belongs.

[0161] Taking a person as an example, the various parts of the target object are related to the category to which the target object belongs, and the category to which the target object belongs can be related to the range of the human body. If the target object is a full-body portrait, the various parts of the target object include all parts of the human body, which include all limbs. The four parts of the human body are the limbs, and each part can also include the head. If the target object is a half-body portrait, the various parts of the target object include all parts of the half-body. If the target object is a head with a face, the various parts of the target object include all parts of the head. If the target object is a face, the various parts of the target object include all parts of the face.

[0162] In some embodiments, marking the new target object using the boundaries of each part containing the new target object to obtain a marked area where the new target object is located includes: using the boundaries of each part containing the new target object as the marking information of the new target object to obtain the marked area where the new target object is located. In this way, using the boundaries or boundary features as the marking information of the new target object can ensure the integrity of the marked area and distinguish between the marked area and the unmarked area.

[0163] In some embodiments, marking the new target object using the boundaries of the parts containing the new target object to obtain a marked area where the new target object is located includes: displaying the boundaries of the parts containing the new target object, using the displayed boundaries as the marking information of the new target object, and using the area within the boundaries as the marked area where the new target object is located. In this way, by displaying the boundaries to form the marking information of the new target object, the integrity of the marked area can be guaranteed, and the marked area can be distinguished from the unmarked area.

[0164] In this embodiment, since the type defined by the new target object itself can include objects within different human body ranges, boundaries are set for the parts contained in the new target object, so that the marking area can be divided according to the new target object. On the premise of ensuring the integrity of the new target object, the corresponding composition can be adaptively adjusted as the new target object is used to form a corresponding new marking area.

[0165] In an exemplary embodiment, Figure 7 As shown, the method specifically includes steps 502 to 710, wherein step 202 includes step 702, step 204 includes step 704, step 206 includes step 708, step 402 includes step 706, and step 404 includes step 708. In this case, the marked area includes content recommended by previewing the real-time frame selection composition. The marking information of the marked area is a marked frame, which is the preview frame. The marked area is located within the preview frame, and the unmarked area is located outside the preview frame. The target object is the subject recognition result, which is the result of identifying the subject in the picture. The magnification method is non-center zoom, i.e., non-center zoom.

[0166] In step 702 , the camera previews and selects the recommended composition content in real time, and the composition selection result is the marked area in the preview frame.

[0167] The frame selection and composition includes: mapping the area of the subject recognition result based on the composition parameters corresponding to the category to which the subject recognition result belongs, and obtaining a composition frame selection result.

[0168] In step 704 , when the user clicks on the marked area in the preview box, the selected content is zoomed in non-center to obtain the marked zoomed area.

[0169] The marked area after zooming in is part of the first screen area, which occupies 80% of the screen, while the surrounding 20% of the content outside the box constitutes the second screen area. This facilitates users to make subtle adjustments when zooming.

[0170] Step 706: When the user clicks on a non-marked area outside the preview box, the clicked content is recognized to obtain a new subject recognition result.

[0171] Step 708: While ensuring the integrity of the subject recognition result, a reasonable composition area is recommended based on the neural network model, and then the process returns to step 702 to perform non-center zooming.

[0172] Among them, the neural network model is obtained based on training algorithm.

[0173] Step 710: After the non-center zoom is completed, if the user is satisfied with the composition result, he or she can take a photo directly.

[0174] In step 712, if you are not satisfied, you can zoom in and out until you get a satisfactory result and then take a photo.

[0175] Among them, the implementation of non-center zoom needs to consider the FOV coverage of cameras with different focal lengths. When the field of view of the zoom magnification includes the field of view of the telephoto camera, the image captured by the telephoto camera is cropped according to the field of view of the main camera, so that the field of view of the image captured by the telephoto camera is adapted to the field of view of the image captured by the main camera, so that the information change during the magnification process is more natural and can express the information more accurately. Thus, a quick and convenient way to compose and take pictures is provided, and real-time composition recommendations are made based on the results of subject recognition. Without moving the phone, the user is guided to complete the operation to obtain the composed photo. Better convey the subject's composition expectations and help the photographer to improve the success rate of helping with shooting as much as possible.

[0176] In an exemplary embodiment, Figure 8 As shown, the method includes:

[0177] Step 802: Display the marked area where the target object is located in the preview image; the marked area includes the target object and the background marked area corresponding to the target object. The background marked area is related to the composition parameters corresponding to the category to which the target object belongs and the size of the target object.

[0178] Step 804: In response to a trigger instruction for the marked area, the marked area is enlarged with the reference position in the marked area as the center to obtain a marked enlarged area; when the center of the marked enlarged area is located at the center of the camera application display screen, the marked enlarged area is displayed; and based on the image data corresponding to the marked enlarged area, a target image is obtained.

[0179] Step 806, in response to the trigger instruction of the non-marked area, the object indicated by the trigger instruction is identified to obtain a new target object; the new target object is marked with the boundaries of each part containing the new target object to obtain the marked area where the new target object is located, and then the step of displaying the marked area where the target object is located in the preview image is executed.

[0180] After displaying the magnified marker area, the method further includes: in a case where the magnified marker area obtained by magnifying the magnified marker area is displayed, in response to the area adjustment instruction, adjusting the content displayed in the first screen area to obtain adjusted content displayed in the first screen area; the adjusted content displayed in the first screen area includes part or all of the image data of the magnified marker area;

[0181] Correspondingly, obtaining a target image based on the image data corresponding to the marked magnified area includes obtaining an adjusted image according to the content displayed in the adjusted first picture area.

[0182] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0183] Based on the same inventive concept, embodiments of the present application also provide an image processing device for implementing the aforementioned image processing method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the following image processing device embodiments can be found in the above-described limitations on the image processing method and will not be further elaborated here.

[0184] In an exemplary embodiment, Figure 9 As shown, an image processing device is provided, the device comprising:

[0185] A marking module 902 is configured to display a marked area where a target object is located in the preview image;

[0186] an enlarging module 904, configured to display an enlarged marked area obtained by enlarging the marked area in response to a trigger instruction on the marked area;

[0187] The generating module 906 is configured to obtain a target image based on the image data corresponding to the marked magnified area.

[0188] In one embodiment, the amplification module 904 is configured to:

[0189] Enlarging the marked area with a reference position in the marked area as the center to obtain an enlarged marked area;

[0190] When the center of the mark enlargement area is located at the center of the camera application display screen, the mark enlargement area is displayed.

[0191] In one embodiment, the amplification module 904 is configured to:

[0192] When the zoom ratio corresponding to the marked area is within the zoom range of the first camera, and the zoom ratio corresponding to the marked magnified area is within the zoom range of the second camera, the image captured by the first camera is magnified and cropped based on the zoom ratio corresponding to the marked magnified area to obtain the marked magnified area;

[0193] The field of view of the first camera is greater than the field of view of the second camera.

[0194] In one embodiment, the marking module 902 is configured to:

[0195] Recognize the preview image to obtain the target object;

[0196] Obtaining the category to which the target object belongs, and obtaining corresponding composition parameters according to the category;

[0197] A marked area where the target object is located is obtained according to the composition parameters and the size of the target object.

[0198] In one embodiment, the composition parameters include a first ratio and a second ratio; the marking module 902 is configured to:

[0199] determining a first distance according to the first ratio and a size of the target object, and determining a first boundary according to a position of the target object and the first distance;

[0200] determining a second distance according to the second ratio and the size of the target object, and determining a second boundary according to the position of the target object and the second distance; the first boundary and the second boundary are located in different directions of the target object;

[0201] A marked area where the target object is located is obtained according to the area between the first boundary and the second boundary.

[0202] In one embodiment, the amplification module 904 is configured to:

[0203] In response to a trigger instruction for the marked area, the marked area is magnified by zooming to obtain a magnified marked area;

[0204] The marked magnified area is displayed.

[0205] In one embodiment, the generating module 906 is configured to:

[0206] The image data corresponding to the marked enlarged area is used as the target image.

[0207] In one embodiment, the camera application display screen includes a first screen area and a second screen area, the first screen area is used to display the marked magnified area, the content displayed in the first screen area is used for imaging, and the content displayed in the second screen area is not used for imaging.

[0208] The amplification module 904 is used to:

[0209] When displaying the enlarged marked area obtained by enlarging the marked area, in response to the area adjustment instruction, adjusting the content displayed in the first screen area to obtain adjusted content displayed in the first screen area; the adjusted content displayed in the first screen area includes part or all of the image data of the enlarged marked area;

[0210] The generating module 906 is used to:

[0211] An adjusted image is obtained according to the content displayed in the first picture area after the adjustment.

[0212] In one embodiment, the marking module 902 is configured to:

[0213] In response to a trigger instruction of the unmarked area, identifying the object indicated by the trigger instruction to obtain a new target object;

[0214] Composing the new target object at the position of the preview image to obtain the marked area where the new target object is located, and then performing the step of displaying the marked area where the target object is located in the preview image.

[0215] In one embodiment, the marking module 902 is configured to:

[0216] The new target object is marked using the boundaries of the parts containing the new target object to obtain a marked area where the new target object is located.

[0217] Each module in the above-mentioned image processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in the electronic device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0218] In an exemplary embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 10As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0219] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0220] In one embodiment, an electronic device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0221] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0222] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0224] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable logic unit (PLC), a data processing logic unit based on quantum computing, an artificial intelligence (AI) processor, and the like.

[0225] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0226] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Displaying the marked area where the target object is located in the preview image; In response to a trigger instruction for the marked area, displaying an enlarged marked area obtained by enlarging the marked area; A target image is obtained based on the image data corresponding to the marked magnified area.

2. The method according to claim 1, characterized in that The displaying of the enlarged marked area obtained by enlarging the marked area includes: Enlarging the marked area with a reference position in the marked area as the center to obtain an enlarged marked area; When the center of the mark enlargement area is located at the center of the camera application display screen, the mark enlargement area is displayed.

3. The method according to claim 2, characterized in that The method further comprises: When the zoom ratio corresponding to the marked area is within the zoom range of the first camera, and the zoom ratio corresponding to the marked magnified area is within the zoom range of the second camera, the image captured by the first camera is magnified and cropped based on the zoom ratio corresponding to the marked magnified area to obtain the marked magnified area; The field of view of the first camera is greater than the field of view of the second camera.

4. The method according to claim 1, wherein The method for determining the marked area includes: Recognize the preview image to obtain the target object; Obtaining the category to which the target object belongs, and obtaining corresponding composition parameters according to the category; A marked area where the target object is located is obtained according to the composition parameters and the size of the target object.

5. The method according to claim 4, characterized in that The composition parameters include a first ratio and a second ratio; and obtaining a marked area where the target object is located according to the composition parameters and the size of the target object includes: determining a first distance according to the first ratio and a size of the target object, and determining a first boundary according to a position of the target object and the first distance; determining a second distance according to the second ratio and the size of the target object, and determining a second boundary according to the position of the target object and the second distance; the first boundary and the second boundary are located in different directions of the target object; A marked area where the target object is located is obtained according to the area between the first boundary and the second boundary.

6. The method according to claim 1, characterized in that The step of displaying the enlarged mark area obtained by enlarging the mark area in response to the trigger instruction for the mark area includes: In response to a trigger instruction for the marked area, the marked area is magnified by zooming to obtain a magnified marked area; The marked magnified area is displayed.

7. The method according to claim 1, characterized in that The step of obtaining a target image based on the image data corresponding to the marked magnified area includes: The image data corresponding to the marked enlarged area is used as the target image.

8. The method according to claim 1, characterized in that The camera application display screen includes a first screen area and a second screen area, the content displayed in the first screen area is used for imaging, and the content displayed in the second screen area is not used for imaging; the method further includes: When displaying the enlarged marked area obtained by enlarging the marked area, in response to the area adjustment instruction, adjusting the content displayed in the first screen area to obtain adjusted content displayed in the first screen area; the adjusted content displayed in the first screen area includes part or all of the image data of the enlarged marked area; The step of obtaining a target image based on the image data corresponding to the marked magnified area includes: An adjusted image is obtained according to the content displayed in the first picture area after the adjustment.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: In response to a trigger instruction of the unmarked area, identifying the object indicated by the trigger instruction to obtain a new target object; Composing the new target object at the position of the preview image to obtain the marked area where the new target object is located, and then performing the step of displaying the marked area where the target object is located in the preview image.

10. The method according to claim 9, characterized in that Composing the image according to the position of the new target object in the preview image to obtain a marked area where the new target object is located includes: The new target object is marked using the boundaries of the parts containing the new target object to obtain a marked area where the new target object is located.

11. An image processing device, characterized in that: The device comprises: a marking module, used for displaying a marked area where a target object is located in a preview image; an enlarging module, configured to display an enlarged marked area obtained by enlarging the marked area in response to a trigger instruction on the marked area; A generating module is used to obtain a target image based on the image data corresponding to the marked magnified area.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images