Image reconstruction method and device and storage medium

Abstract

The invention relates to an image reconstruction method and device and a storage medium. The image reconstruction method comprises the following steps: acquiring a first image input by a user; based on an image reconstruction model and the first image, the first image is amplified, a preset number of second images are obtained, and different images in the preset number of second images have different amplification proportions relative to the first image. Through the image reconstruction method disclosed by the invention, image reconstruction is carried out based on the image uploaded by the user, and the image which comprises a plurality of image proportions and achieves the composition aesthetic effect is output, so that the user demand is met.

Description

Technical Field

[0001] The present disclosure relates to the field of terminals, and in particular, to an image reconstruction method, apparatus, and storage medium. Background Art

[0002] With the rise of intelligent terminal photography, more and more people have joined the ranks of using intelligent terminals to take pictures. However, during the process of taking pictures, users need a certain understanding of composition aesthetics to take pictures with beautiful compositions. Therefore, technologies for optimizing the composition of the pictures taken by users have emerged.

[0003] In the related art, most of the composition optimizations are to perform significant cutting on the image so that the cut image has composition aesthetics. However, there are still some drawbacks, such as being unable to optimize the composition of an image containing a partially missing subject and being unable to place the subject in the image in an aesthetic position, etc. How to perform better composition optimization on the pictures taken by users is an urgent problem in this field. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides an image reconstruction method, apparatus, and storage medium.

[0005] According to the first aspect of the embodiments of the present disclosure, an image reconstruction method is provided, including: obtaining a first image input by a user; based on an image reconstruction model and the first image, amplifying the first image to obtain a predetermined number of second images, and different images among the predetermined number of second images have different amplification ratios relative to the first image.

[0006] In an implementation manner, the amplifying the first image based on the image reconstruction model and the first image to obtain a predetermined number of second images includes: identifying a main image in the first image based on the first image and a first model in the image reconstruction model to obtain a main identification result, where the first model is used for main identification; obtaining an initial amplification parameter based on the main identification result, and amplifying the first image based on the initial amplification parameter to obtain an initial amplified image; amplifying the initial amplified image according to a predetermined amplification ratio corresponding to the predetermined number to obtain the corresponding predetermined number of second images.

[0007] In an implementation manner, the obtaining an initial amplification parameter based on the main identification result includes: in response to the main identification result indicating that no main image is identified, inputting the first image into a second model in the image reconstruction model; obtaining a first amplification parameter for amplifying the first image as the initial amplification parameter, where the second model is used for determining an image amplification parameter.

[0008] In one embodiment, obtaining the initial amplification parameters based on the subject recognition result includes: if the subject recognition result indicates that a subject image is recognized and the proportion of the area of the subject image in the first image is greater than or equal to a preset proportion threshold, the following method is used to obtain the initial amplification parameters: input the first image into a second model in the image reconstruction model to obtain a first amplification parameter for amplifying the first image, and obtain a first distance between the boundary of the subject image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a second amplification parameter for amplifying the first image, and obtain a second distance between the center of gravity point of the subject image and a preset high-quality point, and obtain a first direction in which the preset high-quality point is located at the center of gravity point of the subject image. Amplify the first image in a direction opposite to the first direction by the second distance to obtain a third amplification parameter for amplifying the first image. The preset high-quality point is a point within the first image and is located on a target line of the first image. The target line is at least one of the following: the golden section line, the rule of thirds line, and the center line; in response to the first amplification parameter, the second amplification parameter, and the third amplification parameter including an amplification length, use the amplification parameter with the largest corresponding amplification length among the first amplification parameter, the second amplification parameter, and the third amplification parameter as the initial amplification parameter.

[0009] In one embodiment, obtaining the initial amplification parameters based on the subject recognition result includes: if the subject recognition result indicates that a subject image is recognized and the proportion of the area of the subject image in the first image is less than a preset proportion threshold, the initial amplification parameters are obtained by the following method: obtain a first distance between the boundary of the subject image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a fourth amplification parameter for amplifying the first image, and obtain a second distance between the center of gravity point of the subject image and a preset high-quality point. Based on the first direction in which the preset high-quality point is located at the center of gravity point of the subject image, amplify the first image in a direction opposite to the first direction by the second distance to obtain a fifth amplification parameter for amplifying the first image; in response to the fourth amplification parameter and the fifth amplification parameter including an amplification length, use the amplification parameter with the largest corresponding amplification length among the fourth amplification parameter and the fifth amplification parameter as the initial amplification parameter.

[0010] In one embodiment, amplifying the initial amplified image according to a predetermined amplification ratio corresponding to the predetermined quantity includes: if the ratio of the initial amplified image does not conform to the predetermined amplification ratio, obtaining the short side of the initial amplified image; distributing the amplification length to two amplification directions of the short side based on a distance ratio and the predetermined amplification ratio, where the distance ratio is the distance ratio between a designated point of the first image and different boundaries of the first image, the designated point including the center of gravity of the main image in the first image or the center point of the first image, and the different boundaries of the first image including the upper boundary and the lower boundary of the first image, or the upper left boundary and the right boundary of the first image; if the second amplified image still does not conform to the predetermined amplification ratio, obtaining the long side of the second amplified image; cropping the long side of the second amplified image, and distributing the cropping length to two cropping directions of the long side based on the distance ratio and the predetermined amplification ratio.

[0011] In one embodiment, distributing the cropping length to two cropping directions of the long side based on the distance ratio and the predetermined amplification ratio includes at least one of the following: if the cropping target is the main object image and the area of the cropped main object image is lower than a first cropping threshold; if performing vertical cropping on the second amplified image and the cropping target is the main object image and the cropping length is greater than a second cropping threshold, starting to crop from the bottom of the second amplified image and keeping the distance between the top of the main object image and the upper boundary of the second amplified image greater than a preset distance threshold until the second amplified image conforms to the predetermined amplification ratio.

[0012] According to a second aspect of the embodiments of the present disclosure, there is provided an image reconstruction device, including a receiving unit configured to obtain a first image input by a user; and a processing unit configured to amplify the first image based on an image reconstruction model and the first image to obtain a predetermined quantity of second images, where different images among the predetermined quantity of second images have different amplification ratios relative to the first image.

[0013] In one embodiment, the processing unit amplifies the first image based on the image reconstruction model and the first image in the following manner to obtain a predetermined quantity of second images: identifying a main image in the first image based on the first image and a first model in the image reconstruction model to obtain a main identification result, where the first model is used for main identification; obtaining initial amplification parameters based on the main identification result, and amplifying the first image based on the initial amplification parameters to obtain an initial amplified image; amplifying the initial amplified image according to a predetermined amplification ratio corresponding to the predetermined quantity to obtain the second images corresponding to the predetermined quantity.

[0014] In one implementation, the processing unit obtains initial amplification parameters based on the subject recognition result in the following manner: in response to the subject recognition result indicating that no subject image is recognized, input the first image into a second model in the image reconstruction model; obtain first amplification parameters for amplifying the first image as the initial amplification parameters, where the second model is used to determine image amplification parameters.

[0015] In one implementation, the processing unit obtains initial amplification parameters based on the subject recognition result in the following manner: if the subject recognition result indicates that a subject image is recognized and the proportion of the area occupied by the subject image in the first image is greater than or equal to a preset proportion threshold, obtain the initial amplification parameters in the following manner: input the first image into a second model in the image reconstruction model, obtain first amplification parameters for amplifying the first image, and obtain a first distance between the boundary of the subject image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain second amplification parameters for amplifying the first image, and obtain a second distance between the center of gravity point of the subject image and a preset high-quality point, and obtain a first direction in which the preset high-quality point is located relative to the center of gravity point of the subject image, and amplify the first image in a direction opposite to the first direction by the second distance to obtain third amplification parameters for amplifying the first image. The preset high-quality point is a point within the first image and is located on a target line of the first image, and the target line is at least one of the following: the golden section line, the thirds line, and the center line; in response to the first amplification parameters, the second amplification parameters, and the third amplification parameters including an amplification length, use the amplification parameter with the largest corresponding amplification length among the first amplification parameters, the second amplification parameters, and the third amplification parameters as the initial amplification parameters.

[0016] In one embodiment, the processing unit obtains initial amplification parameters based on the subject recognition result in the following manner: If the subject recognition result indicates that a subject image is recognized and the proportion of the area occupied by the subject image in the first image is less than a preset proportion threshold, the initial amplification parameters are obtained in the following manner: Obtain a first distance between the boundary of the subject image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a fourth amplification parameter for amplifying the first image, and obtain a second distance between the centroid point of the subject image and a preset high-quality point. Based on the fact that the preset high-quality point is in a first direction from the centroid point of the subject image, amplify the first image in a direction opposite to the first direction by the second distance to obtain a fifth amplification parameter for amplifying the first image; In response to the fact that the fourth amplification parameter and the fifth amplification parameter include amplification lengths, use the amplification parameter with the largest corresponding amplification length among the fourth amplification parameter and the fifth amplification parameter as the initial amplification parameter.

[0017] In one embodiment, the processing unit amplifies the initial amplified image according to a corresponding predetermined amplification ratio for the predetermined number: If the ratio of the initial amplified image does not conform to the predetermined amplification ratio, obtain the short side of the initial amplified image; Based on a distance ratio and a predetermined amplification ratio, distribute the amplification length to two amplification directions of the short side to obtain a second amplified image. The distance ratio is the distance ratio between a designated point of the first image and different boundaries of the first image. The designated point includes the centroid point of the subject image in the first image or the center point of the first image. The different boundaries of the first image include the upper boundary and the lower boundary of the first image, or the upper left boundary and the right boundary of the first image; If the second amplified image still does not conform to the predetermined amplification ratio, obtain the long side of the second amplified image; Crop the long side of the second amplified image, and based on the distance ratio and the predetermined amplification ratio, distribute the cropping length to two cropping directions of the long side.

[0018] In one embodiment, the processing unit distributes the cropping length to two cropping directions of the long side based on the distance ratio and the predetermined amplification ratio, including at least one of the following: If the cropping target is the subject image, the area of the cropped subject image is lower than a first cropping threshold; If the second amplified image is longitudinally cropped and the cropping target is the subject image and the cropping length is greater than a second cropping threshold, start cropping from the bottom of the second amplified image and keep the distance between the top of the subject image and the upper boundary of the second amplified image greater than a preset distance threshold until the second amplified image conforms to the predetermined amplification ratio.

[0019] According to a third aspect of the embodiments of the present disclosure, there is provided an image reconstruction apparatus, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to: execute the image reconstruction method described in the first aspect or any one of the implementation manners of the first aspect.

[0020] According to a fourth aspect of the embodiments of the present disclosure, there is provided a storage medium storing instructions, which when executed by a processor of a terminal, enable the terminal to execute the image reconstruction method described in the first aspect or any one of the implementation manners of the first aspect.

[0021] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: Based on the first image input by the user and the image reconstruction model, the first image is amplified, and the second image is output to the user. The second image is amplified compared with the first image and has multiple different amplification ratios, meeting the user's need to amplify the image and amplify multiple ratios, providing convenience for the user and improving the user experience.

[0022] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0024] Figure 1 is a flowchart of an image reconstruction method shown according to an exemplary embodiment.

[0025] Figure 2 is a flowchart of a method for amplifying a first image based on an image reconstruction model shown according to an exemplary embodiment.

[0026] Figure 3 is a flowchart of a method for obtaining initial amplification parameters based on the subject recognition result shown according to an exemplary embodiment.

[0027] Figure 4 is a flowchart of a method for obtaining initial amplification parameters based on the subject recognition result shown according to another exemplary embodiment.

[0028] Figure 5 is a flowchart of a method for obtaining initial amplification parameters based on the subject recognition result shown according to an exemplary embodiment.

[0029] Figure 6 is a flowchart of a method for amplifying an initial amplified image shown according to an exemplary embodiment.

[0030] Figure 7 is a flowchart of a method for cropping a second amplified image shown according to an exemplary embodiment.

[0031] Figure 8 is a schematic diagram of an image reconstruction method shown according to an exemplary embodiment.

[0032] Figure 9 is a block diagram of an image reconstruction device shown according to an exemplary embodiment.

[0033] Figure 10 is a block diagram of a device for image reconstruction shown according to an exemplary embodiment. Detailed implementation manners

[0034] Here, the exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0035] The image reconstruction method provided by the embodiments of the present disclosure is applied to the scenario where a user beautifies the composition of a user image on an electronic device. Among them, the electronic device may be a terminal. The terminal may be a mobile phone, a tablet computer, or the like.

[0036] In daily life, users take pictures by using a terminal and select good-looking pictures taken for browsing and sharing. However, ordinary users do not have rich photography skills and cannot take pictures with compositional aesthetics. Therefore, the pictures taken by ordinary users often have problems such as the person not being in the center of the picture, the person being too far away from the shooting distance, and part of the person's body missing. Therefore, the related art has beautified the composition of the pictures taken by users. Among them, most of the optimization means are through intelligent cropping. By identifying the main object in the image and cutting the edge of the main object, the main object is highlighted to meet the requirements of compositional aesthetics. However, the related art still has the disadvantages of a single method for reconstructing user images and poor quality of image reconstruction.

[0037] In view of this, an image reconstruction method provided by the present disclosure is that an electronic device receives an image uploaded by a user and calls an image reconstruction model to perform amplification processing on the image uploaded by the user to obtain a target image that highlights the main object of the image, thereby improving the user experience.

[0038] Figure 1 is a flowchart of an image reconstruction method shown according to an exemplary embodiment. As Figure 1 shown, it includes the following steps.

[0039] In step S11, a first image input by the user is obtained.

[0040] In step S12, based on the image reconstruction model and the first image, the first image is amplified to obtain a predetermined number of second images, and different images among the predetermined number of second images have different amplification ratios relative to the first image.

[0041] In the embodiments of the present disclosure, the first image uploaded by the user is amplified through the image reconstruction model, and several amplified second images with different ratios are output to the user. It can be understood that based on the first image input by the user and the image reconstruction model, the first image is amplified, and the second image is output to the user. The second image is amplified compared with the first image and has multiple different amplification ratios, meeting the user's need for amplifying the image with multiple ratios, providing convenience for the user, and improving the user experience.

[0042] For example, for the first image, the related art can only segment the image to highlight the main object in the first image. The present disclosure can also amplify the first image, that is, amplify the boundary of the first image to obtain a second image, so that the center of gravity of the main object in the second image is located at positions such as the golden section line, highlighting the main object in the second image and achieving the effect of composition aesthetics. Users often have different ratio requirements for the captured image, such as the size of 16:9, 4:3, 1:1, etc. The present disclosure can amplify the first image according to the preset ratio of the user. For example, if the ratio of the first image is 2:3, the first image is amplified to 4:3, 1:1, 16:9, etc., and output to the user to meet the user's need for amplified images with multiple ratios.

[0043] Next, a method for amplifying the first image based on the image reconstruction model will be described.

[0044] Figure 2 is a flowchart of a method for amplifying a first image based on an image reconstruction model shown according to an exemplary embodiment. As Figure 2 shown, it includes the following steps.

[0045] In step S21, based on the first image and the first model in the image reconstruction model, the main image in the first image is recognized to obtain a main recognition result, and the first model is used for main recognition.

[0046] In the embodiments of the present disclosure, an image reconstruction model is used to perform amplification processing on an image. Among them, the image reconstruction model can be composed of multiple neural network models, and each neural network model respectively implements part of the functions of image amplification. For example, the first model in the image reconstruction model is used to obtain the main object of the image, and the second model in the image reconstruction model is used to amplify the image.

[0047] In the embodiments of the present disclosure, the first model is used to obtain the main image in the image. Among them, the first model can be a saliency segmentation model, such as a lightweight network (such as MobileNet) model. By using the saliency segmentation model, the area of the main object in the image can be recognized based on the image uploaded by the user, so as to obtain the main object image.

[0048] In the embodiments of the present disclosure, the size of the image input to the saliency segmentation model is a preset threshold, such as a size of 512*512, and the size of the image output by the saliency segmentation model is also equal to the fixed threshold of the input image. When the size of the image data input to the model does not meet the preset threshold, scaling (resize) is required, that is, an image with a size larger than the preset threshold needs to be reduced, and an image with a size smaller than the preset threshold needs to be enlarged.

[0049] In the embodiments of the present disclosure, the saliency segmentation model is trained with training data. Among them, the training data contains the data of the mask of the main object in the image, that is, the area inside the contour of the main object and the contour of the main object in the image.

[0050] In the embodiments of the present disclosure, the training process of training the saliency segmentation model with training data can be: based on the image uploaded by the user, the saliency segmentation model recognizes the main object in the image and outputs a mask corresponding to the image size, and calculates the focal loss function between the mask output by the model and the ground truth mask, and uses the RMSprop (Root Mean Square Propagation) optimizer for iterative update. Among them, the purpose of using the RMSprop optimizer is to update the parameters of the neural network.

[0051] In step S22, based on the main object recognition result, an initial amplification parameter is obtained, and the first image is amplified based on the initial amplification parameter to obtain an initial amplified image.

[0052] In the embodiments of the present disclosure, by using the main object recognition of the first image, an initial amplification parameter for amplifying the first image can be obtained, and the first image is amplified into an initial amplified image according to the initial amplification parameter.

[0053] In the embodiments of the present disclosure, relevant image filling techniques such as a generative diffusion model are used to fill the amplified area of the amplified image.

[0054] In step S23, the initial amplified image is amplified according to a corresponding predetermined amplification ratio for a predetermined number of times to obtain a second image corresponding to the predetermined number of times.

[0055] In the embodiments of the present disclosure, by performing secondary amplification on the initial amplified image according to a preset amplification ratio, a second image can be obtained.

[0056] In the embodiments of the present disclosure, the image ratio of the initial amplified image may not meet the predetermined amplification ratio. Therefore, it is necessary to perform re - amplification to reach the predetermined amplification ratio. For example, if the image ratio of the initial amplified image is A:B and the predetermined amplification ratio is C:B, and C:B is greater than A:B, then the initial amplified image needs to be amplified.

[0057] Next, a method for obtaining initial amplification parameters based on the subject recognition result will be explained.

[0058] Figure 3 is a flowchart of a method for obtaining initial amplification parameters based on the subject recognition result shown according to an exemplary embodiment. As Figure 3 shown, it includes the following steps.

[0059] In step S31, in response to the subject recognition result indicating that no subject image is recognized, the first image is input into the second model in the image reconstruction model.

[0060] In step S32, a first amplification parameter for amplifying the first image is obtained as the initial amplification parameter. The second model is used to determine the image amplification parameter.

[0061] In the embodiments of the present disclosure, if the saliency segmentation model fails to recognize the subject in the first image, the first image is input into the second model to obtain the first amplification parameter.

[0062] In the embodiments of the present disclosure, the second model is used to amplify the first image to obtain the amplification parameter. Among them, the second model can be an image amplification model, such as the RegNet model.

[0063] In the embodiments of the related art, when performing saliency composition on some images, for example, an image in which the subject in the first image only shows a part of the body, the subject after saliency composition still only has a part of the body. In the embodiments of the present disclosure, after using the saliency segmentation model to segment the image subject from the first image, the first image is amplified and filled, so that the quality of the reconstructed image is better. For example, a user uploads an image, and the subject is the user himself. It should be noted that the subject in this image only includes a part of the user's body. The image amplification model can amplify and fill the first image, so that the subject is repaired completely and the user is in a beautifully composed position in the image.

[0064] In the embodiments of the present disclosure, the size of the image input to the image amplification model is a preset threshold, for example, it can be a size of 256*256. The output data of the image amplification model is the initial amplification parameter.

[0065] In the embodiments of the present disclosure, the training data for training the image amplification model can be a data set containing a main body frame, and the main body frame is the outer contour of the main object in the outlined image.

[0066] In the embodiments of the present disclosure, before the image amplification model is trained using the data set, the data set needs to be preprocessed. The preprocessing process can be: during training, a margin is extended outward from the main body frame to intercept the main object area, and the intercepted area is randomly cut to ensure that the length and width of the cut image are not less than a part of the length and width of the intercepted area, for example, 50%. The length and width of the intercepted area are used as the ground truth. It can be understood that the ground truth is the original image size. The purpose of preprocessing the data set is to make the trained image amplification model of higher quality and be able to perform higher-quality amplification according to the main object in the image uploaded by the user.

[0067] In the embodiments of the present disclosure, the process of training the image amplification model using the data set containing the main body frame can be: using the cut image, the image amplification model calculates the amplification ratios required in the four directions of the main object, multiplies the obtained ratios by the length and width of the input image, calculates the mean absolute error (L1 loss) with the ground truth, and uses the Adaptive Moment Estimation (Adam) optimizer for iterative update. Among them, the purpose of using the Adam optimizer is to update the parameters of the neural network.

[0068] In the embodiments of the present disclosure, the purpose of using the image amplification model to amplify the image is to make the model use pictures containing various types of main objects during training, and the goal is to restore the original size of the main object through amplification. Through this learning method, the model can learn the structural information of different main objects, so that in the inference stage, for an input image containing a large target main object, the amplification direction and length of the image can be output more accurately.

[0069] Next, an explanation of another method for obtaining the initial amplification parameter based on the main object recognition result will be given.

[0070] Figure 4 is a flowchart of a method for obtaining the initial amplification parameter based on the main object recognition result shown in another exemplary embodiment, as Figure 4 shown, including the following steps.

[0071] In step S41, it is detected that the subject recognition result indicates that the subject image is recognized, and the proportion of the area occupied by the subject image in the first image is greater than or equal to a preset proportion threshold.

[0072] In the embodiments of the present disclosure, the subject image parameter compared with the preset proportion threshold may be the area of the subject image, and the preset threshold may be one-half of the area of the first image.

[0073] In step S42, the first image is input into the second model in the image reconstruction model to obtain a first amplification parameter for amplifying the first image.

[0074] In the embodiments of the present disclosure, after the saliency segmentation model segments the subject in the first image, it compares the subject image parameter with the preset threshold. When the subject image parameter is greater than the preset threshold, the first image is input into the image amplification model to obtain the first amplification parameter.

[0075] In step S43, a first distance between the boundary of the subject image and the boundary of the first image is obtained. If the first distance is less than a preset distance threshold, the first image is amplified based on the first distance to obtain a second amplification parameter for amplifying the first image.

[0076] In the embodiments of the present disclosure, the distance from each side of the subject image in the first image to the corresponding boundary of the first image is obtained respectively. If the obtained distance is lower than the preset distance threshold, the first image is amplified to obtain a second amplification parameter, so that the distance from each side of the subject image in the first image to the corresponding boundary of the first image is greater than the distance threshold.

[0077] In the embodiments of the present disclosure, amplifying the first image based on the second amplification parameter can prevent the subject in the first image from being located on the image boundary, meeting the requirements of composition aesthetics.

[0078] In step S44, a second distance between the centroid point of the subject image and a preset high-quality point is obtained, and a first direction in which the preset high-quality point is located relative to the centroid point of the subject image is obtained. The first image is amplified by the second distance in the direction opposite to the first direction to obtain a third amplification parameter for amplifying the first image.

[0079] Among them, the preset high-quality point is a point within the first image and located on the target line of the first image. The target line is at least one of the following: the golden section line, the rule of thirds line, and the center line.

[0080] In the embodiments of the present disclosure, the centroid point of the main image in the first image is obtained, and at least one of the following preset high-quality points is obtained: points on the trisecting line, points on the central line, and points on the golden section line. Based on the distance between the preset high-quality point and the centroid point of the main image and the first direction in which the preset high-quality point is located with respect to the centroid point of the main image, the first image is amplified in the opposite direction of the first direction to obtain a third amplification parameter.

[0081] In the embodiments of the present disclosure, by amplifying the first image in the opposite direction of the first direction, the centroid point of the main object can be located on the preset high-quality point, ensuring the requirements of compositional aesthetics.

[0082] In step S45, in response to the first amplification parameter, the second amplification parameter, and the third amplification parameter including an amplification length, the amplification parameter with the largest corresponding amplification length among the first amplification parameter, the second amplification parameter, and the third amplification parameter is used as the initial amplification parameter.

[0083] In the embodiments of the present disclosure, the amplification lengths of the obtained first amplification parameter, second amplification parameter, and third amplification parameter are compared, and the amplification parameter with the longest amplification length is selected as the initial amplification parameter.

[0084] In the embodiments of the present disclosure, using the amplification parameter with the longest amplification length as the initial amplification parameter can make the amplification effect of the initial amplified image better.

[0085] Next, another method for obtaining the initial amplification parameter based on the main object recognition result is explained.

[0086] Figure 5 is a flowchart of a method for obtaining an initial amplification parameter based on the main object recognition result shown in an exemplary embodiment, as Figure 5 shown, and includes the following steps.

[0087] In step S51, it is detected that the main object recognition result indicates that the main image is recognized, and the proportion of the area occupied by the main image in the first image is less than a preset proportion threshold.

[0088] In the embodiments of the present disclosure, the main object image parameter compared with the preset proportion threshold may be the area of the main object image, and the preset threshold may be one-half of the area of the first image.

[0089] In step S52, the first distance between the boundary of the main image and the boundary of the first image is obtained. If the first distance is less than a preset distance threshold, the first image is amplified based on the first distance to obtain a fourth amplification parameter for amplifying the first image.

[0090] In the embodiments of the present disclosure, the distances from each side of the main image in the first image to the corresponding boundaries of the first image are obtained. If the obtained distances are lower than a preset distance threshold, the first image is amplified to obtain a fourth amplification parameter, so that the distances from each side of the main image in the first image to the corresponding boundaries of the first image are greater than the distance threshold.

[0091] In the embodiments of the present disclosure, amplifying the first image based on the fourth amplification parameter can prevent the main object in the first image from being located on the image boundary, meeting the requirements of composition aesthetics.

[0092] In step S53, the second distance between the centroid point of the main image and a preset high-quality point is obtained. Based on the fact that the preset high-quality point is in the first direction of the centroid point of the main image, the first image is amplified by the second distance in the direction opposite to the first direction to obtain a fifth amplification parameter for amplifying the first image.

[0093] In the embodiments of the present disclosure, the centroid point of the main image in the first image is obtained, and at least one of the following preset high-quality points is obtained: points on the third dividing line, points on the central line, and points on the golden section line. Based on the distance between the preset high-quality point and the centroid point and the first direction in which the preset high-quality point is located with respect to the centroid point of the main image, the first image is amplified in the direction opposite to the first direction to obtain a fifth amplification parameter.

[0094] In the embodiments of the present disclosure, by amplifying the first image in the direction opposite to the first direction, the centroid point of the main object can be located on the preset high-quality point, meeting the requirements of composition aesthetics.

[0095] In step S54, in response to the fourth amplification parameter and the fifth amplification parameter including amplification lengths, the amplification parameter with the largest corresponding amplification length in the fourth amplification parameter and the fifth amplification parameter is used as the initial amplification parameter.

[0096] In the embodiments of the present disclosure, the amplification lengths of the obtained fourth amplification parameter and fifth amplification parameter are compared, and the amplification parameter with the longest amplification length is selected as the initial amplification parameter.

[0097] In the embodiments of the present disclosure, using the amplification parameter with the longest amplification length as the initial amplification parameter can make the amplification effect of the initial amplified image better.

[0098] The following explains a method for amplifying an initial amplified image.

[0099] Figure 6 is a flowchart of a method for amplifying an initial amplified image shown according to an exemplary embodiment, as Figure 6 shown, and includes the following steps.

[0100] In step S61, if the initial amplified image ratio does not meet the predetermined amplification ratio, obtain the short side of the initial amplified image; based on the distance ratio and the predetermined amplification ratio, distribute the amplification length to the two amplification directions of the short side to obtain a second amplified image.

[0101] Among them, the distance ratio is the distance ratio between a specified point of the first image and different boundaries of the first image. The specified point includes the center of gravity of the main image in the first image or the center point of the first image. Different boundaries of the first image include the upper boundary and the lower boundary of the first image, or the upper left boundary and the right boundary of the first image.

[0102] In the embodiments of the present disclosure, the image ratio of the initial amplified image may not meet the predetermined amplification ratio. Then, it is necessary to amplify the short side of the initial amplified image, and distribute the length to be amplified to the two directions of the short side according to the distance ratio and the predetermined amplification ratio to obtain a second amplified image. It can be understood that the short side of the image can be the left and right boundaries or the upper and lower boundaries.

[0103] In the embodiments of the present disclosure, the amplification criterion for the short side of the image is as follows: if the initial amplified image contains a main object, calculate the distances from the center of gravity of the main object to the four sides of the amplified image, namely the upper, lower, left, and right sides. Obtain the distance from the center of gravity to the upper boundary, obtain the distance from the center of gravity to the lower boundary, and compare the distances from the center of gravity to the two sides to obtain the upper and lower distance ratio. Or obtain the distance from the center of gravity to the left boundary, obtain the distance from the center of gravity to the right boundary, and compare the distances from the center of gravity to the two sides to obtain the left and right distance ratio. The left and right distance ratio and the upper and lower distance ratio are collectively referred to as the distance ratio. Compare the obtained distance ratio with the preset amplification ratio, and distribute the amplification length proportionally to the left and right or upper and lower two amplification directions.

[0104] It can be understood that the basis for judging the up, down, left, and right of the image is as follows: when the scene in the figure can be observed by the observer standing upright, the scene in the figure conforms to the influence of gravity. At this time, the position affected by gravity of animals, plants, or the scene is the lower direction of the image. For example, the direction pointed by the feet of an animal or the roots of a plant is the lower direction of the image, and the direction opposite to the lower direction of the image is the upper direction of the image. The direction of the image on the right side of the observer is the right direction of the image, and the direction of the image on the left side of the observer is the left direction of the image.

[0105] If the initial amplified image does not contain the main object, calculate the distances from the center point of the initial amplified image to the upper, lower, left, and right sides of the amplified image respectively. Obtain the distance from the center point to the upper boundary, obtain the distance from the center point to the lower boundary, compare the distances from the center point to the two sides, and obtain the upper-lower distance ratio. Alternatively, obtain the distance from the center point to the left boundary, obtain the distance from the center point to the right boundary, compare the distances from the center point to the two sides, and obtain the left-right distance ratio. The left-right distance ratio and the upper-lower distance ratio are collectively referred to as the distance ratio. Compare the obtained distance ratio with the preset amplification ratio, and distribute the amplification length proportionally to the left-right or upper-lower two amplification directions.

[0106] It can be understood that the basis for judging the up, down, left, and right of the image is as follows: when the scene in the figure can be observed upright by the observer, the scene in the figure conforms to the influence of gravity. At this time, the position where the animals, plants, or the scene is affected by gravity is the lower direction of the image. For example, the direction pointed by the animal's feet or the plant's roots is the lower direction of the image, and the direction opposite to the lower direction of the image is the upper direction of the image. The direction of the image on the right hand side of the observer is the right direction of the image, and the direction of the image on the left hand side of the observer is the left direction of the image.

[0107] In step S62, if the second amplified image still does not meet the predetermined amplification ratio, obtain the long side of the second amplified image; crop the long side of the second amplified image, and based on the distance ratio and the predetermined amplification ratio, distribute the cropping length to the two cropping directions of the long side.

[0108] In the embodiments of the present disclosure, if the image ratio of the second amplified image still does not meet the predetermined amplification ratio, it is necessary to crop the long side of the second amplified image, and distribute the length to be cropped to the two directions of the long side according to the distance ratio and the predetermined amplification ratio. It can be understood that the long side of the image can be the left and right boundaries or the upper and lower boundaries.

[0109] In the embodiments of the present disclosure, if the initial amplified image still does not meet the predetermined amplification ratio after amplification, it is necessary to crop the amplified initial amplified image. It can be understood that the initial amplified image cannot always be amplified to reach the predetermined amplification ratio. If the amplification length is too large, the quality of subsequent image filling cannot be guaranteed. Therefore, cropping is required. For example, the image ratio of the initial amplified image is X:Y. After amplifying the initial amplified image, the image ratio reaches 2X:Y. Compared with the preset amplification ratio A:Y, 2X:Y is still less than A:Y. Therefore, the ratio of the distances from the center point or the center of gravity of the main object of the amplified initial amplified image to the two boundaries with a length of Y can be calculated, and the cropping length is distributed to the two boundary directions with a length of Y according to the ratio.

[0110] The following explains a method for cropping the second amplified image.

[0111] Figure 7It is a flowchart of a method for cropping a second amplified image shown according to an exemplary embodiment. As Figure 7 shown, it includes the following steps.

[0112] In step S71, if the cropping target is the main object image, the area of the cropped main object image is lower than the first cropping threshold.

[0113] In the embodiments of the present disclosure, when cropping the second amplified image, it is necessary to avoid cropping the main object part. If the main object must be cropped, the area of the cropped main object should be lower than a certain threshold.

[0114] In step S72, if the second amplified image is vertically cropped and the cropping target is the main object image and the cropping length is greater than the second cropping threshold, start cropping from the bottom of the second amplified image, and keep the distance between the top of the main object image and the upper boundary of the second amplified image greater than the preset distance threshold until the second amplified image meets the predetermined amplification ratio.

[0115] In the embodiments of the present disclosure, the bottom of the second amplified image is equivalent to the downward direction of the second amplified image. The top of the main object image is in the same direction as the top of the second amplified image, and the top of the second amplified image is equivalent to the upward direction of the second amplified image. It can be understood that the basis for judging the up, down, left, and right of the image is: when the scene in the figure can be observed by the observer standing upright, the scene in the figure conforms to the influence of gravity. At this time, the position affected by gravity of animals, plants, or the scene is the downward direction of the image. For example, the direction pointed by the feet of animals or the roots of plants is the downward direction of the image, and the direction opposite to the downward direction of the image is the upward direction of the image. The direction of the image located on the right hand side of the observer is the right direction of the image, and the direction of the image located on the left hand side of the observer is the left direction of the image.

[0116] In the embodiments of the present disclosure, if the second amplified image needs to be vertically cropped by a certain length and the main object needs to be cropped, start cropping from the bottom of the second amplified image while ensuring a certain distance between the top of the main object image and the upper boundary of the second amplified image until the predetermined amplification ratio is met. It can be understood that the "while ensuring a certain distance between the top of the main object image and the upper boundary of the second amplified image" mentioned in the above steps can be understood as follows: if it is necessary to ensure that the distance between the top of the main object image and the upper boundary of the second amplified image is E / F of the vertical height of the main object image, then during the process of cropping from the bottom of the second amplified image, if the bottom of the main object is cropped, it is necessary to simultaneously reduce the distance between the top of the main object and the upper boundary of the second amplified image to ensure that this distance is E / F of the vertical height of the cropped main object.

[0117] Next, an explanatory illustration of a schematic diagram of an image reconstruction method will be given.

[0118] Figure 8It is a schematic diagram of an image reconstruction method shown according to an exemplary embodiment. As Figure 8 shown, for the user input image, through the saliency segmentation model, the main object in the first image is obtained, and the ratio of the main object to the first image, the centroid, and the distances of the main object from the four boundaries of the first image are calculated. If the size of the main object is greater than the preset ratio threshold, the first image is input into the image augmentation model to obtain the augmentation direction and length of the current first image. If no main object is detected, the first image is also input into the image augmentation model to obtain the augmentation direction and length of the first image.

[0119] If a main object is detected, calculate the distance between the boundary of the main object and the boundary of the first image. If the distance is less than a certain threshold, obtain the augmentation direction and length of the first image by augmenting in the direction of this boundary.

[0120] If a main object is detected, calculate the high-quality point closest to the centroid of the current main object, and obtain the augmentation direction and length of the first image by augmenting in the opposite direction of the moving direction of the main object towards the high-quality point.

[0121] According to the augmentation direction and length calculated in the above steps, retain the maximum augmentation lengths in the four directions of up, down, left, and right of the image to obtain the initial augmented image, i.e., the augmentation mask. If a main object is detected, calculate the distances from the centroid point of the main object to the four sides of the initial augmented image in the up, down, left, and right directions, and calculate the distance ratios. That is, obtain the distance from the centroid point to the upper boundary, obtain the distance from the centroid point to the lower boundary, and compare the distances from the centroid point to the two sides to obtain the up-down distance ratio. Or obtain the distance from the centroid point to the left boundary, obtain the distance from the centroid point to the right boundary, and compare the distances from the centroid point to the two sides to obtain the left-right distance ratio. The left-right distance ratio and the up-down distance ratio are collectively referred to as the distance ratio.

[0122] If no main object is detected, calculate the distances from the center point of the original image to the four sides of the initial augmented image in the up, down, left, and right directions, and calculate the distance ratios. That is, obtain the distance from the center point to the upper boundary, obtain the distance from the center point to the lower boundary, and compare the distances from the center point to the two sides to obtain the up-down distance ratio. Or obtain the distance from the center point to the left boundary, obtain the distance from the center point to the right boundary, and compare the distances from the center point to the two sides to obtain the left-right distance ratio. The left-right distance ratio and the up-down distance ratio are collectively referred to as the distance ratio.

[0123] The initial amplified image is further amplified and cropped according to the predefined output image ratio. The short side of the initial amplified image is amplified according to the required output image ratio. If the initial amplified image needs to be amplified horizontally or vertically by a certain length, the amplified length is proportionally distributed to the left and right or upper and lower amplification directions according to the ratio of the distance between the center of gravity or the center point and the left and right boundaries or the upper and lower boundaries of the initial amplified image calculated in the previous step. If the second amplified image still does not meet the required ratio, the long side is cropped. The principle of cropping is to try not to crop the main object; if the second amplified image needs to be cropped vertically, when the main object needs to be cropped, the cropping starts from the bottom, and at the same time, it is ensured that the top of the main object is at a certain distance from the upper boundary. If the second amplified image needs to be cropped vertically by a certain length, the cropping length is proportionally distributed to the upper and lower or left and right directions according to the upper and lower distance ratio or the left and right distance ratio calculated in the previous step. The cropped second amplified image is image filled to output a plurality of target images of preset ratios. It can be understood that the steps of further amplifying and cropping the initial amplified image are all included in the block diagram represented by the amplification mask in the figure.

[0124] Among them, the white area in the box diagram represented by the main object in the figure is the main object area, the black area in the box diagram represented by the main object in the figure is the background area, the white area in the box diagram represented by the initial amplification image in the figure is the original image area, the black area in the box diagram represented by the initial amplification image in the figure is the amplification area, the white area in the box diagram represented by the amplification mask in the figure is the original image area, and the black area in the box diagram represented by the amplification mask in the figure is the amplification area.

[0125] Based on the same concept, the embodiment of the present disclosure also provides an image reconstruction device.

[0126] It is understandable that, in order to realize the above functions, the image reconstruction device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0127] Figure 9 is a block diagram of an image reconstruction device according to an exemplary embodiment. Figure 9 The device 100 includes a receiving unit 101 and a processing unit 102.

[0128] A receiving unit 101, configured to obtain a first image input by a user;

[0129] A processing unit 102, configured to amplify the first image based on an image reconstruction model and the first image to obtain a predetermined number of second images, where different images among the predetermined number of second images have different amplification ratios relative to the first image.

[0130] In an embodiment of the present disclosure, the processing unit 102 is configured to identify a main image in the first image based on the first image and a first model in the image reconstruction model to obtain a main body recognition result, where the first model is used for main body recognition; based on the main body recognition result, obtain an initial amplification parameter, and amplify the first image based on the initial amplification parameter to obtain an initial amplified image; amplify the initial amplified image according to corresponding predetermined amplification ratios of a predetermined number to obtain corresponding second images of the predetermined number.

[0131] In an embodiment of the present disclosure, the processing unit 102 is configured to, in response to the main body recognition result indicating that no main image is recognized, input the first image into a second model in the image reconstruction model; obtain a first amplification parameter for amplifying the first image as the initial amplification parameter, where the second model is used to determine an image amplification parameter.

[0132] In an embodiment of the present disclosure, the processing unit 102 is configured to, if the main body recognition result indicates that a main image is recognized and the proportion of the area of the main image in the first image is greater than or equal to a preset proportion threshold, adopt the following method to obtain an initial amplification parameter: input the first image into a second model in the image reconstruction model to obtain a first amplification parameter for amplifying the first image, and obtain a first distance between the boundary of the main image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a second amplification parameter for amplifying the first image, and obtain a second distance between the center of gravity point of the main image and a preset high-quality point, and obtain a first direction in which the preset high-quality point is located at the center of gravity point of the main image, and amplify the first image in a direction opposite to the first direction by the second distance to obtain a third amplification parameter for amplifying the first image. The preset high-quality point is a point within the first image and is located on a target line of the first image, and the target line is at least one of the following: a golden section line, a thirds line, and a center line; in response to the first amplification parameter, the second amplification parameter, and the third amplification parameter including an amplification length, use the amplification parameter with the largest amplification length corresponding to the first amplification parameter, the second amplification parameter, and the third amplification parameter as the initial amplification parameter.

[0133] In an embodiment of the present disclosure, the processing unit 102 is configured to, if the subject recognition result indicates that the subject image is recognized and the proportion of the area occupied by the subject image in the first image is less than a preset proportion threshold, obtain initial amplification parameters in the following manner: obtain a first distance between the boundary of the subject image and the boundary of the first image, if the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a fourth amplification parameter for amplifying the first image, and obtain a second distance between the center of gravity point of the subject image and a preset high-quality point, and amplify the first image by the second distance in a direction opposite to the first direction based on the fact that the preset high-quality point is in the first direction of the center of gravity point of the subject image to obtain a fifth amplification parameter for amplifying the first image; in response to the fourth amplification parameter and the fifth amplification parameter including amplification lengths, use the amplification parameter with the largest corresponding amplification length among the fourth amplification parameter and the fifth amplification parameter as the initial amplification parameter.

[0134] In an embodiment of the present disclosure, the processing unit 102 is configured to, if the proportion of the initial amplified image does not meet a predetermined amplification proportion, obtain the short side of the initial amplified image; based on a distance ratio and the predetermined amplification proportion, distribute the amplification length to two amplification directions of the short side to obtain a second amplified image, the distance ratio is the distance ratio between a designated point of the first image and different boundaries of the first image, the designated point includes the center of gravity point of the subject image in the first image or the center point of the first image, and different boundaries of the first image include the upper boundary and the lower boundary of the first image, or the upper left boundary and the right boundary of the first image; if the second amplified image still does not meet the predetermined amplification proportion, obtain the long side of the second amplified image; crop the long side of the second amplified image, and distribute the cropping length to two cropping directions based on the distance ratio and the predetermined amplification proportion.

[0135] In an embodiment of the present disclosure, the processing unit 102 is configured to distribute the cropping length to two cropping directions of the long side based on the distance ratio and the predetermined amplification proportion, including at least one of the following: if the cropping target is the subject object image, the area of the cropped subject object image is lower than a first cropping threshold; if performing vertical cropping on the second amplified image and the cropping target is the subject object image and the cropping length is greater than a second cropping threshold, start cropping from the bottom of the second amplified image and keep the distance between the top of the subject object image and the upper boundary of the second amplified image greater than a preset distance threshold until the second amplified image meets the predetermined amplification proportion.

[0136] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0137] Figure 10FIG. 0 is a block diagram of an apparatus 200 for image reconstruction according to an exemplary embodiment. For example, the apparatus 200 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0138] Referring Figure 10 , the apparatus 200 may include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, an input / output (I / O) interface 212, a sensor component 214, and a communication component 216.

[0139] The processing component 202 generally controls the overall operation of the apparatus 200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to complete all or part of the steps of the above-described methods. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0140] The memory 204 is configured to store various types of data to support the operation of the apparatus 200. Examples of such data include instructions for any application or method operating on the apparatus 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0141] The power component 206 provides power to the various components of the apparatus 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the apparatus 200.

[0142] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0143] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC) that is configured to receive external audio signals when the device 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 further includes a speaker for outputting audio signals.

[0144] The I / O interface 212 provides an interface between the processing component 202 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0145] The sensor component 214 includes one or more sensors for providing a status assessment of various aspects of the device 200. For example, the sensor component 214 can detect the on / off state of the device 200, the relative positioning of components, such as the display and the keypad of the device 200. The sensor component 214 can also detect a change in the position of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200, and the temperature change of the device 200. The sensor component 214 can include a proximity sensor that is configured to detect the presence of nearby objects without any physical contact. The sensor component 214 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 214 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0146] The communication component 216 is configured to facilitate communication, either wired or wirelessly, between the device 200 and other devices. The device 200 may access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0147] In an exemplary embodiment, the device 200 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described method.

[0148] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 204 including instructions, is also provided. The above instructions may be executed by the processor 220 of the device 200 to complete the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, among others.

[0149] It can be understood that "a plurality of" in the present disclosure means two or more, and other quantifiers are similar thereto. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms of "a", "the", and "said" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0150] It can be further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other and do not indicate a specific order or importance level. In fact, the expressions such as "first" and "second" can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0151] It can be further understood that, unless otherwise specified, "connection" includes both direct connection without other components between the two and indirect connection with other elements between the two.

[0152] It can be further understood that although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be construed as requiring these operations to be performed in the specific order shown or in a serial order, or requiring all the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0153] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure.

[0154] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image reconstruction method, characterized in that, it includes: obtaining a first image input by a user; based on an image reconstruction model and the first image, amplifying the first image to obtain a predetermined number of second images, and different images among the predetermined number of second images have different amplification ratios relative to the first image.

2. The method according to claim 1, characterized in that, the step of amplifying the first image based on the image reconstruction model and the first image to obtain a predetermined number of second images includes: identifying a main image in the first image based on the first image and a first model in the image reconstruction model to obtain a main identification result, where the first model is used for main identification; obtaining initial amplification parameters based on the main identification result, and amplifying the first image based on the initial amplification parameters to obtain an initial amplified image; amplifying the initial amplified image according to a predetermined amplification ratio corresponding to the predetermined number to obtain the second images corresponding to the predetermined number.

3. The method according to claim 2, characterized in that, the step of obtaining initial amplification parameters based on the main identification result includes: in response to the main identification result indicating that no main image is identified, inputting the first image into a second model in the image reconstruction model; obtaining first amplification parameters for amplifying the first image as the initial amplification parameters, where the second model is used to determine image amplification parameters.

4. The method according to claim 2, characterized in that, the step of obtaining initial amplification parameters based on the main identification result includes: if the main identification result indicates that a main image is identified and the proportion of the area occupied by the main image in the first image is greater than or equal to a preset proportion threshold, the following method is used to obtain initial amplification parameters: inputting the first image into a second model in the image reconstruction model to obtain first amplification parameters for amplifying the first image, and obtaining a first distance between the boundary of the main image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplifying the first image based on the first distance to obtain second amplification parameters for amplifying the first image, and obtaining a second distance between the center of gravity point of the main image and a preset high-quality point, and obtaining a first direction in which the preset high-quality point is located at the center of gravity point of the main image, and amplifying the first image by the second distance in a direction opposite to the first direction to obtain third amplification parameters for amplifying the first image, where the preset high-quality point is a point within the first image and is located on a target line of the first image, and the target line is at least one of the following: the golden section line, the thirds line, and the center line; in response to the first amplification parameters, the second amplification parameters, and the third amplification parameters including an amplification length, taking the amplification parameter with the largest amplification length among the first amplification parameters, the second amplification parameters, and the third amplification parameters as the initial amplification parameters.

5. The method according to claim 2, characterized in that, Based on the subject recognition result, obtaining initial amplification parameters, including: If the subject recognition result indicates that a subject image is recognized and the area ratio of the subject image in the first image is less than a preset ratio threshold, the initial amplification parameters are obtained in the following manner: Obtain a first distance between the boundary of the subject image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a fourth amplification parameter for amplifying the first image, and Obtain a second distance between the centroid point of the subject image and a preset high-quality point. Based on the fact that the preset high-quality point is in a first direction from the centroid point of the subject image, amplify the first image in a direction opposite to the first direction by the second distance to obtain a fifth amplification parameter for amplifying the first image; In response to the fact that the fourth amplification parameter and the fifth amplification parameter include amplification lengths, use the amplification parameter with the largest corresponding amplification length among the fourth amplification parameter and the fifth amplification parameter as the initial amplification parameter.

6. The method according to claim 2, wherein, amplifying the initial amplified image according to the corresponding predetermined amplification ratios for the predetermined number, includes: If the ratio of the initial amplified image does not conform to the predetermined amplification ratio, obtain the short side of the initial amplified image; based on a distance ratio and the predetermined amplification ratio, distribute the amplification length to two amplification directions of the short side to obtain a second amplified image, where the distance ratio is the distance ratio between a specified point of the first image and different boundaries of the first image, the specified point includes the centroid point of the subject image in the first image or the center point of the first image, and the different boundaries of the first image include the upper boundary and the lower boundary of the first image, or the upper left boundary and the right boundary of the first image; If the second amplified image still does not conform to the predetermined amplification ratio, obtain the long side of the second amplified image; Crop the long side of the second amplified image, and based on the distance ratio and the predetermined amplification ratio, distribute the cropping length to two cropping directions of the long side.

7. The method according to claim 6, wherein, distributing the cropping length to two cropping directions of the long side based on the distance ratio and the predetermined amplification ratio includes at least one of the following: If the cropping target is the subject object image, the area of the cropped subject object image is lower than a first cropping threshold; If performing vertical cropping on the second amplified image and the cropping target is the subject object image and the cropping length is greater than a second cropping threshold, start cropping from the bottom of the second amplified image and keep the distance between the top of the subject object image and the upper boundary of the second amplified image greater than a preset distance threshold until the second amplified image conforms to the predetermined amplification ratio.

8. An image reconstruction device, wherein, comprising: a receiving unit, configured to obtain a first image input by a user; A processing unit is used to amplify the first image based on an image reconstruction model and the first image to obtain a predetermined number of second images, wherein different images in the predetermined number of second images have different amplification ratios relative to the first image.

9. The device according to claim 8, It is characterized in that The processing unit amplifies the first image based on the image reconstruction model and the first image to obtain a predetermined number of second images in the following manner: Based on the first image and a first model in the image reconstruction model, a subject image in the first image is identified to obtain a subject identification result, wherein the first model is used for subject identification; Based on the subject recognition result, initial amplification parameters are obtained, and the first image is amplified based on the initial amplification parameters to obtain an initial amplified image; The initial amplified image is amplified according to a predetermined amplification ratio corresponding to the predetermined number to obtain a second image corresponding to the predetermined number.

10. The device according to claim 9, It is characterized in that The processing unit obtains initial amplification parameters based on the subject identification result in the following manner: In response to the subject recognition result indicating that the subject image is not recognized, inputting the first image into a second model in the image reconstruction model; A first amplification parameter for amplifying the first image is obtained as the initial amplification parameter, and the second model is used to determine the image amplification parameter.

11. The device according to claim 9, It is characterized in that The processing unit obtains initial amplification parameters based on the subject identification result in the following manner: If the subject recognition result indicates that the subject image is recognized, and the area ratio of the subject image in the first image is greater than or equal to a preset ratio threshold, the initial expansion parameters are obtained in the following manner: The first image is input into a second model in the image reconstruction model to obtain a first amplification parameter for amplifying the first image, and A first distance between a boundary of the subject image and a boundary of the first image is obtained, and if the first distance is less than a preset distance threshold, the first image is enlarged based on the first distance to obtain a second enlargement parameter for enlarging the first image, and A second distance between the center of gravity of the subject image and a preset high-quality point is obtained, and a first direction in which the preset high-quality point is located at the center of gravity of the subject image is obtained, and the first image is amplified by the second distance in a direction opposite to the first direction to obtain a third amplification parameter for amplifying the first image, wherein the preset high-quality point is a point in the first image and is located on a target straight line of the first image, and the target straight line is at least one of the following: a golden section line, a third line, and a center line; In response to the first amplification parameter, the second amplification parameter and the third amplification parameter including an amplification length, the amplification parameter corresponding to the largest amplification length among the first amplification parameter, the second amplification parameter and the third amplification parameter is used as the initial amplification parameter.

12. The device according to claim 9, It is characterized in that The processing unit obtains initial amplification parameters based on the subject recognition result in the following manner: If the subject recognition result indicates that a subject image is recognized, and the proportion of the area occupied by the subject image in the first image is less than a preset proportion threshold, the initial amplification parameters are obtained in the following manner: Obtain a first distance between the boundary of the subject image and the boundary of the first image. If the first distance is less than a preset distance threshold, amplify the first image based on the first distance to obtain a fourth amplification parameter for amplifying the first image, and Obtain a second distance between the center of gravity point of the subject image and a preset high-quality point. Based on the fact that the preset high-quality point is in a first direction from the center of gravity point of the subject image, amplify the first image in a direction opposite to the first direction by the second distance to obtain a fifth amplification parameter for amplifying the first image; In response to the fact that the fourth amplification parameter and the fifth amplification parameter include amplification lengths, use the amplification parameter with the largest corresponding amplification length among the fourth amplification parameter and the fifth amplification parameter as the initial amplification parameter.

13. The apparatus according to claim 9, wherein, the processing unit amplifies the initial amplified image according to the preset amplification ratios corresponding to the predetermined number in the following manner: If the ratio of the initial amplified image does not conform to the preset amplification ratio, obtain the short side of the initial amplified image; based on a distance ratio and the preset amplification ratio, distribute the amplification length to two amplification directions of the short side to obtain a second amplified image, where the distance ratio is the distance ratio between a specified point of the first image and different boundaries of the first image, the specified point includes the center of gravity point of the subject image in the first image or the center point of the first image, and the different boundaries of the first image include the upper boundary and the lower boundary of the first image, or the upper left boundary and the right boundary of the first image; If the second amplified image still does not conform to the preset amplification ratio, obtain the long side of the second amplified image; Crop the long side of the second amplified image, and based on the distance ratio and the preset amplification ratio, distribute the cropping length to two cropping directions of the long side.

14. The apparatus according to claim 13, wherein, the processing unit distributes the cropping length to two cropping directions of the long side based on the distance ratio and the preset amplification ratio in the following manner, including at least one of the following: If the cropping target is the subject object image, the area of the cropped subject object image is lower than a first cropping threshold; If performing vertical cropping on the second amplified image and the cropping target is the subject object image and the cropping length is greater than a second cropping threshold, start cropping from the bottom of the second amplified image, and keep the distance between the top of the subject object image and the upper boundary of the second amplified image greater than a preset distance threshold until the second amplified image conforms to the preset amplification ratio.

15. An image reconstruction device, wherein, comprises: a processor; a memory for storing instructions executable by the processor; Among them, the processor is configured to execute the image reconstruction method according to any one of claims 1 to 7.

16. A storage medium, characterized in that, instructions are stored in the storage medium, and when the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the image reconstruction method according to any one of claims 1 to 7.

Images