Method, device, camera and storage medium for generating thumbnail

Abstract

The present disclosure relates to a method, device, camera and storage medium for generating thumbnails, and relates to the field of image processing technology, and solves the technical problems in the related art that thumbnails cannot highlight the salient areas of a target image, cannot fully display the main content of a target image, and cannot ensure the rationality of the distribution of objects in the thumbnail. The method includes: acquiring a target image; performing object detection on the target image, and cropping the target image according to the detected objects to obtain a plurality of cropped images; inputting each cropped image into a selection model to obtain an output score of the selection model for each cropped image, wherein the selection model is obtained by training based on an image sample set, wherein the image sample set includes a plurality of sample pairs, each sample pair includes an image sample and a score corresponding to the image sample; determining a target cropped image according to the output score, and using the target cropped image as a thumbnail of the target image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, camera and storage medium for generating thumbnails. Background Art

[0002] At present, when previewing a large number of pictures in mobile terminal applications such as photo albums and file managers, thumbnails of the pictures are usually generated to provide previews for users. For example, the overview page of the camera album is usually a square thumbnail display. At this time, we need to crop the original picture to the required size.

[0003] In the related art, the middle area of ​​the image is cropped. Blindly cropping the center area of ​​the image will cause the main object of the image to be cut off, and the significant area of ​​the image cannot be highlighted. The thumbnail display image cannot fully display the content of the original image. Even if the main object is not cut off in an extreme way, blindly cropping the center area of ​​the image cannot guarantee the rationality of the distribution of objects in the image. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, camera and storage medium for generating thumbnails.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for generating a thumbnail is provided, the method comprising:

[0006] Get the target image;

[0007] Performing object detection on the target image, and cropping the target image according to the detected objects to obtain a plurality of cropped images;

[0008] Inputting each of the cropped images into a selection model to obtain an output score of the selection model for each of the cropped images, wherein the selection model is trained based on an image sample set, wherein the image sample set includes a plurality of sample pairs, and each of the sample pairs includes an image sample and a score corresponding to the image sample;

[0009] A target cropped image is determined according to the output score, and the target cropped image is used as a thumbnail of the target image.

[0010] Optionally, performing object detection on the target image and cropping the target image according to the detected objects to obtain a plurality of cropped images includes:

[0011] Performing object detection on the target image, and determining a main area of ​​the target image according to the detected main object;

[0012] Determine a cropping frame size according to the subject area and the target image;

[0013] The target image is cropped multiple times according to the cropping frame size to obtain the multiple cropped images.

[0014] Optionally, performing object detection on the target image and determining a main area of ​​the target image according to the detected main object includes:

[0015] A minimum bounding box of the detected main object set is determined, and an image area within the minimum bounding box is used as the main body area.

[0016] Optionally, the step of cropping the target image multiple times according to the cropping frame size to obtain the multiple cropped images includes:

[0017] Determining a plurality of cropping positions of the cropping frame according to the center position of the main body area and the size of the cropping frame, wherein the cropping frame has a standard position point at each cropping position that coincides with the center position point of the main body area;

[0018] The target image is cropped at each cropping position according to the cropping frame size to obtain a plurality of cropped images, wherein the cropping frame size enables the cropping frame to fully cover the subject area during each cropping.

[0019] Optionally, performing object detection on the target image and determining a main area of ​​the target image according to the detected main object includes:

[0020] When it is detected that the target image contains a face, and the area of ​​the face detection frame accounts for a larger proportion of the target image than a preset threshold, determining that the face in the face detection frame is the main object, and the main body area includes the area where the main object is located;

[0021] When it is detected that the target image does not contain a face, or that the target image contains a face and the area of ​​the face detection frame accounts for less than the preset threshold in the target image, target detection is performed on the target object, and it is determined that the object in the target detection frame belongs to the main object, and the main area includes the area where the main object is located.

[0022] Optionally, the cropping frame is a square cropping frame, and determining the size of the cropping frame according to the size of the subject area and the size of the target image includes:

[0023] The side length L of the trimming frame is determined by the following calculation formula;

[0024] L=min(w, max(2 / 3*w, 2*c));

[0025] Wherein, w represents the minimum side length of the target image, and c represents the minimum side length of the main area.

[0026] Optionally, acquiring the target image includes:

[0027] Get the image preview request sent by the client;

[0028] After determining a target cropped image according to the output score and using the target cropped image as a thumbnail of the target image, the method further includes:

[0029] A message for responding to the image preview request is sent to the client, wherein the message includes the thumbnail.

[0030] According to a second aspect of an embodiment of the present disclosure, a device for generating a thumbnail is provided, the device comprising:

[0031] An image acquisition module is configured to acquire a target image;

[0032] a processing module configured to perform object detection on the target image, and to crop the target image according to the detected objects to obtain a plurality of cropped images;

[0033] A first execution module is configured to input each of the cropped images into a selection model to obtain an output score of the selection model for each of the cropped images, wherein the selection model is trained based on an image sample set, wherein the image sample set includes a plurality of sample pairs, each sample pair includes an image sample and a score corresponding to the image sample;

[0034] The second execution module is configured to determine a target cropped image according to the output score, and use the target cropped image as a thumbnail of the target image.

[0035] According to a third aspect of an embodiment of the present disclosure, a camera is provided, comprising the above-mentioned device for generating thumbnails.

[0036] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for generating thumbnails provided in the first aspect of the present disclosure are implemented.

[0037] According to a fifth aspect of an embodiment of the present disclosure, there is provided a device for generating a thumbnail, the device comprising: a processor;

[0038] a memory for storing processor-executable instructions;

[0039] When the processor executes the instructions stored in the memory, it can execute the method for generating thumbnails provided in the first aspect of the present disclosure.

[0040] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: object detection is performed on a target image, and multiple cropped images are obtained by cropping the target image according to the detected objects, thereby ensuring the integrity of the main object in the image; the image with the highest score among the cropped images is used as the target cropped image, and the target cropped image is used as the thumbnail of the target image, thereby ensuring the rationality of the distribution of objects in the image and the thumbnail can best display the content of the target image.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] Figure 1 This is a schematic diagram before cropping the central area of ​​the image;

[0044] Figure 2 This is a schematic diagram after cropping the central area of ​​the image;

[0045] Figure 3 The figure is a flowchart of a method for generating a thumbnail according to an exemplary embodiment.

[0046] Figure 4 is a flowchart of step S120 according to an exemplary embodiment.

[0047] Figure 5 is a schematic diagram of a cropping frame according to an exemplary embodiment.

[0048] Figure 6 FIG. 4 is a schematic diagram showing a target cropping area according to an exemplary embodiment.

[0049] Figure 7 The figure is a flowchart of a training selection model according to an exemplary embodiment.

[0050] Figure 8 The invention is a block diagram showing a device for generating a thumbnail according to an exemplary embodiment.

[0051] Fig. 9 The invention is a block diagram showing a device for generating a thumbnail according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] When previewing a large number of pictures in mobile terminal applications such as photo albums and file managers, thumbnails of the pictures are usually generated to provide previews for users. The size of the picture is generally determined by the camera, and the most common aspect ratios are 4:3, 3:4, 9:16, 16:9, 1:1, 2:3, 3:2, etc. However, in many application scenarios, when displaying photos, the original size of the photos is not directly displayed. For example, the overview page of the camera album is usually a rectangular thumbnail display. In this case, we need to crop the original image to the required size.

[0053] In the related art, the middle area of ​​the image is cropped, such as Figure 1 The diagram is shown in Figure 1. Figure 2 As shown in the schematic diagram, blindly cropping the center area of ​​the image will result in the main object of the image being cut off, and the significant area of ​​the image cannot be highlighted. The thumbnail cannot fully display the content of the original image. Even if the main object is not cut off in an extreme way, blindly cropping the center area of ​​the image cannot guarantee the rationality of the distribution of objects in the image.

[0054] In view of the deficiencies of the related art, the present disclosure proposes a method, device, camera and storage medium for generating thumbnails. The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0055] Figure 3 is a flowchart of a method for generating a thumbnail according to an exemplary embodiment, wherein the method for generating a thumbnail can be used in a device with a camera function, and the device can be a camera, a video camera, a mobile phone, etc., and the present disclosure does not limit this. Figure 3 As shown, the method comprises the following steps:

[0056] In step S110, a target image is acquired;

[0057] In step S120, object detection is performed on the target image, and a plurality of cropped images are obtained by cropping the target image according to the detected objects;

[0058] In step S130, each of the cropped images is input into a selection model to obtain an output score of the selection model for each of the cropped images, wherein the selection model is trained based on an image sample set, wherein the image sample set includes a plurality of sample pairs, and each of the sample pairs includes an image sample and a score corresponding to the image sample;

[0059] In step S140, a target cropped image is determined according to the output score, and the target cropped image is used as a thumbnail of the target image.

[0060] Specifically, the scores corresponding to the image samples can be used to measure the matching degree between the distribution of the main object in the standard image sample and the standard distribution. The higher the score, the closer the distribution of the main object is to the standard distribution, that is, the more it matches the standard distribution; the standard distribution can be the best distribution state of the thumbnail.

[0061] Optional, such as Figure 4 As shown, in step S120, object detection is performed on the target image, and multiple cropped images are obtained by cropping the target image according to the detected objects. The specific process can be shown in the following steps:

[0062] Step S1201: performing object detection on the target image, and determining a main area of ​​the target image according to the detected main object;

[0063] Step S1202: determining a cropping frame size according to the subject area and the target image;

[0064] Step S1203: the cropping frame size is used to crop the target image multiple times to obtain multiple cropped images.

[0065] The object detection may include face detection on the target image; or may include detection of display objects in the target image, such as plants, animals, buildings, etc., according to actual thumbnail display requirements.

[0066] Optionally, the step of cropping the target image multiple times according to the cropping frame size to obtain the multiple cropped images may include:

[0067] Determining a plurality of cropping positions of the cropping frame according to the center position of the main body area and the size of the cropping frame, wherein the cropping frame has a standard position point at each cropping position that coincides with the center position point of the main body area;

[0068] The target image is cropped at each cropping position according to the cropping frame size to obtain a plurality of cropped images, wherein the cropping frame size enables the cropping frame to fully cover the subject area during each cropping.

[0069] Specifically, the standard points of the cropping frame may be placed at the center points of the main body area as the cropping positions of the cropping frame.

[0070] Specifically, the cropping frame may be arranged in a nine-square grid, and the plurality of standard position points may include: a plurality of position points evenly distributed around the central grid in the nine-square grid, such as 4 or 8, but not limited thereto. That is, the cropping frame may have a plurality of standard points, such as 9 standard points, but not limited thereto.

[0071] For example, Figure 5 As shown, each position point is numbered in sequence, position point No. 9 is located at the center of the center grid in the nine-square grid, and the eight position points evenly distributed around position point No. 9 are position point No. 1, position point No. 2, position point No. 3, position point No. 4, position point No. 5, position point No. 6, position point No. 7, and position point No. 8, which are arranged in sequence on the center grid. The cropping position can be obtained by sequentially aligning the above-mentioned position points of the cropping frame with the center point of the above-mentioned main body area. For example, when position point 1 of the cropping frame coincides with the center point of the main body area, a cropping position is obtained, and the target image is cropped at the cropping position to obtain a cropped image; then, position point 2 of the cropping frame is overlapped with the center point of the main body area to obtain a cropping position, and the target image is cropped at the cropping position to obtain a second cropped image; and so on, when position point 9 of the cropping frame coincides with the center point of the main body area, a cropping position is obtained, and the target image is cropped at the cropping position to obtain the 9th cropped image, that is, 9 cropped images are obtained.

[0072] Optionally, performing object detection on the target image and determining a main area of ​​the target image according to the detected main object may include:

[0073] A minimum bounding box of the detected main object set is determined, and an image area within the minimum bounding box is used as the main body area.

[0074] Optionally, in an implementable exemplary embodiment, the minimum enclosing frame may be a rectangular frame or a square frame, but is not limited thereto.

[0075] For example, when the target image contains one main object, the detection frame of the main object is taken as the main area S; when there are multiple main objects in the target image, the minimum enclosing frames of the multiple main objects are calculated as the main area S, and the center point of the main area is set as the main center C(a, b).

[0076] Optionally, performing object detection on the target image and determining a main area of ​​the target image according to the detected main object may include:

[0077] When it is detected that the target image contains a face, and the area of ​​the face detection frame accounts for a larger proportion of the target image than a preset threshold, determining that the face in the face detection frame is the main object, and the main body area includes the area where the main object is located;

[0078] When it is detected that the target image does not contain a face, or that the target image contains a face and the area of ​​the face detection frame accounts for less than the preset threshold in the target image, target detection is performed on the target object, and it is determined that the object in the target detection frame belongs to the main object, and the main area includes the area where the main object is located.

[0079] The preset threshold value may be preset according to the display requirements of the thumbnails, and this embodiment does not specifically limit this.

[0080] For example, when object detection is performed on a target image, and a face is contained in the target image, and the area of ​​the face detection frame accounts for more than 5% of the target image, all faces in the target object are detected, and it is determined that all faces in the target detection frame belong to the main object, and the area containing the face in the target image is the main area;

[0081] When the target image does not contain a human face, or the target image contains a human face and the area of ​​the face detection frame accounts for less than 5% of the target image, target detection is performed on the animals or plants in the target object, and it is determined that all animals or plants in the target detection frame belong to the main object, and the area in the target image containing the animals or plants is the main area.

[0082] Optionally, in an implementable exemplary embodiment, the cropping frame may be a square cropping frame.

[0083] Optionally, determining the cropping frame according to the size of the subject area and the size of the target image may include:

[0084] The side length L of the trimming frame is determined by the following calculation formula;

[0085] L=min(w, max(2 / 3*w, 2*c));

[0086] Wherein, w represents the minimum side length of the target image, and c represents the minimum side length of the main area.

[0087] Optionally, after the cropping frame is determined, a plurality of cropping positions of the cropping frame are determined according to a center position point of the main body area. Figure 6 is a schematic diagram showing a target cropping area according to an exemplary embodiment. Figure 6 As shown, the position points of the cropping frame, such as No. 1-9, are sequentially overlapped with the center point of the above-mentioned main body area to calculate the cropping area.

[0088] In this embodiment, the coordinates of the upper left corner of the cropping area are (x1, y1), the coordinates of the lower right corner are (x2, y2), the coordinates of the center area are (a, b), and L represents the side length of the cropping frame;

[0089] When the center point coincides with position point 1, the coordinates of the upper left corner of the cropping frame are x1=max (a-1 / 3L, 0), y1=max(b-1 / 3L); the coordinates of the lower right corner of the cropping frame are x2=min(w, a+2 / 3L), y2=min(h, b+2 / 3L); Figure 6 As shown, the cropping frame ensures the integrity of the main object, the position of each point of the nine-square grid of the cropped area (circular area), and the rationality of the object distribution in the cropping area.

[0090] When the center point coincides with position point No. 9, the coordinates of the upper left corner of the cropping frame are x3=max (a-1 / 2L, 0), y3=max(b-1 / 2L); the coordinates of the lower right corner of the cropping frame are x4=min(w, a+1 / 2L), y4=min(h, b+1 / 2L); similarly, when the center point coincides with position points No. 2-8 respectively, the corresponding coordinates of the upper left corner and the lower right corner of the cropping frame can be obtained to determine multiple cropping positions of the cropping frame.

[0091] Optionally, the selected model may be a convolutional neural network, and the training process of the selected model may include:

[0092] The convolutional application network is iterated for a preset number of times, and the following operations are performed in each iteration:

[0093] Inputting the image sample into a convolutional neural network, and calculating the prediction score of the image sample output by the output layer of the convolutional neural network;

[0094] Determining a loss value for this iteration according to the predicted score of the image sample and the score of the image sample;

[0095] Update the network parameters of the convolutional neural network according to the loss value.

[0096] Figure 7 is a flowchart of a training selection model according to an exemplary embodiment. Figure 7 As shown, in step S130, the training process of the selected model may include:

[0097] Step S13011: Obtain an image sample set;

[0098] Step S13012: inputting the image sample into a convolutional neural network, and calculating the prediction score of the image sample output by the output layer of the convolutional neural network;

[0099] Step S13013: determining the loss value of this iteration according to the prediction score of the image sample and the score of the image sample;

[0100] Step S13014: updating the network parameters of the convolutional neural network according to the loss value;

[0101] Step S13015: When the number of iterations reaches the preset number of iterations, a trained selection model is obtained.

[0102] Among them, the preset number of falls can be set according to the number of images in the image training set; it can also be set according to the error size between the actual matching degree of the image and the corresponding standard matching degree; the embodiment of the present disclosure does not make specific limitations on this.

[0103] In this embodiment, a convolutional neural network resNet-34 is taken as an example for explanation. First, an image sample is obtained, wherein the image sample set includes a plurality of sample pairs, each of which includes an image sample and a prediction score corresponding to the image sample, and the resNet-34 initialization model parameters are initialized;

[0104] Then the image samples are input into the resNet-34 initialization model to obtain the scores of each image sample output by the output layer of the resNet-34 initialization model;

[0105] Calculate the error and error gradient between the score of each image sample and the corresponding predicted score in the image training set;

[0106] Finally, the score of each image sample and the error of the corresponding predicted score and the error gradient are back-propagated to the previous layer of the resNet-34 initialization model, and the weights of the resNet-34 initialization model are updated according to the error and error gradient to complete the first iteration;

[0107] Input the image sample to the resNet-34 initialized model after the first iteration for the next iteration;

[0108] Until the number of iterations reaches 200, a trained selection model is obtained.

[0109] Optionally, acquiring the target image may include:

[0110] Get the image preview request sent by the client;

[0111] After determining a target cropped image according to the output score and using the target cropped image as a thumbnail of the target image, the method further includes:

[0112] A message for responding to the image preview request is sent to the client, wherein the message includes the thumbnail.

[0113] For example, after obtaining an image preview request sent by a client, a thumbnail of the image is obtained by using the method for generating thumbnails disclosed in this embodiment, and then in response to the image preview request, information including the thumbnail is sent to the client, and the client displays the thumbnail in the received information.

[0114] In the method for generating thumbnails provided by the present embodiment, multiple cropped images are obtained from the target object based on object detection of the target object, thereby ensuring the integrity of the main object in the cropped image; then, based on the scores corresponding to the image samples, the cropped image with the highest score is determined from each cropped image as the target cropped image, thereby ensuring the rationality of the object distribution in the thumbnail, and the target cropped image is used as the thumbnail of the target image, so that the thumbnail can best display the main object in the target image and highlight the salient area in the target image.

[0115] Figure 8 is a block diagram of a device for generating thumbnails according to an exemplary embodiment. The device for generating thumbnails can implement part or all of a device with a camera function in the form of software, hardware, or a combination of both. Figure 8 As shown, the device 800 for generating thumbnails includes:

[0116] The image acquisition module 801 is configured to acquire a target image;

[0117] The processing module 802 is configured to perform object detection on the target image, and crop the target image according to the detected objects to obtain a plurality of cropped images;

[0118] A first execution module 803 is configured to input each of the cropped images into a selection model to obtain an output score of the selection model for each of the cropped images, wherein the selection model is trained based on an image sample set, wherein the image sample set includes a plurality of sample pairs, each sample pair includes an image sample and a score corresponding to the image sample;

[0119] The second execution module 804 is configured to determine a target cropped image according to the output score, and use the target cropped image as a thumbnail of the target image.

[0120] The device for generating thumbnails provided in the present embodiment obtains multiple cropped images from the target object based on object detection of the target object, thereby ensuring the integrity of the main object in the cropped images; then, based on the scores corresponding to the image samples, the cropped image with the highest score is determined from each cropped image as the target cropped image, thereby ensuring the rationality of the object distribution in the thumbnail, and the target cropped image is used as the thumbnail of the target image, so that the thumbnail can best display the main object in the target image and highlight the salient area in the target image.

[0121] Optionally, the processing module 802 may perform object detection on the target image, and determine a main area of ​​the target image according to the detected main object;

[0122] Determine a cropping frame size according to the subject area and the target image;

[0123] The target image is cropped multiple times according to the cropping frame size to obtain the multiple cropped images.

[0124] Optionally, the processing module 802 may also determine a minimum bounding box of the detected main object set, and use the image area within the minimum bounding box as the main body area.

[0125] Optionally, the processing module 802 may specifically determine a plurality of cropping positions of the cropping frame according to the center position of the main body region and the size of the cropping frame, wherein the cropping frame has a standard position point at each cropping position that coincides with the center position point of the main body region;

[0126] The target image is cropped at each cropping position according to the cropping frame size to obtain a plurality of cropped images, wherein the cropping frame size enables the cropping frame to fully cover the subject area during each cropping.

[0127] Optionally, the processing module 802 may specifically determine that the face in the face detection frame is the main object, and the main body area includes the area where the main object is located, when detecting that the target image contains a face and the area of ​​the face detection frame accounts for a larger proportion of the target image than a preset threshold;

[0128] When it is detected that the target image does not contain a face, or that the target image contains a face and the area of ​​the face detection frame accounts for less than the preset threshold in the target image, target detection is performed on the target object, and it is determined that the object in the target detection frame belongs to the main object, and the main area includes the area where the main object is located.

[0129] Optionally, the first execution module 803 can obtain an image sample set; input the image samples into a convolutional neural network, and calculate the predicted scores of the image samples output by the output layer of the convolutional neural network; determine the loss value of this iteration based on the predicted scores of the image samples and the scores of the image samples; update the network parameters of the convolutional neural network based on the loss value and obtain a trained selection model when the number of iterations reaches a preset number of iterations.

[0130] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0131] The present disclosure also provides a camera, which includes the above-mentioned device for generating thumbnails. The camera can obtain multiple cropped images from the target object according to the object detection of the target object through the device for generating thumbnails, thereby ensuring the integrity of the main object in the cropped images; and then determine the cropped image with the highest score from each cropped image as the target cropped image according to the scores corresponding to the image samples, thereby ensuring the rationality of the object distribution in the thumbnail, and use the target cropped image as the thumbnail of the target image, so that the thumbnail can best display the main object in the target image and highlight the salient area in the target image.

[0132] Regarding the device for generating thumbnails included in the camera in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the method for generating thumbnails provided by the present disclosure are implemented.

[0134] Specifically, the computer-readable storage medium can be a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, a server, etc.

[0135] Regarding the computer-readable storage medium in the above embodiment, the method steps when the computer program stored thereon is executed have been described in detail in the embodiment of the method and will not be elaborated here.

[0136] The present disclosure also provides a device for generating a thumbnail, which may be a computer, a platform device, etc. The device for generating a thumbnail includes:

[0137] processor;

[0138] a memory for storing processor-executable instructions;

[0139] When the processor executes the instructions stored in the memory, the steps of the method for generating thumbnails provided by the present disclosure may be performed.

[0140] The device for generating thumbnails obtains multiple cropped images from the target object based on object detection of the target object, thereby ensuring the integrity of the main object in the cropped images; then, based on the scores corresponding to the image samples, the cropped image with the highest score is determined from each cropped image as the target cropped image, thereby ensuring the rationality of the object distribution in the thumbnail, and the target cropped image is used as the thumbnail of the target image, so that the thumbnail can best display the main object in the target image and highlight the salient area in the target image.

[0141] Fig. 9 FIG. 9 is a block diagram of a device 900 for generating a thumbnail according to an exemplary embodiment. Fig. 9 As shown, the apparatus 900 for generating thumbnails may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0142] Among them, the processing component 902 generally controls the overall operation of the device 900, such as video recording operation, photo operation, etc. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method for generating thumbnails. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0143] The memory 904 is configured to store various types of data to support operations on the device 900. Examples of such data include instructions for any application or method operating on the device 900, feature information of an object (such as face information, feature information of a building, species information of an animal, etc.), preset thresholds of a proportion, etc. The memory 904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0144] The power component 906 provides power to the various components of the device 900. The power component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.

[0145] The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to display thumbnails. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel, thereby changing the thumbnails displayed on the touch screen. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0146] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a start button, and a lock button.

[0147] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the device 900. For example, the sensor assembly 914 can detect the open / closed state of the device 900 and the relative positioning of the components.

[0148] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0149] In an exemplary embodiment, the device 900 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned method of generating thumbnails.

[0150] The method, device, camera and storage medium for generating thumbnails provided by the present disclosure perform object detection on a target image, and obtain multiple cropped images by cropping the target image according to the detected objects, so as to adaptively crop a picture of a required size and ensure the integrity of the main object in the image; determine the image with the highest score from each of the cropped images as the target cropped image, and use the target cropped image as the thumbnail of the target image, so as to highlight the salient area of ​​the target image and fully display the main object to be presented in the target image, thereby ensuring the rationality of the distribution of objects in the image and the thumbnail can best display the main object of the target image.

[0151] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0152] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating a thumbnail, characterized in that: The method comprises: Get the target image; Performing object detection on the target image, and cropping the target image according to the detected objects to obtain a plurality of cropped images; Inputting each of the cropped images into a selection model to obtain an output score of the selection model for each of the cropped images, wherein the selection model is trained based on an image sample set, wherein the image sample set includes a plurality of sample pairs, each of the sample pairs includes an image sample and a score corresponding to the image sample, and the score corresponding to the image sample represents a matching degree between a distribution of a main object in the image sample and a standard distribution; A target cropped image is determined according to the output score, and the target cropped image is used as a thumbnail of the target image.

2. The method according to claim 1, characterized in that The performing object detection on the target image and cropping the target image according to the detected objects to obtain a plurality of cropped images includes: Performing object detection on the target image, and determining a main area of ​​the target image according to the detected main object; Determine a cropping frame size according to the subject area and the target image; The target image is cropped multiple times according to the cropping frame size to obtain the multiple cropped images.

3. The method according to claim 2, characterized in that The performing object detection on the target image and determining the main area of ​​the target image according to the detected main object includes: A minimum bounding box of the detected main object set is determined, and an image area within the minimum bounding box is used as the main body area.

4. The method according to claim 2, characterized in that: The step of cropping the target image multiple times according to the cropping frame size to obtain the multiple cropped images includes: Determining a plurality of cropping positions of the cropping frame according to the center position of the main body area and the size of the cropping frame, wherein the cropping frame has a standard position point at each cropping position that coincides with the center position point of the main body area; The target image is cropped at each cropping position according to the cropping frame size to obtain a plurality of cropped images, wherein the cropping frame size enables the cropping frame to fully cover the subject area during each cropping.

5. The method according to claim 3, characterized in that: The performing object detection on the target image and determining the main area of ​​the target image according to the detected main object includes: When it is detected that the target image contains a face, and the area of ​​the face detection frame accounts for a larger proportion of the target image than a preset threshold, determining that the face in the face detection frame is the main object, and the main body area includes the area where the main object is located; When it is detected that the target image does not contain a face, or that the target image contains a face and the area of ​​the face detection frame accounts for less than the preset threshold in the target image, target detection is performed on the target object, and it is determined that the object in the target detection frame belongs to the main object, and the main area includes the area where the main object is located.

6. The method according to claim 4, characterized in that The cropping frame is a square cropping frame, and determining the size of the cropping frame according to the size of the subject area and the size of the target image includes: The side length L of the trimming frame is determined by the following calculation formula; L=min(w,max(2 / 3·w,2c)); Wherein, w represents the minimum side length of the target image, and c represents the minimum side length of the main area.

7. The method according to any one of claims 1 to 4, characterized in that: The acquiring of the target image comprises: Get the image preview request sent by the client; After determining a target cropped image according to the output score and using the target cropped image as a thumbnail of the target image, the method further includes: A message for responding to the image preview request is sent to the client, wherein the message includes the thumbnail.

8. A device for generating thumbnails, characterized in that: The device comprises: An image acquisition module is configured to acquire a target image; a processing module configured to perform object detection on the target image, and to crop the target image according to the detected objects to obtain a plurality of cropped images; a first execution module, configured to input each of the cropped images into a selection model to obtain an output score of the selection model for each of the cropped images, wherein the selection model is trained based on an image sample set, wherein the image sample set includes a plurality of sample pairs, each sample pair includes an image sample and a score corresponding to the image sample, and the score corresponding to the image sample represents a degree of matching between a distribution of a main object in the image sample and a standard distribution; The second execution module is configured to determine a target cropped image according to the output score, and use the target cropped image as a thumbnail of the target image.

9. A camera, characterized in that: The camera comprises the device for generating thumbnails as claimed in claim 8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method for generating thumbnails described in any one of claims 1 to 7 is implemented.

11. A device for generating thumbnails, characterized in that: include: processor; a memory for storing processor-executable instructions; When the processor executes the instructions stored in the memory, the method for generating thumbnails as described in any one of claims 1 to 7 can be implemented.

Images