Method, device, electronic device and storage medium for determining aesthetics

Abstract

The present application discloses a method, device, electronic device and storage medium for determining aesthetics. The method comprises: obtaining an image to be recognized, wherein the image to be recognized contains characters; segmenting the image to be recognized based on the characters in the image to be recognized to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized; obtaining a first standard character image corresponding to the first character image for the first character image; determining the aesthetics of the characters in the first character image based on the first character image and the first standard character image, thereby improving the accuracy of determining the aesthetics of the characters.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for determining aesthetics. Background Art

[0002] At present, when performing text recognition (optical character recognition, OCR) on an image, generally only the text information in the image is obtained, and the aesthetics of the text information is not obtained. However, text information such as handwriting generally has the need to judge the aesthetics. However, the aesthetics of text information such as handwriting is generally judged based on human subjective consciousness, or directly based on simple judgment criteria (such as whether it is neat), which makes the accuracy of determining the aesthetics very low. Summary of the invention

[0003] The embodiments of the present application provide a method, device, electronic device and storage medium for determining aesthetics, which improves the accuracy of determining the aesthetics of characters by judging the aesthetics of characters in a character image in combination with a standard character image.

[0004] In a first aspect, an embodiment of the present application provides a method for determining aesthetics, the method comprising:

[0005] Acquire an image to be recognized, wherein the image to be recognized contains characters;

[0006] Based on the characters in the image to be recognized, the image to be recognized is segmented to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized;

[0007] Acquire a first standard character image corresponding to the first character image;

[0008] Based on the first character image and the first standard character image, the aesthetics of the characters in the first character image are determined.

[0009] In a second aspect, an embodiment of the present application provides a device for determining aesthetics, the device comprising: an acquisition unit and a processing unit;

[0010] An acquisition unit, used for acquiring an image to be recognized, wherein the image to be recognized contains characters;

[0011] A processing unit, configured to segment the image to be recognized based on the characters in the image to be recognized to obtain a first character image, wherein the first character image includes part or all of the characters in the image to be recognized;

[0012] The processing unit is further used to obtain a first standard character image corresponding to the first character image;

[0013] The processing unit is further used to determine the aesthetics of the characters in the first character image based on the first character image and the first standard character image.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the processor being connected to the memory, the memory being used to store a computer program, and the processor being used to execute the computer program stored in the memory, so that the electronic device executes the method of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method of the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and is computer-operable to cause the computer to execute the method of the first aspect.

[0017] Implementing the embodiments of the present application has the following beneficial effects:

[0018] By acquiring an image to be recognized; then segmenting the image to be recognized based on the characters in the image to be recognized to obtain a first character image, wherein the first character image includes a character; then acquiring a first standard character image corresponding to the first character image; based on the first character image and the first standard character image, determining the aesthetics of the characters in the first character image, thereby improving the accuracy of determining the aesthetics of the characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of a scene for determining aesthetics provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of another scene for determining aesthetics provided in an embodiment of the present application;

[0022] Figure 3 A flow chart of a method for determining aesthetics provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of segmenting an image to be recognized based on characters in the image to be recognized to obtain a first character image provided by an embodiment of the present application;

[0024] Figure 5 A schematic diagram of a model training for determining aesthetics provided in an embodiment of the present application;

[0025] Figure 6 A block diagram of the functional units of a device for determining aesthetics provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", "third" and "fourth" etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] See also Figure 1 , Figure 1 A schematic diagram of a scene for determining aesthetics provided in an embodiment of the present application.

[0031] The scenario includes a terminal device, which may be a computer, tablet computer, smart phone or other device; a target application may be installed on the terminal device, and a virtual function button is set in the target application. When a user touches the virtual function button, the beauty of the characters in the image may be determined according to the beauty determination method of the present application, that is, the function implemented by the virtual function button corresponds to the beauty determination method of the present application. It should be noted that the characters involved in the present application may include handwritten text, numbers, symbols, letters, etc., and the present application mainly uses handwritten text as an example for explanation.

[0032] For example, when a user touches a virtual function button on a terminal device, that is, Figure 1 When the “determine” in the above text is selected, the beauty of the handwritten font in the image can be determined according to the beauty determination method of the present application, specifically:

[0033] The terminal device first obtains the image to be recognized. It should be noted that the image to be recognized can be a frame image in the video or a single frame image, and the image to be recognized can be a single frame image obtained from other devices or a frame image in the video, or a single frame image stored locally in the terminal device or a frame image in the video; then, based on the characters in the image to be recognized, the image to be recognized is segmented to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized, and then a first standard character image corresponding to the first character image is obtained; based on the first character image and the first standard character image, the beauty of the characters in the first character image is obtained, and then the overall beauty of the handwritten font in the image to be recognized can be obtained, that is, Figure 1 "Aesthetics: A" in the video.

[0034] After determining the overall aesthetics of the handwritten text in the image to be recognized, the terminal device may display the overall aesthetics of the handwritten text in the image to be recognized, such as Figure 1 "Aesthetics: A" is shown; of course, the terminal device can also determine the overall aesthetics of the handwritten text in the image to be identified as a feature label of the image to be identified, and can perform a series of post-applications based on the feature label, such as image recommendation, search, etc.

[0035] In addition, the aesthetics determination method of the present application can also be applied in OCR text recognition scenarios, such as Figure 2 As shown, Figure 2 A schematic diagram of another scene for determining aesthetics provided in an embodiment of the present application, such as Figure 2 As shown, when the user touches the virtual function button on the terminal device, that is, Figure 2During the "recognition" shown, the terminal device performs OCR text recognition on the image to be recognized and determines the beauty of the handwritten font in the image to be recognized (i.e., "the unique poem composed by a primary school student") according to the beauty determination method of the present application, and then outputs the font obtained by OCR text recognition (i.e., "the unique poem composed by a primary school student") while also outputting the beauty of the handwritten font as "A".

[0036] It can be seen that in the embodiment of the present application, an image to be recognized is obtained; then, based on the characters in the image to be recognized, the image to be recognized is segmented to obtain a first character image, wherein the first character image includes a character; then, a first standard character image corresponding to the first character image is obtained; based on the first character image and the first standard character image, the aesthetics of the characters in the first character image is determined, thereby improving the accuracy of determining the aesthetics of the characters.

[0037] See also Figure 3 , Figure 3 A flowchart of a method for determining aesthetics provided in an embodiment of the present application is provided, and the method includes but is not limited to steps 301-304:

[0038] 301: Obtain an image to be recognized.

[0039] In an embodiment of the present application, the image to be recognized contains characters, and the number of characters can be one or more, which is not limited by the present application; the image to be recognized can be a frame image in a video, or a single frame image, and the image to be recognized can be a single frame image obtained from other devices or a frame image in a video, or a single frame image locally stored in the terminal device or a frame image in a video, which is not limited by the present application.

[0040] 302: Segment the image to be recognized based on the characters in the image to be recognized to obtain a first character image.

[0041] In an embodiment of the present application, the first character image includes part or all of the characters in the image to be recognized, that is, the characters in the image to be recognized may include one or more. When there is only one character in the image to be recognized, a character image is obtained after the image to be recognized is segmented. At this time, the first character image is the one character image, and the characters in the first character image are all the characters in the image to be recognized; when there are multiple characters in the image to be recognized, one or more character images can be obtained by segmenting the image to be recognized. If there is one character image, the first character image is the one character image, which means that the first character image includes all the characters in the image to be recognized. If there are multiple character images, all the characters contained in the multiple character images are all the characters in the image to be recognized. At this time, the first character image is any one of the multiple character images and includes part of the characters in the image to be recognized, and the present application does not make any limitation.

[0042] In an optional embodiment, step 302 may specifically include: performing character detection on the image to be recognized to obtain a character area; segmenting the image to be recognized based on the pixel coordinates of the character area in the image to be recognized to obtain a character area image; performing character recognition on the character area image to obtain the characters in the image to be recognized; segmenting the character area image based on the characters in the image to be recognized to obtain a first character image, wherein the character area contains all the characters in the image to be recognized, and then the number of characters in the first character image obtained after segmenting the character area image based on the characters in the image to be recognized can be one or more.

[0043] For example, see Figure 4 , Figure 4 A schematic diagram of an embodiment of the present application provides a method for segmenting an image to be recognized based on characters in the image to be recognized to obtain a first character image. Figure 4 As shown in the figure, there are multiple characters in the image to be recognized, namely, "a poem written by a primary school student". Then, after character detection is performed on the image to be recognized to obtain the character area, the image to be recognized is segmented based on the pixel coordinates of the character area in the image to be recognized to obtain Figure 4 The character area image shown (because the characters "primary school students' poems" in the image to be recognized occupy two lines in total, Figure 4The corresponding character region image shown has two regions. One character region image includes "written by primary school students", and the other includes "quatrain". That is to say, after obtaining the character region, the number of lines occupied by the characters in the character region and the pixel coordinates of the characters in each line in the image to be recognized can be detected, and the image to be recognized can be segmented to obtain the character region image corresponding to each line); then character recognition is performed on these two character region images to obtain "written by primary school students" and "quatrain" respectively, that is, the characters in the image to be recognized are obtained; then based on the characters in the image to be recognized, that is, "written by primary school students" and "quatrain", the character region image is segmented, and multiple character images can be obtained, that is Figure 4 The images shown respectively including the characters "small", "school", "student", "compiled", "of", "quatrain", "poem". Then the first character image can be any one of the multiple character images. At this time, the number of characters in the first character image is one, and this application does not make a limitation;

[0044] Of course, in an alternative embodiment, based on Figure 4 The embodiment shown, this application segments the character region image based on the characters in the image to be recognized, that is, "written by primary school students" and "quatrain", and the obtained character images can be multiple character images respectively including the characters "primary school students", "compiled of", "quatrain". At this time, the first character image is any one of the multiple character images and the number of characters is multiple. If the first character image is a character image containing "primary school students", then this application segments the character image containing "primary school students" to obtain sub-character images respectively containing "small", "school", "student", and then determines the aesthetics corresponding to each sub-character image, and then averages the aesthetics corresponding to each sub-character image to obtain the overall aesthetics of the first character image, that is, the image containing the characters "primary school students", and this application does not make a limitation.

[0045] 303: Obtain a first standard character image corresponding to the first character image.

[0046] In the embodiment of this application, a standard character image library will be constructed in advance, and then the first standard character image corresponding to the first character image can be obtained from the standard character image library. For example, when the number of characters in the first character image is one, the number of corresponding first standard character images is one. When the number of characters in the first character image is multiple, the number of corresponding first standard character images is multiple and they correspond to the characters in the first character image respectively.

[0047] 304: Based on the first character image and the first standard character image, determine the aesthetics of the characters in the first character image.

[0048] In the embodiments of the present application, it should be noted that each first character image and the first standard character image can also be binarized to obtain a corresponding binarized image, and then the aesthetics of the character can be determined based on the binarized first character image and the first standard character image. In an optional embodiment, if there are multiple characters in the image to be recognized, and if there are also multiple character images obtained by segmentation, for any one of the multiple character images, i.e., the first character image, if there are multiple characters in the first character image, the aesthetics of each character in the first character image can be determined based on the aesthetics determination method provided in the embodiments of the present application, and then the overall aesthetics of the first character image can be obtained based on the average of the aesthetics of each character, and then the overall aesthetics corresponding to each character image can be averaged to obtain the total aesthetics corresponding to the image to be recognized, and the total aesthetics can be determined as the overall aesthetics of the characters in the image to be recognized.

[0049] In an optional embodiment, step 304 may specifically include: extracting features from the first character image to obtain a first feature; then extracting features from the first standard character image to obtain a second feature; then fusing the first feature with the first feature to obtain a third feature corresponding to the first character image; and then determining the aesthetics corresponding to the first character image based on the third feature.

[0050] Exemplarily, feature extraction of the first character image and the first standard character image to obtain the first feature and the second feature respectively can be performed through a trained feature extraction network. Specifically, the feature extraction network includes a convolution layer, a normalization layer, an activation function layer, a deconvolution layer, a second convolution layer, a second deconvolution layer and a pooling layer. For example, the first character image is convolved based on the convolution layer to obtain a vector Q, and then the vector Q is normalized based on the normalization layer to obtain a vector Q1, and then the vector Q1 is activated based on the activation function layer to obtain a vector Q3, and then the vector Q3 is deconvolved based on the deconvolution layer to obtain a vector Q4; then convolution, normalization, activation and deconvolution are performed in sequence based on the vector Q4 to obtain a vector Q5, and then based on the pooling layer The vector Q5 is downsampled to obtain a first feature corresponding to the first character image. Similarly, the principle of extracting the second feature corresponding to the first standard character image is similar to the first feature and will not be repeated. Then, the first feature and the second feature can be fused based on the fully connected layer to obtain a third feature. Then, based on the third feature and the trained aesthetics determination model, the aesthetics of the characters in the first character image are predicted. That is, the third feature is input into the trained aesthetics determination model, and the aesthetics of the characters in the first character image are output. That is, in step 304, the aesthetics of the characters in the first character image are determined based on the first character image and the first standard character image. This is performed by the aesthetics determination model, and the training of the aesthetics determination model includes the following steps:

[0051] First, the image to be trained is obtained, wherein the relevant interpretation of the image to be trained can refer to the interpretation of the image to be recognized; then, based on the characters in the image to be trained, the image to be trained is segmented to determine the second character image, wherein the second character image includes one character, and it should be noted that the principle of determining the second character image is similar to the principle of determining the first character image, which will not be repeated; then, a second standard character image corresponding to the second character image is obtained, wherein the principle of obtaining the corresponding second standard character image is similar to the principle of obtaining the corresponding first standard character image; finally, based on the second character image and the second standard character image, the aesthetics determination model is trained, specifically:

[0052] Based on the feature extraction network, feature extraction is performed on the second character image to obtain a fourth feature; and based on the feature extraction network, feature extraction is performed on the second standard character image to obtain a fifth feature; the fourth feature and the fifth feature are merged to obtain a sixth feature corresponding to the second character image, wherein the principle of obtaining the sixth feature is similar to the principle of obtaining the third feature, which will not be repeated; then, based on the sixth feature, the first loss corresponding to the second character image is determined. Specifically, based on the sixth feature, the classification result is determined, for example, the level or score of the aesthetics is pre-set, such as distinguishing by level, including six levels of "excellent, better, medium, good, poor, and very poor", and then the corresponding probability of the sixth feature under these six levels is predicted, and the one with the largest probability is taken as the classification result, and then based on the classification result, the aesthetics corresponding to the second character image is determined, for example, the aesthetics is "medium", and then based on the aesthetics corresponding to the second character image and the label corresponding to the second character image, the first loss corresponding to the second character image is determined. The present application does not limit the type of the first loss.

[0053] In addition, based on the sixth feature and the second character image, a second loss corresponding to the second character image is determined. Specifically, the sixth feature is upsampled to obtain a deformation field corresponding to the second character image. For example, a feature map with the same size as the second character image is obtained by upsampling the sixth feature, and then a deformation field is calculated based on the feature map and the second character image, wherein the deformation field corresponding to the second character image represents the pixel offset between the character in the second character image and the standard character in the second standard character image, wherein the pixel offset can be the pixel offset between the character in the second character image and the standard character in the second standard character image in the image coordinate system in the x-axis direction and the y-axis direction; then the deformation field corresponding to the second character image and the pixels of the character in the second character image are added to determine the third character image; based on the third character image and the second standard character image, a second loss corresponding to the second character image is determined; finally, based on the first loss and the second loss, the aesthetics determination model is trained, such as weighting the first loss and the second loss to obtain the target loss, and the aesthetics determination model is iteratively trained based on the target loss until the target loss is less than a preset threshold.

[0054] For example, see Figure 5 , Figure 5 This is a schematic diagram of a model training for determining aesthetics provided in an embodiment of the present application. Figure 5 As shown, the second character image and the second standard character image are both binary images. The second character image and the second standard character image are respectively input into the feature extraction network to obtain the fourth feature and the fifth feature respectively. Then, the fourth feature and the fifth feature are fused to obtain the sixth feature. The classification result is determined based on the sixth feature, and the first loss is determined based on the classification result. At the same time, the fused sixth feature is upsampled to obtain the deformation field. Then, the deformation field and the corresponding pixels of the second character image are added to obtain the third character image. Then, the second loss is calculated based on the third character image and the second standard character image, and the target loss is obtained based on the first loss and the second loss. The aesthetics determination model is iteratively trained based on the target loss.

[0055] It should be noted that in the feature extraction network training process in the beauty determination model, a contrastive learning twin network is used. When learning to extract features from images, the distance between similar samples is reduced as much as possible and the distance between dissimilar samples is increased. In addition, in the training process of the beauty determination model, after determining the first loss based on the sixth feature, a contrast-assisted learning method is added, that is, the model learns the change from handwritten characters to corresponding standard characters, that is, the deformation field after upsampling the sixth feature obtains a predicted third character image, and the second loss is determined based on the third character image and the second standard character image, and the target loss is determined based on the first loss and the second loss. The iteration based on the target loss allows the model to gradually and accurately learn the difference between handwritten characters and standard characters, so that the model pays more attention to the difference between handwritten characters and standard characters when determining the beauty of handwritten characters, thereby improving the accuracy of determining the beauty of handwritten characters.

[0056] See also Figure 6 , Figure 6 This is a block diagram of the functional units of a device for determining aesthetics provided in an embodiment of the present application. The device for determining aesthetics 600 includes: an acquisition unit 601 and a processing unit 602;

[0057] An acquisition unit 601 is used to acquire an image to be recognized, wherein the image to be recognized contains characters;

[0058] A processing unit 602 is used to segment the image to be recognized based on the characters in the image to be recognized to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized;

[0059] The processing unit 602 is further configured to obtain a first standard character image corresponding to the first character image;

[0060] The processing unit 602 is further configured to determine the aesthetics of the characters in the first character image based on the first character image and the first standard character image.

[0061] In one embodiment of the present application, in terms of segmenting the image to be recognized based on the characters in the image to be recognized to obtain the first character image, the processing unit 602 is specifically configured to:

[0062] Perform character detection on the image to be recognized to obtain the character area;

[0063] Based on the pixel coordinates of the character area in the image to be recognized, the image to be recognized is segmented to obtain a character area image;

[0064] Performing character recognition on the character area image to obtain the characters in the image to be recognized;

[0065] Based on the characters in the image to be recognized, the character area image is segmented to obtain a first character image.

[0066] In one embodiment of the present application, in determining the aesthetics of the characters in the first character image based on the first character image and the first standard character image, the processing unit 602 is specifically configured to:

[0067] Extracting features from the first character image to obtain a first feature;

[0068] Extracting features from the first standard character image to obtain a second feature;

[0069] fusing the first feature with the second feature to obtain a third feature corresponding to the first character image;

[0070] Based on the third feature, the aesthetics degree corresponding to the first character image is determined.

[0071] In one embodiment of the present application, based on the first character image and the first standard character image, determining the aesthetics of the characters in the first character image is performed by an aesthetics determination model, and the processing unit 602 is specifically configured to:

[0072] Get the image to be trained;

[0073] Based on the characters in the image to be trained, the image to be trained is split to determine a second character image, wherein the second character image includes one character;

[0074] Acquire a second standard character image corresponding to the second character image;

[0075] The aesthetics determination model is trained based on the second character image and the second standard character image.

[0076] In one embodiment of the present application, in terms of training the aesthetics determination model based on the second character image and the second standard character image, the processing unit 602 is specifically configured to:

[0077] Extracting features from the second character image to obtain a fourth feature;

[0078] Perform feature extraction on the second standard character image to obtain a fifth feature;

[0079] The fourth feature and the fifth feature are combined to obtain a sixth feature corresponding to the second character image;

[0080] Based on the sixth feature, determining a first loss corresponding to the second character image;

[0081] determining a second loss corresponding to the second character image based on the sixth feature and the second character image;

[0082] Based on the first loss and the second loss, the beauty determination model is trained.

[0083] In one embodiment of the present application, in determining the first loss corresponding to the second character image based on the sixth feature, the processing unit 602 is specifically configured to:

[0084] Based on the sixth feature, determining a classification result;

[0085] Based on the classification result, determining the aesthetics corresponding to the second character image;

[0086] Based on the aesthetics corresponding to the second character image and the label corresponding to the second character image, a first loss corresponding to the second character image is determined.

[0087] In one embodiment of the present application, in determining the second loss corresponding to the second character image based on the sixth feature and the second character image, the processing unit 602 is specifically configured to:

[0088] Upsampling the sixth feature to obtain a deformation field corresponding to the second character image, wherein the deformation field corresponding to the second character image represents an offset between a character in the second character image and a standard character in the second standard character image;

[0089] Adding the deformation field corresponding to the second character image and the pixels of the character in the second character image to determine a third character image;

[0090] Based on the third character image and the second standard character image, a second loss corresponding to the second character image is determined.

[0091] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 includes a transceiver 701, a processor 702 and a memory 703. They are connected via a bus 704. The memory 703 is used to store computer programs and data, and can transmit the data stored in the memory 703 to the processor 702.

[0092] The processor 702 is used to read the computer program in the memory 703 and perform the following operations:

[0093] Controlling the transceiver 701 to obtain an image to be recognized, wherein the image to be recognized contains characters;

[0094] Based on the characters in the image to be recognized, the image to be recognized is segmented to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized;

[0095] Acquire a first standard character image corresponding to the first character image;

[0096] Based on the first character image and the first standard character image, the aesthetics of the characters in the first character image are determined.

[0097] In one embodiment of the present application, in terms of segmenting the image to be recognized based on the characters in the image to be recognized to obtain the first character image, the processor 702 is specifically configured to perform the following steps:

[0098] Perform character detection on the image to be recognized to obtain the character area;

[0099] Based on the pixel coordinates of the character area in the image to be recognized, the image to be recognized is segmented to obtain a character area image;

[0100] Performing character recognition on the character area image to obtain the characters in the image to be recognized;

[0101] Based on the characters in the image to be recognized, the character area image is segmented to obtain a first character image.

[0102] In one embodiment of the present application, in determining the aesthetics of the characters in the first character image based on the first character image and the first standard character image, the processor 702 is specifically configured to perform the following steps:

[0103] Extracting features from the first character image to obtain a first feature;

[0104] Extracting features from the first standard character image to obtain a second feature;

[0105] fusing the first feature with the second feature to obtain a third feature corresponding to the first character image;

[0106] Based on the third feature, the aesthetics degree corresponding to the first character image is determined.

[0107] In one embodiment of the present application, based on the first character image and the first standard character image, determining the aesthetics of the characters in the first character image is performed by a aesthetics determination model, and the processor 702 is specifically configured to perform the following steps:

[0108] Get the image to be trained;

[0109] Based on the characters in the image to be trained, the image to be trained is split to determine a second character image, wherein the second character image includes one character;

[0110] Acquire a second standard character image corresponding to the second character image;

[0111] The aesthetics determination model is trained based on the second character image and the second standard character image.

[0112] In one embodiment of the present application, in terms of training the aesthetics determination model based on the second character image and the second standard character image, the processor 702 is specifically configured to perform the following steps:

[0113] Extracting features from the second character image to obtain a fourth feature;

[0114] Perform feature extraction on the second standard character image to obtain a fifth feature;

[0115] The fourth feature and the fifth feature are combined to obtain a sixth feature corresponding to the second character image;

[0116] Based on the sixth feature, determining a first loss corresponding to the second character image;

[0117] determining a second loss corresponding to the second character image based on the sixth feature and the second character image;

[0118] Based on the first loss and the second loss, the beauty determination model is trained.

[0119] In one embodiment of the present application, in determining the first loss corresponding to the second character image based on the sixth feature, the processor 702 is specifically configured to perform the following steps:

[0120] Based on the sixth feature, determining a classification result;

[0121] Based on the classification result, determining the aesthetics corresponding to the second character image;

[0122] Based on the aesthetics corresponding to the second character image and the label corresponding to the second character image, a first loss corresponding to the second character image is determined.

[0123] In one embodiment of the present application, in determining the second loss corresponding to the second character image based on the sixth feature and the second character image, the processor 702 is specifically configured to perform the following steps:

[0124] Upsampling the sixth feature to obtain a deformation field corresponding to the second character image, wherein the deformation field corresponding to the second character image represents an offset between a character in the second character image and a standard character in the second standard character image;

[0125] Adding the deformation field corresponding to the second character image and the pixels of the character in the second character image to determine a third character image;

[0126] Based on the third character image and the second standard character image, a second loss corresponding to the second character image is determined.

[0127] Specifically, the transceiver 701 may be Figure 6 The acquisition unit 601 of the apparatus 600 for determining the aesthetics of the embodiment, the processor 702 may be Figure 6 The processing unit 602 of the apparatus 600 for determining aesthetics of an embodiment.

[0128] It should be understood that the electronic devices in this application may include smart phones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices MID (Mobile Internet Devices, MID for short) or wearable devices, etc. The above electronic devices are only examples, not exhaustive, and include but are not limited to the above electronic devices. In practical applications, the above electronic devices may also include: smart vehicle terminals, computer equipment, etc.

[0129] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all of the steps of any one of the methods for determining aesthetics as described in the above method embodiments.

[0130] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any one of the methods for determining aesthetics as recorded in the above method embodiments.

[0131] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0132] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0133] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0134] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0136] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0137] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0138] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for determining aesthetics, characterized in that: The method comprises: Acquire an image to be recognized, wherein the image to be recognized contains characters; Based on the characters in the image to be recognized, segment the image to be recognized to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized; Acquire a first standard character image corresponding to the first character image; Based on the first character image and the first standard character image, the aesthetics of the characters in the first character image is determined, wherein the aesthetics of the characters in the first character image is performed by a beauty determination model, and the training of the beauty determination model includes the following steps: segmenting the training image to determine the second character image; acquiring the second standard character image corresponding to the second character image; performing feature extraction on the second character image to obtain a fourth feature; performing feature extraction on the second standard character image to obtain a fifth feature; fusing the fourth feature with the fifth feature to obtain a sixth feature corresponding to the second character image; determining a first loss corresponding to the second character image based on the sixth feature; upsampling the sixth feature to obtain a deformation field corresponding to the second character image, wherein the deformation field corresponding to the second character image represents the pixel offset between the characters in the second character image and the standard characters in the second standard character image; adding the deformation field corresponding to the second character image and the pixels of the characters in the second character image to determine the third character image; determining a second loss corresponding to the second character image based on the third character image and the second standard character image; and training the beauty determination model based on the first loss and the second loss.

2. The method according to claim 1, characterized in that The step of segmenting the image to be recognized based on the characters in the image to be recognized to obtain a first character image includes: Performing character detection on the image to be recognized to obtain a character area; Segmenting the image to be recognized based on the pixel coordinates of the character area in the image to be recognized to obtain a character area image; Performing character recognition on the character area image to obtain characters in the image to be recognized; Based on the characters in the image to be recognized, the character area image is segmented to obtain the first character image.

3. The method according to claim 1 or 2, characterized in that: The determining, based on the first character image and the first standard character image, the aesthetics of the characters in the first character image includes: Performing feature extraction on the first character image to obtain a first feature; Extracting features from the first standard character image to obtain a second feature; The first feature and the second feature are merged to obtain a third feature corresponding to the first character image; Based on the third feature, the aesthetic degree corresponding to the first character image is determined.

4. The method according to claim 1 or 2, characterized in that: The determining, based on the sixth feature, a first loss corresponding to the second character image comprises: Based on the sixth feature, determining a classification result; Based on the classification result, determining the aesthetics corresponding to the second character image; A first loss corresponding to the second character image is determined based on the aesthetics corresponding to the second character image and the label corresponding to the second character image.

5. A device for determining aesthetics, characterized in that: The device comprises: an acquisition unit and a processing unit; The acquisition unit is used to acquire an image to be recognized, wherein the image to be recognized contains characters; The processing unit is used to segment the image to be recognized based on the characters in the image to be recognized to obtain a first character image, wherein the first character image contains part or all of the characters in the image to be recognized; The processing unit is further used to obtain a first standard character image corresponding to the first character image; The processing unit is further used to determine the aesthetics of the characters in the first character image based on the first character image and the first standard character image, wherein the aesthetics of the characters in the first character image is performed by a beauty determination model, and the training of the beauty determination model includes the following steps: segmenting the training image to determine the second character image; acquiring the second standard character image corresponding to the second character image; performing feature extraction on the second character image to obtain a fourth feature; performing feature extraction on the second standard character image to obtain a fifth feature; fusing the fourth feature with the fifth feature to obtain a sixth feature corresponding to the second character image; determining a first loss corresponding to the second character image based on the sixth feature; upsampling the sixth feature to obtain a deformation field corresponding to the second character image, wherein the deformation field corresponding to the second character image represents the pixel offset between the characters in the second character image and the standard characters in the second standard character image; adding the deformation field corresponding to the second character image and the pixels of the characters in the second character image to determine the third character image; determining a second loss corresponding to the second character image based on the third character image and the second standard character image; and training the beauty determination model based on the first loss and the second loss.

6. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images