Player skin generation method and device, and storage medium

Abstract

The invention provides a player skin generation method and device and a storage medium, and the method comprises the steps: inputting an image material uploaded by a user into a target image encoder, and outputting to obtain an image semantic feature of the image material; inputting the image semantic features into an image recognition model, recognizing visual elements contained in the image semantic features through the image recognition model, predicting the region type to which the recognized visual elements belong from the foreground region and the background region, and outputting a visual element recognition result corresponding to the obtained image material; according to the visual elements belonging to the foreground area and the visual elements belonging to the background area in the visual element identification result, respectively generating foreground skin and background skin corresponding to the preset player; and combining the foreground skin with the background skin to obtain the player skin of the preset player. Therefore, the flexibility of the player skin generation mode is effectively improved, and the personalized customization requirements of different users can be met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a player skin generation method and device and a storage medium. BACKGROUND

[0002] In the existing audio playing platform, a user can purchase a player skin that he likes from a skin store, or select a basic player skin from a plurality of basic player skins provided by the audio playing platform, and obtain a customized player skin by adjusting skin parameters such as color parameters and button combination modes in the basic player skin. However, the types of player skins that a user can purchase are limited, and the player skin customized by the user cannot get rid of the original skin framework of the basic player skin, which leads to that the existing player skin generation method is not flexible enough and cannot meet the individual customization needs of different users. SUMMARY

[0003] Therefore, the present application provides a player skin generation method, device and storage medium, which generates a player skin matched with an image material uploaded by a user, effectively improves the flexibility of the player skin generation method, and is beneficial to meet the individual customization needs of different users.

[0004] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows.

[0005] In a first aspect, the embodiments of the present application provide a player skin generation method, which comprises: obtaining an image material uploaded by a user, and inputting the image material into a pre-trained target image encoder, performing semantic feature extraction on a foreground region and a background region in the image material through the target image encoder, and outputting image semantic features of the image material; inputting the image semantic features into a pre-trained image recognition model, identifying visual elements contained in the image semantic features through the image recognition model, and predicting a region type to which the identified visual elements belong from the foreground region and the background region, and outputting a visual element recognition result corresponding to the image material; wherein the visual element recognition result includes a plurality of visual elements identified in the image material and region types to which the plurality of visual elements belong, respectively; generating a foreground skin corresponding to a preset player according to the visual elements belonging to the foreground region in the visual element recognition result; generating a background skin corresponding to the preset player according to the visual elements belonging to the background region in the visual element recognition result; combine the foreground skin and the background skin to obtain a player skin corresponding to the preset player.

[0006] In a second aspect, an embodiment of the present application provides a player skin generation apparatus, the player skin generation apparatus comprising: an image input module configured to acquire an image material uploaded by a user and input the image material into a target image encoder pre-trained, extract semantic features of a foreground region and a background region in the image material through the target image encoder, and output image semantic features of the image material; an image analysis module configured to input the image semantic features into an image recognition model pre-trained, recognize visual elements contained in the image semantic features through the image recognition model, and predict a region type to which the recognized visual elements belong from the foreground region and the background region, and output a visual element recognition result corresponding to the image material; wherein the visual element recognition result comprises a plurality of visual elements recognized in the image material and region types to which the plurality of visual elements respectively belong; a first generation module configured to generate a foreground skin corresponding to a preset player according to visual elements belonging to the foreground region in the visual element recognition result; a second generation module configured to generate a background skin corresponding to the preset player according to visual elements belonging to the background region in the visual element recognition result; a combination module configured to combine the foreground skin and the background skin to obtain a player skin corresponding to the preset player.

[0007] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements steps of the player skin generation method when executing the computer program.

[0008] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement steps of the player skin generation method.

[0009] The technical scheme provided by the embodiments of the present application can have the following beneficial effects: The embodiment of the present application provides a player skin generation method, device and storage medium, image material uploaded by a user is acquired, and the image material is input into a target image encoder which is pre-trained, semantic feature extraction is performed on a foreground region and a background region in the image material through the target image encoder, and image semantic features of the image material are output; the image semantic features are input into an image recognition model which is pre-trained, a visual element contained in the image semantic features is recognized through the image recognition model, and a region type to which the recognized visual element belongs is predicted from the foreground region and the background region, and a visual element recognition result corresponding to the image material is output; a foreground skin corresponding to a preset player is generated according to the visual element in the visual element recognition result that belongs to the foreground region; a background skin corresponding to the preset player is generated according to the visual element in the visual element recognition result that belongs to the background region; and the foreground skin and the background skin are combined to obtain a player skin corresponding to the preset player. In this way, the player skin matching the image material uploaded by the user is generated, the flexibility of the player skin generation mode is effectively improved, and the personalized customization demand of different users can be met. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0011] Figure 1 A flowchart of a player skin generation method provided by the embodiment of the present application is shown; Figure 2a A schematic diagram of a foreground image material provided by the embodiment of the present application is shown; Figure 2b A schematic diagram of a foreground skin provided by the embodiment of the present application is shown; Figure 3a A schematic diagram of a background image material provided by the embodiment of the present application is shown; Figure 3b A schematic diagram of a background skin provided by the embodiment of the present application is shown; Figure 4 A schematic diagram of a player skin provided by the embodiment of the present application is shown; Figure 5 A structural schematic diagram of a player skin generation device provided by the embodiment of the present application is shown; Figure 6 A structural schematic diagram of an electronic device 600 provided by the embodiment of the present application is shown. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0013] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0015] In existing audio playback platforms, users can purchase their favorite player skins from a skin store; alternatively, they can choose a basic player skin from several provided by the platform and customize it by adjusting parameters such as color and button combinations. However, the variety of player skins available for purchase is limited, and user-customized skins cannot escape the original framework of the basic player skin. This results in an inflexible approach to player skin generation, failing to meet the personalized customization needs of diverse users.

[0016] Based on this, embodiments of this application provide a method, device, and storage medium for generating player skins. By generating player skins that match image materials uploaded by users, the flexibility of player skin generation is effectively improved, which is beneficial to meeting the personalized customization needs of different users.

[0017] In an embodiment of the present application, the method for generating a player skin can be applied to an audio playing application, which can run on a terminal device or a server. The terminal device can be a local terminal device. When the audio playing application runs on the server, the method for generating a player skin can be implemented and executed based on a cloud interaction system, which includes a server and a client device (i.e., a terminal device).

[0018] To make the embodiments of the present application more comprehensible, a method for generating a player skin, a device, and a storage medium provided by the embodiments of the present application are described in detail below.

[0019] Referring to FIG. 1, Figure 1 Figure 1 FIG. 1 shows a flowchart of a method for generating a player skin provided by an embodiment of the present application. The method includes steps S101-S105. Specifically, S101, an image material uploaded by a user is obtained and input into a target image encoder that has been pre-trained. Semantic feature extraction is performed on a foreground region and a background region in the image material by the target image encoder, and image semantic features of the image material are output.

[0020] S102, the image semantic features are input into an image recognition model that has been pre-trained. Visual elements contained in the image semantic features are recognized by the image recognition model, and a region type to which the recognized visual elements belong is predicted from the foreground region and the background region. A visual element recognition result corresponding to the image material is output.

[0021] S103, a foreground skin corresponding to a preset player is generated according to the visual elements in the visual element recognition result that belong to the foreground region.

[0022] S104, a background skin corresponding to the preset player is generated according to the visual elements in the visual element recognition result that belong to the background region.

[0023] S105, the foreground skin and the background skin are combined to obtain a player skin corresponding to the preset player.

[0024] ​The method for generating the player skin provided in the embodiments of the present application comprises the following steps: obtaining an image material uploaded by a user, inputting the image material into a pre-trained target image encoder, performing semantic feature extraction on a foreground region and a background region in the image material by the target image encoder, and outputting image semantic features of the image material; inputting the image semantic features into a pre-trained image recognition model, identifying visual elements contained in the image semantic features by the image recognition model, and predicting a region type to which the identified visual elements belong from the foreground region and the background region, and outputting a visual element recognition result corresponding to the image material; generating a foreground skin corresponding to a preset player according to the visual elements belonging to the foreground region in the visual element recognition result; generating a background skin corresponding to the preset player according to the visual elements belonging to the background region in the visual element recognition result; and combining the foreground skin and the background skin to obtain a player skin corresponding to the preset player. In this way, the player skin matching the image material uploaded by the user is generated, the flexibility of the player skin generation mode is effectively improved, and the personalized customization requirements of different users can be met.

[0025] The steps in the method for generating the player skin provided in the embodiments of the present application will be described below. In S101, an image material uploaded by a user is obtained, and the image material is input into a pre-trained target image encoder, semantic feature extraction is performed on a foreground region and a background region in the image material by the target image encoder, and image semantic features of the image material are output.

[0026] Here, the user can use an audio playing application on the terminal device, and select and play audio in the audio playing application. On the terminal device side, the user can upload a single static image in any format (such as a jpeg image or a png image) as an image material used for generating a player skin subsequently, and on the server side of the audio playing application, image materials uploaded by different users can be received, the image materials uploaded by different users are stored in a temporary buffer area, a unique image identifier is generated for each image material uploaded by a user, and different image materials uploaded by different users are distinguished in this way.

[0027] In the embodiments of the present application, as an optional embodiment, for each image material uploaded by a user, the server can further perform intelligent auditing on the image material from the dimensions of whether the image material contains risk elements (such as violent elements, prohibited elements, etc.), image quality of the image material, and the like after obtaining the image material uploaded by the user, and take the image material that passes the intelligent auditing as qualified image material that can be used for generating a player skin in the subsequent stage, and return the image material that fails the intelligent auditing to the user node, so that the user can rectify the currently returned image material, thereby facilitating the skin quality of the player skin generated in the subsequent stage.

[0028] Here, for the image material that passes the intelligent auditing, the server can perform in-depth analysis on the image material from the following two aspects: on the one hand, the server needs to perform in-depth analysis on the image content of the image material to determine the visual elements such as color tone, image style, entity object, image scene contained in the image material, so as to serve as the relevant description information of the specific player skin generated in the subsequent stage (equivalent to determining what kind of player skin should be generated in the subsequent stage according to the determined visual elements); on the other hand, the server also needs to identify the foreground region and the background region of the image material, so as to determine the region type to which each visual element belongs (i.e., whether each visual element determined above belongs to the foreground region or the background region), so as to guide the generation of the foreground skin suitable for the foreground component of the player according to the visual elements belonging to the foreground region, and guide the generation of the background skin suitable for the background component of the player according to the visual elements belonging to the background region.

[0029] Specifically, in step S101, the target image encoder can be a pre-trained image encoder capable of completing an image segmentation task (for example, the image encoder in a pre-trained image segmentation model can be obtained as the target image encoder), at this time, based on the fact that the target image encoder belongs to an image encoder capable of completing an image segmentation task, the server inputs the image material that passes the intelligent auditing into the pre-trained target image encoder, and can perform semantic feature extraction on the foreground region and the background region in the image material through the target image encoder to output the image semantic features of the image material (i.e., the image encoding result of the image material), that is, the output image semantic features contain both the foreground semantic information of the image material (semantic information for describing the image content of the foreground region in the image material) and the background semantic information of the image material (semantic information for describing the image content of the background region in the image material).

[0030] It should be noted that in the embodiments of the present application, only the target image encoder needs to be ensured to be an image encoder capable of completing image segmentation tasks, and the specific encoder structure of the target image encoder is not limited in the embodiments of the present application.

[0031] In S102, the image semantic feature is input into a pre-trained image recognition model, the visual elements contained in the image semantic feature are recognized by the image recognition model, and the type of the region to which the recognized visual elements belong is predicted from the foreground region and the background region, and a visual element recognition result corresponding to the image material is output.

[0032] Here, when the visual elements contained in the input image semantic feature are recognized by the image recognition model, the image recognition model performs multi-classification prediction on the input image semantic feature based on a plurality of preset visual element labels, respectively predicts the probability value of each visual element label indicated in the image semantic feature, and determines the specific visual element contained in the image semantic feature according to the predicted probability value.

[0033] It should be noted that the plurality of visual element labels include but are not limited to a plurality of color tone labels (such as blue tone, green tone, red tone, and mixed tone), a plurality of image style labels (such as dream style, luxurious style, simple style, and pastoral style), a plurality of entity object labels (such as cat, dog, butterfly, human, flower, and tree object, and basketball and bookshelf object), and a plurality of image scene labels (such as park scene, campus scene, office scene, and stage scene); the specific label type and the specific number of the plurality of preset visual element labels are not limited in the embodiments of the present application.

[0034] Specifically, when determining the specific visual element contained in the image semantic feature, as an optional embodiment, the image recognition model can determine the visual element indicated by the visual element label with the highest prediction probability value from the plurality of visual element labels of the same type as the corresponding prediction probability value of each visual element label, as the specific visual element contained in the image semantic feature in the plurality of visual elements of the type.

[0035] For example, the plurality of visual element labels of the same type are a plurality of color tone labels belonging to the color tone type, if the image recognition model predicts that the probability value of the blue tone visual element indicated by the blue tone label contained in the image semantic feature is the highest, the image recognition model can determine that the color tone contained in the image semantic feature is blue tone.

[0036] Specifically, as another optional embodiment, when determining the specific visual elements contained in the image semantic feature, the image recognition model can also sort the visual element labels of the same type according to the prediction probability values corresponding to each visual element label in descending order of the prediction probability values, and determine the visual elements indicated by the top n visual element labels as the specific visual elements contained in the image semantic feature in the multiple visual elements of the type; wherein the value of n can be flexibly set according to the actual player skin generation requirements, and the specific value of n is not limited by the embodiments of the present application.

[0037] For example, if the image recognition model predicts that the probability of containing a cat in the image semantic feature is 60%, the probability of containing a dog is 40%, the probability of containing a butterfly is 80%, and the probability of containing a human is 20%, and the value of n is 2, it can be determined that the entity objects contained in the image semantic feature are butterflies and cats.

[0038] It should be noted that, as described in the above step S101, since the above image semantic feature contains both the foreground semantic information of the above image material and the background semantic information of the above image material, after identifying each specific visual element contained in the above image semantic feature (i.e. the above image material) by the image recognition model, the specific region type to which each identified visual element belongs (i.e. classifying and predicting whether each identified visual element belongs to the foreground region or the background region) can be further determined according to the foreground semantic information and the background semantic information contained in the above image semantic feature. At this time, the visual element recognition result obtained includes both each specific visual element identified and the specific region type to which each identified visual element belongs.

[0039] S103, generating a foreground skin corresponding to the preset player according to the visual elements belonging to the foreground region in the visual element recognition result.

[0040] Here, the preset player can be an audio player in an audio playing application, wherein the preset player as a whole can be divided into a foreground component and a background component.

[0041] Specifically, when generating the foreground skin suitable for the above foreground component (i.e. the foreground skin corresponding to the preset player), as an optional embodiment, the above foreground skin can be generated by the method shown in steps a1-a3 as follows: Step a1, determining the first semantic information corresponding to the visual element according to the visual element belonging to the foreground region in the visual element recognition result.

[0042] Here, in generating the above-mentioned foreground skin, an existing text-to-image model (i.e., a model capable of outputting an image matching the input text description information according to the input text description information) can be used; wherein, in order to enable the text-to-image model to output the foreground skin suitable for the above-mentioned foreground component, the text description information required to be input when generating the foreground skin can be divided into: first text description information for guiding the generation of skin content in the foreground skin, and second text description information for guiding the generation of skin style in the foreground skin.

[0043] Specifically, the visual elements in the above-mentioned visual element recognition result belonging to the foreground region are denoted as first visual elements, and it can be known from the foregoing step S102 that each first visual element corresponds to a visual element label, and at this time, the visual element label corresponding to each first visual element can be directly obtained as the above-mentioned first semantic information, wherein the first semantic information is equivalent to the above-mentioned first text description information for guiding the generation of skin content in the foreground skin.

[0044] Step a2, determining second semantic information corresponding to the preset component style of the foreground component according to the preset component style of the foreground component in the preset player.

[0045] Here, the above-mentioned second semantic information is text description information for describing the preset component style of the above-mentioned foreground component, that is, the above-mentioned second semantic information is equivalent to the above-mentioned second text description information for guiding the generation of skin style in the foreground skin.

[0046] It should be noted that the specific information content of the above-mentioned second semantic information can be determined according to the preset component style of the above-mentioned foreground component, that is, when the preset component style of the foreground component in the preset player is unchanged, the above-mentioned second semantic information determined when generating the above-mentioned foreground skin for different image materials uploaded by the user is unchanged, and the above-mentioned second semantic information is irrelevant to the image material uploaded by the user. The specific information content of the above-mentioned second semantic information is not limited in the embodiments of the present application.

[0047] Step a3, inputting the first semantic information and the second semantic information into a text-to-image model, and outputting, by the text-to-image model, a foreground image material matching the first semantic information and the second semantic information as the foreground skin.

[0048] It should be noted that the above-mentioned text-to-image model can be an existing open source text-to-image model, or a text-to-image model trained according to collected training data, and the specific model structure of the above-mentioned text-to-image model is not limited in the embodiments of the present application.

[0049] For example, the first semantic information is "blue color flying butterfly, white background", and the second semantic information is "foreground pendant, high quality". Figure 2a A schematic diagram of a foreground image material is shown in FIG. 1B. Figure 2a As shown in FIG. 1B, the first semantic information "blue color flying butterfly, white background" and the second semantic information "foreground pendant, high quality" are input into the text-to-image model, and the foreground image material matching the first semantic information and the second semantic information can be output by the text-to-image model.

[0050] Specifically, although the foreground image material generated according to the method shown in steps a1-a3 can meet the basic skin requirements of the foreground component in the preset player, there is still room for improvement in terms of aesthetics. Based on this, after step a3 is executed, the foreground image material can be further processed according to the method shown in step a4 below to obtain a foreground skin with better visual display effect: Step a4, identifying and segmenting the salient target in the foreground image material as the foreground skin.

[0051] Here, by performing salient target detection on the foreground image material, the local image region belonging to the salient target can be identified from the foreground image material. By performing automatic matting processing on the identified local image region, the salient target in the foreground image material can be segmented as the foreground skin.

[0052] For example, as shown in FIG. 1C, Figure 2a As shown in FIG. 1C, the foreground image material is segmented into a local image region where the butterfly is located and a local image region where the butterfly is not located. Figure 2b A schematic diagram of a foreground skin is shown in FIG. 1D. Figure 2b As shown in FIG. 1D, by performing salient target detection on the foreground image material shown in FIG. 1C, the local image region belonging to the salient target can be identified from the foreground image material. By performing automatic matting processing on the identified local image region, the butterfly (i.e., the salient target) in the foreground image material can be segmented as the foreground skin shown in FIG. 1D. Figure 2a Figure 2b

[0053] S104, generating a background skin corresponding to the preset player according to the visual elements belonging to the background region in the visual element recognition result.

[0054] Specifically, when generating a background skin suitable for the background component (i.e., a background skin corresponding to the preset player), as an optional embodiment, the background skin can be generated according to the method shown in steps b1-a3 as follows: ​​Step b1, determining third semantic information corresponding to the visual element according to the visual element in the visual element recognition result belonging to the background region.

[0055] Here, when generating the above-mentioned background skin, the existing text-to-image model can also be used; wherein, in order to enable the text-to-image model to output the background skin suitable for the above-mentioned background component, therefore when generating the background skin, the text description information input to the text-to-image model can be divided into: the third type of text description information for guiding the skin content in the generated background skin, and the fourth type of text description information for guiding the skin style in the generated background skin.

[0056] Specifically, the visual element in the visual element recognition result belonging to the background region is denoted as the second visual element, and it can be known from the foregoing step S102 that each second visual element corresponds to a visual element label, and at this time, the visual element label corresponding to each second visual element can be directly obtained as the third semantic information, wherein the third semantic information is equivalent to the third type of text description information for guiding the skin content in the generated background skin.

[0057] Step b2, determining fourth semantic information corresponding to the preset component style of the background component according to the preset component style of the background component in the preset player.

[0058] Here, the fourth semantic information is the text description information for describing the preset component style of the background component, that is, the fourth semantic information is equivalent to the fourth type of text description information for guiding the skin style in the generated background skin.

[0059] It should be noted that the specific information content of the fourth semantic information can be determined according to the preset component style of the background component, that is, when the preset component style of the background component in the preset player is unchanged, the fourth semantic information determined when generating the background skin for different image materials uploaded by the user is unchanged, and the fourth semantic information is irrelevant to the image material uploaded by the user. The specific information content of the fourth semantic information is not limited in the embodiment of the application.

[0060] Step b3, inputting the third semantic information and the fourth semantic information into the text-to-image model, and outputting a background image material matched with the third semantic information and the fourth semantic information as the background skin through the text-to-image model.

[0061] It should be noted that the text-to-image model can be an existing open source text-to-image model, or a text-to-image model trained according to collected training data, and the specific model structure of the text-to-image model is not limited in the embodiment of the application.

[0062] For example, the third semantic information is "dream style blue tone, blue butterfly, background dream blue", and the fourth semantic information is "head portrait frame circular, black fill in the middle of the head portrait frame circular, high quality". Figure 3a A schematic diagram of a background image material is shown, as shown in Figure 3a The third semantic information "dream style blue tone, blue butterfly, background dream blue" and the fourth semantic information "head portrait frame circular, black fill in the middle of the head portrait frame circular, high quality" are input into the text-to-image model, and the background image material matching the third semantic information and the fourth semantic information can be output by the text-to-image model.

[0063] Specifically, considering that the background image material generated according to the method shown in the foregoing steps b1-b3 can meet the basic skin requirements of the background component in the preset player, but its aesthetic level still has room for improvement, based on this, after executing the foregoing step b3, the foregoing background image material can be further processed according to the method shown in the following steps b4-b5 to obtain a background skin with better visual display effect: Step b4, identifying a target image area for displaying an audio cover of the audio to be played from the background image material.

[0064] For example, as shown in Figure 3a The circular area filled with black in the middle of the background image material is the target image area for displaying the audio cover of the audio to be played, wherein when the user switches the audio to be played, the audio cover displayed in the target image area will also be switched accordingly, therefore, when generating the background skin, the foregoing target image area in the background image material can be automatically processed for matting, and the image content displayed in the target image area is removed from the background image material.

[0065] Step b5, removing the image content displayed in the target image area from the background image material to obtain the remaining background image material as the background skin.

[0066] Specifically, by automatically processing the foregoing target image area in the background image material for matting, the image content displayed in the target image area can be removed from the background image material, and at this time, the target image area processed for matting in the remaining background image material can form a transparent area, so that the audio cover of the specific audio played in real time can be displayed in the transparent area in subsequent steps.

[0067] For example, as shown in Figure 3a For example, as shown in Figure 3bA schematic diagram of a background skin is shown, wherein, Figure 3a The circular region filled with black in the background image material is the target image region for displaying the audio cover of the audio to be played, at this time, by performing automatic matting processing on the target image region, the background skin as shown in Figure 3b

[0068] S105, combining the foreground skin and the background skin to obtain the player skin corresponding to the preset player.

[0069] Here, after generating the foreground skin suitable for the foreground component and the background skin suitable for the background component according to the method shown in steps S103-S104, the foreground skin and the background skin can be combined according to the relative position relationship between the foreground component and the background component in the preset player (i.e., applying the foreground skin to the foreground component and applying the background skin to the background component), so as to obtain the player skin suitable for the whole preset player.

[0070] An exemplary description is given with the foreground skin as shown in Figure 2b and the background skin as shown in Figure 3b Figure 4 A schematic diagram of a player skin is shown, as shown in Figure 4 The foreground skin as shown in Figure 2b is applied to the foreground component, and the background skin as shown in Figure 3b is applied to the background component, so as to obtain the player skin suitable for the whole preset player, at this time, the audio cover of the currently played audio can be displayed in the transparent target image region in the background skin as shown in Figure 3b

[0071] The specific implementation process of each step in the embodiments of the present application will be described in detail as follows: For the specific implementation process of step S101, as known from the foregoing content of step S101, after obtaining the image material uploaded by the user, the server side of the audio playing application can perform intelligent auditing on the image material uploaded by the user, wherein, as an optional embodiment, the presence or absence of risk elements (i.e., whether there is a security risk) in the image material can be audited intelligently according to the method shown in steps c1-c3, specifically: Step c1, performing multi-modal detection on the presence or absence of security risks in the image material from the detection dimensions respectively indicated by a plurality of different modalities, to obtain the security risk detection result of the image material.

[0072] ​​​Here, the plurality of different modalities include, but are not limited to, a first modality to which the image data belongs, and a second modality to which the text data belongs. The specific modality type and the specific number of modalities are not limited in the embodiments of the present application.

[0073] Specifically, taking the first modality and the second modality as examples of the plurality of different modalities, the image detection model can be used to perform image recognition on whether there is a pixel-level risk element (such as a violent element, a prohibited element, etc.) in the image material from the image detection dimension indicated by the first modality, to obtain a first safety risk detection result at the pixel level. The image-text joint understanding model can be used to detect whether there is a semantic-level risk element in the image material from the semantic detection dimension indicated by the second modality (that is, to perform semantic analysis on the image content of the image material to detect whether there is risk image content matching the semantic and risk elements in the image material), to obtain a second safety risk detection result at the semantic level. Thus, the first safety risk detection result and the second safety risk detection result are used as the safety risk detection result of the image material.

[0074] Step c2, if it is determined according to the safety risk detection result that the image material has a safety risk, displaying first prompt information on the terminal device of the user.

[0075] Here, the first prompt information is used to prompt the user to replace the currently uploaded image material. That is, when a risk element is detected from the image material (that is, it is determined according to the safety risk detection result that the image material has a safety risk), the first prompt information can be displayed on the terminal device of the user to prompt the user that the currently uploaded image material has a risk element, and recommend the user to replace the currently uploaded image material.

[0076] Step c3, if it is determined according to the safety risk detection result that the image material does not have a safety risk, displaying second prompt information on the terminal device of the user.

[0077] Here, the second prompt information is used to prompt the user that the currently uploaded image material has passed the image review, that is, when no risk element is detected from the image material (that is, it is determined according to the safety risk detection result that the image material does not have a safety risk), the second prompt information can be displayed on the terminal device of the user to prompt the user that the currently uploaded image material has passed the image review and can be used to generate a player skin.

[0078] In the embodiments of the present application, it is considered that when the image quality of the image material uploaded by the user is low, it may cause the visual display effect of the subsequently generated player skin to be poor (for example, low definition, dark color tone, etc.), at this time, as another optional embodiment, after the image material uploaded by the user passes the image review of the above-mentioned security risk dimension (that is, it is determined according to the above-mentioned security risk detection result that the image material does not have security risks), the image quality of the image material can also be reviewed according to the method shown in steps d1-d3, and the specific steps are as follows: Step d1, input the image material into a pre-trained image quality evaluation model, and evaluate the image quality of the image material from multiple preset dimensions by the image quality evaluation model, and output the image quality score of the image material.

[0079] Here, the above-mentioned image quality evaluation model can be a no-reference image quality evaluation model; wherein the above-mentioned multiple preset dimensions include but are not limited to: definition, color balance, composition integrity, etc. The specific dimension types and specific dimension numbers of the above-mentioned multiple preset dimensions are not limited in the embodiments of the present application.

[0080] Specifically, taking the above-mentioned multiple preset dimensions including definition, color balance, and composition integrity as an example, input the image material into a pre-trained image quality evaluation model, and the image quality evaluation model can quantitatively score the input image material from the above-mentioned three preset dimensions, and output the sum of the scores of the image material in the above-mentioned three preset dimensions as the image quality score of the image material.

[0081] It should be noted that in actual application, dedicated preset dimensions (for example, whether the main body area is located in the center of the image, whether there is a large area of solid color area shielding the turntable, etc.) can be flexibly added according to the specific player skin generation requirements of the preset player, so as to flexibly adjust the image quality review standard of the image material.

[0082] Step d2, if the image quality score is lower than the preset review score, display the first prompt information on the terminal device of the user.

[0083] Here, if the above-mentioned image quality score is lower than the preset review score, it indicates that the image quality of the image material currently uploaded by the user is low and insufficient to generate a player skin with a visual display effect meeting the requirements, at this time, the above-mentioned first prompt information can be displayed on the terminal device of the user to prompt the user that the image quality of the image material currently uploaded by the user is low, and recommend the user to replace the image material currently uploaded.

[0084] Step d3, if the image quality score is higher than or equal to the preset review score, displaying the second prompt information on the terminal device of the user.

[0085] Here, if the image quality score is higher than or equal to the preset review score, it means that the image quality of the image material uploaded by the user is high enough to generate a player skin that meets the demand of visual display effect, and at this time, the second prompt information can be displayed on the terminal device of the user to prompt the user that the image material uploaded by the user has passed the image review and can be used to generate a player skin.

[0086] For the target image encoder and the image recognition model in steps S101-S102, in an embodiment of the present application, as an optional embodiment, the image encoder and the image recognition model in the image segmentation model can be jointly trained by the joint training method shown in steps e1-e6 to obtain the trained target image encoder and image recognition model, and specifically: Step e1, a plurality of sample images are collected, and the visual elements contained in the plurality of sample images and the region types to which the visual elements belong are marked respectively to obtain image marking results corresponding to the plurality of sample images respectively.

[0087] Here, when marking the visual elements contained in the sample images, the visual elements that can be marked can refer to the visual element labels in the foregoing step S102; wherein the visual elements that can be marked include but are not limited to: image tone, image style, entity object appearing in the sample image, image scene, etc.

[0088] Specifically, for each visual element contained in the sample image, the region type to which the visual element belongs can also be marked according to whether the visual element belongs to the foreground region of the sample image or the background image region of the sample image; for example, taking the classification label of belonging to the foreground region as 1 and the classification label of belonging to the background region as 0 as an example, if a cat is contained in the background region of the sample image, the region type to which the visual element of the cat belongs can be marked as 0, and if a butterfly is contained in the foreground region of the sample image, the region type to which the visual element of the butterfly belongs can be marked as 1.

[0089] Step e2, inputting the sample image into the image encoder of the image segmentation model, performing semantic feature extraction on the foreground region and the background region in the sample image by the image encoder, and outputting the image semantic features of the sample image.

[0090] It should be noted that the image segmentation model described above can be a SAM2 segmentation model, or can be an image segmentation model of other structures. The specific model structure of the image segmentation model described above is not limited in the embodiments of the present application.

[0091] Specifically, the specific implementation of step e2 can refer to the specific implementation of the foregoing step S101, and repeated parts will not be described here.

[0092] Step e3, input the image semantic features of the sample image into the image decoder of the image segmentation model, perform semantic segmentation on the foreground region and the background region in the sample image through the image decoder, and output the semantic segmentation prediction result of the sample image.

[0093] Here, in the embodiments of the present application, the sample image can be predicted by two parallel branches constituted by the image decoder and the untrained image recognition model (i.e. the original image recognition model in the subsequent step e4) after the image encoder described above (at this time, the input of the two parallel branches is the image semantic features of the sample image output by the image decoder); wherein the auxiliary branch where the image decoder is located is used for segmentation prediction of the foreground region in the sample image (which is equivalent to predicting the foreground region and the background region in the sample image), and the main branch where the original image recognition model is located is used for multi-classification prediction of the visual elements contained in the sample image and the region type to which the visual elements belong. At this time, the segmentation prediction result (i.e. the semantic segmentation prediction result) obtained in the auxiliary branch can be used to assist in determining the region type to which the recognized visual elements belong when the multi-classification prediction is performed in the main branch, so that in the model application stage, the auxiliary branch can be removed, and only the main branch constituted by the trained image decoder and the image recognition model can be retained.

[0094] Step e4, input the image semantic features of the sample image into the original image recognition model, identify the visual elements contained in the input image semantic features through the original image recognition model, and predict the region type to which the recognized visual elements belong from the foreground region and the background region, and output the visual element recognition result corresponding to the sample image.

[0095] Here, the original image recognition model can be a CNN model, or can be an image recognition model of other structures. The specific model structure of the original image recognition model is not limited in the embodiments of the present application.

[0096] Specifically, the specific implementation of step e4 can refer to the specific implementation of the foregoing step S102, and repeated parts will not be described here.

[0097] Step e5, jointly training the image segmentation model and the original image recognition model according to a first prediction loss between the visual element recognition result corresponding to the sample image and the image labeling result of the sample image, and a second prediction loss between the semantic segmentation prediction result of the sample image and the image labeling result of the sample image, until the image segmentation model and the original image recognition model reach convergence.

[0098] Specifically, during the joint training, the model training loss corresponding to the main branch part is the first prediction loss, and the model training loss corresponding to the auxiliary branch part is the second prediction loss (equivalent to the prediction loss between the foreground region recognized in the semantic segmentation result and the foreground region in the image labeling result); wherein, during the joint training, a higher weight coefficient can be configured for the model training loss (i.e. the first prediction loss) corresponding to the main branch part, and a lower weight coefficient can be configured for the model training loss (i.e. the second prediction loss) corresponding to the auxiliary branch part, so as to calculate the overall model training loss of the joint training by weighted summation, and by continuously adjusting the model parameters of the image segmentation model and the original image recognition model, until the overall model training loss reaches the minimum, i.e. the image segmentation model and the original image recognition model reach convergence.

[0099] Step e6, obtaining the image encoder in the image segmentation model reaching the convergence as the trained target image encoder, and obtaining the original image recognition model reaching the convergence as the trained image recognition model.

[0100] Here, after the image segmentation model and the original image recognition model reach convergence, the main branch part corresponding to the trained image encoder (i.e. the target image encoder) and the trained original image recognition model (i.e. the image recognition model) can be retained as the model part actually used in the model application stage.

[0101] In the embodiment of the present application, after applying the generated player skin to the preset player, as an optional embodiment, the rotation speed of the audio cover displayed by the background skin part, the swing amplitude of the foreground component and other dynamic parameters can also be adjusted according to the audio features of the target audio played in real time in the preset player, so as to realize the technical effect that the visual display effect of the player can change synchronously with the rhythm change of the played audio, and enable the user to experience different visual feelings when listening to different songs.

[0102] Specifically, in the embodiment of the present application, the rotation speed of the audio cover displayed by the background skin part can be dynamically adjusted according to the following steps f1-f3: Step f1, in response to the application of the player skin on the preset player, configure the skin of the background component in the preset player as the background skin, and configure the skin of the foreground component in the preset player as the foreground skin.

[0103] Here, after the generation of the player skin, the user can select to apply the currently generated player skin in a preset player (such as a vinyl player in the audio playing application) in the audio playing application, at this time, the terminal device can configure the skin of the background component in the preset player as the background skin in the player skin, and configure the skin of the foreground component in the preset player as the foreground skin in the player skin in response to the application of the player skin on the preset player.

[0104] Step f2, in response to the playing of the target audio through the preset player, display the audio cover of the target audio in the blank area of the background skin.

[0105] Here, as shown in Figure 4 , when playing the target audio (i.e. the audio currently played by the preset player) using the preset player, the audio cover of the target audio can be displayed in the transparent area (i.e. the blank area) in the background skin as shown in Figure 3b .

[0106] Step f3, according to the rhythm transformation feature of the target audio, control the audio cover to rotate at a rotation speed matching the rhythm transformation feature.

[0107] Here, the rhythm transformation feature can be the BPM of the target audio; as an optional embodiment, the audio spectrum of the target audio can be extracted based on the short-time Fourier transform, and the beat period BPM (i.e. the rhythm transformation feature) of the target audio can be calculated in the following formula 1 by using the librosa.beat_track algorithm, specifically: Formula 1; Wherein, t represents the average beat interval of the target audio, and the unit is second.

[0108] Specifically, the matching relationship between the rotation speed of the audio cover and the BPM (i.e. the rhythm transformation feature) of the target audio is that the higher the BPM of the target audio, the faster the rotation speed of the audio cover; as an optional embodiment, the rotation speed rpm matching the BPM of the target audio can be calculated in the following formula 2: Formula 2; Wherein, is the basic rotation speed corresponding to 60 BPM.

[0109] It should be noted that when the rotation speed rpm matched with the BPM of the target audio is calculated according to the calculation method shown in the above formula 2, when the BPM of the target audio is increased to 180 BPM, the rotation speed of the audio cover matched with the target audio is also increased to 49.95 rpm, at this time, in order to prevent the rotation speed from exceeding the visually acceptable expectation, as an optional embodiment, an upper limit value of the rotation speed rpm can also be set, for example, the upper limit of the rotation speed rpm can be set to 45 rpm, so that the rotation speed of the audio cover matched with the target audio in the BPM range (60-180) can be mapped into the rotation speed range (33.3-45) through the prior transformation.

[0110] Specifically, in the embodiment of the present application, the swing amplitude of the foreground component can be dynamically adjusted according to the method shown in the following steps g1-g2: Step g1, in response to the application of the player skin on the preset player, the skin of the background component in the preset player is configured as the background skin, and the skin of the foreground component in the preset player is configured as the foreground skin.

[0111] Here, the specific implementation of step g1 is the same as that of the foregoing step f1, and the repeated parts will not be described here.

[0112] Step g2, in response to playing the target audio through the preset player, according to the rhythm intensity feature of the target audio, the foreground component is controlled to swing according to the swing amplitude matched with the rhythm intensity feature.

[0113] Here, the STE (Short-Time Energy, audio short-time energy) and ZCR (Zero-Crossing Rate, zero-crossing rate) of the audio signal in the target audio can be determined from the mel spectrum of the target audio, at this time, the rhythm intensity feature with a value range of 0-1 and capable of representing the average rhythm intensity of the target audio can be calculated according to the normalization calculation method shown in the following formula 3 , specifically: Formula 3; Wherein, STE represents the audio short-time energy of the audio signal in the observation interval; ZCR represents the zero-crossing rate of the audio signal in the observation interval; represents the maximum value of the audio short-time energy of the audio signal in the observation interval; represents the maximum value of the zero-crossing rate of the audio signal in the observation interval.

[0114] Specifically, as an optional embodiment, to keep the swing range of the foreground component after dynamic adjustment within the visual acceptable range, the swing range of the foreground component can also be limited within the visual acceptable range corresponding to 10-25 in the following formula 4: swing Formula 4; wherein, represents the average rhythm strength of the target audio, and the value range is between 0-1 (which can be calculated by referring to the aforementioned formula 3).

[0115] Based on the above-mentioned player skin generation method provided by the embodiments of the present application, the image material uploaded by the user is obtained, and the image material is input into the pre-trained target image encoder. The semantic features of the foreground region and the background region in the image material are extracted by the target image encoder, and the image semantic features of the image material are output. The image semantic features are input into the pre-trained image recognition model, the visual elements contained in the image semantic features are recognized by the image recognition model, and the region type to which the recognized visual elements belong is predicted from the foreground region and the background region. The visual element recognition result corresponding to the image material is output. According to the visual elements in the visual element recognition result belonging to the foreground region, the foreground skin corresponding to the preset player is generated. According to the visual elements in the visual element recognition result belonging to the background region, the background skin corresponding to the preset player is generated. The foreground skin and the background skin are combined to obtain the player skin corresponding to the preset player. In this way, the player skin matching the image material uploaded by the user is generated, which effectively improves the flexibility of the player skin generation method and is conducive to meeting the individual customization needs of different users.

[0116] Based on the same inventive concept, the present application also provides a player skin generation device corresponding to the above-mentioned player skin generation method. Since the principle of solving problems of the player skin generation device in the embodiments of the present application is similar to that of the above-mentioned player skin generation method in the embodiments of the present application, the implementation of the player skin generation device can be referred to the implementation of the above-mentioned player skin generation method, and the repeated parts will not be described in detail.

[0117] Referring to Figure 5 It is shown that Figure 5 The structure of a player skin generation device provided by the embodiments of the present application is shown, wherein the generation device comprises: An image input module 501 is configured to obtain image material uploaded by a user, and input the image material into a pre-trained target image encoder. The semantic features of the foreground region and the background region in the image material are extracted by the target image encoder, and the image semantic features of the image material are output. The image analysis module 502 is configured to input the image semantic features into a pre-trained image recognition model, identify visual elements contained in the image semantic features through the image recognition model, and predict a region type to which the identified visual elements belong from the foreground region and the background region, and output a visual element recognition result corresponding to the image material, wherein the visual element recognition result includes a plurality of visual elements identified in the image material and region types to which the plurality of visual elements respectively belong. The first generation module 503 is configured to generate a foreground skin corresponding to a preset player according to visual elements belonging to the foreground region in the visual element recognition result. The second generation module 504 is configured to generate a background skin corresponding to the preset player according to visual elements belonging to the background region in the visual element recognition result. The combination module 505 is configured to combine the foreground skin and the background skin to obtain a player skin corresponding to the preset player.

[0118] In an optional implementation, after the image input module 501 obtains the image material uploaded by the user, the image input module 501 is further configured to: perform multi-modal detection on whether the image material has a security risk from detection dimensions respectively indicated by a plurality of different modalities, and obtain a security risk detection result of the image material. If it is determined according to the security risk detection result that the image material has a security risk, display first prompt information on the terminal device of the user; wherein the first prompt information is used to prompt the user to replace the currently uploaded image material. If it is determined according to the security risk detection result that the image material does not have a security risk, display second prompt information on the terminal device of the user; wherein the second prompt information is used to prompt the user that the image material currently uploaded by the user has passed image review.

[0119] In an optional implementation, after the image input module 501 determines that the image material does not have a security risk according to the security risk detection result, the image input module 501 is further configured to: input the image material into a pre-trained image quality evaluation model, perform image quality evaluation on the image material from a plurality of preset dimensions through the image quality evaluation model, and output an image quality score of the image material. If the image quality score is lower than a preset review score, display the first prompt information on the terminal device of the user. If the image quality score is higher than or equal to the preset review score, the second prompt information is displayed on the terminal device of the user.

[0120] In an optional implementation, the generation apparatus further comprises a training module, wherein the training module is configured to jointly train the target image encoder and the image recognition model by using the following method: a plurality of sample images are collected, and visual elements contained in the plurality of sample images and region types to which the visual elements belong are marked respectively to obtain image marking results corresponding to the plurality of sample images respectively; the sample image is input into an image encoder of an image segmentation model, semantic features of the foreground region and the background region in the sample image are extracted by using the image encoder, and image semantic features of the sample image are output; the image semantic features of the sample image are input into an image decoder of the image segmentation model, semantic segmentation of the foreground region and the background region in the sample image is performed by using the image decoder, and a semantic segmentation prediction result of the sample image is output; the image semantic features of the sample image are input into an original image recognition model, visual elements contained in the input image semantic features are recognized by using the original image recognition model, and region types to which the recognized visual elements belong are predicted from the foreground region and the background region, and a visual element recognition result corresponding to the sample image is output; the image segmentation model and the original image recognition model are jointly trained according to a first prediction loss between the visual element recognition result corresponding to the sample image and the image marking result of the sample image and a second prediction loss between the semantic segmentation prediction result of the sample image and the image marking result of the sample image until the image segmentation model and the original image recognition model reach convergence; the image encoder in the image segmentation model reaching convergence is obtained as the target image encoder trained well, and the original image recognition model reaching convergence is obtained as the image recognition model trained well.

[0121] In an optional implementation, when the visual elements belonging to the foreground region in the visual element recognition result are used to generate the foreground skin corresponding to the preset player, the first generation module 503 is configured to: determine first semantic information corresponding to the visual element according to the visual element belonging to the foreground region in the visual element recognition result; determine second semantic information corresponding to the preset component style of the foreground component according to the preset component style of the foreground component in the preset player; input the first semantic information and the second semantic information into a text-to-image model, and output, by the text-to-image model, a foreground image material matching the first semantic information and the second semantic information as the foreground skin.

[0122] In an optional implementation, after the text-to-image model outputs the foreground image material matching the first semantic information and the second semantic information, the first generation module 503 is configured to: identify and segment a salient target in the foreground image material as the foreground skin.

[0123] In an optional implementation, when the foreground skin is generated according to the visual elements belonging to the background region in the visual element identification result, the second generation module 504 is configured to: determine third semantic information corresponding to the visual element according to the visual element belonging to the background region in the visual element identification result; determine fourth semantic information corresponding to a preset component style of a background component in the preset player according to the preset component style of the background component; input the third semantic information and the fourth semantic information into a text-to-image model, and output, by the text-to-image model, a background image material matching the third semantic information and the fourth semantic information as the background skin.

[0124] In an optional implementation, after the text-to-image model outputs the background image material matching the third semantic information and the fourth semantic information, the second generation module 504 is configured to: identify a target image region for displaying an audio cover of a to-be-played audio from the background image material; remove image content displayed in the target image region from the background image material to obtain a remaining background image material as the background skin.

[0125] In an optional implementation, the generation apparatus further includes a first control module, where the first control module is configured to: in response to application of the player skin on the preset player, configure a skin of a background component in the preset player as the background skin, and configure a skin of a foreground component in the preset player as the foreground skin; in response to playing of a target audio by the preset player, display an audio cover of the target audio in a blank region of the background skin; According to a rhythm transformation feature of the target audio, the audio cover is controlled to rotate at a rotation speed matching the rhythm transformation feature.

[0126] In an optional implementation, the generation apparatus further includes a second control module, where the second control module is configured to: In response to application of the player skin on the preset player, configure a skin of a background component in the preset player as the background skin, and configure a skin of a foreground component in the preset player as the foreground skin. In response to playing target audio through the preset player, according to a rhythm intensity feature of the target audio, control the foreground component to swing at a swing amplitude matching the rhythm intensity feature.

[0127] Based on the generation apparatus of the player skin provided in the embodiments of the present application, the image material uploaded by the user is obtained, and the image material is input into the target image encoder pre-trained, the semantic feature extraction of the foreground region and the background region in the image material is performed through the target image encoder, and the image semantic feature of the image material is output. The image semantic feature is input into the image recognition model pre-trained, the visual elements contained in the image semantic feature are recognized through the image recognition model, and the region type to which the recognized visual elements belong is predicted from the foreground region and the background region, and the visual element recognition result corresponding to the image material is output. The foreground skin corresponding to the preset player is generated according to the visual elements in the foreground region in the visual element recognition result. The background skin corresponding to the preset player is generated according to the visual elements in the background region in the visual element recognition result. The foreground skin and the background skin are combined to obtain the player skin corresponding to the preset player. In this way, the player skin matching the image material uploaded by the user is generated, and the flexibility of the player skin generation mode is effectively improved, which is conducive to meeting the individual customization needs of different users.

[0128] Based on the same inventive concept, the present application also provides an electronic device corresponding to the above-mentioned generation method of the player skin. Since the principle of solving problems in the electronic device in the embodiments of the present application is similar to that of the above-mentioned generation method of the player skin in the embodiments of the present application, the implementation of the electronic device can be referred to the implementation of the above-mentioned generation method of the player skin, and the repeated parts will not be described here.

[0129] Figure 6A structural schematic diagram of an electronic device 600 provided for an embodiment of the present application includes a processor 601, a memory 602, and a bus 603. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs a method for generating a player skin as in an embodiment, the processor 601 communicates with the memory 602 through the bus 603. The processor 601 executes the machine-readable instructions, wherein the processor 601 implements the following steps when executing the machine-readable instructions, in particular: obtain an image material uploaded by a user, and input the image material into a pre-trained target image encoder, perform semantic feature extraction on a foreground region and a background region in the image material through the target image encoder, and output image semantic features of the image material; input the image semantic features into a pre-trained image recognition model, recognize visual elements contained in the image semantic features through the image recognition model, and predict a region type to which the recognized visual elements belong from the foreground region and the background region, and output a visual element recognition result corresponding to the image material; wherein the visual element recognition result includes a plurality of visual elements recognized in the image material and region types to which the plurality of visual elements respectively belong; generate a foreground skin corresponding to a preset player according to visual elements belonging to the foreground region in the visual element recognition result; generate a background skin corresponding to the preset player according to visual elements belonging to the background region in the visual element recognition result; combine the foreground skin and the background skin to obtain a player skin corresponding to the preset player.

[0130] In an optional implementation, after obtaining the image material uploaded by the user, the processor 601 is further configured to: perform multi-modal detection on whether the image material has a security risk from detection dimensions respectively indicated by a plurality of different modalities, and obtain a security risk detection result of the image material; if it is determined according to the security risk detection result that the image material has a security risk, display first prompt information on a terminal device of the user; wherein the first prompt information is used to prompt the user to replace the currently uploaded image material; if it is determined according to the security risk detection result that the image material does not have a security risk, display second prompt information on the terminal device of the user; wherein the second prompt information is used to prompt the user that the image material currently uploaded by the user has passed image review.

[0131] In an optional implementation, after determining that the image material does not have a security risk according to the security risk detection result, the processor 601 is further configured to: input the image material into a pre-trained image quality evaluation model, perform image quality evaluation on the image material from multiple preset dimensions through the image quality evaluation model, and output an image quality score of the image material; if the image quality score is lower than a preset review score, display the first prompt information on the terminal device of the user; if the image quality score is higher than or equal to the preset review score, display the second prompt information on the terminal device of the user.

[0132] In an optional implementation, the processor 601 is configured to jointly train the target image encoder and the image recognition model by using the following method: collect multiple sample images, and mark visual elements contained in the multiple sample images and region types to which the visual elements belong, to obtain image marking results corresponding to the multiple sample images respectively; input the sample images into an image encoder of an image segmentation model, perform semantic feature extraction on foreground regions and background regions in the sample images through the image encoder, and output image semantic features of the sample images; input the image semantic features of the sample images into an image decoder of the image segmentation model, perform semantic segmentation on the foreground regions and the background regions in the sample images through the image decoder, and output semantic segmentation prediction results of the sample images; input the image semantic features of the sample images into an original image recognition model, recognize visual elements contained in the input image semantic features through the original image recognition model, and predict region types to which the recognized visual elements belong from the foreground regions and the background regions, to output visual element recognition results corresponding to the sample images; jointly train the image segmentation model and the original image recognition model according to a first prediction loss between the visual element recognition results corresponding to the sample images and the image marking results of the sample images, and a second prediction loss between the semantic segmentation prediction results of the sample images and the image marking results of the sample images, until the image segmentation model and the original image recognition model converge; obtain an image encoder in the image segmentation model that has converged as the trained target image encoder, and obtain the original image recognition model that has converged as the trained image recognition model.

[0133] In an optional implementation, when the foreground skin corresponding to the preset player is generated according to the visual elements belonging to the foreground region in the visual element identification result, the processor 601 is configured to: determine first semantic information corresponding to the visual element according to the visual element belonging to the foreground region in the visual element identification result; determine second semantic information corresponding to the preset component style of the foreground component according to the preset component style of the foreground component in the preset player; input the first semantic information and the second semantic information into a text-to-image model, and output foreground image materials matched with the first semantic information and the second semantic information as the foreground skin through the text-to-image model.

[0134] In an optional implementation, after the foreground image materials matched with the first semantic information and the second semantic information are output through the text-to-image model, the processor 601 is configured to: identify and segment a salient target in the foreground image materials as the foreground skin.

[0135] In an optional implementation, when the background skin corresponding to the preset player is generated according to the visual elements belonging to the background region in the visual element identification result, the processor 601 is configured to: determine third semantic information corresponding to the visual element according to the visual element belonging to the background region in the visual element identification result; determine fourth semantic information corresponding to the preset component style of the background component according to the preset component style of the background component in the preset player; input the third semantic information and the fourth semantic information into a text-to-image model, and output background image materials matched with the third semantic information and the fourth semantic information as the background skin through the text-to-image model.

[0136] In an optional implementation, after the background image materials matched with the third semantic information and the fourth semantic information are output through the text-to-image model, the processor 601 is configured to: identify a target image region for displaying an audio cover of to-be-played audio from the background image materials; remove image content displayed in the target image region from the background image materials to obtain remaining background image materials as the background skin.

[0137] In an optional implementation, the processor 601 is further configured to: in response to application of the player skin on the preset player, configure a skin of a background component in the preset player as the background skin, and configure a skin of a foreground component in the preset player as the foreground skin; in response to playing target audio through the preset player, display an audio cover of the target audio in a blank area of the background skin; in accordance with a rhythm transformation feature of the target audio, control the audio cover to rotate at a rotation speed matching the rhythm transformation feature.

[0138] In an optional implementation, the processor 601 is further configured to: in response to application of the player skin on the preset player, configure a skin of a background component in the preset player as the background skin, and configure a skin of a foreground component in the preset player as the foreground skin; in response to playing target audio through the preset player, in accordance with a rhythm intensity feature of the target audio, control the foreground component to swing at a swing amplitude matching the rhythm intensity feature.

[0139] By means of the electronic device provided in the embodiments of the present application, the image material uploaded by the user is acquired, and the image material is input into the target image encoder pre-trained, the semantic feature of the foreground region and the background region in the image material is extracted through the target image encoder, and the image semantic feature of the image material is output. The image semantic feature is input into the image recognition model pre-trained, the visual elements contained in the image semantic feature are recognized through the image recognition model, and the region type to which the recognized visual elements belong is predicted from the foreground region and the background region, and the visual element recognition result corresponding to the image material is output. The foreground skin corresponding to the preset player is generated according to the visual elements belonging to the foreground region in the visual element recognition result. The background skin corresponding to the preset player is generated according to the visual elements belonging to the background region in the visual element recognition result. The foreground skin and the background skin are combined to obtain the player skin corresponding to the preset player. In this way, the player skin matching the image material uploaded by the user is generated, the flexibility of the player skin generation mode is effectively improved, and it is beneficial to meet the individual customization needs of different users.

[0140] Based on the same inventive concept, the embodiments of the present application further provide a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, the processor performs the following steps: obtaining an image material uploaded by a user, and inputting the image material into a pre-trained target image encoder, performing semantic feature extraction on a foreground region and a background region in the image material by the target image encoder, and outputting image semantic features of the image material; inputting the image semantic features into a pre-trained image recognition model, identifying visual elements contained in the image semantic features by the image recognition model, and predicting a region type to which the identified visual elements belong from the foreground region and the background region, and outputting a visual element recognition result corresponding to the image material; wherein the visual element recognition result includes a plurality of visual elements identified in the image material and region types to which the plurality of visual elements belong, respectively; generating a foreground skin corresponding to a preset player according to visual elements belonging to the foreground region in the visual element recognition result; generating a background skin corresponding to the preset player according to visual elements belonging to the background region in the visual element recognition result; combining the foreground skin and the background skin to obtain a player skin corresponding to the preset player.

[0141] In an optional implementation, after obtaining the image material uploaded by the user, the processor is further configured to: performing multi-modal detection on whether the image material has a security risk from detection dimensions respectively indicated by a plurality of different modalities, and obtaining a security risk detection result of the image material; if it is determined according to the security risk detection result that the image material has a security risk, displaying first prompt information on the terminal device of the user; wherein the first prompt information is used to prompt the user to replace the currently uploaded image material; if it is determined according to the security risk detection result that the image material has no security risk, displaying second prompt information on the terminal device of the user; wherein the second prompt information is used to prompt the user that the image material currently uploaded by the user has passed image review.

[0142] In an optional implementation, after determining that the image material has no security risk according to the security risk detection result, the processor is further configured to: inputting the image material into a pre-trained image quality evaluation model, performing image quality evaluation on the image material from a plurality of preset dimensions by the image quality evaluation model, and outputting an image quality score of the image material; if the image quality score is lower than a preset review score, displaying the first prompt information on the terminal device of the user; If the image quality score is higher than or equal to the preset review score, the second prompt information is displayed on the terminal device of the user.

[0143] In an optional implementation, the processor is configured to obtain the target image encoder and the image recognition model through joint training by the following method: A plurality of sample images are collected, and visual elements contained in the plurality of sample images and region types to which the visual elements belong are marked respectively to obtain image marking results corresponding to the plurality of sample images respectively; The sample image is input into an image encoder of an image segmentation model, and semantic feature extraction is performed on the foreground region and the background region in the sample image by the image encoder to output image semantic features of the sample image; The image semantic features of the sample image are input into an image decoder of the image segmentation model, and semantic segmentation is performed on the foreground region and the background region in the sample image by the image decoder to output a semantic segmentation prediction result of the sample image; The image semantic features of the sample image are input into an original image recognition model, the original image recognition model is used to recognize visual elements contained in the input image semantic features, and region types to which the recognized visual elements belong are predicted from the foreground region and the background region to output a visual element recognition result corresponding to the sample image; According to a first prediction loss between the visual element recognition result corresponding to the sample image and the image marking result of the sample image, and a second prediction loss between the semantic segmentation prediction result of the sample image and the image marking result of the sample image, the image segmentation model and the original image recognition model are jointly trained until the image segmentation model and the original image recognition model reach convergence; The image encoder in the image segmentation model that reaches convergence is obtained as the target image encoder that is trained, and the original image recognition model that reaches convergence is obtained as the image recognition model that is trained.

[0144] In an optional implementation, when the visual elements belonging to the foreground region in the visual element recognition result are used to generate a foreground skin corresponding to a preset player, the processor is configured to: According to the visual elements belonging to the foreground region in the visual element recognition result, first semantic information corresponding to the visual elements is determined; According to a preset component style of a foreground component in the preset player, second semantic information corresponding to the preset component style of the foreground component is determined; input the first semantic information and the second semantic information into a text-to-image model, and output, by the text-to-image model, a foreground image material matching the first semantic information and the second semantic information as the foreground skin.

[0145] In an optional implementation, after the text-to-image model outputs the foreground image material matching the first semantic information and the second semantic information, the processor is configured to: identify and segment, from the foreground image material, a salient target in the foreground image material as the foreground skin.

[0146] In an optional implementation, when the foreground skin is generated according to the visual elements belonging to the background region in the visual element identification result, the processor is configured to: determine, according to the visual elements belonging to the background region in the visual element identification result, third semantic information corresponding to the visual elements; determine, according to a preset component style of a background component in the preset player, fourth semantic information corresponding to the preset component style of the background component; input the third semantic information and the fourth semantic information into a text-to-image model, and output, by the text-to-image model, a background image material matching the third semantic information and the fourth semantic information as the background skin.

[0147] In an optional implementation, after the text-to-image model outputs the background image material matching the third semantic information and the fourth semantic information, the processor is configured to: identify, from the background image material, a target image region for displaying an audio cover of to-be-played audio; remove, from the background image material, image content displayed in the target image region, to obtain remaining background image material as the background skin.

[0148] In an optional implementation, the processor is further configured to: in response to application of the player skin on the preset player, configure a skin of a background component in the preset player as the background skin, and configure a skin of a foreground component in the preset player as the foreground skin; in response to playing of target audio by the preset player, display an audio cover of the target audio in a blank region of the background skin; control the audio cover to rotate at a speed matching a rhythm transformation feature of the target audio according to the rhythm transformation feature.

[0149] In an optional implementation, the processor is further configured to: in response to application of the player skin on the preset player, configure a skin of a background component in the preset player as the background skin, and configure a skin of a foreground component in the preset player as the foreground skin; in response to playing target audio through the preset player, control the foreground component to swing according to a swing amplitude matched with a rhythm intensity feature of the target audio, according to the rhythm intensity feature of the target audio.

[0150] According to the above computer readable storage medium provided in the embodiments of the present application, the image material uploaded by the user is acquired, and the image material is input into the pre-trained target image encoder. The semantic feature extraction is performed on the foreground region and the background region in the image material through the target image encoder, and the image semantic feature of the image material is output. The image semantic feature is input into the pre-trained image recognition model. The visual elements contained in the image semantic feature are recognized through the image recognition model, and the region type to which the recognized visual elements belong is predicted from the foreground region and the background region. The visual element recognition result corresponding to the image material is output. The foreground skin corresponding to the preset player is generated according to the visual elements belonging to the foreground region in the visual element recognition result. The background skin corresponding to the preset player is generated according to the visual elements belonging to the background region in the visual element recognition result. The foreground skin and the background skin are combined to obtain the player skin corresponding to the preset player. In this way, the player skin matched with the image material uploaded by the user is generated, and the flexibility of the player skin generation mode is effectively improved, which is beneficial to meet the individual customization needs of different users.

[0151] In the embodiments of the present application, the computer readable storage medium can also execute other machine readable instructions when executed by the processor, to perform the generation method of the player skin as described in other embodiments. For specific steps and principles of the generation method of the player skin, refer to the description of the method embodiment, which will not be repeated here.

[0152] In the embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. The system embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, and can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0154] In addition, the functional units in the embodiments provided in the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0155] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product, and the computer software product is stored in a storage medium, including a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various program code storage media.

[0156] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings, in addition, the terms "first", "second", "third" and the like are only used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0157] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, used to illustrate the technical solutions of the present application, and not to limit them, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any skilled person in the art can modify or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating a player skin, characterized in that, The generation method includes: The system acquires image materials uploaded by users and inputs the image materials into a pre-trained target image encoder. The target image encoder extracts semantic features from the foreground and background regions of the image materials and outputs the image semantic features of the image materials. The image semantic features are input into a pre-trained image recognition model. The image recognition model identifies the visual elements contained in the image semantic features and predicts the region type to which the identified visual elements belong from the foreground region and the background region. The visual element recognition result corresponding to the image material is then output. The visual element recognition result includes multiple visual elements identified in the image material and the region type to which each of the multiple visual elements belongs. Based on the visual elements belonging to the foreground region in the visual element recognition results, generate a foreground skin corresponding to the preset player; Based on the visual elements belonging to the background area in the visual element recognition results, generate the background skin corresponding to the preset player; The foreground skin and the background skin are combined to obtain the player skin corresponding to the preset player.

2. The generation method according to claim 1, characterized in that, After obtaining the image materials uploaded by the user, the generation method further includes: Multimodal detection is performed on the image material to determine whether there are security risks, based on detection dimensions indicated by multiple different modalities, and the security risk detection results of the image material are obtained. If the security risk detection results indicate that the image material poses a security risk, a first prompt message is displayed on the user's terminal device; wherein, the first prompt message is used to prompt the user to replace the currently uploaded image material; If the security risk detection results determine that the image material does not pose a security risk, a second prompt message is displayed on the user's terminal device; wherein the second prompt message is used to indicate to the user that the currently uploaded image material has passed image review.

3. The generation method according to claim 2, characterized in that, After determining that the image material does not pose a security risk based on the security risk detection result, the generation method further includes: The image material is input into a pre-trained image quality assessment model, which evaluates the image quality of the image material from multiple preset dimensions and outputs an image quality score for the image material. If the image quality score is lower than the preset review score, the first prompt message will be displayed on the user's terminal device; If the image quality score is higher than or equal to the preset review score, the second prompt message will be displayed on the user's terminal device.

4. The generation method according to claim 1, characterized in that, The step of generating a foreground skin corresponding to a preset player based on visual elements belonging to the foreground region in the visual element recognition results includes: Based on the visual elements belonging to the foreground region in the visual element recognition results, determine the first semantic information corresponding to the visual element; Based on the preset component style of the foreground component in the preset player, determine the second semantic information corresponding to the preset component style of the foreground component; The first semantic information and the second semantic information are input into the text-based image model, and the text-based image model outputs a foreground image material that matches the first semantic information and the second semantic information as the foreground skin.

5. The generation method according to claim 4, characterized in that, After outputting foreground image material that matches the first semantic information and the second semantic information through the text-based image model, the generation method further includes: From the foreground image material, salient targets in the foreground image material are identified and segmented as the foreground skin.

6. The generation method according to claim 1, characterized in that, The step of generating a background skin corresponding to the preset player based on visual elements belonging to the background region in the visual element recognition results includes: Based on the visual elements belonging to the background region in the visual element recognition results, determine the third semantic information corresponding to the visual element; Based on the preset component style of the background component in the preset player, determine the fourth semantic information corresponding to the preset component style of the background component; The third semantic information and the fourth semantic information are input into the text-to-image model, and the text-to-image model outputs a background image material that matches the third semantic information and the fourth semantic information as the background skin.

7. The generation method according to claim 6, characterized in that, After outputting background image material that matches the third semantic information and the fourth semantic information through the text-based image model, the generation method further includes: From the background image material, identify the target image area for displaying the audio cover of the audio to be played; The image content displayed in the target image area is removed from the background image material to obtain the remaining background image material as the background skin.

8. The generation method according to claim 1, characterized in that, The generation method further includes: In response to the application of the player skin on the preset player, the skin of the background component in the preset player is configured as the background skin, and the skin of the foreground component in the preset player is configured as the foreground skin; In response to playing target audio through the preset player, the audio cover of the target audio is displayed in the blank area of ​​the background skin; Based on the rhythmic change characteristics of the target audio, the audio cover is controlled to rotate at a speed matching the rhythmic change characteristics.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the player skin generation method as described in any one of claims 1 to 8 are performed.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the player skin generation method as described in any one of claims 1 to 8.

Images