Data processing method and device and storage medium

Receive parsing instructions through the server, obtain media file content information and send target information to the target model, solving the problem of low efficiency when viewing and sharing media files, and achieving rapid understanding of media file content and providing recommended content and prompt information, improving User experience.

CN119011523APending Publication Date: 2024-11-22BEIJING ZITIAO NETWORK TECH CO LTD

Patent Information

Application Number
CN202311433841.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-10-31
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

After users share media files, viewing media files is less efficient, especially audio or video files, which requires users to consume more time, and fast forwarding or skipping some content may lead to the omission of critical information.

Method used

Provide a data processing method, the server receives parsing instructions sent by the client, obtains the content information of the media file, sends the target information to the target model, and sends display information to the client according to the output results, including converting the audio data into content information , extract subtitle files, generate prompt information and recommended content, so that users can understand media file content more efficiently.

Benefits of technology

By automatically parsing media files, users can quickly understand the content of media files, improve viewing efficiency, avoid the omission of key information, and provide recommended content and prompt information, enhancing the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011523A_ABST
    Figure CN119011523A_ABST
Patent Text Reader

Abstract

The invention provides a data processing method and device and a storage medium. The data processing method and device are used for automatically analyzing a media file sent by an IM software client. Specifically, a server side of IM software can receive an analysis instruction which is sent by a client side and aims at a target media file. After the analysis instruction is obtained, the content information corresponding to the target media file can be obtained. And then, target information generated according to the content information can be sent to the target model, so that the target model performs analysis based on the target information, and an output result is returned to the server. And after the output result is obtained, the server side can send the display information to the client side according to the output result. In this way, the user can further understand the content of the media file by means of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device and storage medium. Background Art

[0002] In some scenarios, there is a phenomenon that users share media files with other users. Among them, the media files may include audio files or video files. Optionally, the user can share the media files by directly sending the media files to other users. Alternatively, the user can also share the media files by sending the webpage corresponding to the media files to other users.

[0003] After acquiring the media files shared by other users, the user needs to view the media files to understand the content of the target media files, which is inefficient. Summary of the invention

[0004] In order to solve the problems of the prior art, the present application provides a data processing method, device and storage medium.

[0005] In a first aspect, the present application provides a data processing method, which is applied to a server and includes:

[0006] Receive a parsing instruction for a target media file sent by a client;

[0007] Based on the parsing instruction, obtaining content information of the target media file;

[0008] Sending target information to a target model, wherein the target information is generated based on the content information;

[0009] Receiving an output result returned by the target model based on the target information;

[0010] Sending display information to the client according to the output result.

[0011] In some possible implementations, obtaining content information corresponding to the target media file includes:

[0012] Acquire the target media file;

[0013] Extracting audio data of the target media file;

[0014] The audio data is converted into the content information.

[0015] In some possible implementations, obtaining content information corresponding to the target media file includes:

[0016] Acquire a target link sent by the client, where the target link is a webpage link corresponding to the target media file;

[0017] According to the target link, obtaining a subtitle file of the target media file;

[0018] The content information is determined according to the subtitle file of the target media file.

[0019] In some possible implementations, the target information includes at least one prompt information;

[0020] Before sending the target information to the target model, the method further includes:

[0021] The at least one prompt information is generated according to the target information.

[0022] In some possible implementations, the at least one prompt information includes N segmented prompt information, where N is a positive integer greater than 1, and generating the at least one prompt information according to the target information includes:

[0023] Segmenting the content information to obtain N segmented content information;

[0024] Determine N pieces of segment prompt information according to the N pieces of segment content information;

[0025] The output result includes output sub-results corresponding to each of the segment prompt information, and the output sub-results corresponding to the segment prompt information correspond to the segment content information.

[0026] In some possible implementations, the segment prompt information further includes structured information corresponding to the N segment content information, and the structured information is used to indicate the structural relationship of the segment text content in the text content.

[0027] In some possible implementations, the at least one prompt information includes first prompt information, and generating the at least one prompt information according to the target information further includes:

[0028] Generate first prompt information based on the output sub-results generated by the N segmented prompt information.

[0029] In some possible implementations, the at least one prompt information includes second prompt information;

[0030] Generating at least one prompt information according to the content information comprises:

[0031] extracting at least one target content information from the content information;

[0032] The second prompt information is generated according to the at least one target content information, the second prompt information is used to instruct the target model to extract information from the target content information, the output result includes an output sub-result corresponding to the second prompt information, and the output sub-result corresponding to the second prompt information corresponds to the target content information.

[0033] In some possible implementations, the at least one prompt information includes third prompt information;

[0034] Generating at least one prompt information according to the content information comprises:

[0035] Acquire historical conversation content, where the historical conversation content includes information about the target media file sent by the client;

[0036] The third prompt information is generated according to the content information and the historical conversation content; the third prompt information is used to instruct the target model to obtain recommended content according to the historical conversation content and the content information, and the output result includes the recommended content.

[0037] In a second aspect, the present application provides a data processing method, which is applied to a client and includes:

[0038] Displaying a first content message, where the first content message is used to share a target media file or a link to the target media file;

[0039] Obtaining a parsing operation triggered for the target media file;

[0040] Send parsing instructions to the server;

[0041] The display information returned by the server is displayed, where the display information is determined according to the content of the target media file.

[0042] In some possible implementations, the parsing operation is triggered by any of the following:

[0043] an intelligent assistant invoking an operation on the target media file; or

[0044] A triggering operation on a control displayed in association with the first content message.

[0045] In some possible implementations, the intelligent assistant invoking operation includes any one of the following:

[0046] An operation of sending the target media file to the intelligent assistant; or

[0047] An operation of sending an IM message for waking up an intelligent assistant in an instant messaging IM group chat session including the intelligent assistant, wherein the first content message is an IM message.

[0048] In some possible implementations, the displaying the display information returned by the server includes:

[0049] generating a second content message according to the first content message, wherein the second content message includes information of the target media file and the presentation information;

[0050] The second content message is displayed at a display position corresponding to the first content message.

[0051] In some possible implementations, the presentation information includes summary information, where the summary information is obtained by summarizing the content of the target media file.

[0052] In some possible implementations, the display information includes recommended content; the recommended content is determined according to historical conversation content and the target media file, and the historical conversation content includes information about the target media file sent by the client.

[0053] In some possible implementations, the displaying the display information returned by the server includes:

[0054] A recommended content control is displayed at an associated display position of the first content message, the recommended content control is used to display the recommended content, and the recommended content control is also used to trigger a recommended dialogue operation, and the recommended dialogue operation is used to send the recommended content to the server.

[0055] In some possible implementations, the method further includes:

[0056] In response to a forwarding operation triggered by the presentation information, the information of the target media file and the presentation information are forwarded.

[0057] In some possible implementations, the displaying of the display information returned by the server further includes:

[0058] At least one operation control is displayed, where the operation control is used to trigger a feedback operation, where the feedback operation is used to express the viewpoint of a user viewing the displayed information or to regenerate the displayed information.

[0059] In a third aspect, the present application provides a data processing device, wherein the method is applied to a server, including:

[0060] A first receiving unit, configured to receive a parsing instruction for a target media file sent by a client;

[0061] An acquisition unit, configured to acquire content information of the target media file based on the parsing instruction;

[0062] A first sending unit, configured to send target information to a target model, wherein the target information is generated based on the content information;

[0063] A second receiving unit, configured to receive an output result returned by the target model based on the target information;

[0064] The second sending unit is used to send display information to the client according to the output result.

[0065] In some possible implementations, the acquisition unit is specifically configured to acquire the target media file; extract audio data of the target media file; and convert the audio data into the content information.

[0066] In some possible implementations, the acquisition unit is specifically used to acquire a target link sent by the client, where the target link is a web page link corresponding to the target media file; acquire a subtitle file of the target media file according to the target link; and determine the content information according to the subtitle file of the target media file.

[0067] In some possible implementations, the target information includes at least one prompt information; the device further includes a prompt information generating unit; the prompt information generating unit is configured to generate the at least one prompt information according to the target information.

[0068] In some possible implementations, the at least one prompt information includes N segmented prompt information, where N is a positive integer greater than 1; the prompt information generation unit is specifically used to segment the content information to obtain N segmented content information; determine N segmented prompt information based on the N segmented content information; and the output result includes an output sub-result corresponding to each segmented prompt information, and the output sub-result corresponding to the segmented prompt information corresponds to the segmented content information.

[0069] In some possible implementations, the segment prompt information further includes structured information corresponding to the N segment content information, and the structured information is used to indicate the structural relationship of the segment text content in the text content.

[0070] In some possible implementations, the at least one prompt information includes first prompt information, and the prompt information generating unit is specifically configured to generate the first prompt information based on the output sub-results generated by the N segmented prompt information.

[0071] In some possible implementations, the at least one prompt information includes second prompt information; the prompt information generation unit is specifically used to extract at least one target content information from the content information; the second prompt information is generated according to the at least one target content information, the second prompt information is used to instruct the target model to extract information from the target content information, and the output result includes an output sub-result corresponding to the second prompt information, and the output sub-result corresponding to the second prompt information corresponds to the target content information.

[0072] In some possible implementations, the at least one prompt information includes third prompt information; the prompt information generation unit is specifically used to obtain historical conversation content, the historical conversation content includes information about the target media file sent by the client; the third prompt information is generated according to the content information and the historical conversation content; the third prompt information is used to instruct the target model to obtain recommended content based on the historical conversation content and the content information, and the output result includes the recommended content.

[0073] In a fourth aspect, the present application provides a data processing device, which is applied to a client and includes:

[0074] A display unit, used to display a first content message, where the first content message is used to share a target media file or a link to the target media file;

[0075] An acquisition unit, configured to acquire a parsing operation triggered for the target media file;

[0076] A sending unit, used for sending a parsing instruction to a server;

[0077] The display unit is further used to display the display information returned by the server, and the display information is determined according to the content of the target media file.

[0078] In some possible implementations, the parsing operation is triggered by any one of the following: an intelligent assistant invoking operation on the target media file; or a triggering operation on a control displayed in association with the first content message.

[0079] In some possible implementations, the smart assistant invoking operation includes any one of the following: an operation of sending the target media file to the smart assistant; or an operation of sending an IM message for invoking the smart assistant in an instant messaging IM group chat session including the smart assistant, wherein the first content message is an IM message.

[0080] In some possible implementations, the display unit is specifically configured to generate a second content message according to the first content message, where the second content message includes information about the target media file and the display information; and display the second content message at a display position corresponding to the first content message.

[0081] In some possible implementations, the presentation information includes summary information, where the summary information is obtained by summarizing the content of the target media file.

[0082] In some possible implementations, the display information includes recommended content; the recommended content is determined according to historical conversation content and the target media file, and the historical conversation content includes information about the target media file sent by the client.

[0083] In some possible implementations, the display unit is further used to display a recommended content control at an associated display position of the first content message, the recommended content control is used to display the recommended content, and the recommended content control is further used to trigger a recommendation dialogue operation, and the recommendation dialogue operation is used to send the recommended content to the server.

[0084] In some possible implementations, the device further includes a forwarding unit, configured to forward the information of the target media file and the presentation information in response to a forwarding operation triggered by the presentation information.

[0085] In some possible implementations, the display unit is further used to display at least one operation control, where the operation control is used to trigger a feedback operation, where the feedback operation is used to express the opinion of a user viewing the display information or to regenerate the display information.

[0086] In a fifth aspect, the present application provides an electronic device, including:

[0087] one or more processors;

[0088] a storage device having one or more programs stored thereon,

[0089] When the one or more programs are executed by the one or more processors, the one or more processors implement any method described in the first aspect, or implement any method described in the second aspect.

[0090] In a sixth aspect, the present application provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements any of the methods described in the first aspect, or implements any of the methods described in the second aspect.

[0091] In a seventh aspect, the present application provides a computer program product. When the computer program product runs on a device, the device executes the method described in the first aspect, or executes the method described in the second aspect.

[0092] The present application provides a data processing method, device and storage medium for automatically parsing media files sent by an IM software client. Specifically, the server of the IM software can receive a parsing instruction for a target media file sent by the client. After obtaining the parsing instruction, the content information corresponding to the target media file can be obtained. Then, the target information generated according to the content information can be sent to the target model so that the target model performs parsing based on the target information and returns the output result to the server. After obtaining the output result, the server can send display information to the client according to the output result. In this way, if the IM client receives the relevant information of the target media file, the user can trigger the parsing instruction on the IM client. According to the parsing instruction triggered by the user, the server can first obtain the content information corresponding to the target media file, and then send the content information to the target model for parsing by the target model. In this way, the user can use the model to further understand the content of the media file. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0094] Figure 1 A schematic diagram of a flow chart of a data processing method provided in an embodiment of the present application;

[0095] Figure 2 Another flowchart of the data processing method provided in the embodiment of the present application;

[0096] Figure 3 A schematic diagram of an exemplary application scenario provided for an embodiment of the present application;

[0097] Figure 4 Schematic diagram of another exemplary application scenario provided for an embodiment of the present application

[0098] Figure 5 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0099] Figure 6 A schematic diagram of another data processing device provided in an embodiment of the present application;

[0100] Figure 7 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0101] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0102] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0103] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0104] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different objects, devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0105] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0106] In some scenarios, a user may share a media file with other users. The user to whom the target media file is shared may view the media file shared by other users, thereby understanding the viewpoints or information that other users want to convey.

[0107] For example, in the process of using IM software for communication, users can share media files with other users through IM software, and can also receive media files shared by other users through IM clients. For example, user A can send a video file or a voice IM message to user B through the IM session between user B and user A on the client used by user A, and user B can receive the video file or voice IM message sent by user A through the client used by user B. For another example, user A can also send an IM message including a link to user B through the client used by user A. User B can click on the link in the IM message on the client to view the video file or audio file corresponding to the link.

[0108] In the above process, user B needs to view the media file shared by user A in order to understand the content of the media file. However, whether it is an audio file or a video file, it takes a lot of time to view the file. In addition, due to the continuous nature of media files, viewing the media file by fast-forwarding or skipping part of the content may cause the omission of key information.

[0109] In order to solve the problems of the prior art, an embodiment of the present application provides a method for processing data, which is described in detail below in conjunction with the drawings in the specification.

[0110] See also Figure 1 , Figure 1 An information interaction diagram of a data processing method provided in an embodiment of the present application, the embodiment of the present application can be applied to an application scenario in which a user uses a server to parse a media file. The data processing method can be executed by a data processing device. The data processing device can be implemented by software.

[0111] Optionally, the data processing device can be integrated into the server side of the software. Among them, the server side of the software can run on a server or a server cluster to provide corresponding services for the client side of the software. The client side of the software runs on a terminal device for use, such as a mobile phone or a computer. If the data processing device is integrated into the server side, the data processing device can communicate with the client side to obtain the parsing instructions sent by the client side, and send output results to the client side. Optionally, the data processing device can also communicate with the target model to send sample data and target request data to the target model. In an embodiment of the present application, the above-mentioned server side can be the server side of the IM software, and the above-mentioned client side can be the client side of the IM software. For the convenience of introduction, the following is an example of the data processing device being integrated into the server side of the IM software.

[0112] like Figure 1 As shown, the method specifically comprises the following steps:

[0113] S101: receiving a parsing instruction for a target media file sent by a client.

[0114] In order to parse the target media file, the server can receive a parsing instruction for the target media file sent by the client. The parsing instruction is sent by the client using the user of the client to indicate the intention of the user to parse the target media file. According to the parsing instruction, the server can call the target model to parse the target media file.

[0115] In the embodiment of the present application, the target media file may be an audio file, a video file, or a link corresponding to a video file or an audio file. In other words, the server may obtain the target media file or the link to the target media file. For example, if user A shares a link to video C with user B, user B may trigger a parsing instruction for video C on the client used by user B.

[0116] Optionally, the parsing instruction may be obtained according to an operation triggered by the user on the client. Two possible implementations of triggering the parsing instruction are schematically introduced below.

[0117] In a first possible implementation, the parsing instruction may be obtained based on the operation triggered on the parsing control. Specifically, the client may display the parsing control to the user on the interface. The user may trigger a click operation or other operation on the parsing control. After obtaining the operation triggered by the user on the parsing control, the client may generate a parsing instruction and send the parsing instruction to the server.

[0118] If the parsing instruction is obtained according to the parsing control, the parsing control may be displayed in association with the target media file, indicating that the parsing instruction for the target media file may be triggered by the parsing control. Optionally, the parsing control may be displayed in association with the target media file.

[0119] In some possible implementations, the parsing control may be displayed in a message card carrying the target media file. A message card is a type of message. In a message card, key information of the message may be organized into a message card according to a preset format and displayed to the user. By displaying the message card to the user, it is convenient for the user to view the key information in the message card. Optionally, if the server is an IM software A-level server, the message card may be an IM message card.

[0120] For example, if the target media file is shared by user A to user B, a message card corresponding to the target media file may be displayed on the client of user B in the IM session corresponding to user A.

[0121] In order to call the target model to parse the target media file, a parsing control may be displayed in the message card. For example, the parsing control may be displayed at the bottom of the message card. Optionally, the message card may also include function indication information. The function indication information is used to indicate the purpose of the parsing control. User B can understand the purpose of the parsing control by reading the function indication information, thereby knowing that the parsing control can be used to call the target model to parse the target media file and trigger the parsing control.

[0122] The message card corresponding to the target media file may include relevant information of the target media file, such as a preview of the target media file, a control for playing the target media file, etc. Optionally, if the target media file is shared in the form of a link, that is, user A sends a link corresponding to the target media file to user B, then the message card may also include a link corresponding to the target media file.

[0123] In a second possible implementation manner, the parsing instruction may be obtained according to the operation of sending a message.

[0124] Specifically, the parsing instruction can be obtained according to the operation of the user sending the target media file to the intelligent assistant. The intelligent assistant can be a pre-set object corresponding to the target model. The message sent by the user to the object can be parsed by the target model.

[0125] For an introduction to this part, please refer to the following text, which will not be repeated here.

[0126] S102: Acquire content information of the target media file based on the parsing instruction.

[0127] After obtaining the parsing instruction, the server may first determine the content information of the target media file. The content information of the target media file corresponds to the audio data of the target media file. Optionally, the content information of the target media file may be text data corresponding to the audio data of the target media file, and the text in the content information is consistent with the content described by the audio data of the target media file. The content information may be text information, which may also be referred to as text content information.

[0128] In the embodiment of the present application, the audio data of the target media file may be converted to obtain content information, and the content information of the existing target media file may also be obtained.

[0129] In a first possible implementation manner, the server may convert the audio data of the target media file to obtain content information of the target media file.

[0130] Specifically, the server may extract audio data from the target media file and convert the audio data into text to obtain content information of the target media file.

[0131] As described above, the target media file can be an audio file or a video file.

[0132] If the target media file is an audio file, then the target media file can be regarded as audio data. The server can parse the audio file to convert the audio file into content information.

[0133] If the target media file is a video file, and the target media file includes image data and audio data, the server can parse the target media file and extract the audio data from the target media file. Then, the server can convert the audio data into text to obtain the content information of the target media file.

[0134] As described above, the server can obtain the target media file sent by the client, and can also obtain the link of the target media file.

[0135] If the server obtains the target media file sent by the client, the server can store the target media file locally on the server. After obtaining the parsing instruction, the server can parse the locally stored target media file to obtain the content information of the target media file.

[0136] If the client sends a link of the target media file to the server, then after obtaining the parsing instruction, the server can first obtain the target media file according to the link of the target media file, and then parse the audio data of the target media file to obtain the content information of the target media file. Optionally, the server corresponding to the link of the target media file may have a communication interface with the server of the IM software. The server may send a request to obtain the target media file through the communication interface.

[0137] In some possible implementations, the server may have the ability to convert audio data. Accordingly, after extracting the audio data of the target media file, the server may parse the audio data locally to obtain text content information corresponding to the target media file.

[0138] Alternatively, in some other possible implementations, the server may also convert text content information by calling a service. For example, the server may be deployed with an interface corresponding to calling an audio-to-text cloud service, and the server may call the audio-to-text cloud service through the interface to convert the audio data of the target media file into text content information.

[0139] S103: Send target information to the target model.

[0140] After obtaining the content information, the server can generate target information based on the content information and send the target information to the target model. The target model can parse the received target information to obtain an output result corresponding to the target information. Since the target information is determined based on the content information, the content information corresponds to the target media file, and accordingly, the output result also corresponds to the target media file.

[0141] Among them, the target model can be a model with natural language processing (NLP) capabilities, such as a language model (LM) and other models. The target model can parse the natural language input to determine the information corresponding to the natural language input. For example, the target model can summarize the natural language input to determine the core content of the natural language input. Alternatively, in some implementations, the target model can also perform other processing according to the instructions, for example, it can recommend other content related to the input according to the recommendation instruction.

[0142] In some possible implementations, the server may send target information to the server via prompt information. For example, the target information may include at least one prompt information. Specifically, the server may generate at least one prompt information based on the content information, and send the at least one prompt information to the target model. The target model may process each prompt information respectively, obtain the processing result corresponding to each prompt information, and return the output result to the server.

[0143] In the embodiment of the present application, the server can generate a variety of different prompt information, and different prompt information can have different functions. According to the input of different prompt information, the processing results returned by the target model can have different functions. The following is a detailed description of five specific implementation methods as examples.

[0144] In the first implementation, the server can generate a full-text prompt message.

[0145] In a first possible implementation, the server can call the target model to summarize the full text of the content information. Accordingly, the server can generate full-text prompt information based on the content information and send the full-text prompt information to the target model. The full-text prompt information includes complete content information.

[0146] The target model can summarize the content information according to the full text prompt information. In this way, the output structure returned by the target model includes the summary of the full text of the content information. According to the output result of the target model, the user can view the relevant information obtained by summarizing the complete target media file.

[0147] In actual scenarios, the amount of data that can be input into the target model may be limited. Accordingly, the amount of data that a prompt message can carry is also limited. If the length of the complete content information exceeds the upper limit of the amount of data that a single prompt message can carry, the server can divide the complete content information into multiple segments of content information, and then carry the multiple segments of content information in multiple full-text prompt messages and send them to the target model. The target model can summarize each full-text prompt message and return multiple intermediate results to the server. The server can then generate a new full-text prompt message based on the intermediate results and send the new prompt message to the target model. Based on the new prompt message, the server can summarize the intermediate results, thereby completing the summary of the full text of the content information. In this way, even if the length of the content information exceeds the length that a single prompt message can carry, the summary of the full text of the content information can be achieved.

[0148] In the second implementation, the server can generate segmented prompt information.

[0149] In some scenarios, the target media file may involve a lot of content. Accordingly, the content information obtained from the target media file may also involve multiple aspects. In this way, if the complete content information is input into the target model, the target model may lose one or several aspects of the content information during the summarization process. In other words, if the complete content information is directly summarized through the target model, the relevant information in the target media file may be lost.

[0150] To this end, in some possible implementations, the server can segment the content information, generate multiple prompt messages based on the segmented content information, and then send the multiple prompt messages to the target model. In this way, the target model can summarize each prompt message separately. Since the content information is segmented, the amount of text content carried in each prompt message is small, and the target model will not lose the information in the prompt during the summarization process.

[0151] In the embodiment of the present application, the information obtained after segmenting the content information can be referred to as segmented content information. The prompt information generated according to the segmented content information can be referred to as segmented prompt information. That is, the content information can be divided into N segmented content information (N is a positive integer greater than 1), and then N segmented prompt information can be obtained according to the segmented content information.

[0152] Optionally, when segmenting the content information, the segmentation can be performed based on the logical relationship between the content information. For example, the part of the content information used to describe the same or similar content can be divided into a segmented content information. Optionally, the content information can be segmented based on keywords, for example, the part including the same keyword can be divided into a segmented content information, or the part with a keyword appearance frequency greater than a preset threshold can be divided into a segmented content information.

[0153] Optionally, the content information may be segmented with reference to the target media file. Since the content information is obtained based on the audio data of the target media file, the segmentation method of the target media file may be used in the same manner as the segmentation method of the content information. For example, the pause in the audio data of the target media file may be monitored, and the content information corresponding to the audio data before the pause and the content information after the pause may be divided into two different segmented content information.

[0154] Optionally, the content information may be segmented with reference to the audio features of the target media file. The audio features of the target media file may be features of the audio data of the target media file, such as the timbre, frequency domain, and noise level of the audio data of the target media file. Optionally, if the target media file includes sounds made by multiple persons, the sounds made by different persons may be divided into different parts according to the timbre of the audio data of the target media file, and each part of the audio data corresponds to a segmented content information.

[0155] It should be noted that in the first implementation manner described above, the content information can also be split into multiple segments of content information. However, in the implementation manner corresponding to the full-text prompt information, the content information is divided according to the length, while in the implementation manner corresponding to the segmented prompt information, the content information can be divided according to the actual situation of the target media file. In this way, the segmented prompt information and even the structured prompt information described below refer to the actual situation of the target media file, and the output result obtained can more accurately describe the target media file.

[0156] If at least one prompt information generated by the server includes N segmented prompt information, then after the target model obtains the N segmented prompt information, it can send the N segmented prompt information to the target model to obtain the output sub-result corresponding to each prompt information.

[0157] In a third implementation, the server may generate structured prompt information. Optionally, the structured prompt information may also be referred to as first prompt information.

[0158] By segmenting the content information, the user's need to view the key information corresponding to each audio data segment in the target media file can be met. However, the user's need to view the complete key information of the target media file cannot be met. For this reason, the server can also generate structured prompt information so that the target model can determine the complete key information of the target media file based on the structured prompt information.

[0159] Specifically, the structured prompt information corresponds to the N segmented content information to ensure that the output result returned by the target model is obtained based on the complete content information. Optionally, the structured prompt information may include the N segmented content information.

[0160] In addition, the structured prompt information includes structured information. The structured information is used to describe the association relationship between the segmented content information. In this way, the target model can determine the meaning and role of the segmented content information in the complete content information based on the association relationship between the segmented content information, thereby determining the key information of the complete target media file.

[0161] Optionally, the structured information may include, for example, a sequence number of the segmented content information. The sequence number is used to indicate the sequence of the segmented content information in the content information. Alternatively, the structured information may also be used to describe the logical relationship between the segmented text contents.

[0162] Optionally, the structured prompt information may include complete content information, and the structured information may be obtained by segmenting the content information. Accordingly, the target model may be better summarized based on the structured prompt information, and the obtained output structure may better reflect the key information of the complete target media file.

[0163] As described above, a prompt message may not be able to carry content information. Therefore, when sending a structured prompt message to a target model, if the amount of content information is too large to carry N segmented content information in the structured prompt message, the server can send the N segmented content information to the target model via multiple sub-prompt messages. Optionally, the sub-prompt message can be a segmented prompt message. Alternatively, the sub-prompt message can also include multiple segmented prompt messages.

[0164] Specifically, the server can split N segmented content information into M sub-prompt information (M is a positive integer greater than 1 and less than N). Each sub-prompt information includes at least one segmented content information and sub-structured information. The sub-structured content is used to describe the association relationship between the segmented content information included in the sub-prompt information. Different sub-prompt information includes different segmented content information.

[0165] That is, the i-th sub-prompt information in the M sub-prompt information includes j segment content information and sub-structured information. The sub-structured information in the i-th sub-prompt information is used to indicate the association relationship between the j segment text contents. Wherein, i is a positive integer less than or equal to M, and j is a positive integer less than N.

[0166] After obtaining the M sub-prompt information, the server can send the M sub-prompt information to the target model. The target model can obtain M sub-output results based on the M sub-prompt information and return them to the server. Each sub-output result corresponds to a sub-prompt information. The server can determine the structured prompt information based on the M sub-output results and the structured information. In this way, the server summarizes the sub-prompt information so that the length of the sub-output result is less than the length of the corresponding sub-prompt information. In this way, the compression of the segmented content information is achieved, so that the structured prompt information can carry the relevant content of multiple segmented content information.

[0167] In a fourth implementation, the server may generate a target extraction prompt message. Optionally, the target extraction prompt message may also serve as a second prompt message.

[0168] In some scenarios, there may be a need to summarize a specific part of the target media file. For example, if the target media file is used to introduce something, then the important content related to the thing in the target media file can be summarized. To this end, the server can generate target extraction prompt information.

[0169] The target extraction prompt information is used to instruct the target model to summarize the special part of the content information. Specifically, the target extraction prompt information includes at least one target content information. The target content information is a part of the content information, and is the part that the target model needs to focus on summarizing. Optionally, if the server determines multiple target content information, the server can carry the multiple target content information in one target extraction prompt information and send it, or it can carry it in multiple target extraction prompt information and send it.

[0170] Specifically, after obtaining the content information corresponding to the target media file, the server may extract the content information to extract the target content information from the content information. In some possible implementations, a rule for extracting the target content information may be preconfigured. The server may extract the target content information according to the preconfigured rule.

[0171] In addition to the target content information, the target extraction prompt information may also include instruction information corresponding to the action of "target extraction" (hereinafter referred to as first instruction information). The first instruction information is used to instruct the target model to summarize the target content information.

[0172] In a fifth implementation, the server may generate recommendation prompt information. Optionally, the recommendation prompt information may also be referred to as third prompt information.

[0173] In some scenarios, there may be a need to make recommendations for users. For example, after viewing the summary corresponding to the target media file, the user may have questions. To this end, the server can predict the questions that the user may ask through the target model and display them to the user, so that the user does not need to re-enter the question. For another example, the user may also want to view media files similar to the target media file. Each time, the server can use the target model to predict the search keywords used by the user when searching for media files and display them to the user.

[0174] To this end, the server can also generate recommendation prompt information. The recommendation prompt information may include historical conversation content and second instruction information. The second instruction information is instruction information corresponding to the action of "recommendation". The second instruction information is used to instruct the target model to make recommendations based on the target text content and historical conversation content. In some possible implementations, the second instruction information may include multiple instruction information. Different instruction information is used to instruct the target model to recommend different content.

[0175] The historical conversation content may be the conversation content between the user and the intelligent assistant. By analyzing the historical conversation content, the target model can determine the user's intention and make recommendations based on the user's intention. Alternatively, the historical conversation content may also be the conversation content between users.

[0176] It is understandable that in actual application scenarios, the server can generate any one or more of the above five prompt messages. In addition, the server can also generate other types of prompt messages, which will not be described here.

[0177] S104: Receive the output result returned by the target model based on the target information.

[0178] After obtaining the target information sent by the server, the target model can process the target information to obtain the corresponding output result and return it to the server. The server can receive the output result returned by the target model based on the target information.

[0179] In the above description, the server can generate multiple prompt information according to the content information and send it to the target model. Accordingly, the output result of the server can include multiple output sub-results. Different output sub-results can correspond to different prompt information. In other words, according to different prompt information, the target model can return different output sub-results.

[0180] The above introduces five possible prompt messages. The following introduces the output sub-results corresponding to these five prompt messages. It can be understood that if the server also generates other types of prompt messages, the target model can return the corresponding output sub-results.

[0181] In the first implementation, the server sends the full-text prompt information to the target model, and the output sub-result includes the output sub-result corresponding to the full-text prompt information. Accordingly, the target model can summarize the content information according to the full-text prompt information. The output sub-result corresponding to the full-text prompt information can include a summary of the complete content information.

[0182] If the server sends full-text prompt information to the target model, the output result returned by the target model may include the output sub-result corresponding to the full-text prompt information, or the output result returned by the target model may be further generated based on the output sub-result corresponding to the full-text prompt information.

[0183] In the second implementation, the server may send N segmented prompt information to the target model, and each segmented prompt information may include a portion of the content information. Then, the output result may be generated based on the output sub-results corresponding to the N segmented prompt information. Specifically, the target model may summarize each segmented prompt information to obtain a summary corresponding to each segmented content information. It is understandable that the output sub-result corresponding to the segmented prompt may include the summary corresponding to the segmented content information in the segmented prompt information.

[0184] If the server sends N segmented prompt information to the target model, the output result returned by the target model may include the output sub-result corresponding to each segmented prompt information, or the output result returned by the target model may be further generated based on the output sub-result corresponding to the segmented prompt information.

[0185] In a third implementation, the server may send structured prompt information to the target model, and the output result includes an output sub-result corresponding to the structured prompt information. The target model may summarize the multiple segmented content information based on the structured information in the structured prompt information. In this way, the output sub-result corresponding to the structured prompt information includes a summary of the complete content information with reference to the structure of the content information.

[0186] If the server sends structured prompt information to the target model, the output result returned by the target model may include the output sub-result corresponding to the structured prompt information, or the output result returned by the target model may be further generated based on the output sub-result corresponding to the structured prompt information.

[0187] In a fourth implementation, the server sends a target extraction prompt message to the target model, and the output result includes an output sub-result corresponding to the target extraction prompt message. The target model can summarize the target content information in the target extraction prompt message according to the first instruction information. In this way, the output sub-result corresponding to the target extraction prompt message includes a summary of the target text content.

[0188] If the server sends target extraction prompt information to the target model, the output result returned by the target model may include the output sub-result corresponding to the target extraction prompt information, or the output result returned by the target model may be further generated based on the output sub-result corresponding to the target extraction prompt information.

[0189] In the fifth implementation, the server sends the recommendation prompt information to the target model, and the output result includes the output sub-result corresponding to the recommendation prompt information. The target model can perform context learning based on the historical conversation content included in the recommendation prompt information, and determine the recommended content recommended to the user after the context learning. The output sub-result corresponding to the recommendation prompt information includes the recommended content recommended to the user.

[0190] If the server sends the recommendation prompt information to the target model, the output result returned by the target model may include the output sub-result corresponding to the recommendation prompt information, or the output result returned by the target model may be further generated based on the output sub-result corresponding to the recommendation prompt information.

[0191] Five types of output sub-results are introduced above. In actual scenarios, the target model can generate corresponding output sub-results based on the prompt information sent by the server. If the server generates multiple prompt information based on a media file, then the target model can obtain multiple output sub-results based on the multiple prompt information, and these multiple output sub-results can be collectively referred to as output results. That is, the output sub-results obtained based on the target media file can be collectively referred to as the output results corresponding to the target media file.

[0192] Alternatively, the server may further process the multiple output sub-results to obtain the final output result. For example, the server may summarize the multiple output sub-results or re-call the target model to summarize the multiple output sub-results.

[0193] S105: Sending display information to the client according to the output result.

[0194] After obtaining the output result returned by the target model, the server can generate display information according to the output result and send the display information to the client so that the display information can be displayed to the user through the client. Optionally, the display information can include summary information. The summary information is obtained by summarizing the content of the target media file.

[0195] Optionally, the client may display the display information to the user at a display position associated with the IM message corresponding to the target media file. For example, if the client displays the target media file to the user via an IM message card, the client may divide a display area for displaying the output result in the IM message card and display the output result in the display area.

[0196] Furthermore, the server can also associate and store the display information with the target media file. In this way, if the server receives an instruction triggered by other clients to parse the target media file, the server can return the stored display information. Alternatively, if the client forwards the target media file to other users, the server can synchronously forward the target media file and the display information.

[0197] Specifically, the server can associate and store the output result to the IM message card corresponding to the target media file. In this way, the target media file is forwarded to other clients, and the server can send the IM message card to the corresponding client to display the target media file and display information to the user.

[0198] The above describes some implementation methods of parsing target media files by calling the target model from the perspective of the server. The following describes the implementation method of triggering the server to parse the target media file from the perspective of the client.

[0199] See also Figure 2 , which is a flow chart of a data processing method provided by the present application. The embodiment of the present application can be applied to an application scenario in which a user uses a client to call a server to parse a media file. The data processing method can be executed by a data processing device. The data processing device can be implemented by running on software.

[0200] Optionally, the data processing device can be integrated into a client of the software. The client of the software can be run on a terminal device to provide corresponding services to the user. In the embodiment of the present application, the above-mentioned server can be a client of the IM software. For ease of introduction, the following description is made by taking the data processing device integrated into the client of the IM software as an example.

[0201] like Figure 2 As shown, the method specifically comprises the following steps:

[0202] S201: Display a first content message.

[0203] In an embodiment of the present application, the target media file to be parsed comes from a client. Specifically, before parsing the target media file, the client may first display a first content message. The first content message is used to share the target media file or a link to the target media file. The link to the target media file may be a URL corresponding to the target media file, for example, a Uniform Resource Locator (URL) corresponding to the target media file.

[0204] Optionally, the first content message may be a message sent by a user using the client through the client, or may be a message sent by other users to the user using the client. That is, the user may send the target media file to other users through the client or share the link of the target media file. Alternatively, the user may also receive the target media file sent by other users or receive the link of the target media file shared by other users through the client. Optionally, the above-mentioned other users may include an intelligent assistant. For the introduction of the intelligent assistant, please refer to the above, which will not be repeated here.

[0205] In some possible implementations, the client is an IM client, and the first content message may be an IM message. Accordingly, the client may display the target media file in the form of an IM message card. The first content message may be an IM message card for displaying the target media file. Optionally, information such as the cover and title of the target media file may be displayed in the IM message card. If the first content message is used to share a link to the target media file, the IM message card may also include a link to the target media file. If the first content message is used to share the target media file, the IM message card may also include a download control, which is used to trigger an operation to download the target media file.

[0206] S202: Acquire a parsing operation triggered for a target media file.

[0207] After the first content message is displayed to the user, the user may trigger a parsing operation for the target media file. According to the parsing operation triggered by the user, the client may parse the target media file by calling the target model through the server.

[0208] In the embodiment of the present application, the user can trigger the parsing operation in a variety of implementations. The following introduces some implementations of triggering the parsing operation.

[0209] In a first possible implementation, the parsing operation may be triggered by an intelligent assistant invoking operation on the target media file.

[0210] Specifically, the user can invoke the intelligent assistant so as to parse the target media file through the intelligent assistant. Optionally, the user can invoke the intelligent assistant by sending a message to the intelligent assistant. That is, the operation of the user invoking the intelligent assistant may include the operation of the user sending a message to the intelligent assistant.

[0211] Optionally, the user sending a message to the intelligent assistant may include the user sending an IM message for waking up the intelligent assistant. Specifically, the user may send an IM message for waking up the intelligent assistant in an IM group chat session including the intelligent assistant. The IM message may include a waking up instruction and the name of the intelligent assistant in the IM group chat. The waking up instruction may be “@”. Accordingly, the above-mentioned first content message may be a message sent in the IM group chat session.

[0212] That is to say, for media files or links to media files shared in an IM group chat session, if the IM group chat session includes an intelligent assistant, the user can send an IM message in the IM group chat session to invoke the intelligent assistant, so that the media files can be parsed by the intelligent assistant.

[0213] Alternatively, the user sending a message to the intelligent assistant may also include the user sending a message in an IM single chat session with the intelligent assistant. That is to say, the operation of the user sending the target media file to the intelligent assistant is equivalent to the intelligent assistant's invoking operation for the target media file. Accordingly, the above-mentioned first content message may be a message sent by the user to the intelligent assistant. After obtaining the target media file or the link to the target media file that the user analyzes with the intelligent assistant, the client may display the first content message and obtain the parsing instruction for the target media file.

[0214] In a second implementation manner, the user may trigger a parsing operation through a space displayed in association with the first content message.

[0215] Specifically, the client may display an operation control in association with the first content message. The user may trigger the parsing operation by triggering the operation control. For example, assuming that the first content message includes an IM message card, the IM message card may include an operation control for triggering the parsing operation, so that the user may trigger the parsing operation through the operation control.

[0216] See also Figure 3 , which is a schematic diagram of an application scenario provided by an embodiment of the present application. For example, when a user uses a client installed on a terminal device (such as a mobile phone or a computer), the client may display the following Figure 3 Display area 300 is shown.

[0217] Specifically, the display area 300 is used to display relevant content in the IM group chat. Figure 3 In the implementation shown, the name of the IM group chat is "learning material exchange group" and the user using the client is user B.

[0218] exist Figure 3In the implementation shown, display area 300 is used for IM message 310 and IM message 320. IM message 310 is an IM message sent by user A in an IM group chat, used to share a video with other users in the IM group chat. IM message 320 is an IM message sent by user B in an IM group chat, used to evoke the intelligent assistant to analyze the video shared by user A. Figure 3 In the corresponding implementation, the title of the video shared by user A is “Learning Video - Explanation and Analysis of AAA Knowledge Points”.

[0219] To indicate the user who sent the IM message, in Figure 3 In the illustrated implementation, the IM message may be displayed in association with the name and avatar of the user who sent the IM message.

[0220] The group chat message 310 is an IM message card, which is used to display the relevant information of the video shared by user A. Specifically, the group chat message 310 includes a link 311 of the target media file and a cover 312 of the target media file. Other users in the group chat can see the cover of the target media file through the group chat message 312, and can also jump to the page corresponding to the video by clicking the link 311.

[0221] In addition, the group chat message 310 also includes a parsing control 313. If the group chat message 310 is regarded as the "first content message" mentioned above, the parsing control 313 can be equivalent to the "control displayed in association with the first content message" mentioned above, and the user can trigger the parsing operation by triggering the parsing space 313.

[0222] exist Figure 3 In the implementation shown, user B can trigger the operation of waking up the intelligent assistant by sending an IM message 320, thereby calling the target model to parse the video shared by user A. That is, if the group chat message 310 is regarded as the "first content message" mentioned above, the IM message 320 can be equivalent to the "IM message sent in the IM group chat session including the intelligent assistant to invoke the intelligent assistant" mentioned above.

[0223] S203: Send a parsing instruction to the server.

[0224] After obtaining the parsing instruction, the client can send the parsing instruction to the server. The server can determine the content information of the target media file based on the parsing execution, generate target information based on the content information and send it to the target model, receive the output result obtained by the server based on the target information, determine the display information based on the output result and return the display information to the server. For an introduction to this part, please refer to Figure 1 The implementation method shown will not be repeated here.

[0225] S204: Display information returned by the display server.

[0226] After obtaining the display information returned by the server, the client can display the display information. The display message can include summary information. The summary information is obtained by summarizing the content of the target media file. In this way, the user can understand the general content of the target media file by viewing the display message. Optionally, the display information can also include other information. For more information, please refer to Figure 1 The introduction of the corresponding embodiments will not be repeated here.

[0227] In an embodiment of the present application, the parsed target media file may be shared through a first content message. Then, when displaying the display information, the display information may be displayed through a content message. Specifically, the client may generate a second content message based on the first content message and the display information, and display the second content message at a display position corresponding to the first content message, thereby displaying the display content through the second content message. For example, if the first content message is an IM message, the client may generate an IM message including the display content based on the first content message, and display the IM message including the display content at a position corresponding to the first content message.

[0228] by Figure 3 After user B sends IM message 320, the client can obtain the display information and display it as follows: Figure 4 The interface shown. Figure 4 In the implementation shown, the client can display Figure 4 Display area 400 is shown.

[0229] Specifically, the display area 400 includes an IM message 410. The IM message 410 is based on the display information and Figure 3 In the implementation shown, IM message 310 is obtained. In addition, the display position of IM message 410 matches IM message 310, which is equivalent to replacing IM message 310 with IM message 410.

[0230] Specifically, the IM message 410 includes a link 411 of the target media file, a cover 412 of the target media file, and display information 413. The display information 413 is obtained based on the content information of the target media file. Figure 4 In a corresponding implementation, the display information 413 includes summary information, and the content of the summary information is “This video explains…” By viewing the display information 413, the user can understand the general content of the target media file.

[0231] In some possible implementations, the user can trigger a forwarding operation based on the displayed information. Figure 4The IM message 410 in the illustrated implementation triggers a forwarding operation. After obtaining the forwarding operation triggered by the user, the information and presentation information of the target media file can be forwarded. Specifically, the IM message 410 can be forwarded as a whole. That is, the IM message 410 can be copied to other IM sessions.

[0232] As described above, the prompt information sent by the server to the target model may include the recommendation prompt information. The output sub-result returned by the target model includes the recommended content recommended to the user. If the output result includes the recommended content, the display information may also include the recommended content. Accordingly, when displaying the display information, the client may display the recommended content control at the associated display position of the first content message. The recommended content control is used to display the recommended content to the user.

[0233] In addition, the recommended content control is also used to trigger the recommended conversation operation. After obtaining the triggered recommended conversation operation, the client can send the recommended content to the server. In other words, the recommended content can be questions predicted by the target model that the user may want to ask after viewing the summary information (or other information). The client can display these questions to the user in the form of a recommended content control. The user can trigger the recommended content control to ask the target model again. In this way, the questions that the user may ask can be predicted without the user having to manually enter the questions.

[0234] For example, in Figure 4 In the corresponding implementation, the display area 400 may also include a recommended content control 420. The recommended content control 420 is used to display the recommended content obtained by the target model. Specifically, the recommended content may include three questions that the user may ask: Question A, Question B, and Question C. The user can ask Question A to the target model by clicking the display area corresponding to Question A in the recommended content control 420; can also ask Question B to the target model by clicking the display area corresponding to Question B in the recommended content control 420; can also ask Question C to the target model by clicking the display area corresponding to Question C in the recommended content control 420.

[0235] In some possible implementations, the user may not be satisfied with the output result obtained by the target model. In this case, the user may need to express his or her opinion and / or instruct to reprocess the content information of the target media file. In some other possible implementations, the user may be very satisfied with the output result obtained by the target model. In this case, the user may also need a channel to feedback his or her opinion.

[0236] To this end, in an embodiment of the present application, the client may further display at least one operation control. The user may trigger a feedback operation through the operation control. The feedback operation may express the views of the user viewing the display information or instruct the server to regenerate the display information. In this way, the user may feedback his or her views to the server or instruct the server to regenerate the display information by triggering the corresponding operation control.

[0237] For example, in Figure 4 In the corresponding implementation, the IM message 410 also includes an operation control 414 and an operation control 415. The operation control 414 is used to trigger a re-parsing operation. The user can instruct to re-parse the content information of the target media file by triggering the operation control 414. The operation control 415 is used for the user to feedback his own views. Specifically, the operation control 415 includes two sub-controls. One of the sub-controls is used to indicate that the user's feedback on the displayed content is satisfactory, and the other sub-control is used to indicate that the user is dissatisfied with the displayed content.

[0238] Based on a data processing method provided in the above method embodiment, the embodiment of the present application also provides a data processing device applied to a server and a data processing device applied to a client. The data processing device will be described below in conjunction with the accompanying drawings.

[0239] See also Figure 5 As shown in FIG. 1 , this figure is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The device is applied to a server, such as Figure 5 As shown, the data processing device includes:

[0240] The first receiving unit 510 is used to receive a parsing instruction for a target media file sent by a client;

[0241] An acquisition unit 520, configured to acquire content information of the target media file based on the parsing instruction;

[0242] A first sending unit 530 is used to send target information to a target model, where the target information is generated based on the content information;

[0243] A second receiving unit 540 is configured to receive an output result returned by the target model based on the target information;

[0244] The second sending unit 550 is configured to send display information to the client according to the output result.

[0245] In some possible implementations, the acquisition unit 520 is specifically configured to acquire the target media file; extract audio data of the target media file; and convert the audio data into the content information.

[0246] In some possible implementations, the acquisition unit 520 is specifically used to acquire a target link sent by the client, where the target link is a web link corresponding to the target media file; acquire a subtitle file of the target media file according to the target link; and determine the content information according to the subtitle file of the target media file.

[0247] In some possible implementations, the target information includes at least one prompt information; the device further includes a prompt information generating unit; the prompt information generating unit is configured to generate the at least one prompt information according to the target information.

[0248] In some possible implementations, the at least one prompt information includes N segmented prompt information, where N is a positive integer greater than 1; the prompt information generation unit is specifically used to segment the content information to obtain N segmented content information; determine N segmented prompt information based on the N segmented content information; and the output result includes an output sub-result corresponding to each segmented prompt information, and the output sub-result corresponding to the segmented prompt information corresponds to the segmented content information.

[0249] In some possible implementations, the segment prompt information further includes structured information corresponding to the N segment content information, and the structured information is used to indicate the structural relationship of the segment text content in the text content.

[0250] In some possible implementations, the at least one prompt information includes first prompt information, and the prompt information generating unit is specifically configured to generate the first prompt information based on the output sub-results generated by the N segmented prompt information.

[0251] In some possible implementations, the at least one prompt information includes second prompt information; the prompt information generation unit is specifically used to extract at least one target content information from the content information; the second prompt information is generated according to the at least one target content information, the second prompt information is used to instruct the target model to extract information from the target content information, and the output result includes an output sub-result corresponding to the second prompt information, and the output sub-result corresponding to the second prompt information corresponds to the target content information.

[0252] In some possible implementations, the at least one prompt information includes third prompt information; the prompt information generation unit is specifically used to obtain historical conversation content, the historical conversation content includes information about the target media file sent by the client; the third prompt information is generated according to the content information and the historical conversation content; the third prompt information is used to instruct the target model to obtain recommended content based on the historical conversation content and the content information, and the output result includes the recommended content.

[0253] See also Figure 6 As shown in FIG. 1 , this figure is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The device is applied to a client, such as Figure 6 As shown, the data processing device includes:

[0254] A display unit 610 is used to display a first content message, where the first content message is used to share a target media file or a link to the target media file;

[0255] An acquisition unit 620 is used to acquire a parsing operation triggered for the target media file;

[0256] The sending unit 630 is used to send a parsing instruction to the server;

[0257] The display unit 610 is further configured to display the display information returned by the server, wherein the display information is determined according to the content of the target media file.

[0258] In some possible implementations, the parsing operation is triggered by any one of the following: an intelligent assistant invoking operation on the target media file; or a triggering operation on a control displayed in association with the first content message.

[0259] In some possible implementations, the smart assistant invoking operation includes any one of the following: an operation of sending the target media file to the smart assistant; or an operation of sending an IM message for invoking the smart assistant in an instant messaging IM group chat session including the smart assistant, wherein the first content message is an IM message.

[0260] In some possible implementations, the display unit 610 is specifically configured to generate a second content message according to the first content message, where the second content message includes information about the target media file and the display information; and display the second content message at a display position corresponding to the first content message.

[0261] In some possible implementations, the presentation information includes summary information, where the summary information is obtained by summarizing the content of the target media file.

[0262] In some possible implementations, the display information includes recommended content; the recommended content is determined according to historical conversation content and the target media file, and the historical conversation content includes information about the target media file sent by the client.

[0263] In some possible implementations, the display unit 310 is further used to display a recommended content control at an associated display position of the first content message, the recommended content control is used to display the recommended content, and the recommended content control is further used to trigger a recommendation dialogue operation, and the recommendation dialogue operation is used to send the recommended content to the server.

[0264] In some possible implementations, the device further includes a forwarding unit, configured to forward the information of the target media file and the presentation information in response to a forwarding operation triggered by the presentation information.

[0265] In some possible implementations, the display unit 610 is further used to display at least one operation control, where the operation control is used to trigger a feedback operation, where the feedback operation is used to express the opinion of a user viewing the display information or to regenerate the display information.

[0266] Based on a data processing method provided by the above method embodiment, the present application also provides an electronic device, including: one or more processors; a storage device, on which one or more programs are stored, when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method as described in any of the above embodiments.

[0267] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present application. The terminal device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (portable android devices), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs (televisions), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0268] like Figure 7As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0269] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0270] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0271] The electronic device provided in the embodiment of the present application and the data processing method provided in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0272] Based on a data processing method provided in the above method embodiment, an embodiment of the present application provides a computer storage medium on which a computer program is stored, wherein when the program is executed by a processor, the data processing method as described in any of the above embodiments is implemented.

[0273] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0274] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0275] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0276] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes the data processing method.

[0277] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0278] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0279] The units involved in the embodiments described in this application may be implemented by software or hardware. The name of a unit / module does not limit the unit itself in some cases. For example, a voice data acquisition module may also be described as a "data acquisition module".

[0280] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0281] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0282] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0283] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0284] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0285] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: The method is applied to the server, and includes: Receive a parsing instruction for a target media file sent by a client; Based on the parsing instruction, obtaining content information of the target media file; Sending target information to a target model, wherein the target information is generated based on the content information; Receiving an output result returned by the target model based on the target information; Sending display information to the client according to the output result.

2. The method according to claim 1, characterized in that The acquiring of the content information corresponding to the target media file includes: Acquire the target media file; Extracting audio data of the target media file; The audio data is converted into the content information.

3. The method according to claim 1, characterized in that The acquiring of the content information corresponding to the target media file includes: Acquire a target link sent by the client, where the target link is a webpage link corresponding to the target media file; According to the target link, obtaining a subtitle file of the target media file; The content information is determined according to the subtitle file of the target media file.

4. The method according to any one of claims 1 to 3, characterized in that: The target information includes at least one prompt information; Before sending the target information to the target model, the method further includes: The at least one prompt information is generated according to the target information.

5. The method according to claim 4, characterized in that The at least one prompt information includes N segmented prompt information, where N is a positive integer greater than 1, and generating the at least one prompt information according to the target information includes: Segmenting the content information to obtain N segmented content information; Determine N pieces of segment prompt information according to the N pieces of segment content information; The output result includes output sub-results corresponding to each of the segment prompt information, and the output sub-results corresponding to the segment prompt information correspond to the segment content information.

6. The method according to claim 5, characterized in that The segmentation prompt information also includes structured information corresponding to the N segmentation content information, and the structured information is used to indicate the structural relationship of the segmentation text content in the text content.

7. The method according to claim 6, characterized in that The at least one prompt information includes first prompt information, and the generating the at least one prompt information according to the target information further includes: Generate first prompt information based on the output sub-results generated by the N segmented prompt information.

8. The method according to claim 4, characterized in that The at least one prompt information includes second prompt information; Generating at least one prompt information according to the content information comprises: extracting at least one target content information from the content information; The second prompt information is generated according to the at least one target content information, the second prompt information is used to instruct the target model to extract information from the target content information, the output result includes an output sub-result corresponding to the second prompt information, and the output sub-result corresponding to the second prompt information corresponds to the target content information.

9. The method according to claim 4, characterized in that The at least one prompt information includes third prompt information; Generating at least one prompt information according to the content information comprises: Acquire historical conversation content, where the historical conversation content includes information about the target media file sent by the client; The third prompt information is generated according to the content information and the historical conversation content; the third prompt information is used to instruct the target model to obtain recommended content according to the historical conversation content and the content information, and the output result includes the recommended content.

10. A data processing method, characterized in that: The method is applied to a client and includes: Displaying a first content message, where the first content message is used to share a target media file or a link to the target media file; Obtaining a parsing operation triggered for the target media file; Send parsing instructions to the server; The display information returned by the server is displayed, where the display information is determined according to the content of the target media file.

11. The method according to claim 10, characterized in that The parsing operation is triggered by any of the following: an intelligent assistant invoking an operation on the target media file; or A triggering operation on a control displayed in association with the first content message.

12. The method according to claim 11, characterized in that The smart assistant awakening operation includes any one of the following: An operation of sending the target media file to the intelligent assistant; or An operation of sending an IM message for waking up an intelligent assistant in an instant messaging IM group chat session including the intelligent assistant, wherein the first content message is an IM message.

13. The method according to claim 10, characterized in that The display information returned by the display server includes: generating a second content message according to the first content message, wherein the second content message includes information of the target media file and the presentation information; The second content message is displayed at a display position corresponding to the first content message.

14. The method according to claim 10, characterized in that The display information includes summary information, and the summary information is obtained by summarizing the content of the target media file.

15. The method according to claim 1, characterized in that The display information includes recommended content; the recommended content is determined according to historical conversation content and the target media file, and the historical conversation content includes information about the target media file sent by the client.

16. The method according to claim 15, characterized in that The display information returned by the display server includes: A recommended content control is displayed at an associated display position of the first content message, the recommended content control is used to display the recommended content, and the recommended content control is also used to trigger a recommended dialogue operation, and the recommended dialogue operation is used to send the recommended content to the server.

17. The method according to claim 10, characterized in that The method further comprises: In response to a forwarding operation triggered by the presentation information, the information of the target media file and the presentation information are forwarded.

18. The method according to claim 13, characterized in that The display information returned by the server also includes: At least one operation control is displayed, where the operation control is used to trigger a feedback operation, where the feedback operation is used to express the viewpoint of a user viewing the displayed information or to regenerate the displayed information.

19. A data processing device, characterized in that: The device is applied to a server, and includes: A first receiving unit, configured to receive a parsing instruction for a target media file sent by a client; An acquisition unit, configured to acquire content information of the target media file based on the parsing instruction; A first sending unit, configured to send target information to a target model, wherein the target information is generated based on the content information; A second receiving unit, configured to receive an output result returned by the target model based on the target information; The second sending unit is used to send display information to the client according to the output result.

20. A data processing device, characterized in that: The device is applied to a client and includes: A display unit, used to display a first content message, where the first content message is used to share a target media file or a link to the target media file; An acquisition unit, configured to acquire a parsing operation triggered for the target media file; A sending unit, used for sending a parsing instruction to a server; The display unit is further used to display the display information returned by the server, and the display information is determined according to the content of the target media file.

21. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 18.

22. A computer readable medium, characterized in that A computer program is stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 18 is implemented.

Citation Information

Patent Citations

  • Visualization of automatic summarization using keyword zoom

    CN108027832A

  • Processing method and device, electronic equipment and medium

    CN114697760A

  • Chat interaction method, electronic equipment and server

    CN114710459A

  • Multimedia content summarization method and system thereof

    US20210201045A1

Cited By

  • Recommended information processing method and device, electronic equipment, storage medium and product

    CN119299417A

  • Interface interaction method and device, equipment and storage medium

    CN120687185A