Video resume generation method and device, storage medium and program product

Abstract

The embodiment of the invention provides a video resume generation method and device, a storage medium and a program product. In the video resume generation method, an automatic voice recognition and video editing process can be executed on an initial resume video of a job seeker in response to a resume editing operation, and after an initial text in the initial resume video is extracted, the initial text is optimized by adopting a target model mode; the accuracy of the text information identified from the initial resume video is effectively improved, so that the copywriting quality of the generated target resume video is improved. Besides, the timestamp corresponding to the target text in the initial resume video can be automatically determined, and the initial resume video is finely and automatically edited based on the timestamp, so that the consistency and strict alignment characteristics of the text content and the video content are ensured. And for job seekers, the high-quality video resumes can be easily and efficiently made without professional video editing skills, so that the editing threshold of the video resumes is greatly reduced.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, device, storage medium and program product for generating a video resume. Background Art

[0002] With the development of Internet technology and video technology and the increasing demand for personalized and efficient communication in modern society, video resumes have become increasingly popular. As an innovative way of self-introduction, video resumes provide job seekers with a channel to intuitively display their personal image, skills, and service attitude. For example, in the domestic service industry, domestic service workers can apply for jobs through video resumes. Video resumes can not only intuitively display the professional skills and service attitude of domestic service workers, but also demonstrate them through real life scenes, allowing employers to more comprehensively understand the work ability and personality characteristics of domestic service workers.

[0003] In the prior art, the generation of video resumes requires manual editing and post-processing by job seekers, which has a high technical threshold. Therefore, a new solution is yet to be proposed. Summary of the invention

[0004] Multiple aspects of the present application provide a video resume generation method, device, storage medium and program product to provide a solution for automatically generating video resumes and lower the threshold for generating video resumes.

[0005] An embodiment of the present application provides a method for generating a video resume, comprising: in response to a resume editing operation, obtaining an initial resume video of a job seeker, wherein the initial resume video contains job introduction information; performing voice recognition on the initial resume video to obtain an initial text corresponding to the job introduction information, and obtaining a timestamp corresponding to the initial text in the initial resume video; calling a target model to optimize the initial text to obtain a target text; determining a timestamp corresponding to the target text in the initial resume video based on a timestamp corresponding to the initial text in the initial resume video and a correspondence between the initial text and the target text; editing the initial resume video based on a timestamp corresponding to the target text in the initial resume video to obtain multiple video clips corresponding to the target text; and synthesizing the multiple video clips to obtain a target resume video.

[0006] Optionally, the target model is a large language model whose parameter amount is greater than a set threshold; calling the target model to optimize the initial text to obtain the target text includes: constructing prompt words according to the initial text; calling the target model according to the prompt words, so that the target model extracts valid text from the initial text under the instruction of the prompt words to obtain the target text.

[0007] Optionally, the timestamp corresponding to the target text in the initial resume video is determined based on the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text, including: identifying multiple groups of sentences with corresponding relationships among the target text sentences contained in the target text and the initial text sentences contained in the initial text, any group of sentences containing an initial text sentence and a target text sentence with a similarity greater than a set similarity threshold; in any group of text sentences, comparing the initial text sentences and the target text sentences word by word to obtain the initial text unit corresponding to each text unit in the target text sentence in the initial text sentence; determining the timestamp corresponding to each text unit in the target text sentence in the initial resume video based on the timestamp corresponding to the initial text unit in the initial resume video.

[0008] Optionally, in the target text sentences contained in the target text and the initial text sentences contained in the initial text, multiple groups of sentences with corresponding relationships are identified, including: determining the paragraph segmentation points in the initial text according to the timestamps corresponding to the text units in the initial text in the initial resume video, and determining the paragraph segmentation points in the target text according to the paragraph segmentation points in the initial text; segmenting the initial text and the target text according to the paragraph segmentation points in the initial text and the paragraph segmentation points in the target text, respectively, to obtain multiple initial text paragraphs and multiple target text paragraphs; performing similarity matching between the multiple initial text paragraphs and the multiple target text paragraphs, to obtain multiple groups of text paragraphs, any group of text paragraphs contains an initial text paragraph and a target text paragraph with a similarity greater than a set similarity threshold; in any group of text paragraphs, segmenting the initial text paragraph and the target text paragraph, respectively, to obtain multiple initial text sentences and multiple target text sentences; performing similarity matching between the multiple initial text sentences and the multiple target text sentences, to obtain multiple groups of sentences with corresponding relationships.

[0009] Optionally, the initial resume video is edited according to the timestamp corresponding to the target text in the initial resume video to obtain multiple video clips corresponding to the target text, including: updating the sentence division position of the target text according to the timestamp corresponding to the target text in the initial resume video to re-divide the sentences in the target text; identifying the text unit at the end of the sentence according to the updated sentence division position, and postponing the end timestamp of the text unit at the end of the sentence; editing the initial resume video according to the timestamp corresponding to the target text in the initial resume video and the postponed end timestamp of the text unit at the end of the sentence to obtain multiple video clips corresponding to the target text.

[0010] Optionally, the sentence division position of the target text is updated according to the timestamp corresponding to the target text in the initial resume video, including: determining the timestamps corresponding to multiple text units in the target text sentence in the initial resume video; for any two adjacent text units among the multiple text units, if the difference between the end timestamp of the previous text unit and the start timestamp of the next text unit is greater than a set time threshold, then the position between the previous text unit and the next text unit is used as the sentence division position.

[0011] Optionally, the initial resume video includes: at least one question information, the at least one question information is used to guide the job seeker to provide job-seeking related introductory information from at least one direction; synthesizing the multiple video clips to obtain a target resume video, including: obtaining the video material corresponding to each of the at least one question information; identifying the display order between the multiple video clips and the at least one question information; according to the display order, synthesizing the video material corresponding to each of the at least one question information with the multiple video clips to obtain the target resume video.

[0012] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the method provided in the embodiment of the present application.

[0013] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which can implement the steps in the method provided in the embodiment of the present application when the computer program is executed by a processor.

[0014] The embodiment of the present application also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the method provided in the embodiment of the present application.

[0015] In the video resume generation method provided by the embodiment of the present application, the resume editing operation can be responded to, and the automatic speech recognition and video editing process can be performed on the initial resume video of the job seeker. After the initial text in the initial resume video is extracted, the initial text is optimized by the target model method, which effectively improves the accuracy of the text information recognized from the initial resume video, thereby facilitating the improvement of the copywriting quality of the generated target resume video. In addition, according to the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text, the timestamp corresponding to the target text in the initial resume video can be determined, and then the initial resume video can be finely and automatically edited according to the timestamp corresponding to the target text, ensuring the consistency and strict alignment characteristics of the text content and the video content, while improving the editing efficiency, ensuring the synchronization effect of the sound and picture of the edited target video. Based on this implementation method, for job seekers, without having professional video editing skills, they can easily and efficiently produce high-quality video resumes, greatly reducing the editing threshold of video resumes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flowchart of a method for generating a video resume provided as an exemplary embodiment of the present application;

[0018] Figure 2 A flowchart of a method for generating a video resume provided by another exemplary embodiment of the present application;

[0019] Figure 3 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0021] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0022] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0023] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the product or system including the element.

[0024] In order to solve the technical problem that the generation of video resumes requires manual editing and post-processing by job seekers, which has a high technical threshold, a solution is provided in some embodiments of the present application. The technical solutions provided in each embodiment of the present application are described in detail below in conjunction with the accompanying drawings.

[0025] Figure 1 is a flow chart of a method for generating a video resume provided by an exemplary embodiment of the present application, which method may include: Figure 1 Steps shown:

[0026] Step 101, in response to a resume editing operation, obtain an initial resume video of a job seeker, wherein the initial resume video includes job application introduction information.

[0027] Step 102: Perform voice recognition on the initial resume video to obtain the initial text corresponding to the job introduction information, and obtain the timestamp corresponding to the initial text in the initial resume video.

[0028] Step 103: Call the target model to optimize the initial text to obtain the target text.

[0029] Step 104: Determine the timestamp corresponding to the target text in the initial resume video based on the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text.

[0030] Step 105: Edit the initial resume video according to the timestamp corresponding to the target text in the initial resume video to obtain multiple video clips corresponding to the target text.

[0031] Step 106: synthesize the multiple video clips to obtain a target resume video.

[0032] This embodiment can be executed by an electronic device, and a tool for executing the video resume generation method can be run on the electronic device, hereinafter referred to as a resume editing tool. The resume editing tool can be a client application, a web program, or a back-end service, which can be determined by the implementation form of the electronic device. For example, in some embodiments, the electronic device is implemented as a terminal device such as a desktop computer, a laptop computer, or a smart phone used by job seekers, and the resume editing tool can be implemented as a client application running on the above-mentioned terminal device or a functional module or plug-in in the client program. In other embodiments, the electronic device is implemented as a server-side device such as a conventional server, a cloud server, or a server array, and the resume editing tool can be implemented as a back-end service running on the server-side device. In some optional embodiments, the resume editing tool can be implemented as a web program with the front end running in a browser on the terminal device and the back end running on the server, and this embodiment does not limit this.

[0033] In step 101, the resume editing operation may be initiated by a user, or may be initiated by a system event, which will be described below in conjunction with different embodiments. In some optional embodiments, the electronic device is implemented as a terminal device, and the resume editing tool may display a resume editing page through the terminal device, which may be an in-application page of a client program, or may be a web page displayed by a browser. The terminal device may detect the resume editing operation by detecting the user's operation on the resume editing page. In this embodiment, the user refers to a job seeker who has a resume editing need, such as a housekeeping practitioner who needs to publish a video resume in a housekeeping service scenario.

[0034] In other optional embodiments, when the electronic device is implemented as a server-side device, a backend service corresponding to the resume editing tool or a backend component corresponding to the resume editing tool runs on the server-side device. The above-mentioned backend service or backend component can provide an API (Application Programming Interface) to the outside world, and can detect resume editing operations by detecting call events to the API. Among them, the call event to the API can be initiated by other applications or systems. For example, in an application scenario in the housekeeping service industry, after the housekeeping client detects that the housekeeping practitioner has uploaded a resume video, it automatically calls the above-mentioned API to initiate a resume editing operation.

[0035] Among them, the initial resume video of the job seeker refers to the resume video to be edited. Optionally, when the resume editing tool runs on the terminal device or partially runs on the terminal device, the resume video uploaded by the job seeker can be obtained as the initial resume video according to the job seeker's operation of uploading the local video, or, according to the job seeker's operation of recording the resume video, the camera component on the terminal device can be called to record the initial resume video for the job seeker. Optionally, when the resume editing tool runs on the server device or partially runs on the server device, the resume video uploaded by the job seeker through the terminal device can be obtained as the initial resume video, or the resume video that the job seeker has posted on the server device can be used as the initial resume video, which is not limited in this embodiment.

[0036] In this embodiment, the initial resume video contains job introduction information, which can introduce the job seeker's basic skills, advantages, and job-seeking needs from one or more aspects. In some scenarios, during the recording of the initial resume video, the digital human can ask questions to guide the job seeker to provide introduction information from different aspects.

[0037] After obtaining the initial resume video, in step 102, the resume editing tool can perform voice recognition on the initial resume video to obtain the initial text corresponding to the job introduction information. During the voice recognition process, the resume editing tool can identify silent segments in the initial resume video, where the silent segments refer to segments without audio data. The resume editing tool can crop video segments that do not contain silent segments from the initial resume video as valid video segments. After obtaining valid video segments, the resume editing tool can use a voice recognition tool to convert the audio in the valid video segment into text. The resume editing tool can use a local machine learning model for audio recognition, or can call the API of a remote voice recognition tool for audio recognition, which is not limited in this embodiment.

[0038] After voice recognition, the resume editing tool can obtain the timestamp corresponding to the initial text in the initial resume video. The timestamp corresponding to the initial text in the initial resume video can be the timestamp corresponding to each text unit in the initial text in the initial resume video. The timestamp corresponding to any text unit includes: a start timestamp and an end timestamp.

[0039] After obtaining the initial text, in step 103, the resume editing tool can optimize the initial text to obtain the target text. In this embodiment, optimizing the initial text refers to adjusting the initial text in terms of structure and / or content to improve the quality of the text. Optionally, the methods for optimizing the initial text may include but are not limited to at least one of the following: checking grammar and spelling in the initial text, simplifying the language of the initial text to improve readability, improving the logic of sentences in the initial text to enhance expression, deleting sensitive words in the initial text, adding keywords that are conducive to promotion to the initial text, removing redundant words in the initial text, and adjusting the structure of sentences in the initial text to enhance logic, which will not be listed one by one.

[0040] In some optional embodiments, when the resume editing tool calls the target model to optimize the initial text and obtains the target text, the target model can be used to optimize the initial text. Optionally, the target model can be a large language model with a parameter amount greater than a set threshold. In this embodiment, the resume editing tool can construct prompt words based on the initial text, and call the target model based on the prompt words, so that the target model extracts valid text from the initial text under the instruction of the prompt words to obtain the target text.

[0041] Among them, the large language model is a natural language processing (NLP) model that has been trained on a large scale. Large language models are usually built based on deep learning technology and trained on large-scale training data sets, so that they can show strong performance when processing natural language tasks. Given a piece of text (ie, context), the large language model will try to predict the words that are most likely to appear next. The number of parameters of the large language model is greater than the set threshold, which is usually on the order of millions or billions. During the training process, these parameters are continuously adjusted and optimized based on the difference between the predicted results of the large language model and the actual results to improve the prediction accuracy of the large language model. In some embodiments, the large language model usually uses an advanced neural network architecture, such as a Transformer architecture, to build a model structure. This model structure enables the large language model to capture complex patterns in the text and handle long-distance dependencies. After sufficient training, the large language model has a strong generation capability and can produce coherent and contextual text content based on given prompts.

[0042] In this embodiment, a large language model pre-trained on a large number of general data sets can be used as the base model of the target model. To adapt to the specific application scenario of the present application, that is, for optimizing the text containing resume introduction information, the pre-trained large language model can be fine-tuned on a large number of data sets containing resume introduction information, so that the fine-tuned large language model is more suitable for performing the task of optimizing the text containing resume introduction information.

[0043] In this implementation, the resume editing tool can quickly optimize the initial text through the target model, and can generate high-quality target text based on the powerful content processing capabilities of the target model.

[0044] After obtaining the optimized target text based on the above-mentioned implementation, in step 104, the resume editing tool can determine the timestamp corresponding to the target text in the initial resume video based on the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text. In this embodiment, the correspondence between the initial text and the target text can be expressed as the correspondence between the initial text unit in the initial text and the target text unit in the target text before and after optimization. For example, after the initial text unit A1 in the initial text is optimized, the target text unit A2 in the target text is obtained, then it can be determined that the initial text unit A1 and the target text unit A2 have a correspondence. Among them, the text unit is the smallest unit that constitutes the text, which can be a single word or a phrase, and this embodiment does not make any restrictions.

[0045] The timestamp corresponding to the initial text in the initial resume video refers to the timestamp corresponding to each initial text unit in the initial text in the initial resume video, for example, the timestamp T1={T1-S, T1-E} of the initial text unit A1, where T1-S is the start timestamp corresponding to the initial text unit A1, and T1-E is the end timestamp corresponding to the initial text unit A1. For any target text unit in the target text unit, the resume editing tool can determine the initial text unit corresponding to the target text unit in the initial text, and use the timestamp corresponding to the initial text unit in the initial resume video as the timestamp corresponding to the target text unit in the initial resume video. For example, based on the correspondence between the initial text unit A1 and the target text unit A2, the timestamp T1={T1-S, T1-E} can be used as the timestamp of the target text unit A2.

[0046] In some optional embodiments, when the resume editing tool determines the timestamp corresponding to the target text in the initial resume video based on the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text, it may first perform a text unit matching operation on the initial text and the target text, and then determine the timestamp corresponding to the target text unit in the initial resume video based on the matching result. A specific exemplary description will be given below.

[0047] Optionally, the resume editing tool may identify multiple groups of sentences having corresponding relationships among the target text sentences contained in the target text and the initial text sentences contained in the initial text, wherein any group of sentences includes an initial text sentence and a target text sentence whose similarity is greater than a set similarity threshold. The initial text sentence and the target text sentence have a corresponding relationship, which means that the target text sentence is obtained by optimizing the initial text sentence.

[0048] In some optional embodiments, before performing voice recognition on the initial resume video, the resume editing tool may identify silent periods in the initial resume video, and segment the initial resume video according to the silent periods to obtain multiple initial video segments. The resume editing tool may perform voice recognition on the multiple initial video segments respectively, obtain initial text paragraphs corresponding to the multiple initial video segments, and optimize the multiple initial text paragraphs respectively. When identifying multiple groups of sentences with corresponding relationships, the resume editing tool may identify sentences with corresponding relationships paragraph by paragraph according to the corresponding relationship between the optimized text paragraphs and the text paragraphs before optimization.

[0049] In other optional embodiments, if the initial resume video is not segmented according to the silent period, the resume editing tool can perform overall voice recognition on the initial resume video to obtain the initial text. In this case, optionally, the resume editing tool can use the method of matching paragraphs first and then matching sentences to determine multiple groups of sentences with corresponding relationships. In the process of identifying multiple groups of sentences with corresponding relationships, the resume editing tool can segment the initial text and the target text respectively to obtain multiple initial text paragraphs and multiple target text paragraphs. Optionally, the resume editing tool can determine the paragraph segmentation points in the initial text according to the timestamps corresponding to the text units in the initial text in the initial resume video. For example, if the difference between the start timestamp of the previous text unit and the end timestamp of the next text unit in any two adjacent text units in the initial text is greater than a certain time threshold, such as greater than 300ms, then the position between the two text units can be determined as the paragraph segmentation point. The resume editing tool can determine the paragraph segmentation points in the target text according to the paragraph segmentation points in the initial text. Optionally, the resume editing tool may determine the context of the paragraph division point in the initial text, and the context may be the first N text units and the last M text units of the paragraph division point, where N and M are positive integers. The resume editing tool may perform context matching in the target text to obtain two groups of text units that match the first N text units and the last M text units. If the two groups of text units are continuous, the position between the two groups of text units may be used as the paragraph division point in the target text unit. The resume editing tool may segment the initial text and the target text according to the paragraph division points in the initial text and the paragraph division points in the target text, respectively, to obtain multiple initial text paragraphs and multiple target text paragraphs.

[0050] The resume editing tool can perform similarity matching between multiple initial text paragraphs and multiple target text paragraphs to obtain multiple groups of text paragraphs, and any group of text paragraphs contains an initial text paragraph and a target text paragraph whose similarity is greater than a set similarity threshold. In any group of text paragraphs, the resume editing tool can segment the initial text paragraphs and the target text paragraphs respectively to obtain multiple initial text sentences and multiple target text sentences. The resume editing tool can perform similarity matching between multiple initial text sentences and multiple target text sentences to obtain multiple groups of sentences with corresponding relationships. Based on this implementation, the sentences in the optimized target text can be accurately aligned with the sentences in the initial text before optimization, so as to determine the timestamps corresponding to the text units in the target text in the initial resume video.

[0051] After identifying multiple groups of sentences with corresponding relationships based on the above implementation, in any group of text sentences, the resume editing tool can segment the initial text sentences and the target text sentences respectively to obtain multiple initial text units and multiple target text units. The resume editing tool can compare the initial text sentences and the target text sentences word by word based on the multiple initial text units and the multiple target text units to obtain the initial text unit corresponding to each text unit in the target text sentence in the initial text sentence, and determine the timestamp corresponding to each text unit in the target text sentence in the initial resume video based on the timestamp corresponding to the initial text unit in the initial resume video.

[0052] After obtaining the timestamp corresponding to the target text in the initial resume video based on the above implementation, in step 105, the resume editing tool can edit the initial resume video according to the timestamp corresponding to the target text in the initial resume video to obtain multiple video clips corresponding to the target text, and the multiple video clips correspond to the target text containing multiple target text units. For example, the video clip corresponding to the timestamp T1={T1-S, T1-E} in the initial resume video can be edited as the video clip corresponding to the target text unit A2.

[0053] In some scenarios, the initial resume video recorded by the job seeker may have text incoherence problems caused by background noise, filler words (such as "ah", "um" and other catchphrases), repeated words, self-correction after errors, and other factors. In some optional embodiments, when the resume editing tool edits the initial resume video according to the timestamp corresponding to the target text in the initial resume video, the sentence division position of the target text can be updated according to the timestamp corresponding to the target text in the initial resume video to re-divide the sentences in the target text. Optionally, the resume editing tool can determine the timestamps corresponding to multiple text units in the target text sentence in the initial resume video. For any two adjacent text units among the multiple text units, if the difference between the end timestamp of the previous text unit and the start timestamp of the next text unit is greater than the set time threshold, the position between the previous text unit and the next text unit can be used as the sentence division position. Among them, the set time threshold can be determined based on the average duration of each text unit spoken by humans. For example, in the Chinese scenario, the set time threshold can be 250ms. That is to say, in two adjacent text units, if the difference between the end timestamp of the previous text unit and the start timestamp of the next text unit is greater than 250ms, it can be considered that the previous text unit and the next text unit are sentence division positions, and the previous text unit and the next text unit belong to different sentences. Based on this method, the sentence boundaries can be readjusted to ensure that the information conveyed by each sentence is more accurate.

[0054] After determining the updated sentence division position based on the above implementation, the resume editing tool can identify the text unit at the end of the sentence according to the updated sentence division position, and postpone the end timestamp of the text unit at the end of the sentence. In this implementation, the resume editing tool can use the text unit before the updated sentence division position as the text unit at the end of the sentence. Assuming that the end timestamp of the text unit A2 at the end of the sentence is T1-E, the resume editing tool can extend the end timestamp backward by a specified duration, for example, it can be extended to T1-E+200ms. Based on this implementation, the pause effect between sentences can be retained, thereby improving the overall viewing experience of the resume video synthesized subsequently. Afterwards, the resume editing tool can edit the initial resume video according to the timestamp corresponding to the target text in the initial resume video and the postponed end timestamp of the text unit at the end of the sentence, and obtain multiple video clips corresponding to the target text. For example, taking the target text unit A2 as an example, the resume editing tool can edit the video clips from T1-S, T1-E+200ms in the initial video as the video clips corresponding to the target text unit A2.

[0055] After obtaining multiple video clips corresponding to the target text based on the above implementation, in step 106, the resume editing tool can synthesize the multiple video clips to obtain the target resume video. In the process of synthesizing the multiple video clips, multimedia editing technology (such as FFmpeg technology) can be called to seamlessly splice the multiple video clips, and speech recognition can be performed on the spliced ​​video again to recognize the audio content in the spliced ​​video as subtitle text. Afterwards, the subtitle text can be synthesized with the spliced ​​video clips to obtain the target video.

[0056] In some optional embodiments, the initial resume video includes: at least one question information, and the at least one question information is used to guide the job seeker to provide job-seeking related introduction information from at least one direction. For example, in some scenarios, when the job seeker records the initial resume video, a digital human can be used to send at least one question information to the user, thereby guiding the job seeker to record a high-quality resume video. Optionally, when the resume editing tool synthesizes multiple video clips to obtain the target resume video, it can obtain the video material corresponding to each of the at least one question information. Among them, the video material can be obtained from a specified material library. After obtaining the video material corresponding to each question information, the resume editing tool can add the subtitle text corresponding to the question information to the video material to form the effect of synchronous display of the question information and the screen. The resume editing tool identifies the display order between the multiple video clips and the at least one question information. Optionally, when the resume editing tool identifies the display order between the multiple video clips and the at least one question information, the start timestamp and the end timestamp corresponding to each question information can be identified in the initial resume video. For any question information, if the start timestamp of the question information is less than the end timestamp of a certain video clip, and the end timestamp of the question information is greater than the start timestamp of the next adjacent video clip, then it can be determined that the display order of the question information is between the two video clips. If the end timestamp of a certain question information is greater than the start timestamp of the first video clip in the target text, then it can be determined that the display order of the question information is: the first display. After determining the display order between multiple video clips and the at least one question information based on the above method, the resume editing tool can synthesize the video material corresponding to each of the at least one question information with the multiple video clips according to the display order to obtain the target resume video.

[0057] In the above-mentioned embodiments of the present application, the resume editing tool can respond to the resume editing operation, perform automated speech recognition and video editing processes on the job seeker's initial resume video, and after extracting the initial text in the initial resume video, optimize the initial text using a target model method, effectively improving the accuracy of the text information recognized from the initial resume video, thereby facilitating the improvement of the copywriting quality of the generated target resume video. In addition, according to the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text, the timestamp corresponding to the target text in the initial resume video can be determined, and then the initial resume video can be finely and automatically edited according to the timestamp corresponding to the target text, ensuring the consistency and strict alignment characteristics of the text content and the video content, while improving the editing efficiency, ensuring the synchronization effect of the sound and picture of the edited target video. Based on this implementation method, for job seekers, without having professional video editing skills, they can easily and efficiently produce high-quality video resumes, greatly reducing the editing threshold of video resumes.

[0058] The following will be combined Figure 2 , the video resume generation method provided in the embodiment of the present application is further illustrated. Figure 2 As shown, the resume editing tool for generating video resumes mainly includes: video segmentation module, text optimization module, question and video correspondence module, video subtitle merging module, and video merging module.

[0059] Among them, the video segmentation module is used to obtain the initial resume video provided by the job seeker (such as a cleaning staff or a housekeeping staff). The initial resume video is obtained by recording the process of the job seeker introducing job-related information and answering questions under the guidance of preset questions. After obtaining the initial resume video, the silent period in the video is identified by judging whether the duration of the segment without effective sound in the video is greater than the set threshold. After identifying the silent period, the silent period can be cropped from the initial resume video to obtain a video segment containing the job seeker's effective answer. FunASR (an automatic speech recognition tool) is used to convert the audio in these video segments into text, and the timestamp corresponding to each word in the text is extracted.

[0060] The text optimization module is used to call a large language model to optimize each segment of text converted from the audio according to the provided prompt words, and match the optimized text with the original text sentence by sentence to determine the timestamp of each converted word. The text optimization module can also re-divide the optimized sentences according to these timestamps.

[0061] Among them, the question-video correspondence module is used to match the video clips with each question according to the start and end time of each question and the corresponding video clip, and generate a transition video for each question to enhance the coherence of the video resume.

[0062] Among them, the video subtitle merging module is used to use FFmpeg technology to seamlessly splice the video clips that have been edited twice, and then use FunASR to perform automatic speech recognition on the spliced ​​video to convert the audio into subtitle text and merge it with the video.

[0063] Among them, the video merging module is used to merge the transition video with the subtitle video using FFmpeg, and finally generate a complete video resume.

[0064] exist Figure 2 In the method shown, the high-precision Chinese speech recognition model FunASR is used to realize automatic speech recognition of video content, and the recognized text is optimized using a large language model to improve the fluency and simplicity of the video resume. The FFmpeg tool is used to automatically crop and splice videos, and finally generate high-quality and professional job seeker video resumes.

[0065] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 104 can be device A; for another example, the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0066] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0068] Figure 3 The structure diagram of an electronic device provided by an exemplary embodiment of the present application is shown, and the electronic device is applicable to the video resume generation method provided by the above embodiment. Figure 3 As shown, the electronic device includes: a memory 301 , a processor 302 and a communication component 303 .

[0069] The memory 301 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method operating on the electronic device.

[0070] The processor 302 is coupled to the memory 301 and is used to execute the computer program in the memory 301, so as to: respond to a resume editing operation, obtain an initial resume video of a job seeker, wherein the initial resume video contains job introduction information; perform voice recognition on the initial resume video to obtain an initial text corresponding to the job introduction information, and obtain a timestamp corresponding to the initial text in the initial resume video; call a target model to optimize the initial text to obtain a target text; determine a timestamp corresponding to the target text in the initial resume video based on a timestamp corresponding to the initial text in the initial resume video and a correspondence between the initial text and the target text; edit the initial resume video based on a timestamp corresponding to the target text in the initial resume video to obtain multiple video clips corresponding to the target text; and perform synthesis processing on the multiple video clips to obtain a target resume video.

[0071] Optionally, the target model is a large language model whose parameter amount is greater than a set threshold; when the processor 302 calls the target model to optimize the initial text to obtain the target text, it is specifically used to: construct prompt words according to the initial text; call the target model according to the prompt words, so that the target model extracts valid text from the initial text under the instruction of the prompt words to obtain the target text.

[0072] Optionally, when the processor 302 determines the timestamp corresponding to the target text in the initial resume video based on the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text, it is specifically used to: identify multiple groups of sentences with corresponding relationships among the target text sentences contained in the target text and the initial text sentences contained in the initial text, any group of sentences containing an initial text sentence and a target text sentence whose similarity is greater than a set similarity threshold; in any group of text sentences, compare the initial text sentences and the target text sentences word by word to obtain the initial text unit corresponding to each text unit in the target text sentence in the initial text sentence; determine the timestamp corresponding to each text unit in the target text sentence in the initial resume video based on the timestamp corresponding to the initial text unit in the initial resume video.

[0073] Optionally, when the processor 302 identifies multiple groups of sentences having a corresponding relationship among the target text sentences contained in the target text and the initial text sentences contained in the initial text, it is specifically used to: determine the paragraph segmentation points in the initial text according to the timestamps corresponding to the text units in the initial text in the initial resume video, and determine the paragraph segmentation points in the target text according to the paragraph segmentation points in the initial text; segment the initial text and the target text according to the paragraph segmentation points in the initial text and the paragraph segmentation points in the target text, respectively, to obtain multiple initial text paragraphs and multiple target text paragraphs; perform similarity matching between the multiple initial text paragraphs and the multiple target text paragraphs, to obtain multiple groups of text paragraphs, any group of text paragraphs contains an initial text paragraph and a target text paragraph whose similarity is greater than a set similarity threshold; in any group of text paragraphs, segment the initial text paragraph and the target text paragraph, respectively, to obtain multiple initial text sentences and multiple target text sentences; perform similarity matching between the multiple initial text sentences and the multiple target text sentences, to obtain multiple groups of sentences having a corresponding relationship.

[0074] Optionally, when the processor 302 clips the initial resume video according to the timestamp corresponding to the target text in the initial resume video to obtain multiple video clips corresponding to the target text, it is specifically used to: update the sentence division position of the target text according to the timestamp corresponding to the target text in the initial resume video to re-divide the sentences in the target text; identify the text unit at the end of the sentence according to the updated sentence division position, and postpone the end timestamp of the text unit at the end of the sentence; clip the initial resume video according to the timestamp corresponding to the target text in the initial resume video and the postponed end timestamp of the text unit at the end of the sentence to obtain multiple video clips corresponding to the target text.

[0075] Optionally, when the processor 302 updates the sentence division position of the target text according to the timestamp corresponding to the target text in the initial resume video, it is specifically used to: determine the timestamps corresponding to multiple text units in the target text sentence in the initial resume video; for any two adjacent text units among the multiple text units, if the difference between the end timestamp of the previous text unit and the start timestamp of the next text unit is greater than a set time threshold, then the position between the previous text unit and the next text unit is used as the sentence division position.

[0076] Optionally, the initial resume video includes: at least one question information, and the at least one question information is used to guide the job seeker to provide job-seeking related introductory information from at least one direction; when the processor 302 synthesizes the multiple video clips to obtain the target resume video, it is specifically used to: obtain the video material corresponding to each of the at least one question information; identify the display order between the multiple video clips and the at least one question information; according to the display order, synthesize the video material corresponding to each of the at least one question information with the multiple video clips to obtain the target resume video.

[0077] Further, if Figure 3 As shown, the electronic device also includes: a power supply component 304, a display component 305, an audio component 306 and other components. Figure 3 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 3 Components shown. Figure 3Components in the dashed box are optional components, not mandatory components, and may depend on the product form of the electronic device. The electronic device of this embodiment may be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or may be a server device such as a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it may include Figure 3 If the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 3 Components within the dashed box.

[0078] Among them, the memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0079] Among them, the communication component 303 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as 2G (such as Global System for Mobile Communications (GSM)), 3G (such as Wideband Code Division Multiple Access (WCDMA), 4G (such as Long Term Evolution (LTE), etc.), 4G+ (such as upgraded version of Long Term Evolution (LTE-Advanced, LTE-A), etc.) or 5G (fifth generation mobile communication technology (5th Generation Mobile Communication Technology)), or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0080] The power supply component 304 is used to provide power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power to the device where the power supply component is located.

[0081] The display assembly includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0082] The audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0083] In this embodiment, in response to the resume editing operation, the automatic speech recognition and video editing process can be performed on the job seeker's initial resume video. After the initial text in the initial resume video is extracted, the initial text is optimized using a target model method, which effectively improves the accuracy of the text information recognized from the initial resume video, thereby facilitating the improvement of the copywriting quality of the generated target resume video. In addition, according to the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text, the timestamp corresponding to the target text in the initial resume video can be determined, and then the initial resume video can be finely and automatically edited according to the timestamp corresponding to the target text, ensuring the consistency and strict alignment characteristics of the text content and the video content, while improving the editing efficiency, ensuring the synchronization effect of the sound and picture of the edited target video. Based on this implementation method, job seekers can easily and efficiently produce high-quality video resumes without having professional video editing skills, which greatly reduces the editing threshold of video resumes.

[0084] Accordingly, the embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed, each step that can be performed by the electronic device in the above method embodiment can be implemented. Among them, the computer-readable storage medium includes volatile or non-volatile or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (Phase-change Random Access Memory, PRAM), static random access memory (SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), other types of random access memory (Random-Access Memory, RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (Digital Video Disc, DVD) or other optical storage, magnetic cassette, tape disk storage or other magnetic storage device or any other non-transmission medium.

[0085] The present application embodiment also provides a computer program product, including: a computer program / instruction, which can implement the steps in the method provided in the present application embodiment when the computer program / instruction is executed by a processor. It should be understood that each process or a combination of multiple processes in the above method flow can be implemented by a computer program or instruction. In addition, these computer programs or instructions can be applied to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device, so that the processor of the general-purpose computer, the special-purpose computer, the embedded processor or other programmable data processing device can be implemented as a device for implementing the corresponding functions in the above method embodiment.

[0086] It should also be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, product or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, product or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, product or device including the element.

[0087] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for generating a video resume, characterized in that: include: In response to the resume editing operation, an initial resume video of the job seeker is obtained, wherein the initial resume video includes job application introduction information; Performing voice recognition on the initial resume video to obtain the initial text corresponding to the job introduction information, and obtaining the timestamp corresponding to the initial text in the initial resume video; Calling the target model to optimize the initial text to obtain a target text; Determine the timestamp corresponding to the target text in the initial resume video according to the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text; According to the timestamp corresponding to the target text in the initial resume video, the initial resume video is edited to obtain multiple video clips corresponding to the target text; The multiple video clips are synthesized to obtain a target resume video.

2. The method according to claim 1, characterized in that The target model is a large language model whose parameter quantity is greater than a set threshold; calling the target model to optimize the initial text to obtain the target text includes: constructing prompt words according to the initial text; The target model is called according to the prompt word, so that the target model extracts valid text from the initial text under the instruction of the prompt word to obtain the target text.

3. The method according to claim 1, characterized in that Determining the timestamp corresponding to the target text in the initial resume video according to the timestamp corresponding to the initial text in the initial resume video and the correspondence between the initial text and the target text includes: Among the target text sentences contained in the target text and the initial text sentences contained in the initial text, identifying a plurality of groups of sentences having a corresponding relationship, any group of sentences including an initial text sentence and a target text sentence whose similarity is greater than a set similarity threshold; In any group of text sentences, the initial text sentence and the target text sentence are compared word by word to obtain the initial text unit corresponding to each text unit in the target text sentence in the initial text sentence; According to the timestamp corresponding to the initial text unit in the initial resume video, the timestamp corresponding to each text unit in the target text sentence in the initial resume video is determined.

4. The method according to claim 3, characterized in that Among the target text sentences included in the target text and the initial text sentences included in the initial text, a plurality of groups of sentences having a corresponding relationship are identified, including: Determine the paragraph segmentation points in the initial text according to the timestamps corresponding to the text units in the initial text in the initial resume video, and determine the paragraph segmentation points in the target text according to the paragraph segmentation points in the initial text; According to the paragraph segmentation points in the initial text and the paragraph segmentation points in the target text, the initial text and the target text are segmented respectively to obtain a plurality of initial text paragraphs and a plurality of target text paragraphs; Performing similarity matching between the multiple initial text paragraphs and the multiple target text paragraphs to obtain multiple groups of text paragraphs, any group of text paragraphs including an initial text paragraph and a target text paragraph with similarity greater than a set similarity threshold; In any group of text paragraphs, the initial text paragraph and the target text paragraph are segmented respectively to obtain a plurality of initial text sentences and a plurality of target text sentences; Similarity matching is performed between the multiple initial text sentences and the multiple target text sentences to obtain multiple groups of sentences with corresponding relationships.

5. The method according to claim 3 or 4, characterized in that: According to the timestamp corresponding to the target text in the initial resume video, the initial resume video is edited to obtain multiple video clips corresponding to the target text, including: According to the timestamp corresponding to the target text in the initial resume video, the sentence division position of the target text is updated to re-divide the sentences in the target text; According to the updated sentence division position, a text unit at the end of the sentence is identified, and an end timestamp of the text unit at the end of the sentence is postponed; According to the timestamp corresponding to the target text in the initial resume video and the delayed end timestamp of the text unit at the end of the sentence, the initial resume video is edited to obtain multiple video clips corresponding to the target text.

6. The method according to claim 5, characterized in that According to the timestamp corresponding to the target text in the initial resume video, the sentence division position of the target text is updated, including: Determine timestamps corresponding to a plurality of text units in the target text sentence in the initial resume video; For any two adjacent text units among the multiple text units, if the difference between the end timestamp of the previous text unit and the start timestamp of the next text unit is greater than a set time threshold, the position between the previous text unit and the next text unit is used as the sentence division position.

7. The method according to any one of claims 1 to 4, characterized in that: The initial resume video includes: at least one question information, the at least one question information is used to guide the job seeker to provide job-seeking related introduction information from at least one direction; The plurality of video clips are synthesized to obtain a target resume video, including: Obtaining video material corresponding to each of the at least one question information; identifying a display order between the plurality of video clips and the at least one question information; According to the display order, the video material corresponding to each of the at least one question information is synthesized with the multiple video clips to obtain the target resume video.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the method described in any one of claims 1 to 7 can be implemented.

10. A computer program product, characterized in that include: A computer program / instruction, which, when executed by a processor, can implement the steps of the method according to any one of claims 1 to 7.

Images