Video generation method, device and electronic equipment
By obtaining the service requests of the user terminal and dynamically synthesize video content, the problem that smart video customer service cannot push personalized videos is solved, and the service effect is improved.
Patent Information
- Application Number
- CN202110256103.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-09
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-03-09
AI Technical Summary
The existing smart video customer service system cannot push personalized videos to users, resulting in poor service effects.
By obtaining the service request of the user terminal, the text content and background pictures of the target service are determined, the text content is converted into pictures and synthesized with the background pictures, the video is generated based on the voice content, and the personalized video is dynamically generated according to the specific needs of the user terminal.
It realizes the generation of matching personalized videos for different user terminals, and improves the service effect of smart video customer service.
Smart Images

Figure CN115052073B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of video technology, and in particular to a video generation method, device and electronic equipment. Background Art
[0002] At present, the customer service system usually implements intelligent video customer service by pre-recording a video clip, and when the user terminal requests the customer service system to provide intelligent customer service, the pre-recorded video clip is played to the user terminal, so that the user terminal can play the pre-recorded video clip. The existing intelligent video customer service has strong limitations and cannot push personalized videos to users. The service effect of intelligent video customer service is poor. Summary of the invention
[0003] The embodiments of the present invention provide a video generation method, device and electronic device to solve the problem that the existing intelligent video customer service has strong limitations and cannot push personalized videos to users, and the service effect of the intelligent video customer service is poor.
[0004] To solve the above technical problems, the present invention is achieved as follows:
[0005] In a first aspect, an embodiment of the present invention provides a video generation method, the method comprising:
[0006] Obtaining a business service request sent by a user terminal, wherein the business service request is used to request a target business service;
[0007] Determine text content corresponding to the target business service, and determine a background image corresponding to the target business service;
[0008] Convert the text content into a first picture, and combine the first picture and the background picture into a second picture;
[0009] The text content is converted into voice content, and a video for returning to the user terminal is generated according to the voice content and the second image.
[0010] Optionally, the determining of text content corresponding to the target business service includes:
[0011] Executing the target business service for the user terminal and obtaining an execution result of the target business service;
[0012] Obtain the text template and index value corresponding to the target business service;
[0013] The execution result is inserted into the position corresponding to the index value in the text template to obtain text content corresponding to the target business service.
[0014] Optionally, converting the text content into a first picture includes:
[0015] Determine target text parameters and target number of lines according to the text content, wherein the target text parameters are used to represent the text size;
[0016] Determine the text area in the picture template according to the target text parameter and the target number of lines, the picture template being the same size as the background picture;
[0017] Determining the text display color according to the color attribute parameters of the position corresponding to the text area in the background image;
[0018] The text content is filled into the text area in the picture template according to the text display parameters to obtain a first picture, wherein the text display parameters include the target text parameters, the target number of lines and the text display color.
[0019] Optionally, determining target text parameters and target number of lines according to the text content includes:
[0020] Determine at least one candidate character parameter and a corresponding line number, wherein the candidate character parameter and the line number satisfy a first condition and a second condition;
[0021] Determine a target text parameter from the at least one candidate text parameter, and determine a target number of lines corresponding to the target text parameter;
[0022] Among them, the first condition is: P*(1+L)≤R y *Y, P are candidate text parameters, L is the number of lines, R y is the preset width ratio, and Y is the width of the background image;
[0023] The second condition is: N is the length of the text content, R x is the preset length ratio, and X is the length of the background image.
[0024] Optionally, the color attribute parameter is an RGB parameter, and determining the text display color according to the color attribute parameter of the position corresponding to the text area in the background image includes:
[0025] Obtain RGB parameters of multiple pixel points at positions corresponding to the text area in the background image;
[0026] Determine average RGB parameters of the plurality of pixels according to the RGB parameters of the plurality of pixels;
[0027] Determine a saturation value and a brightness value according to the average RGB parameters of the plurality of pixels;
[0028] The text display color is determined based on a first threshold, wherein the first threshold is the sum of a product of a first preset coefficient and the saturation value and a product of a second preset coefficient and the brightness value.
[0029] Optionally, synthesizing the first picture and the background picture into a second picture includes:
[0030] Merging the first picture and the background picture to obtain a merged picture;
[0031] Extracting a plurality of edge pixel points at positions corresponding to the text area in the merged image based on an edge detection algorithm;
[0032] Obtaining grayscale values of a target edge pixel and a plurality of adjacent pixels adjacent to the target edge pixel, wherein the target edge pixel is any one of the plurality of edge pixels;
[0033] Based on the grayscale values of the target edge pixel point and the multiple adjacent pixel points, weighted average filtering is performed on the target edge pixel point in the merged image to obtain the second image.
[0034] Optionally, performing weighted average filtering on the target edge pixel in the merged image based on the grayscale values of the target edge pixel and the plurality of adjacent pixels includes:
[0035] Determine a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels;
[0036] Performing a Hadamard product operation based on an RGB parameter matrix and a weight matrix to obtain a weighted RGB parameter matrix, wherein the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel points and the RGB parameters of the plurality of adjacent pixel points, and the weight matrix is determined based on the difference matrix;
[0037] Determine a merged image after weighted average filtering, wherein RGB parameters of the target edge pixel points in the merged image after weighted average filtering are determined based on the weighted RGB parameter matrix.
[0038] In a second aspect, an embodiment of the present invention provides a video generating device, the device comprising:
[0039] An acquisition module, used to acquire a business service request sent by a user terminal, wherein the business service request is used to request a target business service;
[0040] A determination module, used to determine text content corresponding to the target business service, and determine a background image corresponding to the target business service;
[0041] A synthesis module, used for converting the text content into a first picture, and synthesizing the first picture and the background picture into a second picture;
[0042] A generation module is used to convert the text content into voice content, and generate a video for returning to the user terminal based on the voice content and the second image.
[0043] Optionally, the determining module is specifically used to:
[0044] Executing the target business service for the user terminal and obtaining an execution result of the target business service;
[0045] Obtain the text template and index value corresponding to the target business service;
[0046] Inserting the execution result at the position corresponding to the index value in the text template to obtain text content corresponding to the target business service;
[0047] A background image corresponding to the target business service is determined.
[0048] Optionally, the synthesis module includes:
[0049] A first determination unit, configured to determine target text parameters and a target number of lines according to the text content, wherein the target text parameters are used to represent the text size;
[0050] A second determining unit, configured to determine a text area in a picture template according to the target text parameter and the target number of lines, wherein the picture template has the same size as the background picture;
[0051] A third determining unit, configured to determine a text display color according to a color attribute parameter of a position corresponding to the text area in the background image;
[0052] A filling unit, configured to fill the text content into the text area in the picture template according to text display parameters to obtain a first picture, wherein the text display parameters include the target text parameter, the target number of lines and the text display color;
[0053] A synthesis unit is used to synthesize the first picture and the background picture into a second picture.
[0054] Optionally, the first determining unit is specifically configured to:
[0055] Determine at least one candidate character parameter and a corresponding line number, wherein the candidate character parameter and the line number satisfy a first condition and a second condition;
[0056] Determine a target text parameter from the at least one candidate text parameter, and determine a target number of lines corresponding to the target text parameter;
[0057] Among them, the first condition is: P*(1+L)≤R y *Y, P are candidate text parameters, L is the number of lines, R y is the preset width ratio, and Y is the width of the background image;
[0058] The second condition is: N is the length of the text content, R x is the preset length ratio, and X is the length of the background image.
[0059] Optionally, the color attribute parameter is an RGB parameter, and the third determining unit is specifically used to:
[0060] Obtain RGB parameters of multiple pixel points at positions corresponding to the text area in the background image;
[0061] Determine average RGB parameters of the plurality of pixels according to the RGB parameters of the plurality of pixels;
[0062] Determine a saturation value and a brightness value according to the average RGB parameters of the plurality of pixels;
[0063] The text display color is determined based on a first threshold, wherein the first threshold is the sum of a product of a first preset coefficient and the saturation value and a product of a second preset coefficient and the brightness value.
[0064] Optionally, the synthesis module includes:
[0065] A conversion unit, used to convert the text content into a first picture;
[0066] a merging unit, configured to merge the first image and the background image to obtain a merged image;
[0067] An extraction unit, configured to extract a plurality of edge pixel points at positions corresponding to the text area in the merged image based on an edge detection algorithm;
[0068] An acquisition unit, used to acquire grayscale values of a target edge pixel point and a plurality of adjacent pixel points adjacent to the target edge pixel point, wherein the target edge pixel point is any one edge pixel point among the plurality of edge pixel points;
[0069] A processing unit is used to perform weighted average filtering on the target edge pixel points in the merged image based on the grayscale values of the target edge pixel points and the multiple adjacent pixel points to obtain the second image.
[0070] Optionally, the processing unit is specifically used for:
[0071] Determine a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels;
[0072] Performing a Hadamard product operation based on an RGB parameter matrix and a weight matrix to obtain a weighted RGB parameter matrix, wherein the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel points and the RGB parameters of the plurality of adjacent pixel points, and the weight matrix is determined based on the difference matrix;
[0073] Determine a merged image after weighted average filtering, wherein RGB parameters of the target edge pixel points in the merged image after weighted average filtering are determined based on the weighted RGB parameter matrix.
[0074] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the video generation method described in the first aspect.
[0075] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video generation method described in the first aspect are implemented.
[0076] In an embodiment of the present invention, a business service request sent by a user terminal is obtained, and the business service request is used to request a target business service; the text content corresponding to the target business service is determined, and the background image corresponding to the target business service is determined; the text content is converted into a first image, and the first image and the background image are combined into a second image; the text content is converted into voice content, and a video for returning to the user terminal is generated based on the voice content and the second image. In this way, for different user terminals, a video matching the user terminal is generated, and personalized videos can be pushed to users, thereby improving the service effect of intelligent video customer service. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0078] Figure 1 is a flow chart of a video generation method provided by an embodiment of the present invention;
[0079] Figure 2 It is a partial flow chart of a video generation method provided by an embodiment of the present invention;
[0080] Figure 3 This is one of the structural schematic diagrams of a video generating device provided by an embodiment of the present invention;
[0081] Figure 4 This is a second structural diagram of a video generating device provided by an embodiment of the present invention;
[0082] Figure 5 This is a third structural diagram of a video generating device provided by an embodiment of the present invention;
[0083] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0085] In the embodiments of the present invention, a video generation method, device and electronic device are proposed to solve the problem that the existing intelligent video customer service has strong limitations and cannot push personalized videos to users, and the service effect of the intelligent video customer service is poor.
[0086] See also Figure 1 , Figure 1 is a flow chart of a video generation method provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0087] Step 101: Acquire a business service request sent by a user terminal, where the business service request is used to request a target business service.
[0088] The business service request sent by the user terminal can be obtained from the Interactive Voice and Video Response (IVVR) system. The business service request can carry a service number for representing the target business service. For example, the user terminal can be a mobile phone, and the target business service can be a business service such as checking the balance of phone charges, checking the traffic and package usage.
[0089] Step 102: Determine text content corresponding to the target business service, and determine a background image corresponding to the target business service.
[0090] Among them, the target business service can be executed for the user terminal, and the execution result of the target business service can be obtained. The text content corresponding to the target business service can include the execution result of the target business service. In addition, the background image corresponding to the target business service can be found in the image template library. For example, an SQL statement can be generated according to the service number used to represent the target business service, and the background image corresponding to the target business service can be found in the image template library. In the image template library, different background images can be stored corresponding to different service numbers.
[0091] Step 103: convert the text content into a first picture, and combine the first picture and the background picture into a second picture.
[0092] The first picture may be a picture including the text content. To facilitate picture synthesis, the first picture may be a picture with a transparent background. The first picture and the background picture may be synthesized into the second picture by merging layers. The second picture may be a picture obtained by superimposing the first picture and the background picture.
[0093] Step 104: Convert the text content into voice content, and generate a video for returning to the user terminal based on the voice content and the second image.
[0094] Among them, the text content can be converted into voice content using a TTS (speech synthesis) interface, and the voice length can be obtained. In practical applications, a picture-to-video tool, such as an ffmpeg tool, can be used to convert the second picture into a video, the length of the video is the length of the voice, and in the process of converting the second picture into a video, a 0.5s blank frame without sound can be added before and after the video, and a transition effect can be added, so that the generated video is smoother. A 0.5s blank can be added before and after the voice content, and the voice content with the blank added can be merged with the converted video to generate a video for returning to the user terminal.
[0095] In an embodiment of the present invention, a business service request sent by a user terminal is obtained, and the business service request is used to request a target business service; the text content corresponding to the target business service is determined, and the background image corresponding to the target business service is determined; the text content is converted into a first image, and the first image and the background image are combined into a second image; the text content is converted into voice content, and a video for returning to the user terminal is generated based on the voice content and the second image. In this way, for different user terminals, a video matching the user terminal is generated, and personalized videos can be pushed to users, thereby improving the service effect of intelligent video customer service.
[0096] Optionally, the determining of text content corresponding to the target business service includes:
[0097] Executing the target business service for the user terminal and obtaining an execution result of the target business service;
[0098] Obtain the text template and index value corresponding to the target business service;
[0099] The execution result is inserted into the position corresponding to the index value in the text template to obtain text content corresponding to the target business service.
[0100] Among them, the business service request may also carry a user terminal identifier, and the user terminal identifier may be used to identify the user terminal. Taking the user terminal as a mobile phone as an example, the user terminal identifier may be a mobile phone number. Taking the target business service as a query service as an example, the execution result of the target business service may be a query result, and the execution of the target business service for the user terminal and obtaining the execution result of the target business service may be to execute the query service for the user terminal and obtain the query result. By way of example, one or more data tables may be obtained according to the service number mapping used to characterize the target business service, an SQL statement may be generated according to the data table and the user terminal identifier, a query may be performed in the database using the SQL statement, and the query result may be obtained. Taking the target business service as a call balance query service as an example, the execution result of the target business service is the call balance.
[0101] In addition, the text template and index value corresponding to the target business service can be found in the text template library. For example, an SQL statement can be generated according to the service number used to represent the target business service, and the text template and index value corresponding to the target business service can be found in the text template library. In the text template library, different text templates and index values can be stored correspondingly for different service numbers.
[0102] For example, the text template can be "Dear Mr. Zhang San, your phone bill balance is yuan", the index value is 15, the execution result is 102.67, and the execution result is inserted at the position corresponding to the index value in the text template. The text content can be: "Dear Mr. Zhang San, your phone bill balance is 102.67 yuan".
[0103] In this implementation, by inserting the execution result at the position corresponding to the index value in the text template, text content corresponding to the target business service is obtained, thereby providing users with personalized data display in the generated video, which has a wider range of usage scenarios.
[0104] Optionally, converting the text content into a first picture includes:
[0105] Determine target text parameters and target number of lines according to the text content, wherein the target text parameters are used to represent the text size;
[0106] Determine the text area in the picture template according to the target text parameter and the target number of lines, the picture template being the same size as the background picture;
[0107] Determining the text display color according to the color attribute parameters of the position corresponding to the text area in the background image;
[0108] The text content is filled into the text area in the picture template according to the text display parameters to obtain a first picture, wherein the text display parameters include the target text parameters, the target number of lines and the text display color.
[0109] Among them, the target text parameter can be expressed as a font size, for example, size 4, size 5, etc.; or the target text parameter can also be described by the number of pixels, and there can be a corresponding relationship between the number of pixels and the font size. For example, the number of pixels corresponding to the size 4 is 18, and the number of pixels corresponding to the size 5 is 14. The picture template can be a picture with a transparent bottom. The text area in the picture template can be represented in the form of coordinates. For example, with the long side below the picture template as the x-axis and the wide side on the left of the picture template as the y-axis, the coordinates of the upper left corner of the text area can be: (0.1X, 0.7Y), and the coordinates of the lower right corner of the text area can be: (0.9X, 0.7Y-(L1+1)*P1), X is the length of the background image, Y is the width of the background image, P1 is the target text parameter, and L1 is the target number of rows.
[0110] In addition, the text content is filled into the text area in the picture template according to the text display parameters to obtain a first picture, in which the text content is arranged in the text area with a target number of rows, displayed in a text display color, and a display size that is a target text parameter.
[0111] In this implementation, the first picture is obtained by filling the text content into the text area in the picture template according to the text display parameters, so that the video screen generated based on the first picture is more beautiful and has better user-friendliness.
[0112] Optionally, determining target text parameters and target number of lines according to the text content includes:
[0113] Determine at least one candidate character parameter and a corresponding line number, wherein the candidate character parameter and the line number satisfy a first condition and a second condition;
[0114] Determine a target text parameter from the at least one candidate text parameter, and determine a target number of lines corresponding to the target text parameter;
[0115] Among them, the first condition is: P*(1+L)≤R y *Y, P are candidate text parameters, L is the number of lines, R y is the preset width ratio, and Y is the width of the background image;
[0116] The second condition is: N is the length of the text content, R x is the preset length ratio, and X is the length of the background image.
[0117] Where L is a positive integer. 0<R x <1, and 0<R y <1. The candidate text parameters can be used to characterize the text size. The candidate text parameters can be expressed as font size, for example, size 4, size 5, etc.; or the candidate text parameters can also be described by the number of pixels, and there can be a corresponding relationship between the number of pixels and the font size. For example, the number of pixels corresponding to size 4 is 18, and the number of pixels corresponding to size 5 is 14. The preset length ratio can be 60%, or 80%, or 90%, etc. The preset width ratio can be 20%, or 40%, or 60%, etc. Taking the target business service as the bank account balance query service as an example, in order to obtain a better display effect, the length and width ratio of the text content to the image template usually does not exceed 80% and 40%. The preset width ratio can be set to 40%, and the preset length ratio can be set to 80%. When L is 1, 2, or 3, the candidate text parameters that meet the first condition and the second condition are calculated respectively.
[0118] In addition, the determination of the target text parameter from the at least one candidate text parameter may be to select the candidate text parameter with the largest value from the at least one candidate text parameter as the target text parameter; or may be to select the candidate text parameter within a preset value range as the target text parameter, and when there are multiple candidate text parameters within the preset value range, the candidate text parameter with the largest value within the preset value range may be selected as the target text parameter. The preset value range may be set according to actual needs. For example, taking the candidate text parameter described by the number of pixels as an example, the preset value range may be set to between 15 and 25.
[0119] In practical applications, such as Figure 2 As shown, the method of determining at least one candidate text parameter and the corresponding number of lines may include the following process:
[0120] Step a: Initialization;
[0121] In the initialization process, the length and width of the background image and the text length N of the text content are obtained, and the initial value of the number of lines L is set to 1;
[0122] Step b: Determine whether the number of rows L is less than or equal to 3; if so, execute step c; if not, end;
[0123] Step c: obtaining candidate text parameters that meet the second condition;
[0124] Step d: determining whether the obtained candidate text parameters meet the first condition; if so, storing the corresponding relationship between the candidate text parameters and the number of lines; if not, executing step e;
[0125] Step e: Add 1 to the value of the number of rows L and execute step b.
[0126] The candidate text parameters and the corresponding number of rows can be stored in the form of a tuple. If there is a candidate text parameter within the preset value range, the candidate text parameter with the largest value within the preset value range can be selected as the target text parameter; if there is no candidate text parameter within the preset value range, the candidate text parameter with the smallest absolute difference from the target value can be selected as the target text parameter. The number of rows corresponding to the selected candidate text parameter can be used as the target number of rows. The target value can be set according to actual needs. For example, taking the candidate text parameter described by the number of pixels as an example, the target value can be 18.
[0127] Taking the example of describing the candidate text parameters with the number of pixels, there may be a corresponding relationship between the number of pixels and the font size. After determining the target text parameters, the text content may be displayed in the image template with the font size corresponding to the target text parameters. The font size corresponding to the number of pixels closest to the target text parameters may be used as the font size corresponding to the target text parameters. The correspondence between the number of pixels and the font size may include the correspondence between Chinese character size, English character size and the number of pixels. When the text content is Chinese, the Chinese character size may be selected; when the text content is English, the English character size may be selected. By way of example, the correspondence between the Chinese character size, English character size and the number of pixels may be as shown in Table 1:
[0128] Table 1
[0129]
[0130] In this implementation, at least one candidate text parameter and the corresponding number of lines are determined, the candidate text parameter and the number of lines satisfy the first condition and the second condition, a target text parameter is determined from the at least one candidate text parameter, and the target number of lines corresponding to the target text parameter is determined. In this way, the text display parameters can be adaptively adjusted based on the size of the background image, so that the generated video has a better display effect.
[0131] Optionally, the color attribute parameter is an RGB parameter, and determining the text display color according to the color attribute parameter of the position corresponding to the text area in the background image includes:
[0132] Obtain RGB parameters of multiple pixel points at positions corresponding to the text area in the background image;
[0133] Determine average RGB parameters of the plurality of pixels according to the RGB parameters of the plurality of pixels;
[0134] Determine a saturation value and a brightness value according to the average RGB parameters of the plurality of pixels;
[0135] The text display color is determined based on a first threshold, wherein the first threshold is the sum of a product of a first preset coefficient and the saturation value and a product of a second preset coefficient and the brightness value.
[0136] Among them, the position corresponding to the text area in the background image can be the position of the text area of the first image in the background image after the first image and the background image are merged. The average RGB parameter of the multiple pixels can be the average value of the RGB parameters of the multiple pixels, and the average RGB parameters of the multiple pixels can include the average R parameter value of the multiple pixels, the average B parameter value of the multiple pixels, and the average G parameter value of the multiple pixels. The average R parameter value of the multiple pixels can be the average value of the R parameter values of the multiple pixels, the average G parameter value of the multiple pixels can be the average value of the G parameter values of the multiple pixels, and the average B parameter value of the multiple pixels can be the average value of the B parameter values of the multiple pixels. The saturation value and the brightness value can be determined according to the average RGB parameters of the multiple pixels by the existing RGB to HSV method.
[0137] In addition, the saturation value (S) and the lightness value (V) are color parameters of the HSV color space. The smaller the saturation value or the larger the lightness value, the lighter the color; the larger the saturation value or the smaller the lightness value, the darker the color. The first preset coefficient can be greater than the second preset coefficient. The first preset coefficient can be 0.7, or 0.8, or 0.9, etc. The second preset coefficient can be 0.3, or 0.2, or 0.1, etc. For example, the first preset coefficient is 0.8 and the second preset coefficient is 0.2. When the first threshold is greater than the preset threshold, it can be judged that the position corresponding to the text area in the background image is dark; when the first threshold is less than or equal to the preset threshold, it can be judged that the position corresponding to the text area in the background image is light. The preset threshold can be set according to the actual situation. When the first preset coefficient is 0.8 and the second preset coefficient is 0.2, the preset threshold can be taken as 200. In order to highlight the text display effect, when it is determined that the position corresponding to the text area in the background image is dark, the text display color can be determined to be light; when it is determined that the position corresponding to the text area in the background image is light, the text display color can be determined to be dark.
[0138] In this implementation, the RGB parameters of multiple pixels at the position corresponding to the text area in the background image are obtained; the average RGB parameters of the multiple pixels are determined based on the RGB parameters of the multiple pixels; the saturation value and the brightness value are determined based on the average RGB parameters of the multiple pixels; and the text display color is determined based on the first threshold. In this way, the text display color can be adaptively determined according to the color of the position corresponding to the text area in the background image, so that the generated video has a better display effect.
[0139] Optionally, synthesizing the first picture and the background picture into a second picture includes:
[0140] Merging the first picture and the background picture to obtain a merged picture;
[0141] Extracting a plurality of edge pixel points at positions corresponding to the text area in the merged image based on an edge detection algorithm;
[0142] Obtaining grayscale values of a target edge pixel and a plurality of adjacent pixels adjacent to the target edge pixel, wherein the target edge pixel is any one of the plurality of edge pixels;
[0143] Based on the grayscale values of the target edge pixel point and the multiple adjacent pixel points, weighted average filtering is performed on the target edge pixel point in the merged image to obtain the second image.
[0144] Among them, the text area can be an area where the text content is located, and specifically, it can be a text area in a picture template used to generate the first picture. The edge detection algorithm can be used to extract the edge of the text area. By way of example, the edge detection algorithm can be a Canny algorithm. The extracted edge pixels can be stored in a list. Before obtaining the grayscale values of the target edge pixel point and the multiple adjacent pixel points adjacent to the target edge pixel point, the image corresponding to the text area in the merged picture can be extracted and converted into a grayscale image. By way of example, the multiple adjacent pixel points adjacent to the target edge pixel point can be 8 adjacent pixel points adjacent to the target edge pixel point. By way of example, the 8 adjacent pixel points can be 8 adjacent pixel points surrounding the target edge pixel point with the target edge pixel point as the center.
[0145] In addition, the weighted average filtering processing of the target edge pixel in the merged image based on the grayscale value of the target edge pixel and the multiple adjacent pixels may be: determining a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale value of the multiple adjacent pixels, performing a Hadamard product operation based on the RGB parameter matrix and the weight matrix to obtain a weighted RGB parameter matrix, the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel and the RGB parameters of the multiple adjacent pixels, the weight matrix is determined based on the difference matrix, and the merged image after the weighted average filtering processing is determined, wherein the RGB parameters of the target edge pixel in the merged image after the weighted average filtering processing are determined based on the weighted RGB parameter matrix; or it may be that the mean of the grayscale values of the multiple adjacent pixels is calculated, and the ratio of the grayscale value of the target edge pixel to the mean is calculated, and the RGB parameters of the target edge pixel in the second image are the product of the RGB parameters of the target edge pixel in the merged image and the ratio.
[0146] In this implementation, weighted average filtering is performed on the target edge pixels in the merged image based on the grayscale values of the target edge pixels and the multiple adjacent pixels, which can make the edge of the text area smoother, thereby achieving a better fusion effect between the text content and the background image.
[0147] Optionally, performing weighted average filtering on the target edge pixel in the merged image based on the grayscale values of the target edge pixel and the plurality of adjacent pixels includes:
[0148] Determine a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels;
[0149] Performing a Hadamard product operation based on an RGB parameter matrix and a weight matrix to obtain a weighted RGB parameter matrix, wherein the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel points and the RGB parameters of the plurality of adjacent pixel points, and the weight matrix is determined based on the difference matrix;
[0150] Determine a merged image after weighted average filtering, wherein RGB parameters of the target edge pixel points in the merged image after weighted average filtering are determined based on the weighted RGB parameter matrix.
[0151] Wherein, taking the case where the plurality of adjacent pixels are 8 adjacent pixels as an example, the difference matrix may be a matrix of 3 rows and 3 columns, the value of the 2nd row and 2nd column in the difference matrix may be 0, which is used to characterize the target edge pixel, and the remaining 8 values are respectively the absolute values of the difference between the grayscale value of the target edge pixel and the grayscale value of the plurality of adjacent pixels. The weight matrix may be a matrix of 3 rows and 3 columns, the value of the 2nd row and 2nd column in the weight matrix may be a first preset value, which is used to characterize the target edge pixel, and the values in the difference matrix are sorted from large to small, and the corresponding positions of the adjacent pixels corresponding to the four larger values in the weight matrix may be set to the second preset value, and the corresponding positions of the adjacent pixels corresponding to the four smaller values in the weight matrix may be set to the third preset value. The first preset value, the second preset value and the third preset value may be set according to actual needs. For example, the first preset value may be 4, the second preset value may be 1, and the third preset value may be 2.
[0152] Furthermore, the RGB parameter matrix may include an R parameter matrix, a B parameter matrix, and a G parameter matrix. Taking the R parameter matrix as an example, the R parameter matrix may be a matrix of 3 rows and 3 columns, and the values in the R parameter matrix are respectively the R parameter values of the target edge pixel and its 8 adjacent pixels, and the B parameter matrix and the G parameter matrix may be deduced in the same manner.
[0153] In addition, the RGB parameters of the target edge pixels in the merged picture after the weighted average filtering process may be the sum of the values in the weighted RGB parameter matrix, and the weighted RGB parameter matrix may include a weighted R parameter matrix, a weighted G parameter matrix, and a weighted B parameter matrix. The R parameter in the RGB parameters of the target edge pixels in the merged picture after the weighted average filtering process may be the sum of the values in the weighted R parameter matrix, the G parameter in the RGB parameters of the target edge pixels may be the sum of the values in the weighted G parameter matrix, and the B parameter in the RGB parameters of the target edge pixels may be the sum of the values in the weighted B parameter matrix.
[0154] As a specific implementation, the RGB values of the target edge pixel and the eight adjacent pixels can be represented by the following matrix:
[0155]
[0156] [0 11 9] is the RGB value of the target edge pixel, where 0 is the R parameter value, 11 is the B parameter value, and 9 is the G parameter value. The positional relationship between the RGB values of the target edge pixel and the 8 adjacent pixels in the matrix can be the same as the positional relationship between the target edge pixel and the 8 adjacent pixels in the image. After conversion to a grayscale image, the grayscale values of the target edge pixel and the 8 adjacent pixels can be:
[0157]
[0158] The absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels is calculated, and the difference matrix may be:
[0159]
[0160] The weight matrix is determined based on the difference matrix, and the weight matrix is:
[0161]
[0162] Perform Hadamard product operations on the R parameter matrix, B parameter matrix, and G parameter matrix with the weight matrix to obtain the weighted R parameter matrix, weighted G parameter matrix, and weighted B parameter matrix:
[0163]
[0164]
[0165]
[0166] The RGB parameters of the target edge pixel in the merged image after the weighted average filtering process may be the sum of the values in the weighted RGB parameter matrix:
[0167] R=10+4+9+10+0+0+10+0+0=43
[0168] G=12+6+11+12+2+0+12+0+0=55
[0169] B=12+6+11+11+2+0+12+0+0=54
[0170] In this implementation, a difference matrix is determined based on the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the multiple adjacent pixel points; a Hadamard product operation is performed based on the RGB parameter matrix and the weight matrix to obtain a weighted RGB parameter matrix, the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel and the RGB parameters of the multiple adjacent pixel points, and the weight matrix is determined based on the difference matrix; a merged image after weighted average filtering is determined, thereby achieving conditional weighted average filtering, with better filtering effect, which can make the edge of the text area smoother.
[0171] As a specific implementation, the video generation method can be applied to a video generation device, such as Figure 3 As shown, the video generation device may include a request processing module, a text-to-picture module, a picture synthesis module, a speech synthesis module and a video synthesis module.
[0172] For example, when a user dials a query phone number through a user terminal to check his / her phone balance, the working process of the video generating device may be as follows:
[0173] After the user dials the inquiry number, the IVVR system is connected, and the user terminal plays a welcome video with a prompt "Press 1 to check the balance of phone charges, press 2 to check the balance of data traffic..." The user presses 1 to check the balance of phone charges;
[0174] The video generation device obtains the service request of the user terminal to query the phone balance through IVVR. The service request carries the service number 1. The mapping table can determine that the service request is a request to query the phone balance. Figure 4As shown, the request processing module of the video generation device includes a request image template library service unit, a request text template library service unit and a request database service unit. The request image template library service unit finds the background image corresponding to the call balance query in the image template library, the request text template library service unit finds the text template and index value corresponding to the call balance query in the text template library, and the request database service unit finds the corresponding call balance in the database according to the user terminal identifier carried in the business service request. The request text template library service unit passes the found text template and index value to the text-to-image module as input, the request database service unit passes the found call balance to the text-to-image module as input, and the request image template library service unit passes the background image to the image synthesis module as input.
[0175] The text-to-image module fills the user information and call balance corresponding to the user terminal identifier into the text template according to the index value, generates the text content "Dear Mr. Zhang San, your call balance is 102.67 yuan", and passes the text content to the speech synthesis module as input. y is 0.3, R x is 0.9, the text length N is 20, the background image length X is 360, and the background image width Y is 480. According to the first and second conditions, we can calculate:
[0176] P×(1+L)≤0.3×480
[0177]
[0178] The value range of L is 1, 2 and 3. When L is 1, P is 16, which is a suitable size. The correspondence between the number of pixels and the font size is found, and the font size is 4. The average HSV value of the background image is [99 132 234], the saturation value is 132, and the lightness value is 234. Taking the first preset coefficient as 0.8, the second preset coefficient as 0.2, and the preset threshold value as 200 as an example, the first threshold is calculated to be 152.4. The first threshold is less than 200, which is determined to be a light background. It can be determined that the text color is dark text, and the text color is selected as black. The text-to-image module generates a first image with a transparent text background according to the size of the background image, and passes it to the image synthesis module as input.
[0179] The picture synthesis module merges the first picture and the background picture, superimposes the picture with a transparent bottom on the background picture to synthesize a merged picture, performs edge processing on the merged picture, extracts the edge of the text using the Canny operator, performs weighted average filtering on the edge pixels, reassigns the RGB value of each edge point to obtain the second picture, and passes the second picture to the video synthesis module as input.
[0180] The speech synthesis module uses the existing TTS interface to synthesize speech content from the text content delivered by the text-to-image module, and passes the synthesized speech content to the video synthesis module as input.
[0181] The video synthesis module synthesizes the video using the existing video synthesis tool according to the second picture transmitted by the picture synthesis module and the voice content transmitted by the voice synthesis module. In order to make the video more viewing-friendly, a transition frame is added to make the picture more soothing and smooth as a whole.
[0182] The video generating device returns the generated video to the IVVR for playing on the user terminal.
[0183] In this implementation, a method for dynamically generating a video is provided, so that the user personalized data obtained in real-time can be dynamically generated and presented in a video, and the determination of the text area, font size, and color makes the overall picture more beautiful. The video generation method of the embodiment of the present invention can be used for personalized data display, has a wider range of applicable scenarios, and has economic value.
[0184] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a video generating device provided by an embodiment of the present invention, such as Figure 5 As shown, the video generating device 200 includes:
[0185] An acquisition module 201 is used to acquire a business service request sent by a user terminal, where the business service request is used to request a target business service;
[0186] A determination module 202, configured to determine text content corresponding to the target business service, and determine a background image corresponding to the target business service;
[0187] A synthesis module 203, used for converting the text content into a first picture, and synthesizing the first picture and the background picture into a second picture;
[0188] The generating module 204 is used to convert the text content into voice content, and generate a video for returning to the user terminal according to the voice content and the second image.
[0189] Optionally, the determining module 202 is specifically configured to:
[0190] Executing the target business service for the user terminal and obtaining an execution result of the target business service;
[0191] Obtain the text template and index value corresponding to the target business service;
[0192] Inserting the execution result at the position corresponding to the index value in the text template to obtain text content corresponding to the target business service;
[0193] A background image corresponding to the target business service is determined.
[0194] Optionally, the synthesis module 203 includes:
[0195] A first determination unit, configured to determine target text parameters and a target number of lines according to the text content, wherein the target text parameters are used to represent the text size;
[0196] A second determining unit, configured to determine a text area in a picture template according to the target text parameter and the target number of lines, wherein the picture template has the same size as the background picture;
[0197] A third determining unit, configured to determine a text display color according to a color attribute parameter of a position corresponding to the text area in the background image;
[0198] A filling unit, configured to fill the text content into the text area in the picture template according to text display parameters to obtain a first picture, wherein the text display parameters include the target text parameter, the target number of lines and the text display color;
[0199] A synthesis unit is used to synthesize the first picture and the background picture into a second picture.
[0200] Optionally, the first determining unit is specifically configured to:
[0201] Determine at least one candidate character parameter and a corresponding line number, wherein the candidate character parameter and the line number satisfy a first condition and a second condition;
[0202] Determine a target text parameter from the at least one candidate text parameter, and determine a target number of lines corresponding to the target text parameter;
[0203] Among them, the first condition is: P*(1+L)≤R y *Y, P are candidate text parameters, L is the number of lines, R y is the preset width ratio, and Y is the width of the background image;
[0204] The second condition is: N is the length of the text content, R x is the preset length ratio, and X is the length of the background image.
[0205] Optionally, the color attribute parameter is an RGB parameter, and the third determining unit is specifically used to:
[0206] Obtain RGB parameters of multiple pixel points at positions corresponding to the text area in the background image;
[0207] Determine average RGB parameters of the plurality of pixels according to the RGB parameters of the plurality of pixels;
[0208] Determine a saturation value and a brightness value according to the average RGB parameters of the plurality of pixels;
[0209] The text display color is determined based on a first threshold, wherein the first threshold is the sum of a product of a first preset coefficient and the saturation value and a product of a second preset coefficient and the brightness value.
[0210] Optionally, the synthesis module 203 includes:
[0211] A conversion unit, used to convert the text content into a first picture;
[0212] a merging unit, configured to merge the first image and the background image to obtain a merged image;
[0213] An extraction unit, configured to extract a plurality of edge pixel points at positions corresponding to the text area in the merged image based on an edge detection algorithm;
[0214] An acquisition unit, used to acquire grayscale values of a target edge pixel point and a plurality of adjacent pixel points adjacent to the target edge pixel point, wherein the target edge pixel point is any one edge pixel point among the plurality of edge pixel points;
[0215] A processing unit is used to perform weighted average filtering on the target edge pixel points in the merged image based on the grayscale values of the target edge pixel points and the multiple adjacent pixel points to obtain the second image.
[0216] Optionally, the processing unit is specifically used for:
[0217] Determine a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels;
[0218] Performing a Hadamard product operation based on an RGB parameter matrix and a weight matrix to obtain a weighted RGB parameter matrix, wherein the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel points and the RGB parameters of the plurality of adjacent pixel points, and the weight matrix is determined based on the difference matrix;
[0219] Determine a merged image after weighted average filtering, wherein RGB parameters of the target edge pixel points in the merged image after weighted average filtering are determined based on the weighted RGB parameter matrix.
[0220] The video generating device can realize Figure 1 The various processes implemented in the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.
[0221] like Figure 6 As shown, an embodiment of the present invention further provides an electronic device 300, including: a processor 301, a memory 302, and a program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, each process of the above-mentioned video generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0222] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned video generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not described here. The computer-readable storage medium is, for example, a ROM, RAM, a magnetic disk or an optical disk.
[0223] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0224] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0225] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.
Claims
1. A video generation method, characterized in that: The method comprises: Obtaining a business service request sent by a user terminal, wherein the business service request is used to request a target business service; Determine text content corresponding to the target business service, and determine a background image corresponding to the target business service; Convert the text content into a first picture, and combine the first picture and the background picture into a second picture; converting the text content into voice content, and generating a video for returning to the user terminal according to the voice content and the second image; The synthesizing the first picture and the background picture into a second picture includes: Merging the first picture and the background picture to obtain a merged picture; Extracting a plurality of edge pixel points at positions corresponding to the text area in the merged image based on an edge detection algorithm; Obtaining grayscale values of a target edge pixel and a plurality of adjacent pixels adjacent to the target edge pixel, wherein the target edge pixel is any one of the plurality of edge pixels; Performing weighted average filtering processing on the target edge pixel in the merged image based on the grayscale values of the target edge pixel and the plurality of adjacent pixel points to obtain the second image; The performing weighted average filtering processing on the target edge pixel in the merged image based on the grayscale values of the target edge pixel and the plurality of adjacent pixel points includes: Determine a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels; Performing a Hadamard product operation based on an RGB parameter matrix and a weight matrix to obtain a weighted RGB parameter matrix, wherein the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel points and the RGB parameters of the plurality of adjacent pixel points, and the weight matrix is determined based on the difference matrix; Determine a merged image after weighted average filtering, wherein RGB parameters of the target edge pixel points in the merged image after weighted average filtering are determined based on the weighted RGB parameter matrix.
2. The method according to claim 1, characterized in that The determining of the text content corresponding to the target business service includes: Executing the target business service for the user terminal and obtaining an execution result of the target business service; Obtain the text template and index value corresponding to the target business service; The execution result is inserted into the position corresponding to the index value in the text template to obtain text content corresponding to the target business service.
3. The method according to claim 1, characterized in that The converting the text content into a first picture includes: Determine target text parameters and target number of lines according to the text content, wherein the target text parameters are used to represent the text size; Determine a text area in a picture template according to the target text parameter and the target number of lines, wherein the picture template has the same size as the background picture; Determining the text display color according to the color attribute parameters of the position corresponding to the text area in the background image; The text content is filled into the text area in the picture template according to the text display parameters to obtain a first picture, wherein the text display parameters include the target text parameters, the target number of lines and the text display color.
4. The method according to claim 3, characterized in that The step of determining target text parameters and target number of lines according to the text content includes: Determine at least one candidate character parameter and a corresponding line number, wherein the candidate character parameter and the line number satisfy a first condition and a second condition; Determine a target text parameter from the at least one candidate text parameter, and determine a target number of lines corresponding to the target text parameter; The first condition is: P is the candidate text parameter, L is the number of lines, is the preset width ratio, and is the width of the background image; The second condition is: N is the text length of the text content, is the preset length ratio, and is the length of the background image.
5. The method according to claim 3, characterized in that: The color attribute parameter is an RGB parameter, and determining the text display color according to the color attribute parameter of the position corresponding to the text area in the background image includes: Obtain RGB parameters of multiple pixel points at positions corresponding to the text area in the background image; Determine average RGB parameters of the plurality of pixels according to the RGB parameters of the plurality of pixels; Determine a saturation value and a brightness value according to the average RGB parameters of the plurality of pixels; The text display color is determined based on a first threshold, wherein the first threshold is the sum of a product of a first preset coefficient and the saturation value and a product of a second preset coefficient and the brightness value.
6. A video generating device, characterized in that: The device comprises: An acquisition module, used to acquire a business service request sent by a user terminal, wherein the business service request is used to request a target business service; A determination module, used to determine text content corresponding to the target business service, and determine a background image corresponding to the target business service; A synthesis module, used for converting the text content into a first picture, and synthesizing the first picture and the background picture into a second picture; A generating module, configured to convert the text content into voice content, and generate a video for returning to the user terminal according to the voice content and the second image; The synthesis module comprises: A conversion unit, used to convert the text content into a first picture; a merging unit, configured to merge the first image and the background image to obtain a merged image; An extraction unit, configured to extract a plurality of edge pixel points at positions corresponding to the text area in the merged image based on an edge detection algorithm; An acquisition unit, used to acquire grayscale values of a target edge pixel point and a plurality of adjacent pixel points adjacent to the target edge pixel point, wherein the target edge pixel point is any one edge pixel point among the plurality of edge pixel points; a processing unit, configured to perform weighted average filtering processing on the target edge pixel in the merged image based on the grayscale values of the target edge pixel and the plurality of adjacent pixels to obtain the second image; The processing unit is specifically used for: Determine a difference matrix according to the absolute value of the difference between the grayscale value of the target edge pixel and the grayscale values of the plurality of adjacent pixels; Performing a Hadamard product operation based on an RGB parameter matrix and a weight matrix to obtain a weighted RGB parameter matrix, wherein the RGB parameter matrix is determined based on the RGB parameters of the target edge pixel points and the RGB parameters of the plurality of adjacent pixel points, and the weight matrix is determined based on the difference matrix; Determine a merged image after weighted average filtering, wherein RGB parameters of the target edge pixel points in the merged image after weighted average filtering are determined based on the weighted RGB parameter matrix.
7. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the video generation method according to any one of claims 1 to 5 are implemented.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video generation method according to any one of claims 1 to 5 are implemented.
