Screen shot generation method and device, computer device and storage medium

Abstract

The application relates to a screenshot generation method and device, computer equipment and a storage medium. The method comprises the following steps: in response to a screenshot operation on target content, video recording is performed on the target content to obtain a target video; a reference frame image is obtained from the target video, and a candidate frame image is determined from the target video starting from a next frame image of the reference frame image; a first image displacement between the reference frame image and the candidate frame image is obtained, and a key frame image is determined based on the first image displacement and the candidate frame image; an overlapping image region between the reference frame image and the key frame image is determined based on the first image displacement, a second image displacement between the reference frame image and the key frame image is determined based on the overlapping image region; a to-be-spliced image region is determined from the key frame image based on the second image displacement, image splicing processing is performed on the to-be-spliced image region, and a content screenshot corresponding to the target content is generated. The method can improve the efficiency of screenshot generation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a screenshot generation method and device, computer equipment and storage medium. BACKGROUND

[0002] In the process of using a smart phone, a user often needs to take a screenshot of the content displayed on the screen of the phone and save it, but there are cases where the current screen cannot display all the content that needs to be saved, in which case a long screenshot of the screen needs to be taken. A long screenshot of the screen aggregates multiple images, which facilitates sharing and is frequently used in daily life.

[0003] In the prior art, a long screenshot is usually taken using the long screenshot function provided by the system of the phone, but the long screenshot function provided by the system of the phone requires system-level permissions and is complicated to operate, resulting in low efficiency of screenshot generation. SUMMARY

[0004] Therefore, it is necessary to provide a screenshot generation method and device, computer equipment, computer readable storage medium and computer program product that can improve the efficiency of screenshot generation.

[0005] In a first aspect, the present application provides a screenshot generation method. The method comprises: in response to a screenshot operation on target content, recording a video of the target content to obtain a target video; obtaining a reference frame image from the target video, and determining a candidate frame image from the target video starting from a next frame image of the reference frame image according to the display order of the video frame images in the target video; obtaining a first image displacement between the reference frame image and the candidate frame image, and determining the candidate frame image as a key frame image in a case where the first image displacement is greater than a displacement threshold; the first image displacement is determined based on a displacement positioning point, and the displacement positioning point is an intersection of different direction lines in an image; determining an overlapping image region between the reference frame image and the key frame image based on the first image displacement, and determining a second image displacement between the reference frame image and the key frame image based on the overlapping image region; determining a to-be-stitched image region from the key frame image based on the second image displacement, performing image stitching processing on the to-be-stitched image region, and generating a content screenshot corresponding to the target content.

[0006] In a second aspect, the present application also provides a screenshot generation apparatus. The apparatus comprises: a video recording module configured to perform video recording on target content to obtain a target video in response to a screenshot operation on the target content; an image acquisition module configured to acquire a reference frame image from the target video, and determine a candidate frame image from the target video starting from a next frame image of the reference frame image according to a display order of video frame images in the target video; a first displacement determination module configured to acquire a first image displacement between the reference frame image and the candidate frame image, and determine the candidate frame image as a key frame image in a case where the first image displacement is greater than a displacement threshold; the first image displacement is determined based on displacement positioning points, and the displacement positioning points are intersections of different direction lines in an image; a second displacement determination module configured to determine an overlapping image region between the reference frame image and the key frame image based on the first image displacement, and determine a second image displacement between the reference frame image and the key frame image based on the overlapping image region; and an image splicing module configured to determine a to-be-spliced image region from the key frame image based on the second image displacement, perform image splicing processing on the to-be-spliced image region, and generate a content screenshot corresponding to the target content.

[0007] In some embodiments, the reference frame image and the candidate frame image are adjacent; the first displacement determination module is further configured to: determine pixel points occupied by foreground content in the reference frame image to obtain foreground pixel points, and determine pixel points occupied by background content in the reference frame image to obtain background pixel points; update pixel values of the foreground pixel points in the reference frame image to first pixel values, and update pixel values of the background pixel points in the reference frame image to second pixel values to generate a mask image corresponding to the reference frame image; the first pixel values and the second pixel values are different; determine foreground pixel points through which a preset line passes in the mask image to obtain first displacement positioning points; and determine the first image displacement between the reference frame image and the candidate frame image based on the first displacement positioning points.

[0008] In some embodiments, the reference frame image and the candidate frame image are not adjacent; the first displacement determination module is further configured to: acquire a video segment with the reference frame image as a starting frame image and the candidate frame image as a terminal frame image, and calculate image displacements between adjacent two frame images in the video segment; and statistically acquire the first image displacement between the reference frame image and the candidate frame image based on the image displacements between adjacent two frame images in the video segment.

[0009] In some embodiments, the second displacement determining module is further configured to: determine, based on the first image displacement, an area of the key frame image that overlaps with the reference frame image to obtain a first overlapping image area; determine, based on the first image displacement, an area of the reference frame image that overlaps with the key frame image to obtain a second overlapping image area; determine a template image area from the first overlapping area and a candidate image area from the second overlapping image area, calculate a similarity between the template image area and the candidate image area to obtain a similarity score; in a case where the similarity score is greater than a similarity threshold, obtain a first position of a top-left vertex of the template image area in the reference frame image, and obtain a second position of the top-left vertex of the template image area in the key frame image; and determine a second image displacement between the reference frame image and the key frame image based on the first position and the second position.

[0010] In some embodiments, the second displacement determining module is further configured to: in a case where the similarity score is less than a similarity threshold, extract feature points of the first overlapping image area to obtain a first feature point set, and extract feature points of the second overlapping image area to obtain a second feature point set; for each first feature point in the first feature point set, determine a second feature point that matches the first feature point from the second feature point set; calculate a vertical distance between each first feature point and the matched second feature point; and determine a second image displacement between the reference frame image and the key frame image based on the vertical distances.

[0011] In some embodiments, the image stitching module is further configured to: stitch the to-be-stitched image area based on a current stitched image to obtain a new current stitched image, and determine the key frame image as a new reference frame image; return to the step of determining a candidate frame image from a next frame image of the reference frame image according to a display order of the video frame images in the target video until the candidate frame image traverses the video frame images in the target video; and determine the current stitched image in a case where the candidate frame image traverses the video frame images in the target video as the content screenshot corresponding to the target content.

[0012] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps in the screenshot generation method described above when executing the computer program.

[0013] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in the screenshot generation method described above.

[0014] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the screenshot generation method described above.

[0015] The screenshot generation method, device, computer device, storage medium and computer program product described above, in response to a screenshot operation on target content, video records the target content to obtain a target video, obtains a reference frame image from the target video, determines a candidate frame image from the target video starting from a next frame image of the reference frame image according to the display order of the video frame image in the target video, obtains a first image displacement between the reference frame image and the candidate frame image, determines the candidate frame image as a key frame image in a case where the first image displacement is greater than a displacement threshold, the first image displacement is determined based on a displacement positioning point, the displacement positioning point is an intersection of different direction lines in an image, determines an overlapping image region between the reference frame image and the key frame image based on the first image displacement, determines a second image displacement between the reference frame image and the key frame image based on the overlapping image region, determines a to-be-stitched image region from the key frame image based on the second image displacement, performs image stitching processing on the to-be-stitched image region, and generates a content screenshot corresponding to the target content. Thus, the screenshot can be automatically generated in the case where the screenshot operation is triggered, thereby improving the efficiency of screenshot generation. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 An application environment diagram of the screenshot generation method in an embodiment;

[0017] Figure 2 A flowchart of the screenshot generation method in an embodiment;

[0018] Figure 3A A flowchart of calculating the image displacement of two adjacent frame images in an embodiment;

[0019] Figure 3B A flowchart of performing image stitching in an embodiment;

[0020] Figure 4 A diagram of image stitching in an embodiment;

[0021] Figure 5 A flowchart of the screenshot generation method in another embodiment;

[0022] Figure 6 A structural block diagram of the screenshot generation device in an embodiment;

[0023] Figure 7 An internal structure diagram of the computer device in an embodiment;

[0024] Figure 8 Figure 1 is a schematic diagram of an internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0025] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0026] The screenshot generation method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server.

[0027] Specifically, the terminal 102, in response to a screenshot operation on target content, performs video recording on the target content to obtain a target video, acquires a reference frame image from the target video, determines a candidate frame image from the target video starting from a next frame image of the reference frame image according to a display order of video frame images in the target video, acquires a first image displacement between the reference frame image and the candidate frame image, and determines the candidate frame image as a key frame image in a case where the first image displacement is greater than a displacement threshold. The terminal 102 determines an overlapping image region between the reference frame image and the key frame image based on the first image displacement, determines a second image displacement between the reference frame image and the key frame image based on the overlapping image region, determines a to-be-stitched image region from the key frame image based on the second image displacement, performs image stitching processing on the to-be-stitched image region, and generates a content screenshot corresponding to the target content. The server 104 can store the content screenshot generated by the terminal 102. For example, after the terminal 102 displays the screenshot, a terminal user can select to upload the screenshot to the server.

[0028] The terminal 102 can be, but is not limited to, a desktop terminal or a mobile terminal. The mobile terminal can be at least one of a mobile phone, a tablet computer, a notebook computer, a smart watch, and the like. The terminal 102 can be a terminal device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0029] Those skilled in the art can understand that Figure 1 The application environment shown in

[0030] In some embodiments, asFigure 2 As shown, a screenshot generation method is provided, and the method is applied to Figure 1 As shown, a screenshot generation method is provided, and the method is applied to

[0031] Step 202, in response to the screenshot operation on the target content, video recording is performed on the target content to obtain a target video.

[0032] The target content is the screen display content that needs to generate a screenshot, for example, it can be the bill content in an application, or it can be the session content of the application. The screenshot operation is the operation that triggers the screenshot. The target video is the video obtained by video recording the target content, and the terminal can generate the screenshot corresponding to the target content based on the target video.

[0033] Specifically, the terminal is installed with a screenshot application, after the terminal detects the screenshot instruction triggered by the screenshot application, the terminal enters the screen recording state, in the screen recording state, in response to the operation of displaying the target content, all or part of the target content is displayed, and in response to the sliding operation on the target content. In response to the "start screenshot recording instruction" on the target content, the terminal performs video recording on the target content, and in the process of recording the video, the terminal can update the displayed content in response to the sliding operation on the target content, to display other content in the target content and record other content in the target content, until the terminal receives the "end screenshot recording instruction", the video recording is ended, and the recorded video is determined as the target video. In the process of video recording the target content by the terminal, the terminal user can perform a sliding operation on the screen display content upwards or downwards.

[0034] Step 204, obtaining a reference frame image from the target video, and determining a candidate frame image from the target video starting from the next frame image of the reference frame image according to the display order of the video frame image in the target video.

[0035] The reference frame image refers to a frame of video frame image as a reference, and the candidate frame image refers to a video frame image determined from the target video starting from the next frame image of the reference frame according to the display order of the video frame image in the target video. The reference frame image is determined from the target video according to the display order of the video frame image in the target video, and the reference frame image can be constantly changed, for example, the first frame in the target video is selected as the reference frame image initially, and then other video frame images in the target video can be selected as the reference frame image.

[0036] The display order is determined according to the display time, each video frame image in the target video has a display time, and the display time refers to the time when the video frame image is displayed. The earlier the display time, the earlier the display order.

[0037] Specifically, the terminal extracts the video frame image from the target video according to the display order of the video frame image in the target video to obtain a reference frame image, and then determines a candidate frame image from the target video according to the display order of the video frame image in the target video starting from a next frame image of the reference frame image. For example, the terminal can take the first frame image of the target video as the reference frame image, and then extract the second frame image of the target video as the candidate frame image, and continue to extract the third frame image of the target video as the candidate frame image in a case where a first image displacement between the first frame image and the second frame image is less than a displacement threshold. The first image displacement is determined based on a displacement positioning point, and the displacement positioning point is an intersection of different direction lines in the image, and the lines in the image include lines that exist in the image and can also include newly added lines in the image.

[0038] In step 206, a first image displacement between the reference frame image and the candidate frame image is obtained, and the candidate frame image is determined as the key frame image in a case where the first image displacement is greater than a displacement threshold. The first image displacement is determined based on a displacement positioning point, and the displacement positioning point is an intersection of different direction lines in the image.

[0039] The first image displacement is an image displacement between the reference frame image and the candidate frame image determined based on a displacement positioning point, and is used to represent a displacement distance of the candidate frame image relative to the reference frame image. The first image displacement is also used to represent a displacement direction, which refers to a sliding direction of the screen content. For example, when the first image displacement has a positive value, the displacement direction is downward sliding, and the key frame image slides into the screen from above the reference frame image. When the first image displacement has a negative value, the displacement direction is upward sliding, and the key frame image slides into the screen from below the reference frame image. The displacement threshold can be preset or set as needed, for example, the displacement threshold can be set as X% of a frame image height h, and the value of X ranges from 0 to 100. The key frame image can be a candidate frame image with a first image displacement greater than the displacement threshold. The displacement positioning point is an intersection of different direction lines in the image, for example, the displacement positioning point can be an intersection of vertical lines and horizontal lines in the image. The displacement positioning point can be a corner point, which belongs to a pixel point. When the pixel point in the vicinity of a region has a large change in gradient direction or gradient amplitude, the pixel point is a corner point. The position of the displacement positioning point in the image can be represented in the form of coordinates, for example, (x, y) is used to represent the position of the displacement positioning point in the image.

[0040] Specifically, the terminal obtains a first image displacement between the reference frame image and the candidate frame image, and then compares the first image displacement with a displacement threshold value. When the first image displacement is greater than the displacement threshold value, the candidate frame image is determined as a key frame image. When the first image displacement is less than the displacement threshold value, it is determined whether the candidate frame image is the last frame in the current displacement direction or the last frame video image of the target video. If the candidate frame image is the last frame in the current displacement direction or the last frame video image of the target video, the candidate frame image is determined as a key frame image. If the candidate frame image is not the last frame video image of the target video, the terminal continues to extract a next frame image of the candidate frame image in the target video according to the display order of the video frame images in the target video, updates the next frame image as the candidate frame image, and obtains a first image displacement between the reference frame image and the candidate frame image.

[0041] In some embodiments, when the reference frame image and the candidate frame image are adjacent, the terminal determines the first image displacement between the reference frame image and the candidate frame image based on displacement positioning points. For example, the terminal takes the first frame video image as the reference frame image and the second frame video image as the candidate frame image. The terminal extracts the displacement positioning points of the reference frame image as first displacement positioning points, and obtains second displacement positioning points based on the first displacement positioning points and the second frame video image, and determines the first image displacement between the reference frame image and the candidate frame image based on the first displacement positioning points and the second displacement positioning points. The first displacement positioning points are determined from the reference frame image, and there can be one or more first displacement positioning points, with more than two first displacement positioning points. Each first displacement positioning point can determine one second displacement positioning point, i.e., each first displacement positioning point corresponds to one second displacement positioning point.

[0042] In some embodiments, the terminal can determine the feature of the first displacement positioning point to obtain a first positioning point feature, determine the position of the first displacement positioning point in the reference frame image to obtain a first positioning point position, for example, the first positioning point position can be a pixel point at the 2nd row and the 3rd column, determine the pixel point at the first positioning point position in the candidate frame image as a center point, determine the pixel points with a distance less than a distance threshold value from the center point as candidate positioning points, determine the candidate positioning point corresponding to the positioning point feature most similar to the first positioning point feature in the candidate positioning point features as the second displacement positioning point corresponding to the first displacement positioning point.

[0043] In some embodiments, the step of determining the feature of the first displacement positioning point comprises: obtaining an image region centered on the first displacement positioning point, performing feature extraction on the obtained image region, and determining the extracted feature as the feature of the first displacement positioning point.

[0044] In some embodiments, when the reference frame image and the candidate frame image are not adjacent, the terminal obtains an image displacement between each two adjacent frame images in the target video, and performs statistics based on the image displacement between each two adjacent frame images to obtain a first image displacement between the reference frame image and the candidate frame image. The method for calculating the image displacement between each two adjacent frame images can refer to the method for calculating the first image displacement when the reference frame image and the candidate frame image are adjacent.

[0045] In step 208, an overlapping image region between the reference frame image and the key frame image is determined based on the first image displacement, and a second image displacement between the reference frame image and the key frame image is determined based on the overlapping image region.

[0046] The overlapping image region is determined based on the first image displacement, the reference frame image and the key image can have an overlapping image region, the second image displacement is an image displacement between the reference frame image and the key frame image determined based on the overlapping image region, and is used to represent a displacement distance of the key frame image relative to the reference frame image. The second image displacement is also used to represent a displacement direction. The displacement direction includes at least one of upward sliding or downward sliding.

[0047] According to the displacement positioning point, the first image displacement is determined, and the calculation speed is fast, thereby improving the efficiency of determining the first image displacement. However, when the reference frame image and the candidate frame image are not adjacent, the image displacement of each two adjacent frame images between the reference frame image and the candidate frame image needs to be calculated, and the statistical process can cause calculation errors, that is, the first image displacement can have certain errors. Therefore, in order to improve the calculation accuracy of the image displacement and reduce errors, the image displacement between the reference frame and the key frame can be accurately calculated, for example, the second image displacement between the reference frame image and the key frame image is determined based on the overlapping image region, so that the image displacement between the reference frame image and the key frame image is recalculated by using the overlapping image region. The second image displacement is more accurate than the first image displacement, thereby improving the accuracy of calculating the image displacement while improving the efficiency of calculating the image displacement.

[0048] Specifically, the terminal intercepts an image region overlapping with the reference frame image in the key frame image based on the first image displacement, and determines the obtained image region as an overlapping image region between the reference frame image and the key frame image. By using the overlapping image region, the second image displacement between the reference frame image and the key frame image can be determined.

[0049] In step 210, a to-be-stitched image region is determined from the key frame image based on the second image displacement, and image stitching processing is performed on the to-be-stitched image region to generate a content screenshot corresponding to the target content.

[0050] The to-be-stitched image region is an image region extracted from the key frame based on the second image displacement, and is used to generate a content screenshot after image stitching. The image stitching processing is an image processing operation of stitching two images to generate one image, and the content screenshot is a screenshot image corresponding to the target content.

[0051] Specifically, the displacement direction represented by the second image displacement can be downward sliding or upward sliding. If the displacement direction is downward sliding, as shown in FIG. 2, the key frame image slides into the screen display area from the top of the reference frame image, and the terminal extracts an image region from the key frame image starting from the top of the key frame image and taking the second image displacement as the image height. The obtained image region is the image region of the key frame image relative to the reference frame image sliding, and the image region is determined as the to-be-stitched image region. If the displacement direction is upward sliding, the key frame image slides into the screen display area from the bottom of the reference frame image, and the terminal extracts an image region from the key frame image starting from the bottom of the key frame image and taking the second image displacement as the image height. The obtained image region is the image region of the key frame image relative to the reference frame image sliding, and the image region is determined as the to-be-stitched image region. The terminal performs image stitching processing on the to-be-stitched image region to generate a content screenshot corresponding to the target content. Figure 4

[0052] In the above screenshot generation method, in response to a screenshot operation on the target content, a target video is obtained by video recording on the target content, a reference frame image is obtained from the target video, a candidate frame image is determined from the target video starting from a next frame image of the reference frame image according to the display order of the video frame images in the target video, a first image displacement between the reference frame image and the candidate frame image is obtained, the candidate frame image is determined as a key frame image in a case where the first image displacement is greater than a displacement threshold, the first image displacement is determined based on a displacement positioning point, the displacement positioning point is an intersection of different direction lines in the image, an overlapping image region between the reference frame image and the key frame image is determined based on the first image displacement, a second image displacement between the reference frame image and the key frame image is determined based on the overlapping image region, a to-be-stitched image region is determined from the key frame image based on the second image displacement, image stitching processing is performed on the to-be-stitched image region, and a content screenshot corresponding to the target content is generated. Thus, the screenshot can be automatically generated in the case where the screenshot operation is triggered, thereby improving the efficiency of screenshot generation.

[0053] ​In one embodiment, the reference frame image is adjacent to the candidate frame image; obtaining the first image displacement between the reference frame image and the candidate frame image comprises: determining foreground pixels in the reference frame image to obtain foreground pixels, and determining background pixels in the reference frame image to obtain background pixels; updating pixel values of the foreground pixels in the reference frame image to first pixel values, and updating pixel values of the background pixels in the reference frame image to second pixel values to generate a mask image corresponding to the reference frame image; the first pixel values are different from the second pixel values; determining foreground pixels through which the preset line passes in the mask image to obtain first displacement positioning points; and determining the first image displacement between the reference frame image and the candidate frame image based on the first displacement positioning points.

[0054] The foreground pixels are pixels occupied by foreground content in the reference frame image, for example, pixels occupied by a moving object in a video frame image, and the background pixels are pixels occupied by background content in the reference frame image. The first pixel values and the second pixel values can be set as needed, and the first pixel values are different from the second pixel values, for example, the first pixel values are 255 and the second pixel values are 0. The mask image is an image obtained by updating pixel values in the reference frame image. The preset line is a straight line in a preset direction, and the preset direction is a direction consistent with the screen sliding direction, including but not limited to at least one of a vertical direction or a horizontal direction, for example, when the screen sliding direction is a vertical sliding direction, the preset line can be a straight line in a vertical direction, and the first displacement positioning points are displacement positioning points determined in the reference frame image. The first displacement positioning points include foreground pixels through which the preset line passes in the mask image, and can also include displacement positioning points obtained from the reference frame image by using other methods.

[0055] Specifically, when the reference frame image is adjacent to the candidate frame image, the reference frame image is the previous frame image of the candidate frame image. The terminal first performs image preprocessing on the reference frame image, and the image preprocessing includes: 1) scaling the video frame image to a unified MxN resolution; 2) converting the scaled video frame image from a color RGB image to a grayscale image; 3) determining the pixel points occupied by the foreground content in the grayscale image corresponding to the reference frame image to obtain the foreground pixel points, and determining the pixel points occupied by the background content in the reference frame image to obtain the background pixel points; updating the pixel value of the foreground pixel points in the reference frame image to a first pixel value, and updating the pixel value of the background pixel points in the reference frame image to a second pixel value, to generate a mask image corresponding to the reference frame image. The terminal adds a preset line to the mask image corresponding to the reference frame image, extracts the corner points in the image to obtain the first displacement positioning point of the reference frame image, then performs the above image preprocessing on the candidate frame image to obtain the mask image corresponding to the candidate frame image, obtains the second displacement positioning point of the candidate frame image according to the first displacement positioning point and the mask image corresponding to the candidate frame image, and calculates the displacement distance between the first displacement positioning point and the corresponding second displacement positioning point, which is determined as the first image displacement between the reference frame image and the candidate frame image. The mask image can also be referred to as a binary image.

[0056] In some embodiments, the foreground content in the video frame image includes a text part, the terminal performs image scaling and grayscale processing on the reference frame image to obtain the MxN resolution grayscale image corresponding to the first frame video image, and then converts the grayscale image into a binary image, displays the text part in the video frame image in black, and displays the background outside the text in white to obtain the binary image corresponding to the first frame video image, i.e., the mask image. The terminal can add a vertical line at a fixed position on the binary image corresponding to the reference frame image, so that the vertical line intersects with the text part and the horizontal line in the image, and the intersection point is determined as the first displacement positioning point.

[0057] In some embodiments, the displacement positioning points can be referred to as corner points. The terminal can perform image preprocessing on the reference frame image to obtain a first image, and perform corner point extraction on the first image using a corner point detector to obtain a first corner point list. The terminal can perform corner point extraction on the first image using a Harris corner point detector, and the position of a corner point in the image can be represented in the form of coordinates, for example, a corner point A in the first corner point list can be represented as (x1, y1). The terminal can perform image preprocessing on the candidate frame image to obtain a second image, and perform corner point tracking update on the second image. The corner point tracking update refers to finding a pixel point with the highest similarity to a corner point A1 in the first corner point list in a certain region within a predetermined radius range with the position of the corner point A1 as the center, and taking the pixel point with the highest similarity as a corner point A2 of the second image. The terminal determines the pixel points with the highest similarity of all the corner points in the first corner point list on the second image to obtain a second updated corner point list of the second image. The terminal calculates the pixel displacement, i.e., the distance, of each corner point in the second updated corner point list relative to the corresponding corner point in the first corner point list to obtain a corner point displacement list, sorts the corner point displacement list in descending order, and then removes the displacement data in the corner point displacement list that is ranked in the front n% of the list and that is ranked in the back n% of the list according to a predetermined proportion n%, to obtain an updated corner point displacement list. The terminal performs mean value calculation on the updated corner point displacement list, and takes the obtained mean value as the first image displacement between the first frame image and the second frame image.

[0058] In some embodiments, the terminal can combine the method of extracting corner points by the corner point detector and the method of extracting corner points by using a predetermined line to generate more first displacement positioning points. Since the text content in the video frame image mainly presents horizontally, increasing the fixed position of the black vertical line and the intersection of the horizontal text content and the horizontal line in the video frame image can generate more high-quality displacement positioning points.

[0059] In some embodiments, when the terminal can only respond to a vertical sliding operation on the target content, the terminal can calculate the corner point displacement using the vertical coordinates of the corner points, and determine the displacement direction according to the positive and negative of the corner point displacement. For example, the corner point A1 in the first corner point list is represented as (x1, y1), and the corner point A2 in the second updated corner point list is represented as (x2, y2). The terminal can calculate the difference y0 between y2 and y1, y0 = y2 - y1, to obtain the corner point displacement between the corner point A1 and the corner point A2. The terminal can take the sliding direction of the screen content as the downward sliding as the positive direction of the displacement direction. When y0 takes a positive value, it indicates that the displacement direction is the positive direction. When y0 takes a negative value, it indicates that the displacement direction is the negative direction.

[0060] In some embodiments, in order to screen reliable corner points, the terminal can perform corner point tracking on the first image based on the second updated corner point list, obtain an updated first corner point check list, calculate the position deviation of the corresponding corner points in the first corner point list and the first corner point check list, obtain a position deviation list, then compare each displacement deviation in the position deviation list with a deviation threshold, when the displacement deviation is greater than the deviation threshold, eliminate the corner points corresponding to the displacement deviation in the first corner point list and the second updated corner point list, until the position deviation list is traversed, and the verified first corner point list and the second updated corner point list are obtained. The position deviation includes at least one of a horizontal displacement deviation or a vertical displacement deviation, and the deviation threshold is a preset displacement deviation threshold. Thus, unreliable corner points are eliminated.

[0061] In some embodiments, the terminal performs corner point extraction and corner point tracking update on the video frame images, and can perform the corner point extraction operation again at intervals of a preset frame number. As shown in Figure 3A The terminal obtains a frame of video frame image in the target video as a current frame image, performs image preprocessing on the current frame image, performs corner point extraction on the current frame image if the current frame image is a first frame image, performs corner point tracking update on the current frame image based on the corner point list of the previous frame image of the current frame image if the current frame image is not the first frame image, obtains an updated corner point list corresponding to the current frame image, obtains the image displacement between the current frame image and the previous frame image based on the corner point list of the previous frame image and the updated corner point list, then judges whether the current frame image and the reference frame image are spaced apart by a preset frame number, performs corner point extraction on the current frame image if the current frame image and the reference frame image are spaced apart by the preset frame number, and obtains the current corner point list of the current frame image, and returns to the step of obtaining a frame of image in the target video as a current frame image if the current frame image and the reference frame image are not spaced apart by the preset frame number, until all video frame images in the target video are traversed. The preset frame number is a preset corner point extraction interval frame number. In this embodiment, the corner point extraction operation is performed on the video frame image at intervals of the preset frame number, which reduces the number of times of corner point extraction and improves the speed of obtaining the first image displacement.

[0062] For example, when the interval preset frame number is 2, the target video has a total of 5 frame video image, the terminal acquires the first frame image in the target video, obtains the first image through image preprocessing, then extracts the corner points of the first image to obtain the first corner point list of the first image, then acquires the second frame image in the target video, obtains the second image through image preprocessing. Based on the first corner point list and the second image, the second updated corner point list is obtained, and based on the first corner point list and the second updated corner point list, the image displacement between the first frame image and the second frame image is obtained. Then the terminal acquires the third frame image, obtains the third image through image preprocessing, obtains the third updated corner point list based on the second updated corner point list and the third image, and obtains the image displacement between the second frame image and the third frame image based on the second updated corner point list and the third updated corner point list. The third frame image is 2 frames apart from the first frame image, so the corner points of the third image are extracted to obtain the third corner point list, and then the terminal continues to acquire the fourth frame image and repeats the above steps until all video frame images of the target video are traversed.

[0063] In this embodiment, by using the preset line through the foreground pixel points of the mask image, the displacement positioning points in the image are increased, and when the first image displacement between the reference frame and the candidate frame image is determined based on the first displacement positioning points, the accuracy of the first image displacement calculation is improved, thereby improving the accuracy of the screenshot generation.

[0064] In one embodiment, the reference frame image and the candidate frame image are not adjacent; acquiring the first image displacement between the reference frame image and the candidate frame image comprises: acquiring a video segment with the reference frame image as the starting frame image and the candidate frame image as the terminal frame image, calculating the image displacement between two adjacent frame images in the video segment; and counting the image displacement between the two adjacent frame images in the video segment to obtain the first image displacement between the reference frame image and the candidate frame image.

[0065] The starting frame image refers to the first frame image of the video segment between the reference frame image and the candidate frame image, and the terminal frame image refers to the last frame image of the video segment between the reference frame image and the candidate frame image. The video segment includes a plurality of video frame images, and the plurality refers to at least three. The first frame image of the video segment is the reference frame image, and the last frame image is the candidate frame image.

[0066] Specifically, the terminal can obtain the image displacement between two adjacent images in the video clip, count the image displacement between the two adjacent images in the video clip, and determine the counting result as the first image displacement between the reference frame image and the candidate frame image. For example, the video clip includes 3 images, the image displacement between the first image and the second image is calculated to obtain image displacement 1, the image displacement between the second image and the third image is calculated to obtain image displacement 2, the sum of image displacement 1 and image displacement 2 is calculated, and the sum result is determined as the first image displacement between the first image and the third image. The method for calculating the image displacement between the two adjacent images can refer to the method for calculating the first image displacement between the two adjacent images.

[0067] In this embodiment, the first image displacement between the non-adjacent reference frame image and the candidate frame image is determined by counting the image displacement between the two adjacent images, the number of image splicing is reduced, and the efficiency of screenshot generation is improved.

[0068] In one embodiment, step 208 further includes: determining an overlapping image area between the reference frame image and the key frame image based on the first image displacement, obtaining a first overlapping image area; determining an overlapping image area between the reference frame image and the key frame image based on the first image displacement, obtaining a second overlapping image area; determining a template image area from the first overlapping area and a candidate image area from the second overlapping image area, calculating the similarity between the template image area and the candidate image area to obtain a similarity score; in the case that the similarity score is greater than a similarity threshold, obtaining the position of the top-left vertex of the template image area in the reference frame image to obtain a first position, and obtaining the position of the top-left vertex of the template image area in the key frame image to obtain a second position; determining the second image displacement between the reference frame image and the key frame image based on the first position and the second position.

[0069] The template image area is an image area determined based on the overlapping image area, which can be used to calculate the similarity between two images, and the similarity is used to represent the similarity between two images. The candidate image area is the image area on the reference frame image that is most similar to the template image area. The similarity score is used to represent the similarity between the template image area and the candidate image area, and the range of the similarity score can be [0, 1]. The similarity threshold is a preset similarity threshold, for example, the similarity threshold can be set to 0.95.

[0070] Specifically, the terminal determines an area overlapping with the reference frame image from the key frame image based on the first image displacement, to obtain a first overlapping image area, and selects an upper-left corner image area according to a preset image size in the first overlapping image area, and determines the image area as a template image area; the preset image size is a preset size of the template image area. Then the terminal determines an area overlapping with the reference frame image from the key frame image based on the first image displacement, to obtain a second overlapping image area. The terminal determines a candidate image area from the second overlapping image area by using the template image area, calculates a similarity between the template image area and the candidate image area, to obtain a similarity score, and in a case where the similarity score is greater than a similarity threshold, obtains a matching point coordinate P(x, y) of an upper-left vertex of the template image area on the reference frame image, obtains a matching point coordinate P1(x1, y1) of the upper-left vertex of the template image area on the key frame image, calculates a vertical displacement d between P(x, y) and P1(x1, y1), d = y1-y, and then takes d as a second image displacement between the reference frame image and the key frame image.

[0071] In the embodiment, the second image displacement between the reference frame image and the key frame image is calculated by determining the template image area, the error caused by determining the first image displacement is reduced, and the to-be-stitched image is obtained based on the second image displacement, so that the accuracy of the screenshot generation is improved.

[0072] In one embodiment, step 208 further includes: in a case where the similarity score is less than the similarity threshold, extracting feature points of the first overlapping image area to obtain a first feature point set, and extracting feature points of the second overlapping image area to obtain a second feature point set; for each first feature point in the first feature point set, determining a second feature point matching the first feature point from the second feature point set; calculating a vertical distance between each first feature point and the matched second feature point; and determining the second image displacement between the reference frame image and the key frame image based on the vertical distances.

[0073] The feature points belong to pixel points, and the feature points are representative points in the image. The first feature points are feature points in the first feature point set, and the second feature points are feature points in the second feature point set. The first feature point set is a feature point set obtained based on the first overlapping image area, and the second feature point set is a pixel point set matched by the first feature point set in the second overlapping image area. The feature matching pair set is obtained based on the first feature point set and the second feature point set, for example, the first feature point set is [p0, p1, p2, …, pn], and the second feature point set is [t0, t1, t2, …, tn], and [p0, t2] can be a feature matching pair.

[0074] Specifically, in the case that the similarity score is less than the similarity threshold, the terminal extracts feature points of the first overlapped image region to obtain a first feature point set [p0, p1, p2, …, pn], and the coordinate of the feature point p0 can be represented as (p0_x, p0_y); and extracts feature points of the second overlapped image region to obtain a second feature point set [t0, t1, t2, …, tn]. The terminal can extract the feature points by using a SIFT (Scale-invariant feature transform) or ORB (Oriented FAST and Rotated BRIEF) algorithm. The terminal determines, for each first feature point in the first feature point set, a second feature point in the second feature point set that matches the first feature point to obtain a feature matching pair, for example, the best matching point of p0 in the first feature point set in the second feature point set is t0, and then [p0, t0] is a feature matching pair. The terminal traverses all the feature points in the first feature point set to obtain a feature matching pair set {[p0, t0], [p1, t1], …, [pn, tn]}. Then the terminal calculates the longitudinal displacement between the two feature points in each feature matching pair to obtain a longitudinal displacement set D = [d0, d1, …, dn], where d0 = p0_y-t0_y, …, dn = pn_y-tn_y. Then the terminal clusters the longitudinal displacement set to obtain a clustering result, and the clustering result contains m categories. The terminal selects, from the m categories, a category with the largest number of elements as a target category, and calculates the mean value of the target category, and takes the mean value as the second image displacement between the key frame image and the reference frame image.

[0075] In some embodiments, for each first feature point in the first feature point set, the terminal can calculate the displacement between the first feature point and each second feature point in the second feature point set, and take the second feature point corresponding to the smallest displacement as the second feature point matched with the first feature point.

[0076] In some embodiments, the terminal clusters the set of longitudinal displacements D, first takes a longitudinal displacement d0 of the set of longitudinal displacements D as a first clustering center, calculates a difference between d1 and d0, when the difference between d1 and d0 is greater than a preset longitudinal displacement, d1 remains in the set of longitudinal displacements D; when the difference between d1 and d0 is less than the preset longitudinal displacement, d1 is put into a first cluster, an updated first cluster is obtained, d1 is deleted from the set of longitudinal displacements D, and the difference between d2 and d0 is continuously calculated, the above steps are repeated, and the first cluster is obtained. The preset longitudinal displacement can be a preset longitudinal displacement threshold. Then, the first longitudinal displacement di in the updated set of longitudinal displacements D is taken, di is the i+1th longitudinal displacement in the original set of longitudinal displacements D, the above clustering steps are repeated, until the set of longitudinal displacements D is traversed, a clustering result is obtained, a class with the most elements in the clustering result is selected as a target class, and a mean value is calculated for the target class, and the mean value is taken as the second image displacement between the key frame image and the reference frame image. For example, the set of longitudinal displacements D = [d0, d1, …, d8], the clustering result can be {[d0, d4, d7}, [d1, d2], [d3, d5, d6, d8]}, the clustering result has three clusters, including a first cluster D0 = [d0, d4, d7], a second cluster D1 = [d1, d2], and a third cluster D2 = [d3, d5, d6, d8], and the third cluster D2 has four elements, so the third cluster is taken as the target class, and a mean value of d3, d5, d6, and d8 is calculated, which is taken as the second image displacement between the key frame image and the reference frame image.

[0077] In this embodiment, the second image displacement is obtained based on the first feature point set and the second feature point set, the second image displacement is more accurate, and the to-be-stitched image is obtained based on the second image displacement, which can improve the accuracy of the generated screenshot.

[0078] In one embodiment, the image stitching processing is performed on the to-be-stitched image region, and the content screenshot corresponding to the target content is generated, including: stitching the to-be-stitched image region based on the current stitching image to obtain a new current stitching image, and determining the key frame image as a new reference frame image; returning to the step of determining the candidate frame image from the next frame image of the reference frame image according to the display order of the video frame images in the target video, until the candidate frame image traverses the video frame images in the target video; determining the current stitching image when the candidate frame image traverses the video frame images in the target video as the content screenshot corresponding to the target content.

[0079] Wherein, with continuous stitching, the current stitching image is continuously changing, and the initial current stitching image can be the first frame image in the target video.

[0080] Specifically, as Figure 3BAs shown, the terminal obtains the image region to be spliced based on the second image displacement and the key frame image, then splices the image region to be spliced on the basis of the current spliced image, if the displacement direction is downward sliding, splices the image region to be spliced above the current spliced image, if the displacement direction is upward sliding, splices the image region to be spliced below the current spliced image, obtains a new current spliced image, and determines the key frame image as a new reference frame image; repeats the steps of determining the candidate frame image from the target video according to the display order of the video frame images in the target video, starting from the next frame image of the reference frame image, until the candidate frame image traverses the video frame images in the target video; and determines the current spliced image in the case that the candidate frame image traverses the video frame images in the target video as the content screenshot corresponding to the target content.

[0081] In some embodiments, as shown in Figure 4 As shown, the video frame images of the target video include an image fixed region and an image content sliding region, during the screen sliding process, the display position of the image fixed region does not change, for example, the image fixed region can be the top navigation bar of the mobile phone page, and the image content sliding region will be updated with the sliding operation. The terminal obtains a frame of video frame image from the target video, needs to extract the image content sliding region of the video frame image, then performs image splicing based on the image content sliding region of the video frame image, and after traversing all the video frame images of the target video, splices the image fixed region on the current spliced image to obtain the content screenshot corresponding to the target content.

[0082] In this embodiment, by splicing the image to be spliced with the current spliced image, updating the current spliced image, and then determining the key frame image as a new reference frame image, all the video frame images of the target video are traversed to obtain the content screenshot corresponding to the target content, and the efficiency of screenshot generation is improved.

[0083] In some embodiments, as shown in Figure 5 As shown, a screenshot generation method is provided, which is taken as an example of application to a terminal and includes the following steps:

[0084] Step 502, in response to a screenshot operation for target content, video recording is performed on the target content to obtain a target video.

[0085] Step 504, a reference frame image is obtained from the target video, and the next frame image of the reference frame image is taken as a candidate frame image.

[0086] Among them, the obtained reference frame image and candidate frame image are image content sliding regions in the corresponding video frame images.

[0087] Step 506, a first image displacement between the reference frame image and the candidate frame image is determined.

[0088] The first image displacement is determined based on image displacement between two adjacent frames of images, and the terminal can calculate the image displacement between two adjacent frames of images in the target video based on the displacement positioning point. The displacement positioning point can be a corner point.

[0089] In step 508, it is determined whether the first image displacement is greater than a displacement threshold. If not, step 510 is performed, and if yes, step 512 is performed.

[0090] In step 510, the next frame of image of the candidate frame of image is updated as the candidate frame of image, and step 506 is returned.

[0091] In step 512, the candidate frame of image is determined as the key frame of image, and the overlapping image region between the reference frame of image and the key frame of image is determined based on the first image displacement.

[0092] In step 514, the overlapping image region between the reference frame of image and the key frame of image is determined based on the first image displacement, and the second image displacement between the reference frame of image and the key frame of image is determined based on the overlapping image region.

[0093] In step 516, the image region to be spliced is determined from the key frame of image based on the second image displacement.

[0094] In step 518, the image region to be spliced is spliced based on the current spliced image to obtain a new current spliced image.

[0095] In step 520, it is determined whether the key frame of image is the last frame of image of the target video. If not, step 522 is performed, and if yes, step 524 is performed.

[0096] In step 522, the key frame of image is taken as the reference frame of image, and step 504 is returned.

[0097] In step 524, the current spliced image is determined as the content screenshot corresponding to the target content.

[0098] In this embodiment, the overlapping image region is determined based on the first image displacement when the first image displacement is greater than the displacement threshold, the second image displacement between the reference frame and the key frame is calculated using the overlapping region, the image splicing processing is performed based on the second image displacement, the number of image splicing is reduced, and the efficiency of screenshot generation is improved. In addition, the calculation error of the first image displacement is corrected by calculating the second image displacement, and the accuracy of screenshot generation is improved.

[0099] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately executed with at least some of the other steps or steps or stages in other steps.

[0100] Based on the same inventive concept, the embodiments of the present application also provide a screenshot generation device for implementing the screenshot generation method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more screenshot generation device embodiments provided below can refer to the limitations of the screenshot generation method described above, which will not be repeated here.

[0101] In one embodiment, as shown in Figure 6 A screenshot generation device is provided, comprising: a video recording module 602, an image acquisition module 604, a first displacement determination module 606, a second displacement determination module 608, and an image stitching module 610, wherein:

[0102] The video recording module 602 is configured to perform video recording on the target content to obtain a target video in response to a screenshot operation on the target content.

[0103] The image acquisition module 604 is configured to acquire a reference frame image from the target video, and determine a candidate frame image from the target video starting from a next frame image of the reference frame image according to the display order of the video frame images in the target video.

[0104] The first displacement determination module 606 is configured to acquire a first image displacement between the reference frame image and the candidate frame image, and determine the candidate frame image as a key frame image in a case where the first image displacement is greater than a displacement threshold. The first image displacement is determined based on displacement positioning points, and the displacement positioning points are intersection points of different direction lines in the image.

[0105] The second displacement determination module 608 is configured to determine an overlapping image region between the reference frame image and the key frame image based on the first image displacement, and determine a second image displacement between the reference frame image and the key frame image based on the overlapping image region.

[0106] The image stitching module 610 is configured to determine a to-be-stitched image region from the key frame image based on the second image displacement, perform image stitching processing on the to-be-stitched image region, and generate a content screenshot corresponding to the target content.

[0107] In some embodiments, the reference frame image is adjacent to the candidate frame image; the first displacement determination module is further configured to: determine pixel points occupied by foreground content in the reference frame image to obtain foreground pixel points, and determine pixel points occupied by background content in the reference frame image to obtain background pixel points; update pixel values of the foreground pixel points in the reference frame image to a first pixel value, and update pixel values of the background pixel points in the reference frame image to a second pixel value to generate a mask image corresponding to the reference frame image; the first pixel value is different from the second pixel value; determine foreground pixel points through which the preset line passes in the mask image to obtain first displacement positioning points; and determine the first image displacement between the reference frame image and the candidate frame image based on the first displacement positioning points.

[0108] In some embodiments, the reference frame image is not adjacent to the candidate frame image; the first displacement determination module is further configured to: obtain a video segment with the reference frame image as a starting frame image and the candidate frame image as a terminal frame image, and calculate image displacements between adjacent two frame images in the video segment; and perform statistics on the image displacements between the adjacent two frame images in the video segment to obtain the first image displacement between the reference frame image and the candidate frame image.

[0109] In some embodiments, the second displacement determination module is further configured to: determine, based on the first image displacement, an image region overlapping with the reference frame image from the key frame image to obtain a first overlapping image region; determine, based on the first image displacement, an image region overlapping with the key frame image from the reference frame image to obtain a second overlapping image region; determine a template image region from the first overlapping image region and a candidate image region from the second overlapping image region; calculate a similarity between the template image region and the candidate image region to obtain a similarity score; in a case where the similarity score is greater than a similarity threshold, obtain a first position of an upper left vertex of the template image region in the reference frame image, and obtain a second position of the upper left vertex of the template image region in the key frame image; and determine the second image displacement between the reference frame image and the key frame image based on the first position and the second position.

[0110] In some embodiments, the second displacement determining module is further configured to: in a case where the similarity score is less than the similarity threshold, extract feature points of the first overlapped image region to obtain a first feature point set, and extract feature points of the second overlapped image region to obtain a second feature point set; for each first feature point in the first feature point set, determine a second feature point matching the first feature point from the second feature point set; calculate a longitudinal distance between each first feature point and the matched second feature point; and determine a second image displacement between the reference frame image and the key frame image based on the longitudinal distances.

[0111] In some embodiments, the image stitching module is further configured to: stitch the to-be-stitched image region based on the current stitched image to obtain a new current stitched image, and determine the key frame image as a new reference frame image; return to the step of determining the candidate frame image from a next frame image of the reference frame image according to the display order of the video frame images in the target video, until the candidate frame image traverses the video frame images in the target video; and determine the current stitched image in the case where the candidate frame image traverses the video frame images in the target video as the content screenshot corresponding to the target content.

[0112] The above modules in the screenshot generation apparatus can be implemented by software, hardware, or a combination thereof, in whole or in part. The above modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0113] In one embodiment, a computer device is provided, which can be a server. An internal structure diagram of the computer device can be as shown in FIG. 8. Figure 7 The computer device includes a processor, a memory, and a network interface connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store related data involved in the screenshot generation method. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a screenshot generation method.

[0114] In one embodiment, a computer device is provided, which can be a terminal. An internal structure diagram of the computer device can be as shown in FIG. 9. Figure 8As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a screenshot generation method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad provided on the shell of the computer device. It can also be an external keyboard, touchpad or mouse, etc.

[0115] Those skilled in the art can understand that, Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0116] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the screenshot generation method described above.

[0117] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the screenshot generation method described above.

[0118] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps of the screenshot generation method described above.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0120] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0121] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0122] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A screenshot generation method, characterized in that, The method includes: In response to a screenshot operation targeting the target content, a video is recorded of the target content to obtain the target video; Obtain reference frame images from the target video; starting from the next frame image of the reference frame image, determine candidate frame images from the target video according to the display order of video frame images in the target video; A first image displacement is obtained between the reference frame image and the candidate frame image. If the first image displacement is greater than a displacement threshold, the candidate frame image is determined as a keyframe image. The first image displacement is determined based on a displacement positioning point, which is the intersection of lines in different directions in the image. The change in gradient direction and gradient magnitude between the displacement positioning point and its adjacent pixel region is greater than the change in gradient direction and gradient magnitude between pixels in the adjacent pixel region. Based on the first image displacement, the overlapping image region between the reference frame image and the key frame image is determined, and based on the overlapping image region, the second image displacement between the reference frame image and the key frame image is determined; Based on the second image displacement, the image region to be stitched is determined from the keyframe image, and the image region to be stitched is processed to generate a content screenshot corresponding to the target content.

2. The method according to claim 1, characterized in that, The reference frame image is adjacent to the candidate frame image; obtaining the first image displacement between the reference frame image and the candidate image includes: The number of pixels occupied by the foreground content in the reference frame image is determined to obtain the foreground pixels, and the number of pixels occupied by the background content in the reference frame image is determined to obtain the background pixels. The pixel values ​​of the foreground pixels in the reference frame image are updated to a first pixel value, and the pixel values ​​of the background pixels in the reference frame image are updated to a second pixel value, thereby generating a mask image corresponding to the reference frame image; the first pixel value and the second pixel value are different. The foreground pixels through which the preset lines pass in the mask image are determined to obtain the first displacement positioning point; based on the first displacement positioning point, the first image displacement between the reference frame image and the candidate frame image is determined.

3. The method according to claim 1, characterized in that, The reference frame image and the candidate frame image are not adjacent; obtaining the first image displacement between the reference frame image and the candidate image includes: Obtain a video segment with a reference frame image as the starting frame image and the candidate frame image as the ending frame image, and calculate the image displacement between two adjacent frames in the video segment; The image displacement between two adjacent frames in the video segment is statistically analyzed to obtain the first image displacement between the reference frame image and the candidate frame image.

4. The method according to claim 1, characterized in that, The step of determining the overlapping image region between the reference frame image and the keyframe image based on the first image displacement, and determining the second image displacement between the reference frame image and the keyframe image based on the overlapping image region, includes: Based on the first image displacement, the region overlapping with the reference frame image is determined from the keyframe image to obtain the first overlapping image region; Based on the first image displacement, the region overlapping with the keyframe image is determined from the reference frame image to obtain the second overlapping image region; A template image region is determined from the first overlapping region, and a candidate image region is determined from the second overlapping image region. The similarity between the template image region and the candidate image region is calculated to obtain a similarity score. If the similarity score is greater than the similarity threshold, the position of the top left vertex of the template image region in the reference frame image is obtained to obtain the first position, and the position of the top left vertex of the template image region in the keyframe image is obtained to obtain the second position. Based on the first position and the second position, a second image displacement between the reference frame image and the key frame image is determined.

5. The method according to claim 4, characterized in that, The method further includes: If the similarity score is less than the similarity threshold, feature points of the first overlapping image region are extracted to obtain a first feature point set, and feature points of the second overlapping image region are extracted to obtain a second feature point set. For each first feature point in the first feature point set, a second feature point matching the first feature point is determined from the second feature point set; Calculate the longitudinal distance between each of the first feature points and the matched second feature points; The second image displacement between the reference frame image and the key frame image is determined based on the longitudinal distances described above.

6. The method according to claim 1, characterized in that, The step of performing image stitching processing on the image region to be stitched to generate a screenshot corresponding to the target content includes: Based on the current stitched image, the image region to be stitched is stitched together to obtain a new current stitched image, and the keyframe image is determined as the new reference frame image; Returning to the next frame image from the reference frame image, and determining candidate frame images from the target video according to the display order of the video frame images in the target video, until the candidate frame images have traversed all the video frame images in the target video; The current stitched image obtained by traversing the video frame images in the target video using candidate frame images is determined as the content screenshot corresponding to the target content.

7. A screenshot generation device, characterized in that, The device includes: The video recording module is used to record the target content in response to a screenshot operation on the target content to obtain the target video; The image acquisition module is used to acquire reference frame images from the target video, and starting from the next frame image of the reference frame image, determine candidate frame images from the target video according to the display order of video frame images in the target video; The first displacement determination module is used to obtain a first image displacement between the reference frame image and the candidate frame image, and to determine the candidate frame image as a keyframe image if the first image displacement is greater than a displacement threshold. The first image displacement is determined based on a displacement positioning point, which is the intersection of lines in different directions in the image. The change in gradient direction and gradient magnitude between the displacement positioning point and its adjacent pixel region is greater than the change in gradient direction and gradient magnitude between pixels in the adjacent pixel region. The second displacement determination module is used to determine the overlapping image region between the reference frame image and the key frame image based on the first image displacement, and to determine the second image displacement between the reference frame image and the key frame image based on the overlapping image region; The image stitching module is used to determine the image region to be stitched from the keyframe image based on the second image displacement, perform image stitching processing on the image region to be stitched, and generate a content screenshot corresponding to the target content.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images