Image processing method, terminal device and computer readable storage medium

Abstract

The application discloses an image processing method, a terminal device and a computer readable storage medium. The image processing method comprises the following steps: preprocessing image information to obtain a bitmap, the image information corresponding to an image frame of a to-be-processed video, a display element of the image frame comprising a main body; generating a main body contour path corresponding to the main body according to the bitmap; determining a background mask region according to a playing window and the main body contour path, the playing window being an actual playing region of the to-be-processed video, the background mask region corresponding to a region in which the main body is not displayed in the image frame; and determining a cutting region according to a display window and the background mask region, the display window being located in the playing window, and the display element in the image frame, except the main body, only displaying a corresponding part located in the display window. The image processing method can realize a frame-out special effect of the main body, does not need to re-edit the video, and can adapt to different display interfaces.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, a terminal device, and a computer-readable storage medium. Background Technology

[0002] In related technologies, to achieve the "out-of-frame" effect of naked-eye 3D in a video, the background outside the video frame needs to be cropped according to the size of the video frame to retain only the main content of the video. White space is left to match the background color of the playback page, creating the effect that the video appears to jump out of the video frame. Therefore, if the display interface changes, the video needs to be re-edited to adapt to the new display. Summary of the Invention

[0003] This invention provides an image processing method, a terminal device, and a computer-readable storage medium.

[0004] An image processing method according to an embodiment of the present invention includes:

[0005] Image information is preprocessed to obtain a bitmap, wherein the image information corresponds to an image frame of the video to be processed, and the display elements of the image frame include the main body;

[0006] Generate a main body outline path corresponding to the main body based on the bitmap;

[0007] The background mask area is determined based on the playback window and the outline path of the main subject. The playback window is the actual playback area of ​​the video to be processed, and the background mask area corresponds to the area in the image frame where the main subject is not displayed.

[0008] The cropping area is determined based on the display window and the background mask area. The display window is located within the playback window, and the display elements in the image frame other than the main body only display the corresponding parts located within the display window.

[0009] The image processing method described above determines the background mask area by using the subject outline path. This ensures that the corresponding cropping area always corresponds to a position outside the display area within the playback window, and the background mask area does not display the subject. After processing the cropping area, the part of the subject located outside the display area can also be displayed within the playback window, while other display elements located outside the display window are not displayed. This achieves the subject out-of-frame effect without requiring video re-editing and can adapt to different display interfaces.

[0010] In some implementations, generating a subject outline path corresponding to the subject based on the bitmap includes:

[0011] Perform contour detection on the bitmap to generate the contour information of the main body;

[0012] Based on the bitmap and the outline information of the subject, the subject is analyzed and processed to obtain the subject outline path. In this way, the subject outline path can be obtained.

[0013] In some embodiments, the image processing method includes:

[0014] Generate an image contour detection object based on the display size of the video to be processed;

[0015] Perform contour detection on the bitmap to generate contour information of the main body, including:

[0016] The pixel intensity of the bitmap is analyzed and processed using the image contour detection object.

[0017] The contour information of the subject is generated based on the analysis results. This allows for convenient acquisition of the subject's contour information.

[0018] In some embodiments, the image processing method includes:

[0019] Create an image detection result processing object;

[0020] Based on the bitmap and the outline information of the subject, the subject is analyzed and processed to obtain the subject outline path, including:

[0021] In the image detection result processing object, the contour information of the bitmap and the subject is analyzed and processed to generate an image contour result object;

[0022] The contour coordinate set of the subject is extracted from the image contour result object to obtain the subject contour path. This allows the subject contour path to be obtained.

[0023] In some implementations, determining the background mask area based on the playback window and the subject outline path includes:

[0024] Generate an inverted Bézier path based on the playback window and the main body outline path;

[0025] The background mask area is determined based on the inverted Bezier path and the mask layer object, where the mask layer object corresponds to the size of the playback window. This allows the background mask area to be obtained.

[0026] In some implementations, generating an inverted Bézier path based on the playback window and the main outline path includes:

[0027] Create a first Bezier path based on the size of the playback window;

[0028] Create a second Bezier path based on the main body contour path;

[0029] Remove the second Bézier path from the first Bézier path to generate the inverted Bézier path;

[0030] Import the inverted Bezier path into the mask layer object to determine the background mask area. This makes it easy to determine the background mask area.

[0031] In some implementations, determining the cropping area based on the display window and the background mask area includes:

[0032] Based on the Bezier path corresponding to the display window, path splicing is performed within the background mask area to determine the cropping area. This allows for the determination of the cropping area.

[0033] In some embodiments, the image processing method includes:

[0034] The contour path set corresponding to the cropped area and the current graphic frame are encapsulated to generate contour timeline information, which includes the time information of the current graphic frame within the video to be processed. This saves computational resources.

[0035] A terminal device according to an embodiment of the present invention includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of any of the above-described image processing methods.

[0036] The image processing method described above determines the background mask area by using the subject outline path. This ensures that the corresponding cropping area always corresponds to a position outside the display area within the playback window, and the background mask area does not display the subject. After processing the cropping area, the part of the subject located outside the display area can also be displayed within the playback window, while other display elements located outside the display window are not displayed. This achieves the effect of the subject going out of the frame, without requiring re-editing the video, and can adapt to different display interfaces.

[0037] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image processing method described in any of the above embodiments.

[0038] The image processing method described above determines the background mask area by using the subject outline path. This ensures that the corresponding cropping area always corresponds to a position outside the display area within the playback window, and the background mask area does not display the subject. After processing the cropping area, the part of the subject located outside the display area can also be displayed within the playback window, while other display elements located outside the display window are not displayed. This achieves the effect of the subject going out of the frame, without requiring re-editing the video, and can adapt to different display interfaces.

[0039] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0040] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0041] Figure 1 This is a flowchart of the image processing method according to an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of a terminal device according to an embodiment of the present invention;

[0043] Figures 3-6 This is a schematic diagram of the playback window according to an embodiment of the present invention;

[0044] Figure 7 and Figure 8 This is a flowchart of the image processing method according to an embodiment of the present invention;

[0045] Figure 9 and Figure 10 This is a schematic diagram of a mobile device according to an embodiment of the present invention.

[0046] Explanation of key component symbols:

[0047] 100 terminal devices;

[0048] Main body 110, main body outline path 120, background mask area 130, cropping area 140;

[0049] Playback window 210, display window 220;

[0050] Mobile device 300. Detailed Implementation

[0051] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0052] Please refer to Figure 1 An image processing method according to an embodiment of the present invention includes:

[0053] 01: Preprocess the image information to obtain a bitmap. The image information corresponds to the image frame of the video to be processed. The display elements of the image frame include the main body 110.

[0054] 02: Generate the main outline path 120 of the corresponding main body 110 based on the bitmap;

[0055] 03: Determine the background mask area 130 based on the playback window 210 and the main body outline path 120. The playback window 210 is the actual playback area of ​​the video to be processed, and the background mask area 130 corresponds to the area in the image frame where the main body 110 is not displayed.

[0056] 04: Determine the cropping area 140 based on the display window 220 and the background mask area 130. The display window 220 is located within the playback window 210. In the image frame, display elements other than the main body 110 are only displayed in the corresponding parts located within the display window 220.

[0057] The image processing method of this invention can be implemented by the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: preprocess image information to obtain a bitmap, the image information corresponding to an image frame of the video to be processed, the display elements of the image frame including the main body 110; generate a main body outline path 120 corresponding to the main body 110 based on the bitmap; determine a background mask area 130 based on the playback window 210 and the main body outline path 120, the playback window 210 being the actual playback area of ​​the video to be processed, the background mask area 130 corresponding to the area in the image frame where the main body 110 is not displayed; determine a cropping area 140 based on the display window 220 and the background mask area 130, the display window 220 being located within the playback window 210, and display elements in the image frame other than the main body 110 only displaying the corresponding parts located within the display window 220.

[0058] The image processing method and terminal device 100 described above determine the background mask area 130 through the main body outline path 120, so that the corresponding cropping area 140 always corresponds to the position outside the display area within the playback window 210, and the background mask area 130 does not display the main body 110. After processing the cropping area 140, the part of the main body 110 located outside the display area can also be displayed within the playback window 210, while other display elements located outside the display window 220 are not displayed, thereby achieving the out-of-frame effect of the main body 110, without the need to re-edit the video, and can adapt to different display interfaces.

[0059] In one implementation, the current frame of the video to be processed can be obtained using a preset pixel image type, and this current frame can be used as an image frame. Before playing the image frame, the image information of the current frame can be determined. The preset pixel image type can be CVPixelBufferRef.

[0060] It is understandable that an image frame can display multiple display elements. When the video to be processed is played, these multiple display elements in the image frame will be displayed accordingly. The main body 110 represents the object that is expected to be highlighted in the video to be processed.

[0061] A bitmap can be a raster image of pixels corresponding to an image frame. Please refer to... Figure 3 In one implementation, upon obtaining image information, a preset filter object can be used to apply an image filter to preprocess the image information, resulting in a bitmap displaying the subject 110. The preset filter object can be a CIFilter. The image filter can be a black and white filter. Preprocessing the image information using a black and white filter ensures that the resulting bitmap contains only black and white pixels, thus simplifying the complexity of the bitmap's image information. Subsequent processing of the bitmap will not increase processing time due to the inclusion of complex image information. The preset filter object can be a CIFilter object. The bitmap can be a CGImageRef bitmap object. Figure 3 In the text, areas where the main body 110 is not displayed are marked in gray, while areas where the main body 110 is displayed are marked in white.

[0062] Please combine Figure 4 Given a bitmap, the main subject 110 in the bitmap can be processed to create a subject outline path 120 corresponding to the main subject 110 in the bitmap. The subject outline path 120 corresponds to the display area occupied by the main subject 110 in the bitmap. Figure 4 In the middle, the main outline path 120 is marked with a shadow.

[0063] Please combine Figure 5After determining the main subject outline path 120, the background mask area 130 needs to be determined based on the playback window 210 and the main subject outline path 120. The playback window 210 is the actual playback area of ​​the video to be processed. The maximum image size of the bitmap matches the window size of the playback window 210, so that the window border of the playback window 210 corresponds to the display boundary of the bitmap. Within the display boundary, the area where the main subject 110 is not displayed can be determined as the background mask area 130. Figure 5 In the middle, the background mask area 130 is marked with a shadow.

[0064] Please combine Figure 6 After determining the background mask area 130, the corresponding cropping area 140 needs to be determined based on the display window 220 and the background mask area 130. The display window 220 is the interface display window for the video to be processed. It should be noted that the playback window 210 is adapted to the video display size, and primarily displays the corresponding portion of the video to be processed within the display window 220. Figure 6 In the playback window 210, the area corresponding to the main subject 110 is partly located within the display window 220 and partly located outside the display window 220. Since the background mask area 130 does not include the area corresponding to the main subject 110, based on the display window 220 and the background mask area 130, the portion outside the display window 220 can be defined as the cropping area 140 within the background mask area 130. After cropping the cropping area 140, when the video to be processed is played, only all display elements corresponding to the display window 220 of the video to be processed, as well as the display elements of the main subject 110 located outside the display window 220, will be displayed. Display elements located outside the display window 220 but not including the main subject 110 will not be displayed. Cropping the cropping area 140 can be used to crop the display elements within the cropping area 140, so that the display elements within the cropping area 140 are not displayed. Figure 6 In the image, the cropped area 140 is marked with a shaded area.

[0065] In addition, the display window 220 can correspond to the default size of the video to be processed. After responding to the request to enlarge the display of the video to be processed, the size of the video to be processed can be enlarged and adapted to the playback window 210, so that all display elements in the video to be processed located within the display window 220 will be displayed, while only the main body 110 will be displayed for display elements located within the playback window 210 but outside the display window 220.

[0066] Terminal device 100 can be a server.

[0067] Please refer to Figure 7In some implementations, step 02 (generating the main outline path 120 of the corresponding main body 110 based on the bitmap) includes:

[0068] 021: Perform contour detection on the bitmap to generate the contour information of the main body 110;

[0069] 022: Based on the bitmap and the outline information of the main body 110, the main body 110 is analyzed and processed to obtain the main body outline path 120.

[0070] The image processing method of this invention can be implemented using the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: perform contour detection on the bitmap to generate contour information of the main body 110; and analyze and process the main body 110 based on the bitmap and the contour information of the main body 110 to obtain the main body contour path 120.

[0071] In this way, the main body outline path 120 can be obtained.

[0072] Specifically, in the contour detection of the bitmap, the features of the bitmap can be learned and analyzed to obtain the contour information of the subject 110. Since the contour information of the subject 110 carries the image features of the subject 110 in the bitmap, the display area of ​​the subject 110 in the bitmap can be determined based on the bitmap and the contour information of the subject 110, and the display area of ​​the subject 110 in the bitmap is determined as the subject contour path 120.

[0073] In some implementations, the image processing method includes:

[0074] Generate image contour detection objects based on the display size of the video to be processed;

[0075] Step 021 (Perform contour detection on the bitmap to generate contour information of the main body 110), includes:

[0076] The pixel intensity of the bitmap is analyzed and processed by detecting objects through image contours.

[0077] Based on the analysis results, the outline information of the main body 110 is generated.

[0078] The image processing method of this invention can be implemented using the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: generate an image contour detection object based on the display window 220; analyze and process the pixel intensity of the bitmap through the image contour detection object; and generate the contour information of the main body 110 based on the analysis results.

[0079] In this way, the outline information of the main body 110 can be easily obtained.

[0080] Specifically, an image contour detection object can be obtained by initializing the detection based on the display size of the video to be processed. Then, in a preset detection mode, the image contour detection object can be used to learn and analyze the pixel intensity of the bitmap. Since the video to be processed has multiple image frames, each image frame can correspond to a bitmap, thus the video to be processed can be analyzed and processed. The image features of the subject 110 in the bitmap can be obtained from the changes of the subject 110 in consecutive image frames, and thus the contour information of the subject 110 can be obtained. The image contour detection object can be VNDetectContoursRequest. The preset detection mode can be the detectsDarkOnLight mode.

[0081] In some implementations, the image processing method includes:

[0082] Create an image detection result processing object;

[0083] Step 022 (based on the bitmap and the outline information of the main body 110, analyze and process the main body 110 to obtain the main body outline path 120), including:

[0084] In the image detection result processing object, the contour information of the bitmap and the main body 110 is analyzed and processed to generate an image contour result object;

[0085] Extract the set of contour coordinates of the subject 110 from the image contour result object to obtain the subject contour path 120.

[0086] The image processing method of this invention can be implemented by the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: create an image detection result processing object; analyze and process the contour information of the bitmap and the subject 110 in the image detection result processing object to generate an image contour result object; and extract the contour coordinate set of the subject 110 in the image contour result object to obtain the subject contour path 120.

[0087] In this way, the main body outline path 120 can be obtained.

[0088] Specifically, in one implementation, after creating the image detection result processing object, a bitmap and the corresponding contour information of the subject 110 can be input into the image detection result processing object. This allows for image contour analysis of the area of ​​the subject 110 in the bitmap, resulting in an image contour result object. Extraction is then performed within the image contour result object to determine the subject contour path 120 using the extracted set of contour coordinates of the main image. The image detection result processing object can be a VNImageRequestHandler. The image contour result object can be a VNContoursObservation. The set of contour coordinates of the subject 110 can be a CGPathRef object. The subject contour path 120 can be a normalizedPath.

[0089] Please refer to Figure 8 In some implementations, step 03 (determining the background mask area 130 based on the playback window 210 and the main outline path 120) includes:

[0090] 031: Generate an inverted Bézier path based on the playback window 210 and the main outline path 120;

[0091] 032: Determine the background mask area 130 based on the inverted Bezier path and the mask layer object, with the mask layer object corresponding to the size of the playback window 210.

[0092] The image processing method of this invention can be implemented by the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: generate an inverted Bézier path based on the playback window 210 and the main outline path 120; and determine a background mask area 130 based on the inverted Bézier path and the mask layer object, wherein the mask layer object corresponds to the size of the playback window 210.

[0093] In this way, the background mask area 130 can be obtained.

[0094] Specifically, playback window 210 corresponds to the actual playback area of ​​the video to be processed, and subject outline path 120 corresponds to the area in the video to be processed where subject 110 is displayed. Through playback window 210 and subject outline path 120, the inverted Bézier path can be determined for all areas in the video to be processed where subject 110 is not displayed. In playback window 210, the mask layer object is combined with the inverted Bézier path in the form of a mask, thereby completing the acquisition of all areas in the video to be processed where subject 110 is not displayed.

[0095] Additionally, the mask layer object can be created from a preset image library based on the corresponding layer class, so that the resulting mask layer object corresponds to the video rectangular area of ​​the playback window 210. The preset image library can be the CoreGraphics underlying graphics library. The corresponding layer class can be the CAShapeLayer class.

[0096] In some implementations, step 031 (generating an inverted Bézier path based on the playback window 210 and the main outline path 120) includes:

[0097] Create a first Bezier path based on the size of the playback window 210;

[0098] Create a second Bezier path based on the main outline path 120;

[0099] Remove the second Bézier path from the first Bézier path to generate a reversed Bézier path;

[0100] Import the inverted Bezier path into the mask layer object and define the background mask area as 130.

[0101] The image processing method of this invention can be implemented using the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: create a first Bezier path based on the size of the playback window 210; create a second Bezier path based on the main outline path 120; remove the second Bezier path from the first Bezier path to generate an inverted Bezier path; import the inverted Bezier path into the mask layer object to determine the background mask area 130.

[0102] This makes it easy to determine the background mask area 130.

[0103] Specifically, a graphic path can be created based on the size of the rectangular player corresponding to the playback window 210, thereby obtaining the first Bézier path of the window border of the playback window 210. After obtaining the main outline path 120, a graphic path can be created through the main outline path 120, thereby obtaining the second Bézier path of the display boundary of the main body 110 in the video to be processed. Using a preset path algorithm, the area corresponding to the second Bézier path can be removed from the area corresponding to the first Bézier path, thereby generating an inverted Bézier path based on the remaining area. After determining the inverted Bézier path, it can be imported into a mask layer object, thereby obtaining the background mask area 130 through a mask. The first and second Bézier paths can be UIBezierPath objects.

[0104] In some implementations, the Vison machine learning image recognition framework can be used to obtain the subject's outline path.

[0105] In some implementations, step 04 (determining the clipping region 140 based on the display window 220 and the background mask region 130) includes:

[0106] Based on the Bezier path corresponding to the display window 220, path splicing is performed within the background mask area 130 to determine the cropping area 140.

[0107] The image processing method of this invention can be implemented by the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: perform path splicing processing within the background mask area 130 according to the Bezier path corresponding to the display window 220, and determine the cropping area 140.

[0108] In this way, the cutting area 140 can be determined.

[0109] Specifically, after determining the background mask area 130, the Bezier path corresponding to the display window 220 can be obtained by creating a path. Then, according to the preset path splicing algorithm, the Bezier path corresponding to the display window 220 is spliced ​​in the background mask area 130 to remove the area of ​​the corresponding display window 220 in the background mask area 130 and finally obtain the cropping area 140.

[0110] Image processing methods include:

[0111] The contour path set corresponding to the cropping region 140 and the current graphic frame are encapsulated to generate contour timeline information, which includes the time information of the current graphic frame within the video to be processed.

[0112] The image processing method of this invention can be implemented by the terminal device 100 of this invention. Specifically, please refer to... Figure 2 The terminal device 100 is used to: encapsulate the set of contour paths corresponding to the cropping area 140 and the current graphic frame to generate contour timeline information, which includes the time information of the current graphic frame in the video to be processed.

[0113] This saves computing resources.

[0114] Specifically, by generating outline timeline information, a caching effect can be achieved. When playing the video to be processed later, the video to be processed can be played directly without needing to be processed again, thus achieving a real-time out-of-frame visual effect. The outline timeline information can be a JSON object.

[0115] Furthermore, by generating contour timeline information, dynamic masking effects can be achieved in response to operations such as progress bar sliding or replaying of the video being processed. Specifically, in Figure 8 and Figure 9 The video to be processed is displayed on the mobile device 300. Figure 8 In this context, the display size of the video to be processed corresponds to the size of the display window 220, with the main body 110 entirely within the display window 220. When it is necessary to enlarge a portion of the video to be processed, a magnification request can be responded to, increasing the display size of the video to be processed so that the enlarged display size corresponds to the size of the playback window 210, such as... Figure 10 As shown, part of the main body 110 is located within the display window 220, while part is located outside the display window 220. Figure 9 and Figure 10 In this case, the corresponding cropping area 140 can be immediately determined through the outline timeline information for cropping processing, so that in the enlarged video to be processed, all display elements located in the display window 220 will still be displayed, while only the main body 110 will be displayed for display elements located outside the display window 220.

[0116] Additionally, please combine Figure 6 In one implementation, a player corresponding to the playback window 210 can be created first, and a background mask area 130 can be superimposed on the player. The superimposed part of the background mask area 130 and the player can be determined as the clipping area 140 for clipping. The clipped area is then exposed to display the display interface of the mobile device 300 as the background, thereby achieving a real-time out-of-frame effect.

[0117] Based on the above, the image processing method of the present invention, compared with related technologies that require video editing to achieve the out-of-frame effect, can achieve the following effects:

[0118] 1. No need to re-edit the video to remove the background that extends beyond the border;

[0119] 2. The size of the video frame can be freely adjusted on the video playback device (iOS mobile app);

[0120] 3. The background of the video playback page can be designed freely and does not need to be the same color as the white space after the video background is cropped;

[0121] 4. This out-of-frame video effect can be implemented on the client side without relying on backend pre-parsing and processing.

[0122] A terminal device 100 according to an embodiment of the present invention includes a processor and a memory. The memory stores a computer program. When the processor executes the computer program, it implements the steps of the image processing method of any of the above embodiments.

[0123] The aforementioned terminal device 100 determines the background mask area 130 through the main body outline path 120, which ensures that the corresponding cropping area 140 always corresponds to the position outside the display area within the playback window 210, and the background mask area 130 does not display the main body 110. After processing the cropping area 140, the part of the main body 110 located outside the display area can also be displayed within the playback window 210, while other display elements located outside the display window 220 are not displayed, thereby achieving the out-of-frame effect of the main body 110 without needing to re-edit the video and adapting to different display interfaces.

[0124] For example, when a computer program is executed, the following steps can be achieved:

[0125] 01: Preprocess the image information to obtain a bitmap. The image information corresponds to the image frame of the video to be processed. The display elements of the image frame include the main body 110.

[0126] 02: Generate the main outline path 120 of the corresponding main body 110 based on the bitmap;

[0127] 03: Determine the background mask area 130 based on the playback window 210 and the main body outline path 120. The playback window 210 is the actual playback area of ​​the video to be processed, and the background mask area 130 corresponds to the area in the image frame where the main body 110 is not displayed.

[0128] 04: Determine the cropping area 140 based on the display window 220 and the background mask area 130. The display window 220 is located within the playback window 210. In the image frame, display elements other than the main body 110 are only displayed in the corresponding parts located within the display window 220.

[0129] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of the image processing method of any of the above embodiments.

[0130] The aforementioned computer-readable storage medium determines the background mask area 130 through the main body outline path 120, ensuring that the corresponding cropping area 140 always corresponds to a position outside the display area within the playback window 210. Furthermore, the background mask area 130 does not display the main body 110. After processing the cropping area 140, the portion of the main body 110 located outside the display area can also be displayed within the playback window 210, while other display elements located outside the display window 220 are not displayed. This achieves the out-of-frame effect of the main body 110 without requiring re-editing of the video and adapts to different display interfaces.

[0131] For example, when a computer program is executed, the following steps can be achieved:

[0132] 01: Preprocess the image information to obtain a bitmap. The image information corresponds to the image frame of the video to be processed. The display elements of the image frame include the main body 110.

[0133] 02: Generate the main outline path 120 of the corresponding main body 110 based on the bitmap;

[0134] 03: Determine the background mask area 130 based on the playback window 210 and the main body outline path 120. The playback window 210 is the actual playback area of ​​the video to be processed, and the background mask area 130 corresponds to the area in the image frame where the main body 110 is not displayed.

[0135] 04: Determine the cropping area 140 based on the display window 220 and the background mask area 130. The display window 220 is located within the playback window 210. In the image frame, display elements other than the main body 110 are only displayed in the corresponding parts located within the display window 220.

[0136] The computer-readable storage medium can be located in the terminal device 100 or in other terminals. The terminal device 100 can communicate with other terminals to obtain the corresponding program.

[0137] It is understood that computer-readable storage media can include: any entity or device capable of carrying computer programs, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. Computer programs include computer program code. Computer program code can be in the form of source code, object code, executable files, or certain intermediate forms, etc. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media.

[0138] In some embodiments of the present invention, the terminal device 100 integrates a processor, a memory, a communication module, etc. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0139] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a system including a processing module or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0141] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0142] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An image processing method, characterized in that, The image processing method includes: Image information is preprocessed to obtain a bitmap, wherein the image information corresponds to an image frame of the video to be processed, and the display elements of the image frame include the main body; Generate a main body outline path corresponding to the main body based on the bitmap; The background mask area is determined based on the playback window and the outline path of the main subject. The playback window is the actual playback area of ​​the video to be processed, and the background mask area corresponds to the area in the image frame where the main subject is not displayed. The cropping area is determined based on the display window and the background mask area. The display window is located within the playback window. In the image frame, display elements other than the main body are only displayed in the corresponding parts located within the display window. The image processing method includes: The contour path set corresponding to the cropped area and the current graphic frame are encapsulated to generate contour timeline information, which includes the time information of the current graphic frame within the video to be processed.

2. The image processing method according to claim 1, characterized in that, Generating a subject outline path corresponding to the subject based on the bitmap includes: Perform contour detection on the bitmap to generate the contour information of the main body; Based on the bitmap and the outline information of the subject, the subject is analyzed and processed to obtain the subject outline path.

3. The image processing method according to claim 2, characterized in that, The image processing method includes: Generate an image contour detection object based on the display size of the video to be processed; Perform contour detection on the bitmap to generate contour information of the main body, including: The pixel intensity of the bitmap is analyzed and processed using the image contour detection object. The contour information of the subject is generated based on the analysis results.

4. The image processing method according to claim 2, characterized in that, The image processing method includes: Create an image detection result processing object; Based on the bitmap and the outline information of the subject, the subject is analyzed and processed to obtain the subject outline path, including: In the image detection result processing object, the contour information of the bitmap and the subject is analyzed and processed to generate an image contour result object; Extract the set of contour coordinates of the subject from the image contour result object to obtain the subject contour path.

5. The image processing method according to claim 1, characterized in that, The background mask area is determined based on the playback window and the main subject outline path, including: Generate an inverted Bézier path based on the playback window and the main body outline path; The background mask area is determined based on the inverted Bezier path and the mask layer object, where the mask layer object corresponds to the size of the playback window.

6. The image processing method according to claim 5, characterized in that, Generate an inverted Bézier path based on the playback window and the main outline path, including: Create a first Bezier path based on the size of the playback window; Create a second Bezier path based on the main body contour path; Remove the second Bézier path from the first Bézier path to generate the inverted Bézier path; Import the inverted Bezier path into the mask layer object to determine the background mask area.

7. The image processing method according to claim 1, characterized in that, The cropping area is determined based on the display window and the background mask area, including: Based on the Bezier path corresponding to the display window, path splicing is performed within the background mask area to determine the cropping area.

8. A terminal device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and when the processor executes the computer program, it implements the steps of the image processing method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image processing method according to any one of claims 1-7.

Images