Image processing method, device, equipment and storage medium

Abstract

The embodiment of the present invention relates to the field of image processing technology, and provides an image processing method, device, equipment and storage medium, and relates to video editing software. The video editing software provides preview controls and drawing special effects. The method includes: creating a timeline object, connecting the timeline object with the preview control to obtain a video base class; adding drawing special effects to the video base class; the video base class is used to indicate the target multimedia resource to be processed; responding to the drawing processing instruction, processing a preset dot matrix sequence on the target multimedia resource through the drawing special effect; and previewing the target multimedia resource after line marking. By combining the video editing capabilities of the video editing software, and through the preview controls and drawing special effects provided to the user on the upper interface, the processing of the preset dot matrix sequence on the multimedia resource is realized through the drawing special effect, and then the line marking operation of the original video or picture is realized, which is convenient for user operation.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, an image processing device, a corresponding electronic device, and a corresponding computer-readable storage medium. Background Art

[0002] In video post-processing software, there is usually a need to add line markings to the original video or picture, such as circling the key points in the video, marking the important parts of the picture with lines, etc.

[0003] In the related technologies of video or image processing, in order to meet the above-mentioned requirements, the operations adopted during processing require decoding the video or image, and then performing line marking processing on the decoded data to export a video or image that meets the user's expectations. However, the operation of performing line marking on the data after decoding is not conducive to use by ordinary users. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide an image processing method, an image processing device, a corresponding electronic device and a corresponding computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0005] An embodiment of the present invention discloses an image processing method, which relates to video editing software. The video editing software provides a preview control and a drawing special effect. The method includes:

[0006] Create a timeline object, and connect the timeline object with the preview control to obtain a video base class;

[0007] Adding the drawing special effect to the video base class; the video base class is used to indicate the target multimedia resource to be processed;

[0008] In response to the drawing processing instruction, a preset dot matrix sequence is processed on the target multimedia resource by the drawing special effect; the preset dot matrix sequence is used to indicate a line mark;

[0009] The target multimedia resource marked by the line is previewed.

[0010] Optionally, the creating of the timeline object, based on connecting the timeline object with the preview control, to obtain a video base class, includes:

[0011] Obtaining a preset software development kit of the video editing software, and initializing the preset software development kit;

[0012] Creating a timeline object based on the initialized preset software development kit; the timeline object is used to add multimedia resources selected by the user;

[0013] The timeline object is connected to the preview control to obtain a video base class.

[0014] Optionally, connecting the timeline object with the preview control includes:

[0015] Obtaining the initialized streaming media context object of the preset software development kit;

[0016] The timeline object and the preview control are passed in as parameters of a preset interface through the streaming media context object to connect the timeline object and the preview control.

[0017] Optionally, after adding the drawing special effect to the video base class, a video processing object is obtained;

[0018] The method further comprises:

[0019] Adding the drawing special effect to the video processing object to obtain a drawing configuration object; the drawing configuration object is used to set drawing processing parameters;

[0020] The drawing processing parameters include at least one of the following: line width, line color, a first dot matrix sequence for a target marking line to be added, and a second dot matrix sequence for an existing historical marking line.

[0021] Optionally, the responding to the drawing processing instruction and processing the preset dot matrix sequence on the target multimedia resource by using the drawing special effect includes:

[0022] In response to the drawing processing instruction, obtaining the drawing processing parameters in the drawing configuration object;

[0023] Through the drawing effect, the preset dot matrix sequence is processed on the target multimedia resource according to the drawing processing parameters.

[0024] Optionally, the preset dot matrix sequence includes the first dot matrix sequence and / or the second dot matrix sequence; and the processing of the preset dot matrix sequence on the target multimedia resource according to the drawing processing parameters by the drawing special effect includes:

[0025] By means of the drawing special effect, the first dot matrix sequence is set on the target multimedia resource with a corresponding target marking line according to the line width and the line color;

[0026] And / or, by using the drawing special effect, clearing the historical marking line corresponding to the second dot matrix sequence on the target multimedia resource.

[0027] Optionally, before the response drawing processing instruction, the method further includes:

[0028] Responding to the drawing processing operation, obtaining drawing processing parameters;

[0029] Fill the drawing processing parameters into the drawing configuration object to generate a drawing processing instruction;

[0030] The drawing processing operation is a mouse operation, and the mouse operation includes a mouse click operation, a mouse release operation and a mouse move operation.

[0031] The embodiment of the present invention further discloses an image processing device, which relates to video editing software. The video editing software provides a preview control and drawing special effects. The device includes:

[0032] A preview control connection module, used for creating a timeline object, and obtaining a video base class based on connecting the timeline object with the preview control;

[0033] A drawing special effect adding module, used for adding the drawing special effect to the video base class; the video base class is used for indicating the target multimedia resource to be processed;

[0034] A dot matrix sequence processing module, used for responding to a drawing processing instruction, and processing a preset dot matrix sequence on the target multimedia resource through the drawing special effect; the preset dot matrix sequence is used for indicating a line mark;

[0035] The effect preview module is used to preview the target multimedia resource after the line marking.

[0036] Optionally, the preview control connection module includes:

[0037] The preview control connection submodule is used to obtain the preset software development kit of the video editing software and initialize the preset software development kit; create a timeline object based on the initialized preset software development kit; the timeline object is used to add multimedia resources selected by the user; and connect the timeline object with the preview control to obtain a video base class.

[0038] Optionally, the preview control connection submodule includes:

[0039] The timeline object connection submodule is used to obtain the streaming media context object of the initialized preset software development kit; through the streaming media context object, the timeline object and the preview control are passed in as parameters of the preset interface to connect the timeline object and the preview control.

[0040] Optionally, after adding the drawing special effect to the video base class, a video processing object is obtained; the device further comprises:

[0041] A drawing configuration object processing module is used to add the drawing special effect to the video processing object to obtain a drawing configuration object; the drawing configuration object is used to set drawing processing parameters; wherein the drawing processing parameters include at least one of the following: line width, line color, a first dot matrix sequence for a target marking line to be added, and a second dot matrix sequence for an existing historical marking line.

[0042] Optionally, the dot matrix sequence processing module includes:

[0043] The dot matrix sequence processing submodule is used to respond to the drawing processing instruction and obtain the drawing processing parameters in the drawing configuration object; through the drawing effect, the preset dot matrix sequence is processed on the target multimedia resource according to the drawing processing parameters.

[0044] Optionally, the preset dot matrix sequence includes the first dot matrix sequence and / or the second dot matrix sequence, and the dot matrix sequence processing submodule includes:

[0045] A dot sequence processing unit is used to set a corresponding target marking line for the first dot sequence on the target multimedia resource according to the line width and the line color through the drawing special effect; and / or, through the drawing special effect, clear the historical marking line corresponding to the second dot sequence on the target multimedia resource.

[0046] Optionally, before the response drawing processing instruction, the device further includes:

[0047] The drawing processing instruction generation module is used to respond to the drawing processing operation and obtain the drawing processing parameters; fill the drawing processing parameters into the drawing configuration object to generate the drawing processing instruction; wherein the drawing processing operation is a mouse operation, and the mouse operation includes a mouse click operation, a mouse release operation and a mouse move operation.

[0048] An embodiment of the present invention further discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements any one of the image processing methods when executed by the processor.

[0049] An embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any one of the image processing methods is implemented.

[0050] The embodiments of the present invention include the following advantages:

[0051] In an embodiment of the present invention, through the preview control and drawing special effects provided by the video editing software, based on connecting the timeline object with the preview control, and adding the drawing special effects to the video base class after the connection, the preset dot matrix sequence is processed on the target multimedia resource through the provided drawing special effects, and the real-time preview of the target multimedia resource after line marking can be realized based on the connection with the preview control. By combining the video editing capabilities of the video editing software, and through the preview control and drawing special effects provided to the user in the upper interface, the preset dot matrix sequence is processed on the multimedia resource through the drawing special effects, and the line marking operation of the original video or picture is realized, which is convenient for user operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flowchart of steps of an image processing method embodiment of the present invention;

[0053] Figure 2 is a flowchart of another image processing method embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of a drawing effect provided by an embodiment of the present invention;

[0055] Figure 4 is a schematic diagram of a scheduling process of a line marking function provided by an embodiment of the present invention;

[0056] Figure 5 It is a structural block diagram of an image processing device embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] The embodiment of the present invention combines the video editing capability of the video editing software, and through the preview control and drawing special effects provided to the user on the upper interface, the processing of the preset dot matrix sequence on the multimedia resource is realized through the drawing special effects, thereby realizing the line marking operation on the original video or picture, that is, while facilitating the user's operation, it can allow the import of any video or picture material to be processed by the preset dot matrix sequence, and achieve the effect of line marking from the functional point of view, and the scope of use is very wide; and, when responding to the drawing processing instruction, the preset dot matrix sequence can be processed through the provided drawing special effects, and then the line marking effect on the target multimedia resource can be presented to the user in real time, so that the user can preview the line marking effect of the video. Further, based on the dot matrix sequence of the target marking line to be added to the target multimedia resource and the existing dot matrix sequence, multiple line markings can be combined to meet user needs. In addition, QT (Qt Toolkit, a cross-platform application development framework) can also be used to develop an upper interface that is easy for users to operate, which is conducive to ordinary users to use the drawing function, that is, the line marking processing operation.

[0059] Reference Figure 1 , shows a flowchart of an image processing method embodiment of the present invention, which may specifically include the following steps:

[0060] Step 101, creating a timeline object, and connecting the timeline object with the preview control to obtain a video base class;

[0061] The image processing method provided by the embodiment of the present invention is applicable to PC-side image processing and special effect processing software, and can perform a marking function of any line combination on any imported original video or picture.

[0062] In an embodiment of the present invention, the image processing and special effects processing software on the PC (Personal Computer) side can provide preview controls and drawing effects with the help of the video editing capabilities of the relevant video editing software, so as to process the preset dot matrix sequence based on the provided preview controls and drawing effects, thereby functionally achieving the effect of increasing line markings.

[0063] In one embodiment of the present invention, the preview control provided is just a control. If the control is not connected to any timeline, no multimedia resources can be displayed on the preview interface presented by the control. In order to avoid a black screen, the preview control can be connected to a timeline object. Specifically, a timeline object can be created and connected to the preview control based on the timeline object, so that the user can preview the multimedia material selected by the user in real time.

[0064] In actual applications, the preview control provided by the video editing software can be used to preview the multimedia materials selected by the user in real time. In order to facilitate the user to preview the multimedia resources in the image processing process in real time, it can respond to the touch operation of the preview control and present the target multimedia resources selected by the user.

[0065] It should be noted that the preview control provided by the video editing software can be located on the upper interface. At this time, the target multimedia resources can be presented on the upper interface. In the subsequent image processing process, the multimedia resources in the image processing process can also be displayed in real time on the upper interface, and then the line marking effect for the target multimedia resources can be presented to the user in real time, so that the user can preview the line marking effect of the video.

[0066] Step 102, adding drawing effects to the video base class;

[0067] In one embodiment of the present invention, after connecting the created timeline object with the preview control, a video base class can be obtained, wherein the created timeline object can be used to add multimedia resources selected by the user, that is, the obtained video base class can be used to indicate the target multimedia resources to be processed. However, since the drawing effects provided by the video editing software are its built-in effects, at this time, drawing effects can be added to the video base class used to point to the target multimedia resources to give the drawing function for the target multimedia resources.

[0068] Step 103, responding to the drawing processing instruction, processing the preset dot matrix sequence on the target multimedia resource by drawing special effects;

[0069] The target multimedia resource pointed to may be a video or a picture that requires line marking.

[0070] After the upper-level interface presents the target multimedia resource with line marking requirements to the user, it can respond to the drawing processing instructions and use the drawing effects provided by the video editing software to process the preset dot matrix sequence on the multimedia resource, thereby functionally achieving the line marking effect.

[0071] Exemplarily, the drawing effect adopted may be the Painting Effect, which is commonly used in graphics processing and video editing software. This effect can be used to paint images, or to paint seamless textures for geometric bodies in the scene. Its working principle can be manifested as attaching a specific object to the path drawn by the brush, and then restoring it through software rendering.

[0072] In actual applications, the drawing processing instructions can be generated through the setting operation of the drawing processing parameters, and the setting operation of the drawing processing parameters can be manifested as the user's input of relevant parameters, or as the user's mouse clicking operation, mouse releasing operation and mouse moving operation on a preset dot matrix sequence on a multimedia resource, and this is not limited by the embodiments of the present invention.

[0073] In an embodiment of the present invention, the drawing processing parameters in the drawing configuration object can be obtained by responding to the drawing processing instruction, and then the preset dot matrix sequence can be processed on the target multimedia resource according to the drawing processing parameters through the drawing special effect.

[0074] In a specific implementation, the processing of the preset dot matrix sequence can be expressed as an addition operation on the preset dot matrix sequence to be added, so as to add corresponding line marks on the target multimedia resource; it can also be expressed as a clearing operation on the existing preset dot matrix sequence, so as to clear part or all of the existing line marks on the target multimedia resource.

[0075] Step 104: preview the target multimedia resource marked with lines.

[0076] Since the preview control connected to the timeline object can preview the multimedia materials selected by the user in real time, the video base class that adds drawing effects can not only provide the built-in drawing effects of the video editing software for line marking processing for the target multimedia resources, but also can provide real-time preview capabilities based on the connection between the timeline object and the preview control. At this time, the target multimedia resources after line marking can be previewed.

[0077] In an embodiment of the present invention, through the preview control and drawing special effects provided by the video editing software, based on connecting the timeline object with the preview control, and adding the drawing special effects to the video base class after the connection, the preset dot matrix sequence is processed on the target multimedia resource through the provided drawing special effects, and the real-time preview of the target multimedia resource after line marking can be realized based on the connection with the preview control. By combining the video editing capabilities of the video editing software, and through the preview control and drawing special effects provided to the user in the upper interface, the preset dot matrix sequence is processed on the multimedia resource through the drawing special effects, and the line marking operation of the original video or picture is realized, which is convenient for user operation.

[0078] Reference Figure 2 , shows a flowchart of another image processing method embodiment of the present invention, involving video editing software, the video editing software provides preview controls and drawing special effects, and specifically may include the following steps:

[0079] Step 201, creating a timeline object based on a preset software development kit of the video editing software, and connecting the timeline object to a preview control;

[0080] In an embodiment of the present invention, the PC-side image processing and special effects processing software can provide preview controls and drawing effects with the help of the video editing capabilities of the relevant video editing software, so as to process the preset dot matrix sequence based on the provided preview controls and drawing effects, thereby functionally achieving the effect of increasing line markings.

[0081] Specifically, the preview controls and drawing effects provided may be provided through a preset software development kit (SDK) of the video editing software.

[0082] In actual applications, the preview control provided, such as the Live window control, is just a control. If the control is not connected to any timeline, no multimedia resources can be displayed on the preview interface presented by it. In order to avoid a black screen, the preview control can be connected to a timeline object so that users can preview their selected multimedia materials in real time.

[0083] In one embodiment of the present invention, in order to connect the preview control with the timeline object, a timeline object for connection may be first created, and then the created timeline object and the preview control are connected to add the multimedia resource selected by the user to the preview control.

[0084] The creation process of the timeline object can be mainly implemented through the preset software development kit of the video editing software. Specifically, the preset software development kit of the video editing software can be obtained, and then the preset software development kit can be initialized, so as to create the timeline object based on the initialized preset software development kit. In a specific implementation, the streaming media context object of the initialized preset software development kit can be obtained, and then the timeline object is created based on the streaming media context object.

[0085] Exemplarily, the streaming media processing module of the preset software development kit SDK is initialized, and the streaming media context Context object (i.e., the CNv Streaming Context object) is obtained. At this time, the function for creating a timeline, such as the create Timeline function, can be called through the aforementioned streaming media context Context object to create a timeline object timeline, thereby realizing the creation of the timeline object.

[0086] The created timeline object can be used to add multimedia resources selected by the user. In a specific implementation, the created timeline object is a data structure describing the timeline, and video tracks and audio tracks can be added to the timeline. Video files (or picture files) and audio files selected by the user can be added to the tracks.

[0087] For the connection between the created timeline object and the preview control, the streaming media context object of the initialized preset software development kit can be obtained, and then the timeline object and the preview control are passed in as parameters of the preset interface through the streaming media context object to connect the timeline object and the preview control.

[0088] For example, the timeline can be connected to the live window through the connection interface of the streaming media context object, such as the connectTimeline With Live window interface, to synchronize the timeline with the real-time data stream or event stream so as to perform corresponding operations or processing on the timeline. In a specific implementation, the timeline and the Live window can be passed in as parameters of the aforementioned interface to achieve the connection between the two. It should be noted that the embodiment of the present invention does not limit the specific connection process within the interface.

[0089] Step 202, adding drawing effects to the video base class;

[0090] After the timeline object is connected to the preview control, that is, after the track and the corresponding multimedia file are added to the timeline object, a video base class (Video Clip) can be obtained, which can be used to carry the drawing effects provided by the video editing software to provide the target multimedia resource to be processed with the ability to use the effects. In one embodiment of the present invention, a drawing effect can be added to the video base class at this time to achieve the effect of line marking in terms of function.

[0091] Among them, the drawing effect used can be a Painting Effect, which can be used to paint images, or to paint seamless textures for geometric bodies in the scene. Specifically, by using the Painting Effect, a specific object can be attached to the path drawn by the brush, and the drawing processing parameters can be defined based on mouse events, so as to process a preset dot matrix sequence on a video or picture.

[0092] Specifically, after adding drawing effects to the video base class, a video processing object (video Fx) can be obtained. The video processing object is the object for subsequent actual drawing processing. At this time, drawing effects can also be added to the video processing object to obtain a drawing configuration object. The obtained drawing configuration object can be mainly used to set drawing processing parameters.

[0093] In a specific implementation, a painting effect is added to the video base class, for example, a built-in video effect named Painting Effect is added to the Video Clip object. At this time, an object representing the effect can be returned, usually called a Video Fx object, which allows further configuration or control of the parameters of the effect. For the configuration of the parameters, it is necessary to use a context or configuration object related to the Painting Effect effect, that is, in actual applications, the Painting Effect effect can be further added to the Video Fx object to obtain a painting configuration object (Painting Effect Context) that can be used to configure the parameters. The object can contain the properties and methods required to control the effect, such as setting the brush style, color, speed, etc. The drawing processing parameters that need to be set can correspondingly include at least one of the following: line width, line color, a first dot matrix sequence for the target marker line to be added, and a second dot matrix sequence for the existing historical marker line. This is not limited in the embodiments of the present invention.

[0094] Step 203, responding to a touch operation on a preview control provided by the upper layer interface, presenting a target multimedia resource;

[0095] After the timeline object is connected to the preview control, since the preview control has been connected to the timeline object to which the track and multimedia file are added, it already has a real-time preview function for multimedia resources. At this time, by responding to the touch operation of the preview control provided by the upper interface, the target multimedia resource selected by the user can be presented to the user on the upper interface, so that the user can preview the multimedia resources in the image processing process in real time.

[0096] Among them, the image processing process can be expressed as a subsequent line marking process. Based on the touch response to the preview control and the presentation of the target multimedia resources, it is ensured that the drawing effect for the target multimedia resources can be presented to the user in real time, so that the user can preview the line marking effect of the video.

[0097] Step 204, responding to the drawing processing instruction, obtaining drawing processing parameters in the drawing configuration object;

[0098] In one embodiment of the present invention, in order to realize the subsequent use of drawing effects on the target multimedia resources, it is possible to respond to the setting operation of the drawing processing parameters, that is, respond to the drawing processing operation, and generate a drawing processing instruction. The drawing processing instruction can be used to instruct the processing of a preset dot matrix sequence on the target multimedia resource through the drawing effects.

[0099] Specifically, the setting of drawing processing parameters involves the definition of brush style, color, speed and other attributes, the acquisition of drawing processing parameters, and the filling of drawing processing parameters in the drawing configuration object, that is, the setting of drawing processing parameters can be implemented in the drawing configuration object.

[0100] Exemplarily, a drawing effect, such as a Painting Effect, can be added to a base class Video Clip for a video that requires line markings, and a Video Fx object (a component for implementing special effects processing of a video) can be obtained. Then, the NvsPaintingEffectContext object Painting EffectContext (i.e., a drawing configuration object) can be obtained through the Video Fx object. At this time, the color of the marking line, the line width, and the dot sequence of the marking can be set through the Painting Effect Context. The dot sequence can be connected into a line to achieve the effect of adding line markings to a video or image.

[0101] The drawing processing parameters may include at least one of the following: line width, line color, a first dot matrix sequence for the target marker line to be added, and a second dot matrix sequence for the existing historical marker line. For example, the above main parameters can be set in the NvsPaintingEffectContext object, for example, Painting EffectContext.SetStrokeWidth can be used to set the line width, Painting EffectContext.SetStrokeGradientColor can be used to set the line color, and Painting EffectContext.AppendStroke(List <point>) can be used to add a dot sequence, Painting EffectContext.RemoveLastStroke can be used to remove the previous marker line, and Painting EffectContext.RemoveAllStroke can be used to clear all marker lines.

[0102] In a preferred embodiment of the present invention, QT can be used to develop an upper interface that is easy for users to operate. For example, a UI interface (User Interface) can be developed based on Qt Widgets, which is helpful for ordinary users to use the video line marking function. Then, based on the UI interface, the setting operation of the drawing processing parameters, that is, when setting the corresponding drawing processing parameters, can be expressed as allowing the user to input the relevant setting parameters. The aforementioned input of the parameters can be expressed as the user's custom input on the UI interface, or as the user's mouse click operation, mouse release operation and mouse movement operation on the preset dot matrix sequence on the multimedia resource, so as to determine the drawing processing parameters to be set based on the response to the aforementioned mouse operation or the response to the parameter input operation, and then generate the drawing processing instruction.

[0103] Exemplarily, since the UI interface is developed based on Qt Widgets, a LineCoverWidget widget interface can be subclassed from QWidget (the base class of all user interface objects in the QT library). The LineCoverWidget class is a custom QWidget subclass, which can be used to cover lines or draw lines, and may have functions such as drawing lines, setting line colors, line thicknesses, and line styles; and the LineCoverWidget widget interface refers to the user interface presented by the custom QWidget object created by the LineCoverWidget class. This interface is a rectangular area in which rectangular graphics can be drawn, that is, the LineCoverWidget class can customize the function of drawing rectangles, so as to display the desired rectangular graphics on the interface; then, in order to add the aforementioned interface to the main window or other container of the QT application to display the drawn rectangle, the LineCoverWidget can be overlaid on a preview control, such as a Livewindow, and the background of the widget interface is set to be transparent, and the size of the widget interface is set to be the same as the size of the Live window.

[0104] In this example, the mouse operation can be combined to obtain some points where the mouse slides over, and the points can be used to collect a dot matrix sequence, and the dot matrix sequence can be used to connect a line to achieve the effect of line marking. Among them, the mouse operation can be mainly used to indicate the drawing processing operation. The mouse operation can include the mouse click operation, the mouse release operation and the mouse move operation, which can be respectively implemented in the mousePressEvent (a function used in QT to process the mouse click event), mouseMoveEvent (a function used in QT to process the mouse move event) and the mouseReleaseEvent event function (a function used in QT to process the mouse release event).

[0105] It should be noted that for drawRect and preview controls, such as Live window overlay, the size and position of the widget interface can always be kept the same as the Live window. The purpose of LineCoverWidget overlay is mainly to facilitate coordinate conversion. Since the Live window previews the video on the timeline object, the size of the timeline refers to the resolution of the final exported video or picture, such as 720P or 1080P. However, Livewindow is just a control, and its size is usually set at the UI layer. The data operations for the drawing effects added to the Video Clip on the timeline are all set accordingly relative to the size of the timeline object. For example, in the video, x is 100 pixels and y is 200 pixels, but it is not necessarily reflected as the pixel position of (100,200) on the UI layer Live window control.

[0106] Then, in order to perform corresponding operations or processing on the timeline, in the process of data operation, the timeline special effects can be converted into control coordinates and the control coordinates can be converted into timeline coordinates. For example, this can be achieved through Map Canonical ToWidget (a function or method usually used to map the Canonical coordinate system to the Widget coordinate system) and MapWidget To Canonical (a method or function used to determine the position and size of the control in the canonical space). If the size of the LineCoverWidget is the same as the size of the Live window, the coordinate data conversion can be performed through the aforementioned Map CanonicalTo Widget and / or Map Widget To Canonical methods. At this time, although the mouse operation is not directly performed on the Live window, because the size and position of the LineCoverWidget are the same as those of the Live window, the aforementioned coordinate conversion method can be directly used for calculation to convert the mouse operation position information on the LineCoverWidget into the position on the timeline, and then pass it to the SDK.

[0107] Step 205 , processing the preset dot matrix sequence on the target multimedia resource according to the drawing processing parameters by drawing special effects.

[0108] The processing performed on the preset dot matrix sequence can be expressed as an addition operation on the preset dot matrix sequence to be added, so as to add corresponding line marks on the target multimedia resource; it can also be expressed as a clearing operation on the existing preset dot matrix sequence, so as to clear part or all of the existing line marks on the target multimedia resource.

[0109] In one embodiment of the present invention, the preset dot matrix sequence may include a first dot matrix sequence for a target marking line to be added, and / or a second dot matrix sequence for an existing historical marking line. In one case, the first dot matrix sequence can be drawn on the target multimedia resource by setting the corresponding target marking line according to the line width and line color by drawing special effects; in another case, the historical marking line corresponding to the second dot matrix sequence can be cleared on the target multimedia resource by drawing special effects.

[0110] In a preferred embodiment of the present invention, the target multimedia resource marked with lines can be previewed.

[0111] Specifically, you can capture some position points of the mouse movement, call MapWidgetToCanonica of Live window, convert the captured position points into a series of position point data on the timeline, and then call PaintingEffectContext.AppendStroke interface to pass these position points to the special effect, so as to realize real-time processing of the position points corresponding to the dot matrix sequence. Figure 3 As shown in the figure, you can preview the line mark where the mouse moves over in real time on the Live window. Finally, you can export the video by calling the compileTimeline interface of the SDK, or export a static image by calling the grabImageFromTimeline interface of the SDK to get the material with the line mark added.

[0112] In an embodiment of the present invention, by combining the video editing capabilities of video editing software and providing preview controls and drawing effects to users on the upper interface, a preset dot matrix sequence is processed on multimedia resources through drawing effects, thereby realizing line marking operations on original videos or pictures, which is convenient for user operation.

[0113] Reference Figure 4 , showing a schematic diagram of the scheduling process of the line marking function provided by an embodiment of the present invention. In the embodiment of the present invention, the PC-side image processing and special effects processing software can provide preview controls and drawing special effects with the help of the video editing capabilities of the relevant video editing software, so as to process the preset dot matrix sequence based on the provided preview controls and drawing special effects, thereby achieving the effect of increasing line markings in terms of functionality.

[0114] Specifically, the preview controls and drawing effects provided can be provided by a preset software development kit SDK of the video editing software. The process of implementing line marking through the aforementioned SDK involves creating a timeline, adding videos or pictures, adding special effects, setting drawing processing parameters, and exporting, which can be specifically performed in the following steps:

[0115] a. Initialization: Initialize the streaming media processing module of the preset software development kit SDK, that is, execute CNvStreaming Context::Init to obtain the streaming media context Context object;

[0116] b. Create a timeline: Call the function for creating a timeline, such as the create Timeline function, through the aforementioned streaming media context object, that is, execute context.create Timeline to obtain the timeline object timeline;

[0117] c. Create a preview control: instantiate the Live window Widget interface provided by the preset software development kit SDK to obtain a preview control Live window for real-time preview of multimedia materials selected by the user;

[0118] d. Preview connection: connect the timeline with the live window through the connection interface of the streaming media context object, such as the connect TimelineWith Live window interface, that is, execute context.connect Timeline With Live window(timeline, Live window) to connect the created timeline object with the preview control;

[0119] e. Create a video track: add a video track and an audio track to the created timeline object timeline, that is, execute timeline.append Video Track to obtain the track Video Track;

[0120] f. Add video or picture: The video file (or picture file) and audio file selected by the user can be added to the above added track, that is, Video Track.append Video Clip (video or picture path) can be executed to obtain the video base class Video Clip;

[0121] g. Adding drawing effects: You can add drawing effects, such as Painting Effect, to the base class Video Clip that has drawing requirements. For example, execute Video Clip.append Raw Built In Video Fx("Painting Effect") to obtain the video processing object Video Fx, which is a component for implementing special effects processing of videos.

[0122] h. Adding drawing effects: You can add drawing effects, such as Painting Effect, to the video processing object Video Fx that actually performs processing. For example, execute Video Clip.getPaintingEffectContext() to obtain the drawing configuration object, that is, the paintingEffectContext object.

[0123] Among them, the above step g is mainly to add the built-in video effect of Painting Effect to the videoClip object. At this time, an object representing the effect can be returned, usually called a videoFx object, which allows further configuration or control of the parameters of the effect; for the configuration of parameters, it is necessary to use the context or configuration object related to the Painting Effect effect, that is, according to the above step h, the Painting Effect effect can be further added to the Video Fx object to obtain a drawing configuration object (Painting Effect Context) that can be used to configure parameters. This object can contain the properties and methods required to control the effect, such as setting the brush style, color, speed, etc. Specifically, the main parameters can be set in the NvsPaintingEffectContext object, for example, Painting EffectContext.SetStrokeWidth can be used to set the line width, Painting EffectContext.SetStrokeGradientColor can be used to set the line color, and Painting EffectContext.AppendStroke(List <point>) can be used to add a dot sequence, Painting EffectContext.RemoveLastStroke can be used to remove the previous marker line, and Painting EffectContext.RemoveAllStroke can be used to clear all marker lines.

[0124] Finally, you can export the processed images or videos through the preset software development kit SDK of the relevant video editing software.

[0125] In an embodiment of the present invention, by combining the video editing capabilities of the video editing software, and through the preview controls and drawing effects provided to the user in the upper interface, the processing of the preset dot matrix sequence on the multimedia resources is realized through the drawing effects, and then the line marking operation of the original video or picture is realized, that is, while facilitating the user's operation, it can allow the import of any video or picture material to be processed by the preset dot matrix sequence, and achieve the effect of line marking from the functional point of view, and the scope of use is very wide; and, when responding to the drawing processing instruction, the preset dot matrix sequence can be processed through the provided drawing effects, and then the line marking effect on the target multimedia resource can be presented to the user in real time, so that the user can preview the line marking effect of the video. Further, based on the dot matrix sequence of the target marking line to be added to the target multimedia resource and the existing dot matrix sequence, multiple line markings can be combined to meet user needs. In addition, QT can also be used to develop an upper interface that is easy for users to operate, which is conducive to ordinary users to use the drawing function, that is, the line marking processing operation.

[0126] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0127] Reference Figure 5 , shows a structural block diagram of an embodiment of an image processing device of the present invention, which relates to video editing software. The video editing software provides preview controls and drawing special effects, and may specifically include the following modules:

[0128] A preview control connection module 501 is used to create a timeline object, and obtain a video base class based on the connection between the timeline object and the preview control;

[0129] A drawing effect adding module 502 is used to add the drawing effect to the video base class; the video base class is used to indicate the target multimedia resource to be processed;

[0130] The dot matrix sequence processing module 503 is used to respond to the drawing processing instruction and process the preset dot matrix sequence on the target multimedia resource through the drawing special effect; the preset dot matrix sequence is used to indicate the line mark;

[0131] The effect preview module 504 is used to preview the target multimedia resource after the line marking.

[0132] In one embodiment of the present invention, the preview control connection module 501 may include the following submodules:

[0133] The preview control connection submodule is used to obtain the preset software development kit of the video editing software and initialize the preset software development kit; create a timeline object based on the initialized preset software development kit; the timeline object is used to add multimedia resources selected by the user; and connect the timeline object with the preview control to obtain a video base class.

[0134] In one embodiment of the present invention, the preview control connection submodule may include the following units:

[0135] The timeline object connection unit is used to obtain the streaming media context object of the initialized preset software development kit; through the streaming media context object, the timeline object and the preview control are passed in as parameters of the preset interface to connect the timeline object and the preview control.

[0136] In an embodiment of the present invention, after adding the drawing special effect to the video base class, a video processing object is obtained; the apparatus provided by the embodiment of the present invention may further include the following modules:

[0137] A drawing configuration object processing module is used to add the drawing special effect to the video processing object to obtain a drawing configuration object; the drawing configuration object is used to set drawing processing parameters; wherein the drawing processing parameters include at least one of the following: line width, line color, a first dot matrix sequence for a target marking line to be added, and a second dot matrix sequence for an existing historical marking line.

[0138] In one embodiment of the present invention, the dot sequence processing module 503 may include the following submodules:

[0139] The dot matrix sequence processing submodule is used to respond to the drawing processing instruction and obtain the drawing processing parameters in the drawing configuration object; through the drawing effect, the preset dot matrix sequence is processed on the target multimedia resource according to the drawing processing parameters.

[0140] In an embodiment of the present invention, the preset dot matrix sequence includes the first dot matrix sequence and / or the second dot matrix sequence, and the dot matrix sequence processing submodule may include the following units:

[0141] A dot sequence processing unit is used to set a corresponding target marking line for the first dot sequence on the target multimedia resource according to the line width and the line color through the drawing special effect; and / or, through the drawing special effect, clear the historical marking line corresponding to the second dot sequence on the target multimedia resource.

[0142] In an embodiment of the present invention, before responding to the drawing processing instruction, the apparatus provided by the embodiment of the present invention may further include the following modules:

[0143] The drawing processing instruction generation module is used to respond to the drawing processing operation and obtain the drawing processing parameters; fill the drawing processing parameters into the drawing configuration object to generate the drawing processing instruction; wherein the drawing processing operation is a mouse operation, and the mouse operation includes a mouse click operation, a mouse release operation and a mouse move operation.

[0144] In an embodiment of the present invention, the image processing device provided by the embodiment of the present invention uses the preview control and drawing special effects provided by the video editing software, based on connecting the timeline object with the preview control, and adding drawing special effects to the video base class after the connection, and then processes the preset dot matrix sequence on the target multimedia resource through the provided drawing special effects, and can realize the real-time preview of the target multimedia resource after line marking based on the connection with the preview control. By combining the video editing capabilities of the video editing software, and through the preview control and drawing special effects provided to the user in the upper interface, the preset dot matrix sequence is processed on the multimedia resource through the drawing special effects, and then the line marking operation of the original video or picture is realized, which is convenient for user operation.

[0145] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0146] An embodiment of the present invention further provides an electronic device, including:

[0147] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0148] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0149] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0150] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0151] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0155] Finally, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0156] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0157] The image processing method, image processing device, corresponding electronic device and corresponding computer-readable storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.< / point> < / point>

Claims

1. An image processing method, characterized in that: The invention relates to video editing software, wherein the video editing software provides a preview control and a drawing special effect, and the method comprises: Create a timeline object, and connect the timeline object with the preview control to obtain a video base class; Adding the drawing special effect to the video base class; the video base class is used to indicate the target multimedia resource to be processed; In response to the drawing processing instruction, a preset dot matrix sequence is processed on the target multimedia resource by the drawing special effect; the preset dot matrix sequence is used to indicate a line mark; Previewing the target multimedia resource marked by the line; creating a timeline object, connecting the timeline object with the preview control to obtain a video base class, including: Obtaining a preset software development kit of the video editing software, and initializing the preset software development kit; Creating a timeline object based on the initialized preset software development kit; the timeline object is used to add multimedia resources selected by the user; The timeline object is connected to the preview control to obtain a video base class.

2. The method according to claim 1, characterized in that The step of connecting the timeline object with the preview control includes: Obtaining the initialized streaming media context object of the preset software development kit; The timeline object and the preview control are passed in as parameters of a preset interface through the streaming media context object to connect the timeline object and the preview control.

3. The method according to claim 1, characterized in that After adding the drawing special effect to the video base class, a video processing object is obtained; the method further comprises: Adding the drawing special effect to the video processing object to obtain a drawing configuration object; the drawing configuration object is used to set drawing processing parameters; The drawing processing parameters include at least one of the following: line width, line color, a first dot matrix sequence for a target marking line to be added, and a second dot matrix sequence for an existing historical marking line.

4. The method according to claim 3, characterized in that The responding drawing processing instruction processes the preset dot matrix sequence on the target multimedia resource by using the drawing special effect, including: In response to the drawing processing instruction, obtaining the drawing processing parameters in the drawing configuration object; Through the drawing effect, the preset dot matrix sequence is processed on the target multimedia resource according to the drawing processing parameters.

5. The method according to claim 4, characterized in that The preset dot matrix sequence includes the first dot matrix sequence and / or the second dot matrix sequence; The processing of the preset dot matrix sequence on the target multimedia resource according to the drawing processing parameters by the drawing special effect includes: By means of the drawing special effect, the first dot matrix sequence is set on the target multimedia resource according to the line width and the line color to form a corresponding target marking line; And / or, by using the drawing special effect, clearing the historical marking line corresponding to the second dot matrix sequence on the target multimedia resource.

6. The method according to any one of claims 1 to 5, characterized in that: Before the response drawing processing instruction, it also includes: Responding to the drawing processing operation, obtaining drawing processing parameters; Fill the drawing processing parameters into the drawing configuration object to generate a drawing processing instruction; The drawing processing operation is a mouse operation, and the mouse operation includes a mouse click operation, a mouse release operation and a mouse move operation.

7. An image processing device, characterized in that: The invention relates to video editing software, wherein the video editing software provides a preview control and drawing special effects, and the device comprises: A preview control connection module is used to create a timeline object, and obtain a video base class based on the connection between the timeline object and the preview control; wherein the creation of the timeline object, the connection between the timeline object and the preview control, and the acquisition of the video base class include: obtaining a preset software development kit of the video editing software, initializing the preset software development kit; creating a timeline object based on the initialized preset software development kit; the timeline object is used to add multimedia resources selected by the user; and the timeline object is connected to the preview control to obtain the video base class; A drawing special effect adding module, used for adding the drawing special effect to the video base class; the video base class is used for indicating the target multimedia resource to be processed; A dot matrix sequence processing module, used for responding to a drawing processing instruction, and processing a preset dot matrix sequence on the target multimedia resource through the drawing special effect; the preset dot matrix sequence is used for indicating a line mark; The effect preview module is used to preview the target multimedia resource after the line marking.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the image processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Images