Mark display method of video content and electronic equipment

Abstract

The invention discloses a video content mark display method and electronic equipment. The method comprises the following steps: acquiring a to-be-processed video; processing the to-be-processed video to obtain at least one image frame; for each image frame, identifying character objects in the image frame, and counting the number of the character objects included in the image frame; a matched first generation strategy is applied, and a visualization graph is generated according to the number of the character objects included in each image frame; the visual graph is used for representing the distribution condition of the character objects in the time dimension; in at least one image frame including the character object, respectively extracting face features included in each image frame; and displaying the visual image in a first display area of a video picture of the video, and displaying a face image represented by the face feature in a second display area of an image picture corresponding to the source image frame of the face feature. By displaying the visual image and the face image on the video picture, the distribution of the character objects is displayed, and the efficiency in the video troubleshooting process is improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method for marking and displaying video content and an electronic device. Background Art

[0002] Video plays an important role in various fields and scenarios. People can find the object they are interested in and the key information of the object by watching the video. For example, you want to quickly find a person in a video, but in a complete video, the person only appears for a few seconds or tens of seconds in a specific time period, and people only pay attention to the situation or scene of these few seconds or tens of seconds.

[0003] Therefore, if you want to quickly locate a person, you need to watch the entire video, which is time-consuming and labor-intensive, and has low screening efficiency. Summary of the invention

[0004] In an exemplary embodiment of the present application, a method and electronic device for marking and displaying video content are provided, which display the distribution of human objects and facial images in a visual manner on the video screen, thereby improving the efficiency of the video screening process as a whole.

[0005] According to a first aspect of an exemplary embodiment, a method for displaying a mark of video content is provided, comprising:

[0006] Get the video to be processed;

[0007] Processing the video to be processed to obtain at least one image frame;

[0008] For each image frame, identifying a human object in the image frame, and counting the number of human objects included in the image frame;

[0009] Applying the first generation strategy of character object matching, generating a visualization graph of character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension;

[0010] In at least one image frame including a person object, extracting facial features included in each image frame respectively;

[0011] The visualization image is displayed in a first display area of ​​the video screen of the video, and the face image represented by the face feature is displayed in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0012] Get the video to be processed;

[0013] Processing the video to be processed to obtain at least one image frame;

[0014] For each image frame, identifying vehicle objects in the image frame, and counting the number of vehicle objects included in the image frame;

[0015] Applying the second generation strategy of vehicle object matching, generating a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of vehicle objects in the time dimension;

[0016] In at least one image frame including a vehicle object, respectively extracting license plate features included in each image frame;

[0017] The visualization image is displayed in a first display area of ​​the video screen of the video, and the license plate number represented by the license plate feature is displayed in a second display area of ​​the image screen corresponding to the source image frame of the license plate feature.

[0018] According to a third aspect of an exemplary embodiment, there is provided an electronic device, including a first display screen and a first processor:

[0019] The first processor is configured to execute:

[0020] Get the video to be processed;

[0021] Processing the video to be processed to obtain at least one image frame;

[0022] For each image frame, identifying a human object in the image frame, and counting the number of human objects included in the image frame;

[0023] Applying the first generation strategy of character object matching, generating a visualization graph of character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension;

[0024] In at least one image frame including a person object, extracting facial features included in each image frame respectively;

[0025] The first display is configured to perform:

[0026] The visualization image is displayed in a first display area of ​​the video screen of the video, and the face image represented by the face feature is displayed in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0027] According to a fourth aspect of the exemplary implementation, there is provided an electronic device, comprising a second display screen and a second processor:

[0028] The second processor is configured to execute:

[0029] Get the video to be processed;

[0030] Processing the video to be processed to obtain at least one image frame;

[0031] For each image frame, identifying vehicle objects in the image frame, and counting the number of vehicle objects included in the image frame;

[0032] Applying the second generation strategy of vehicle object matching, generating a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of vehicle objects in the time dimension;

[0033] In at least one image frame including a vehicle object, respectively extracting license plate features included in each image frame;

[0034] The second display is configured to perform:

[0035] The visualization image is displayed in a first display area of ​​the video screen of the video, and the license plate number represented by the license plate feature is displayed in a second display area of ​​the image screen corresponding to the source image frame of the license plate feature.

[0036] According to a fifth aspect of an exemplary implementation, a method and apparatus for displaying a mark of video content is provided, comprising:

[0037] A first video acquisition unit is used to: acquire a video to be processed;

[0038] A first video parsing unit is used to: process the video to be processed to obtain at least one image frame;

[0039] The first target recognition unit is used to: for each image frame, recognize a human object in the image frame, and count the number of human objects included in the image frame;

[0040] A first visualization graph generating unit is used to: apply a first generation strategy for character object matching, and generate a visualization graph of character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension;

[0041] A facial feature extraction unit, used to: extract facial features included in each image frame in at least one image frame including a human object;

[0042] The first display unit is used to: display the visualization image in a first display area of ​​the video screen of the video, and display the face image represented by the face feature in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0043] According to a sixth aspect of an exemplary implementation, a method and apparatus for displaying a mark of video content is provided, comprising:

[0044] The second video acquisition unit is used to: acquire the video to be processed;

[0045] A second video parsing unit is used to: process the video to be processed to obtain at least one image frame;

[0046] A second target recognition unit is used to: for each image frame, recognize a vehicle object in the image frame, and count the number of vehicle objects included in the image frame;

[0047] A second visualization graph generating unit is used to: apply a second generation strategy for vehicle object matching to generate a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of vehicle objects in a time dimension;

[0048] A license plate feature extraction unit is used to: extract the license plate features included in each image frame in at least one image frame including a vehicle object;

[0049] The second display unit is used to: display the visualization image in the first display area of ​​the video screen of the video, and display the license plate number represented by the license plate feature in the second display area of ​​the image screen corresponding to the source image frame of the license plate feature.

[0050] According to a seventh aspect of the exemplary implementation, a computer storage medium is provided, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the method for marking and displaying video content of the first aspect or the second aspect.

[0051] In an embodiment of the present application, first, for each image frame of at least one image frame obtained by processing the video to be processed, the person object in the image frame is identified, and the number of person objects included in the image frame is counted. Secondly, the first generation strategy of person object matching is applied, and a visualization diagram of person object matching is generated according to the number of person objects included in each image frame. The visualization diagram is used for the distribution of person objects in the time dimension, and the facial features included in each image frame are extracted in at least one image frame including the person object. Finally, the visualization diagram is displayed in the first display area of ​​the video screen of the video, and the face image represented by the facial features is displayed in the second display area of ​​the image screen corresponding to the source image frame of the facial features. Therefore, by displaying the visualization diagram in the video screen, the distribution of person objects in the video can be intuitively displayed, and the time point with the peak frequency of the person objects can be quickly located or the time period with the person objects, thereby improving the efficiency of the video screening process. By directly displaying the facial image, the person can also be quickly identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A schematic diagram of a monitoring screen in which a target object is a person provided in an embodiment of the present application is exemplarily shown;

[0054] Figure 2 A schematic diagram of a monitoring screen in which a target object is a vehicle provided in an embodiment of the application is exemplarily shown;

[0055] Figure 3 A flowchart of a method for marking and displaying video content provided by an embodiment of the present application is exemplarily shown;

[0056] Figure 4 A schematic diagram of a face image watermark provided by an embodiment of the present application is exemplarily shown;

[0057] Figure 5 A schematic diagram of displaying a visual image watermark provided by an embodiment of the present application is exemplarily shown;

[0058] Figure 6 A schematic diagram exemplarily shows a screen jump process provided by an embodiment of the present application;

[0059] Figure 7 A schematic diagram of a hidden visual image watermark provided by an embodiment of the present application is exemplarily shown;

[0060] Figure 8 A flowchart of a method for displaying a video with a visual image watermark provided by an embodiment of the present application is exemplified;

[0061] Fig. 9 A flowchart of another method for marking and displaying video content provided by an embodiment of the present application is exemplified;

[0062] Fig.10 A schematic diagram of a license plate number watermark provided in an embodiment of the present application is exemplarily shown;

[0063] Fig.11 Another schematic diagram of displaying a visual image watermark provided by an embodiment of the present application is exemplarily shown;

[0064] Fig.12 A schematic diagram exemplarily shows another screen jump process provided by an embodiment of the present application;

[0065] Fig.13A schematic diagram of another hidden visual image watermark provided by an embodiment of the present application is exemplarily shown;

[0066] Fig.14 A schematic diagram of the structure of a marking display device for video content provided by an embodiment of the present application is exemplarily shown;

[0067] Fig.15 A schematic diagram showing the structure of another video content marking display device provided in an embodiment of the present application is exemplified;

[0068] Fig.16 The following is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application;

[0069] Fig.17 The schematic diagram of the structure of another electronic device provided in an embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0071] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0072] (1) Heat map: A heat map that is visualized through a density function to represent the density of points in a map. It is mainly divided into map heat maps, web page heat maps, APP heat maps, and heat maps in business data analysis. The heat map in business data analysis can include heat maps based on geographic location and heat maps based on time. For example, a sales volume heat map based on the time dimension can represent the relationship between time and sales volume. The heat map in the embodiment of the present application can be a time-based heat map.

[0073] Video plays an important role in various fields and scenarios, and has become an indispensable technical means in people's work and study. Taking public safety construction as an example, people's demand for public safety is constantly increasing, and public safety construction is constantly improving. Video can be used as a means of public safety and social order management. People can find the object they are concerned about and the key information of the object by watching the video. However, in a complete video, the key information may only be a few seconds or tens of seconds in a specific time period. People only pay attention to the situation or scene of these few seconds or tens of seconds. Therefore, if you want to quickly locate the key information, you need to complete the video investigation with your naked eyes. Watching the entire video is time-consuming and labor-intensive, and the investigation efficiency is low.

[0074] The target object here may be a person, a vehicle, an animal or other movable object, and the other movable object may be, for example, a mobile robot. In the embodiment of the present application, a person object and a vehicle object are respectively used as examples for description.

[0075] To this end, an embodiment of the present application provides a method for marking and displaying video content, in which the video is processed into image frames, and then the character objects in each image frame are identified, and the number of character objects included in each image frame is counted. Based on the number of character objects included in each image frame, a visualization graph of the character objects is generated, and the facial features in each image frame are extracted, and each visualization graph is added to the video data of the video as watermark content for display. In this way, since the visualization graph can represent the distribution of character objects in the time dimension, in the process of troubleshooting the video, it is possible to determine in which time period of the video the character objects appear based on the visualization graph, and then jump to the video of the corresponding time period, thereby improving the troubleshooting efficiency. Displaying facial images can quickly locate whether the person is the person you want to find, further improving the troubleshooting efficiency.

[0076] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0077] Usually, the surveillance images show the time and location of the surveillance images. When you want to find the person or vehicle of interest, you need to watch it frame by frame, wasting time and energy. Figure 1 A schematic diagram of a monitoring screen of a person provided in an embodiment of the present application, Figure 2 A schematic diagram of a vehicle monitoring screen provided in an embodiment of the present application.

[0078] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0079] Combine the following Figure 1 and Figure 2 The application scenario shown is shown in Figure 3 A flowchart of a method for marking and displaying video content is shown. Figure 3 In the embodiment, the target object is a person object, and the technical solution provided in the embodiment of the present application is described.

[0080] S301: Obtain the video to be processed.

[0081] S302: Process the video to be processed to obtain at least one image frame.

[0082] S303: For each image frame, identify the human objects in the image frame, and count the number of human objects included in the image frame.

[0083] S304: Apply the first generation strategy for character object matching to generate a visualization graph of character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of character objects in the time dimension.

[0084] S305: In at least one image frame including a person object, extract facial features included in each image frame.

[0085] S306: Display the visualization image in a first display area of ​​the video screen of the video, and display the face image represented by the face feature in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0086] In an embodiment of the present application, first, for each image frame of at least one image frame obtained by processing the video to be processed, the person object in the image frame is identified, and the number of person objects included in the image frame is counted. Secondly, the first generation strategy of person object matching is applied, and a visualization diagram of person object matching is generated according to the number of person objects included in each image frame. The visualization diagram is used for the distribution of person objects in the time dimension, and the facial features included in each image frame are extracted in at least one image frame including the person object. Finally, the visualization diagram is displayed in the first display area of ​​the video screen of the video, and the face image represented by the facial features is displayed in the second display area of ​​the image screen corresponding to the source image frame of the facial features. Therefore, by displaying the visualization diagram in the video screen, the distribution of person objects in the video can be intuitively displayed, and the time point with the peak frequency of the person objects can be quickly located or the time period with the person objects, thereby improving the efficiency of the video screening process. By directly displaying the facial image, the person can also be quickly identified.

[0087] In S301, the video to be processed may be a surveillance video captured by a surveillance camera at a traffic intersection, a traffic section, or a public place. The public place may be a shopping mall, a park, or a residential entrance. The scene of the video screening may be to find a target person or a target vehicle within a set time period, or to find a lost elderly person, child, or pet.

[0088] For example, investigators want to confirm whether there is a white Y-brand vehicle among the vehicles entering and exiting the gate of XX community between 14:00:41 and 14:10:41 on March 7, 2023. In this example, the video to be processed is the video captured by the surveillance camera installed at the gate of XX community during this time period.

[0089] In S302, a video parsing method in related art is applied to process the video to be processed to obtain at least one video frame. The video parsing method here can be any one of related art and is not limited.

[0090] In S303, for each image frame, a human object in the image frame is identified, and the number of human objects included in the image frame is counted.

[0091] Wherein, a moving target detection algorithm can be applied based on a Gaussian mixture detection model to identify human objects. Exemplarily, a total of 15,000 image frames can be processed from the image frames to be processed in 10 minutes, and the number of human objects included in each of the 15,000 image frames is counted.

[0092] In S304, in order to distinguish different types of target objects, different visualization map generation strategies are set. For example, if the target object is a person, the generation strategy is to generate a time-based heat map; if the target object is a vehicle, the generation strategy is to generate a bar map; if the person object is an animal, the generation strategy is to generate a scatter plot. Generally, a visualization map can be generated for a person object in a video to be processed, and the visualization map shows the distribution of the person object in the time dimension.

[0093] Exemplarily, the process of generating a visualization graph of a person object may be implemented through steps A1 to A2.

[0094] A1: According to the first generation strategy of character object matching, the graphic shape and color of the visualization graph are determined.

[0095] For example, in the first generation strategy for matching a person object, the graphic shape is a bar graph and the graphic color is red, that is, the presentation color of the visualization graph is red.

[0096] A2: The number of human objects included in each image frame is used as the vertical coordinate, and the timestamp information of each image frame is used as the horizontal coordinate. According to the corresponding relationship between the number and the timestamp information, the shape and color of the graphic are used to generate a visualization diagram of the human object matching.

[0097] For example, there are 15,000 image frames with a frame rate of 25 frames per second. In the timestamp information of the horizontal axis of the visualization graph, the time interval between two adjacent times is 0.04s. Image frame 1 includes 1 person, and the coordinate information corresponding to image frame 1 is (0.04s, 1); image frame 2 includes 0 people, and the coordinate information corresponding to image frame 2 is (0.08s, 0). According to this principle, 1,500 sets of coordinate information can generate a visualization of the person object. Among them, the graphic shape and color of the visualization graph can be set according to the generation strategy. With such a design, the visualization graph can represent the distribution of the person object in the time dimension.

[0098] In S305, the above embodiment displays the visualization of the person object as a watermark on the video screen, and the user can check according to the distribution of the person object in the time dimension represented by the visualization. In addition, in order to further improve the checking efficiency, the face image of each image frame can also be added to the video screen as a watermark.

[0099] Specifically, in at least one image frame including a person object, facial features included in each image frame are extracted respectively, so that a facial image represented by the facial features is displayed without manually enlarging the face, thereby further improving the screening efficiency.

[0100] In S306, since after determining that the target object is a human object, the visualization image of the human object generated by the entire video to be processed is one, the visualization image can be displayed in the video screen of the video, that is, the display screen of each image frame in the video screen displays the visualization image. For the display of the face image, only the image screen corresponding to the source image frame of the face feature is displayed. For example, if the 30th frame to the 40th frame all include a group of face features, the face is displayed in the image screen of the 30th frame to the 40th frame.

[0101] Exemplarily, the display form may be based on watermark technology, that is, the visualization image may be added as a watermark to the video data, and the facial features may be added as a watermark to the corresponding image frame. In order to distinguish the display, the visualization image is displayed in the first display area, and the facial image is displayed in the second area. The first display area may be the bottom of the video screen, and the second display area may be the right side of the video screen.

[0102] In a specific example, Figure 4 This is a schematic diagram of a face image watermark provided in an embodiment of the present application. With such a design, the detailed features of the person's face can be intuitively seen without magnifying the face to determine the person's identity.

[0103] On the basis of the above embodiments, after obtaining the visualization graph, the visualization graph can be displayed in any of the following ways: timed display, static display or dynamic display. Timed display can be pre-set to display in several time periods during the entire video playback process; static display can be displayed in a fixed display area without changing; dynamic display can be set to adjust the display effect by setting the transparency of the display.

[0104] For example, the following two display processes are described:

[0105] Mode B1: default display, that is, setting the visualization diagram to be displayed in the corresponding display area by default;

[0106] Mode B2: triggering display, that is, displaying according to the user's operation. For example, displaying a first icon for displaying a visualization image, and in response to the user's operation on the first icon, displaying the visualization image in the corresponding display area.

[0107] In a specific example, Figure 5 A schematic diagram of displaying a visual image watermark provided in an embodiment of the present application, wherein: Figure 5 (a) in which the first icon of "display graphic watermark" is displayed, in response to the user's Figure 5 The operation of the first icon in (a) shows Figure 5 Visualization diagram in (b).

[0108] In addition, during the display of video screens, jumping to the video screen that the user is interested in can be achieved in the following two ways:

[0109] Method C1:

[0110] A target playback node is determined according to at least one visualization diagram; and in response to a user's dragging operation on the current playback node, a current display screen is jumped to a video screen corresponding to the target playback node.

[0111] In this method, the visualization graph is displayed in the video screen, and the user can determine the target playback node by observing the visualization graph. For example, if the target playback node is 3 minutes and 15 seconds, and the number of nearby vehicles at this moment is relatively large, then in response to the user's dragging operation on the current playback node, the dragging operation stops at the playback segment represented by the target playback node. In this way, the current display screen jumps to the video screen corresponding to the 3 minutes and 15 seconds.

[0112] In a specific example, Figure 6 A schematic diagram of a screen jump process provided in an embodiment of the present application, wherein in response to a user's Figure 6 The dragging operation of the playback progress bar in (a) shows Figure 6The video frame corresponding to the target playback node in (b) therein.

[0113] Method C2:

[0114] In response to the user's click operation on the visualization graph, jump the currently displayed frame to the video frame corresponding to the playback node indicated by the click operation.

[0115] In this method, the click operation can be a click on the position of 3 minutes and 15 seconds on the visualization graph, and then the currently displayed frame can jump to the video frame corresponding to 3 minutes and 15 seconds.

[0116] In addition, to improve the user's viewing experience of the video frame, during the video playback, if the user has no troubleshooting requirements, the visualization graph can be hidden at this time. Therefore, for the watermark hiding operation for the user, the visualization graph can be hidden and displayed. In a specific example, Figure 7 is a schematic diagram of hiding the visualization graph watermark provided by an embodiment of the present application, where Figure 7 the icon of "hide watermark" is shown in (a) therein, in response to the user's operation on Figure 7 the icon in (a) therein, in Figure 7 the visualization graph is hidden in (b) therein.

[0117] The embodiments of the present application help the troubleshooting personnel quickly and efficiently troubleshoot the distribution time points and heat distribution of the human objects in the video, reduce the cumbersome troubleshooting work during the video troubleshooting process, focus on the time points when the moving targets appear and the peak time periods when the moving targets appear, filter out useless video data, quickly complete the video troubleshooting work, improve the video troubleshooting efficiency, and reduce the labor cost.

[0118] For the display method of the key materials as watermarks in the embodiments of the present application, they can also be displayed by default, or can be displayed through the user's trigger operation, and can also be hidden according to the user's hiding operation. Refer to the description of the display method when the visualization graph is used as a watermark. In addition, in the case of quickly locating the human object, directly display the key materials such as the face or license plate number, without the need to play the video frame, improving the troubleshooting efficiency.

[0119] It should be noted that Figure 4-Figure 7 the visualization graph in therein is shown in the form of a line graph, and can also be shown in other forms such as a heat map. Usually, each image frame corresponds to a line, and the length of the line is proportional to the number of human objects. The number can also be displayed on the visualization graph, Figure 4-Figure 7 the number of lines in therein is also an example, without forming a specific limitation.

[0120] To make the technical solution of the present application more perfect, Figure 8A flowchart of a method for displaying a video with a visual watermark provided in an embodiment of the present application. Figure 8 At least the following steps are included:

[0121] S801: Obtain the video to be processed.

[0122] S802: Process the video to be processed to obtain at least one image frame.

[0123] S803: For each image frame, identify the human objects in the image frame, and count the number of human objects included in the image frame.

[0124] S804: Determine the graphic shape and graphic color of the visualization graph of the character object according to the first generation strategy for character object matching.

[0125] S805: Using the number of human objects included in each image frame as the vertical coordinate and the timestamp information of each image frame as the horizontal coordinate, and generating a visualization diagram of human object matching according to the correspondence between the number and the timestamp information, the shape and color of the graphics.

[0126] S806: In at least one image frame including a person object, extract facial features included in each image frame respectively.

[0127] S807: Display the visualization image in a first display area of ​​the video screen of the video, and display the face image represented by the face feature in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0128] S808-1: Determine a target playback node according to the visualization graph; in response to a user's dragging operation on the current playback node, jump the current display screen to the video screen corresponding to the target playback node.

[0129] S808-2: In response to a user clicking operation on the visualization graph, the current display screen is jumped to the video screen corresponding to the playback node indicated by the click operation.

[0130] Among them, S808 - 1 and S808 - 2 are two different processes of jumping the current display screen to the video screen of the target playback node, and either one of them can complete the process.

[0131] S809: During video playback, in response to a watermark hiding operation by the user, the visualization image is hidden and displayed.

[0132] In the embodiment of the present application, during the video screening process, based on the complexity of the screening video, there is a certain tendency to screen the target object in the video, which may involve the screening of specific human objects such as faces and vehicles. For such scenes, the detection of human objects in the video is optimized while increasing the distribution of the detection of specific human objects. The present application is an embodiment that is also applicable to the scene of diverse display of thermal distribution of videos when the emphasis of human objects is different during the screening process. When the video is playing, the user can select human objects for identification and thermal display according to their own screening focus, and at the same time optimize and increase the situation of identifying multiple human objects, and provide different types of thermal display methods, such as thermal images, colors, etc.

[0133] In the above embodiment, the target object is described by taking a person as an example. In actual application, the target object may also be a vehicle. Fig. 9 A method for displaying a video content mark is provided in an embodiment of the present application. Fig. 9 At least the following steps are included:

[0134] S901: Obtain the video to be processed.

[0135] S902: Process the video to be processed to obtain at least one image frame.

[0136] S903: For each image frame, identify vehicle objects in the image frame, and count the number of vehicle objects included in the image frame.

[0137] S904: Apply the second generation strategy for vehicle object matching to generate a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame.

[0138] Among them, the visualization diagram is used to represent the distribution of vehicle objects in the time dimension.

[0139] S905: In at least one image frame including a vehicle object, extract license plate features included in each image frame respectively.

[0140] S906: Displaying a visualization image in a first display area of ​​a video screen of the video, and displaying a license plate number represented by the license plate feature in a second display area of ​​an image screen corresponding to a source image frame of the license plate feature.

[0141] Optionally, a second generation strategy for vehicle object matching is applied to generate a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame, including:

[0142] According to the second generation strategy of vehicle object matching, the graphic shape and graphic color of the visualization map of the vehicle object are determined; the number of vehicle objects included in each image frame is used as the vertical coordinate, and the timestamp information of each image frame is used as the horizontal coordinate, and according to the correspondence between the number and the timestamp information, the visualization map of the vehicle object matching is generated according to the graphic shape and graphic color.

[0143] Optionally, displaying a visualization image in a first display area of ​​a video screen includes: displaying a first icon for displaying the visualization image; and responding to a user's operation on the first icon.

[0144] Optionally, the method further includes: determining a target playback node according to the visualization graph; and in response to a user dragging operation on the current playback node, jumping the current display screen to a video screen corresponding to the target playback node.

[0145] Optionally, the method further includes: in response to a user clicking operation on the visualization image, jumping the current display screen to a video screen corresponding to the playback node represented by the click operation.

[0146] Optionally, the method further includes: during video playback, in response to a watermark hiding operation by a user, hiding and displaying the visualization image.

[0147] Optionally, the display mode of the visualization graph includes any one of timed display, static display or dynamic display.

[0148] The implementation of each step in the embodiment of the present application can refer to the implementation process of the character object, which will not be repeated here. In the actual application process, when a person and a vehicle appear at the same time, the corresponding visualization image, face image and license plate number of the two can be displayed synchronously to ensure that they do not block each other.

[0149] Fig.10 A schematic diagram of displaying a license plate number watermark provided in an embodiment of the present application; Fig.11 A schematic diagram of another method for displaying a visual image watermark provided in an embodiment of the present application; Fig.12 A schematic diagram of another screen jump process provided in an embodiment of the present application; Fig.13 A schematic diagram of another hidden visual image watermark provided in an embodiment of the present application. Figure 10-13 The detailed display process of can be found in the above-mentioned embodiment and will not be described here in detail.

[0150] In an embodiment of the present application, first, for each image frame of at least one image frame obtained by processing the video to be processed, the vehicle object in the image frame is identified, and the number of vehicle objects included in the image frame is counted. Secondly, the first generation strategy of vehicle object matching is applied, and a visualization diagram of vehicle object matching is generated according to the number of vehicle objects included in each image frame. The visualization diagram is used for the distribution of vehicle objects in the time dimension, and the facial features included in each image frame are extracted in at least one image frame including the vehicle object. Finally, the visualization diagram is displayed in the first display area of ​​the video screen of the video, and the facial image represented by the facial features is displayed in the second display area of ​​the image screen corresponding to the source image frame of the facial features. Therefore, by displaying the visualization diagram in the video screen, the distribution of vehicle objects in the video can be intuitively displayed, and the time point with the peak frequency of vehicle objects or the time period when vehicle objects appear can be quickly located, thereby improving the efficiency of the video screening process. The license plate number can also be directly displayed to quickly identify the vehicle.

[0151] like Fig.14 As shown, based on the same inventive concept, an embodiment of the present application provides a marking and displaying device for video content, which is used to execute a marking and displaying method for video content whose target object is a person object. The device includes a first video acquisition unit 141, a first video parsing unit 142, a first target recognition unit 143, a first visualization graph generation unit 144, a face feature extraction unit 145 and a first display unit 146.

[0152] The first video acquisition unit 141 is used to: acquire the video to be processed;

[0153] The first video parsing unit 142 is used to: process the video to be processed to obtain at least one image frame;

[0154] The first object recognition unit 143 is used to: for each image frame, recognize a human object in the image frame, and count the number of human objects included in the image frame;

[0155] The first visualization graph generating unit 144 applies a first generating strategy for character object matching to generate a visualization graph of character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension;

[0156] The face feature extraction unit 145 is used to: extract the face features included in each image frame in at least one image frame including the human object;

[0157] The first display unit 146 is used to: display the visualization image in a first display area of ​​the video screen of the video, and display the face image represented by the face feature in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0158] In an optional implementation, the first visualization graph generating unit 144 is specifically configured to:

[0159] Determine the graphic shape and graphic color of the visualization graph of the human object according to the first generation strategy for matching the human object;

[0160] The number of human objects included in each image frame is used as the vertical coordinate, and the timestamp information of each image frame is used as the horizontal coordinate. According to the correspondence between the number and the timestamp information, a visualization diagram of human object matching is generated according to the graphic shape and color.

[0161] In an optional implementation, the first display unit 146 is specifically used for:

[0162] displaying a first icon for displaying a visualization;

[0163] Respond to the user's operation on the first icon.

[0164] In an optional implementation, the method further includes a first processing unit, configured to:

[0165] Determine the target playback node based on the visualization graph;

[0166] In response to the user's dragging operation on the current playback node, the current display screen jumps to the video screen corresponding to the target playback node.

[0167] In an optional implementation, the first processing unit is further configured to:

[0168] In response to a user's click operation on the visualization graph, the current display screen jumps to the video screen corresponding to the playback node represented by the click operation.

[0169] In an optional implementation, the first processing unit is further configured to:

[0170] During video playback, in response to the user's watermark hiding operation, the visualization image is hidden and displayed.

[0171] In an optional implementation, the display mode of the visualization diagram includes any one of timed display, static display or dynamic display.

[0172] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0173] like Fig.15 As shown, based on the same inventive concept, an embodiment of the present application provides a marking display device for video content, which is used to execute a marking display method for video content whose target object is a vehicle object. The device includes a second video acquisition unit 151, a second video parsing unit 152, a second target recognition unit 153, a second visualization graph generation unit 154, a license plate feature extraction unit 155 and a second display unit 156.

[0174] The second video acquisition unit 151 is used to: acquire the video to be processed;

[0175] The second video parsing unit 152 is used to: process the video to be processed to obtain at least one image frame;

[0176] The second target recognition unit 153 is used to: for each image frame, recognize the vehicle object in the image frame, and count the number of vehicle objects included in the image frame;

[0177] The second visualization graph generating unit 154 is used to: apply the second generation strategy of vehicle object matching, and generate a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of vehicle objects in the time dimension;

[0178] The license plate feature extraction unit 155 is used to: extract the license plate features included in each image frame in at least one image frame including the vehicle object;

[0179] The second display unit 156 is used to: display the visualization image in the first display area of ​​the video screen of the video, and display the license plate number represented by the license plate feature in the second display area of ​​the image screen corresponding to the source image frame of the license plate feature.

[0180] In an optional implementation, the second visualization graph generating unit 154 is specifically configured to:

[0181] Determine the graphic shape and graphic color of the visualization graph of the human object according to the first generation strategy for matching the human object;

[0182] The number of human objects included in each image frame is used as the vertical coordinate, and the timestamp information of each image frame is used as the horizontal coordinate. According to the correspondence between the number and the timestamp information, a visualization diagram of human object matching is generated according to the graphic shape and color.

[0183] In an optional implementation, the second display unit 156 is specifically used for:

[0184] displaying a first icon for displaying a visualization;

[0185] Respond to the user's operation on the first icon.

[0186] In an optional implementation, the method further includes a second processing unit, configured to:

[0187] Determine the target playback node based on the visualization graph;

[0188] In response to the user's dragging operation on the current playback node, the current display screen jumps to the video screen corresponding to the target playback node.

[0189] In an optional implementation, the second processing unit is further configured to:

[0190] In response to a user's click operation on the visualization graph, the current display screen jumps to the video screen corresponding to the playback node represented by the click operation.

[0191] In an optional implementation, the second processing unit is further configured to:

[0192] During video playback, in response to the user's watermark hiding operation, the visualization image is hidden and displayed.

[0193] In an optional implementation, the display mode of the visualization diagram includes any one of timed display, static display or dynamic display.

[0194] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0195] like Fig.16 As shown, based on the same inventive concept, an embodiment of the present application provides an electronic device for executing a marking display method for video content whose target object is a person object. The electronic device includes a first display screen 161 and a first processor 162.

[0196] The first processor 162 is configured to execute:

[0197] Get the video to be processed;

[0198] Processing the video to be processed to obtain at least one image frame;

[0199] For each image frame, identifying a human object in the image frame, and counting the number of human objects included in the image frame;

[0200] Applying the first generation strategy of character object matching, generating a visualization graph of character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension;

[0201] In at least one image frame including a person object, extracting facial features included in each image frame respectively;

[0202] The visualization image is displayed in a first display area of ​​the video screen of the video, and the face image represented by the face feature is displayed in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0203] In an optional implementation, the first processor 162 is specifically configured to execute:

[0204] Determine the graphic shape and graphic color of the visualization graph of the human object according to the first generation strategy for matching the human object;

[0205] The number of human objects included in each image frame is used as the vertical coordinate, and the timestamp information of each image frame is used as the horizontal coordinate. According to the correspondence between the number and the timestamp information, a visualization diagram of human object matching is generated according to the graphic shape and color.

[0206] In an optional implementation, the first processor 162 is further configured to execute:

[0207] displaying a first icon for displaying a visualization;

[0208] Respond to the user's operation on the first icon.

[0209] In an optional implementation, the first processor 162 is further configured to execute:

[0210] Determine the target playback node based on the visualization graph;

[0211] In response to the user's dragging operation on the current playback node, the current display screen jumps to the video screen corresponding to the target playback node.

[0212] In an optional implementation, the first processor 162 is further configured to execute:

[0213] In response to a user's click operation on the visualization graph, the current display screen jumps to the video screen corresponding to the playback node represented by the click operation.

[0214] In an optional implementation, the first processor 162 is further configured to execute:

[0215] During video playback, in response to the user's watermark hiding operation, the visualization image is hidden and displayed.

[0216] In an optional implementation, the display mode of the visualization diagram includes any one of timed display, static display or dynamic display.

[0217] like Fig.17 As shown, based on the same inventive concept, an embodiment of the present application provides an electronic device for executing a marking display method for video content whose target object is a vehicle object. The electronic device includes a second display screen 171 and a second processor 172.

[0218] The first processor 172 is configured to execute:

[0219] Get the video to be processed;

[0220] Processing the video to be processed to obtain at least one image frame;

[0221] For each image frame, identifying vehicle objects in the image frame, and counting the number of vehicle objects included in the image frame;

[0222] Applying the second generation strategy of vehicle object matching, generating a visualization graph of vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of vehicle objects in the time dimension;

[0223] In at least one image frame including a vehicle object, extracting facial features included in each image frame respectively;

[0224] The visualization image is displayed in a first display area of ​​the video screen of the video, and the face image represented by the face feature is displayed in a second display area of ​​the image screen corresponding to the source image frame of the face feature.

[0225] In an optional implementation, the second processor 172 is specifically configured to execute:

[0226] Determining a graphic shape and a graphic color of a visualization of a vehicle object according to a first generation strategy for matching the vehicle object;

[0227] The number of vehicle objects included in each image frame is used as the ordinate, and the timestamp information of each image frame is used as the abscissa. According to the correspondence between the number and the timestamp information, a visualization diagram of vehicle object matching is generated according to the shape and color of the graphic.

[0228] In an optional implementation, the second processor 172 is further configured to execute:

[0229] displaying a first icon for displaying a visualization;

[0230] Respond to the user's operation on the first icon.

[0231] In an optional implementation, the second processor 172 is further configured to execute:

[0232] Determine the target playback node based on the visualization graph;

[0233] In response to the user's dragging operation on the current playback node, the current display screen jumps to the video screen corresponding to the target playback node.

[0234] In an optional implementation, the second processor 172 is further configured to execute:

[0235] In response to a user's click operation on the visualization graph, the current display screen jumps to the video screen corresponding to the playback node represented by the click operation.

[0236] In an optional implementation, the second processor 172 is further configured to execute:

[0237] During video playback, in response to the user's watermark hiding operation, the visualization image is hidden and displayed.

[0238] In an optional implementation, the display mode of the visualization diagram includes any one of timed display, static display or dynamic display.

[0239] The embodiment of the present application also provides a computer storage medium, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned method for marking and displaying video content.

[0240] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0241] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0242] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0243] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0244] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for displaying a mark of video content, It is characterized in that include: Get the video to be processed; Processing the video to be processed to obtain at least one image frame; For each image frame, identifying a human object in the image frame, and counting the number of human objects included in the image frame; Applying a first generation strategy for character object matching, generating a visualization graph of the character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension; In at least one image frame including a person object, extracting facial features included in each image frame respectively; The visualization image is displayed in a first display area of ​​a video screen of the video, and a facial image represented by the facial feature is displayed in a second display area of ​​an image screen corresponding to a source image frame of the facial feature.

2. The method according to claim 1, It is characterized in that The first generation strategy of applying character object matching is used to generate a visualization graph of character object matching according to the number of character objects included in each image frame, including: Determining a graphic shape and a graphic color of a visualization graph of the character object according to a first generation strategy for matching the character object; The number of the character objects included in each of the image frames is used as the vertical coordinate, and the timestamp information of each image frame is used as the horizontal coordinate. According to the corresponding relationship between the number and the timestamp information, a visualization diagram of the character object matching is generated according to the graphic shape and graphic color.

3. The method according to claim 1, It is characterized in that The displaying of the visualization image in the first display area of ​​the video screen includes: displaying a first icon for displaying the visualization; Respond to the user's operation on the first icon.

4. The method according to claim 1, It is characterized in that The method further comprises: Determine a target playback node according to the visualization graph; In response to the user's dragging operation on the current playback node, the current display screen jumps to the video screen corresponding to the target playback node.

5. The method according to claim 1, It is characterized in that The method further comprises: In response to a user clicking operation on the visualization diagram, the current display screen is jumped to the video screen corresponding to the playback node represented by the click operation.

6. The method according to claim 1, It is characterized in that The method further comprises: During the video playback, in response to the user's watermark hiding operation, the visualization image is hidden and displayed.

7. The method according to claim 1, It is characterized in that The display mode of the visualization diagram includes any one of timed display, static display or dynamic display.

8. A method for displaying a mark of video content, It is characterized in that include: Get the video to be processed; Processing the video to be processed to obtain at least one image frame; For each image frame, identifying a vehicle object in the image frame, and counting the number of vehicle objects included in the image frame; Applying the second generation strategy of vehicle object matching, generating a visualization graph of the vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of the vehicle objects in the time dimension; In at least one image frame including a vehicle object, respectively extracting license plate features included in each image frame; The visualization image is displayed in a first display area of ​​a video screen of the video, and the license plate number represented by the license plate feature is displayed in a second display area of ​​an image screen corresponding to a source image frame of the license plate feature.

9. An electronic device, It is characterized in that Including display and processor: The processor is configured to perform: Get the video to be processed; Processing the video to be processed to obtain at least one image frame; For each image frame, identifying a human object in the image frame, and counting the number of human objects included in the image frame; Applying a first generation strategy for character object matching, generating a visualization graph of the character object matching according to the number of character objects included in each image frame; wherein the visualization graph is used to represent the distribution of the character objects in the time dimension; In at least one image frame including a person object, extracting facial features included in each image frame respectively; The display screen is configured to perform: The visualization image is displayed in a first display area of ​​a video screen of the video, and a facial image represented by the facial feature is displayed in a second display area of ​​an image screen corresponding to a source image frame of the facial feature.

10. An electronic device, It is characterized in that Includes display and processor; The processor is configured to perform: Get the video to be processed; Processing the video to be processed to obtain at least one image frame; For each image frame, identifying a vehicle object in the image frame, and counting the number of vehicle objects included in the image frame; Applying the second generation strategy of vehicle object matching, generating a visualization graph of the vehicle object matching according to the number of vehicle objects included in each image frame; wherein the visualization graph is used to represent the distribution of the vehicle objects in the time dimension; In at least one image frame including a vehicle object, respectively extracting license plate features included in each image frame; The display screen is configured to perform: The visualization image is displayed in a first display area of ​​a video screen of the video, and the license plate number represented by the license plate feature is displayed in a second display area of ​​an image screen corresponding to a source image frame of the license plate feature.

Images