Monitoring video log recording method and device, medium and computer program product

Abstract

The invention discloses a monitoring video log recording method and device, a medium and a computer program product, and relates to the technical field of video monitoring, and the method comprises the steps: extracting video frames from a monitoring video according to the current sampling frequency, and constructing a detection video frame set according to the sequence of time information corresponding to the extracted video frames; performing target object identification on the video frames in the detection video frame set, and determining the video frame in which the target object is identified as a target video frame; wherein the target object comprises any one or a combination of several of a portrait, a mark and a text; and generating a log record corresponding to the target video frame according to the time information of the target video frame, the equipment information of the monitoring equipment corresponding to the target video frame and the identified basic information of the target object. According to the method and the device, recording of the video content of the monitoring video is realized, and the retrieval efficiency of the monitoring video is improved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and more specifically, to a surveillance video log recording method, device, medium and computer program product. Background Art

[0002] With the continuous development of society and people's increasing attention to security, there is a demand for video surveillance in many places. And as the video becomes clearer, it also brings other problems. The space occupied is getting larger and larger, which increases the difficulty of video storage and video analysis. When you need to search for certain specific video clips or content through the stored videos, you need to read a large amount of video and then perform video analysis. When the amount of video is large, the time cost is very high.

[0003] Therefore, how to improve the efficiency of surveillance video retrieval is a technical problem that technicians in this field need to solve. Summary of the invention

[0004] The purpose of the present application is to provide a monitoring video log recording method, device, medium and computer program product, which improves the monitoring video retrieval efficiency.

[0005] To achieve the above objectives, the present application provides a monitoring video log recording method, comprising:

[0006] Extract video frames from the surveillance video according to the current sampling frequency, and construct a detection video frame set according to the order of time information corresponding to the extracted video frames;

[0007] Identify the target object on the video frames in the detection video frame set, and determine the video frame in which the target object is identified as the target video frame; wherein the target object includes any one or a combination of any one of a portrait, a logo, and a text;

[0008] A log record corresponding to the target video frame is generated according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the identified basic information of the target object.

[0009] Before extracting video frames from the surveillance video according to the current sampling frequency to obtain a detection video frame set, the method further includes:

[0010] When the current sampling frequency needs to be updated, the number of continuous video frames in the detection video frame set is determined; wherein the continuous video frame is a video frame whose degree of change relative to the previous video frame in the detection video frame set is less than a preset value;

[0011] Determine whether the number of continuous video frames is greater than the current video frame rate; if so, determine the current sampling frequency as the current video frame rate; if not, determine the current sampling frequency as the number of continuous video frames.

[0012] Among them, it also includes:

[0013] The current sampling frequency is updated according to the current update frequency; wherein the current update frequency is a preset multiple of the current sampling frequency.

[0014] The step of identifying the target object on the video frames in the detection video frame set includes:

[0015] Extracting image features of the video frames in the detection video frame set, and performing feature representation on the image features to obtain image feature representation;

[0016] performing feature matching between the image feature representation and standard feature representations of different types of target objects;

[0017] Different types of target objects in the video frame and basic information of the target objects are identified based on feature matching results using classification models corresponding to different types of target objects.

[0018] Wherein, if a human portrait is recognized in the target video frame, before generating a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the basic information of the recognized target object, the method further includes:

[0019] Matching in a portrait library according to the portrait information of the portrait;

[0020] If the match is successful, the portrait name of the matched target portrait is obtained, and the portrait name is added to the portrait information;

[0021] If the match fails, generating a portrait name according to the portrait information, and adding the portrait name to the portrait information;

[0022] Correspondingly, generating a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the basic information of the identified target object includes:

[0023] A log record corresponding to the target video frame is generated according to time information of the target video frame, device information of a monitoring device corresponding to the target video frame, and portrait information of the portrait.

[0024] Among them, it also includes:

[0025] Determine a first position of the portrait in the target video frame, determine a second position of the portrait in a subsequent video frame of the target video frame in the detection video frame set, and compare the first position with the second position;

[0026] If the second position is close to the center of the target video frame relative to the first position, the portrait event is determined as an entry event; if the second position is far from the center of the target video frame relative to the first position, the portrait event is determined as a departure event;

[0027] The portrait event is added to the log record.

[0028] Among them, it also includes:

[0029] Determine a cabinet label and a position label in the cabinet in the target video frame, determine a first color block color of a position corresponding to the position label in a video frame preceding the target video frame in the detection video frame set, and a second color block color of the position label at a position corresponding to the target video frame;

[0030] If the first color block is white and the second color block is black, the current operation behavior is determined as putting the device on the shelf; if the first color block is black and the second color block is white, the current operation behavior is determined as removing the device from the shelf;

[0031] The current operation behavior is added to the log record.

[0032] Wherein, after adding the current operation behavior to the log record, it also includes:

[0033] The cabinet tag and the location tag are added to the log record as the device location information of the current operation behavior.

[0034] Among them, it also includes:

[0035] If the current operation behavior is to remove the device from the shelf, an alarm message is generated according to the location tag.

[0036] Among them, it also includes:

[0037] Identify the color of the color block contained in the portrait in the target video frame and the number of color blocks corresponding to the color block color;

[0038] Comparing the color of the color blocks contained in the portrait in the target video frame and the number of color blocks corresponding to the color of the color blocks in the previous video frame of the target video frame in the detection video frame set, so as to determine the degree of change of the target video frame relative to the portrait in the previous video frame;

[0039] If the degree of change of the portrait reaches a first threshold, an alarm message is generated according to the degree of change of the portrait.

[0040] Wherein, if a logo is recognized in the target video frame, the method further includes:

[0041] Determine the device manufacturer information corresponding to the flag, and add the device manufacturer information to the log record.

[0042] Among them, it also includes:

[0043] A target index item is determined, a target record corresponding to the target index item is matched in the log record, and the target record is integrated in chronological order.

[0044] To achieve the above objectives, the present application provides a surveillance video log recording device, comprising:

[0045] An extraction module is used to extract video frames from the surveillance video according to the current sampling frequency, and to construct a detection video frame set according to the order of time information corresponding to the extracted video frames;

[0046] an identification module, configured to identify a target object in the video frames in the detection video frame set, and determine the video frame in which the target object is identified as a target video frame; wherein the target object includes any one or a combination of a portrait, a logo, and a text;

[0047] The recording module is used to generate a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the basic information of the identified target object.

[0048] To achieve the above objectives, the present application provides an electronic device, including:

[0049] Memory for storing computer programs;

[0050] A processor is used to implement the steps of the monitoring video log recording method as described above when executing the computer program.

[0051] To achieve the above objectives, the present application provides a non-volatile storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the monitoring video log recording method as described above are implemented.

[0052] To achieve the above objectives, the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned monitoring video log recording method when executed by a processor.

[0053] It can be seen from the above scheme that a monitoring video log recording method provided by the present application includes: extracting video frames from the monitoring video according to the current sampling frequency, and constructing a detection video frame set in the order of time information corresponding to the extracted video frames; identifying the target object for the video frames in the detection video frame set, and determining the video frame in which the target object is identified as the target video frame; wherein the target object includes any one or a combination of any several of a portrait, a logo, and a text; and generating a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the basic information of the identified target object.

[0054] The surveillance video log recording method provided by the present application extracts key frames from the surveillance video at a set sampling frequency, and then identifies target objects on these video frames, including portraits, logos and texts, and records them in a log in a structured manner, thereby recording the video content of the surveillance video and facilitating users to understand the video content. By quickly retrieving the log, the retrieval efficiency of the surveillance video can be significantly improved. The present application also discloses a surveillance video log recording device, an electronic device, a non-volatile storage medium and a computer program product, which can also achieve the above-mentioned technical effects.

[0055] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific implementation methods, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the drawings:

[0057] Figure 1 is a flowchart of a monitoring video log recording method according to an exemplary embodiment;

[0058] Figure 2 is a flow chart of another monitoring video log recording method according to an exemplary embodiment;

[0059] Figure 3 is a structural diagram of a surveillance video log recording device according to an exemplary embodiment;

[0060] Figure 4The figure is a structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0062] In the related art, the log recording method of the monitoring video is as follows: the user chooses to start recording through the client, and when the client opens the real-time video stream, it synchronously sends a first asynchronous request to the video monitoring server; the video monitoring server receives the first asynchronous request, records the current time as the video stream opening time, generates a monitoring log code, and records the video stream opening time and the monitoring log code as a monitoring log, and the video monitoring server sends the monitoring log code to the client; the client receives the monitoring log code, and performs corresponding processing on the opened real-time video stream and the monitoring log code; the user chooses to end recording through the client, and the client closes the currently opened real-time video stream, and synchronously sends a second asynchronous request to the video monitoring server, and the parameters of the second asynchronous request include the monitoring log code; the video monitoring server receives the second asynchronous request, records the current time as the video stream closing time, searches for the corresponding monitoring log according to the monitoring log code, and updates it to the corresponding monitoring log. The video surveillance server can find the corresponding surveillance log according to the query request and send it to the client. The client selects a surveillance log from the corresponding surveillance log, obtains its device code, video stream start time, and video stream end time, and can directly call the corresponding video for playback. In this way, the selected video can be played more quickly according to the query result to review the occurrence of the event. It can be seen that the log recording method of surveillance video in the related art only records the log of the video itself being operated, and cannot record the log of the video content, and the efficiency of video content retrieval is low.

[0063] Therefore, this application extracts key frames from the surveillance video at a set sampling frequency, and then performs target object recognition on these video frames, including portraits, logos and texts, and records them in a structured manner in the log, thereby recording the video content of the surveillance video and facilitating users to understand the video content. By quickly retrieving the log, the retrieval efficiency of the surveillance video can be significantly improved.

[0064] The embodiment of the present application discloses a monitoring video log recording method, which improves the monitoring video retrieval efficiency.

[0065] See also Figure 1 , according to an exemplary embodiment, a flowchart of a monitoring video log recording method is shown, such as Figure 1 As shown, including:

[0066] S101: extracting video frames from the surveillance video according to the current sampling frequency to obtain a detection video frame set;

[0067] In this step, during the real-time shooting of the surveillance video, a frame of image is extracted at a certain time interval according to the current sampling frequency to form a set of multiple continuous images, i.e., a detection video frame set, which are used for subsequent detection analysis. It can be understood that the current sampling frequency in this embodiment can be preset to a fixed value, or can be adaptively and flexibly adjusted according to the actual situation of the surveillance video.

[0068] As a feasible implementation method, before extracting video frames from the surveillance video according to the current sampling frequency to obtain a detection video frame set, it also includes: when the current sampling frequency needs to be updated, determining the number of continuous video frames in the detection video frame set; wherein the continuous video frame is a video frame whose degree of change relative to the previous video frame in the detection video frame set is less than a preset value; judging whether the number of continuous video frames is greater than the current video frame rate; if so, determining the current sampling frequency as the current video frame rate; if not, determining the current sampling frequency as the number of continuous video frames.

[0069] In a specific implementation, when the current sampling frequency needs to be adjusted, the continuous frames in the video frame set are analyzed and detected to identify those "continuous video frames" whose degree of change from the previous frame is less than a preset threshold. If the number of continuous video frames exceeds the current video frame rate, it means that the video content has not changed much, and the current video frame rate is used as the current sampling frequency; if the number of continuous video frames is less than or equal to the current video frame rate, it means that the video content is relatively static, and the number of continuous video frames is used as the current sampling frequency.

[0070] For example, the current video frame rate is 60 Hz. When the change degree of N consecutive video frames is less than 5% of the entire video frame, if N Hz is less than or equal to 60 Hz, the current sampling frequency is set to N Hz; if N Hz is greater than 60 Hz, the current sampling frequency is set to 60 Hz.

[0071] As a feasible implementation method, the degree of change of the current video frame relative to the previous video frame is determined as follows: the pixel value of each pixel in the current video frame and the pixel value of each pixel in the previous video frame are counted, and the pixel values ​​of the pixels at corresponding positions in the current video frame and the previous video frame are compared. If the difference between the two is less than or equal to a preset pixel threshold, the pixel point is determined as an unchanged pixel point, otherwise the pixel point is determined as a changed pixel point, and the ratio of the number of changed pixels in the current video frame to the number of pixels contained in the entire current video frame is used as the degree of change of the current video frame relative to the previous video frame.

[0072] This implementation effectively quantifies the degree of change between video frames by accurately counting and comparing the pixel value changes of corresponding pixels in consecutive video frames. By setting a preset pixel threshold, the system can distinguish which pixels are unchanged and which are changed, thereby calculating the proportion of changed pixels. This method not only improves the sensitivity and recognition accuracy of changes in video content, but also provides an objective metric for video processing and analysis by quantifying the degree of change.

[0073] As a feasible implementation, it also includes: updating the current sampling frequency according to the current update frequency; wherein the current update frequency is a preset multiple of the current sampling frequency. In a specific implementation, the current update frequency of the current sampling frequency is a preset multiple of the current sampling frequency, so as to minimize the distortion of the current sampling frequency caused by the update mechanism of the current sampling frequency and reduce the accuracy of log records. For example, the current update frequency is determined to be twice the current sampling frequency, that is, the current update frequency is set to 2NHz, and when the current video frame rate is 60Hz, the current update frequency is set to 120HZ.

[0074] S102: Identify a target object on the video frames in the detection video frame set, and determine the video frame in which the target object is identified as a target video frame; wherein the target object includes any one of a portrait, a logo, and text, or any combination of several of them;

[0075] In this step, the target object recognition technology is used to analyze each frame of the video in the detection video frame set to find and identify video frames containing specific target objects such as portraits, logos or text. Once these target objects are successfully identified in the video frame, the corresponding frame will be marked as a "target video frame", and the information related to the target object will be identified in the target video frame. The target video frame and the information related to the target object therein will serve as the basis for subsequent monitoring video log recording.

[0076] As a feasible implementation method, target objects are identified in video frames in the detection video frame set, including: extracting image features of the video frames in the detection video frame set, and performing feature representation on the image features to obtain image feature representation; performing feature matching on the image feature representation and standard feature representations of different types of target objects; and using classification models corresponding to different types of target objects to identify different types of target objects in the video frames and basic information of the target objects based on the feature matching results.

[0077] In a specific implementation, when a portrait, logo or text is recognized from a video frame, the above content is parsed into natural language through an artificial intelligence recognition algorithm. The principles of image recognition include image preprocessing, feature extraction, feature representation, feature matching, classification and decision-making, and result output. Image preprocessing: preprocessing the original image, including image denoising, resizing, graying and other operations. Feature extraction: the core step of image recognition, the purpose is to extract the most representative and distinguishing features from the image to describe the key information of the image. Common feature methods include edge detection, corner point extraction, texture analysis, color histogram, etc. Feature representation: after feature extraction, the extracted features need to be quantified and represented in a form that can be processed by the computer. It can be a feature vector or feature description. Feature matching: compare and match the extracted features with known patterns. By comparing the distance, similarity or relative position between two feature vectors, determine whether they represent the same object or scene. Common feature matching algorithms include nearest neighbor algorithm, support vector machine, etc. Classification and decision-making: Based on the results of feature matching, images are classified and decided through trained machine learning models or deep learning models. Common classification algorithms include decision trees, random forests, convolutional neural networks, etc. Result output: Based on the recognition results, the results are displayed to the user or used as input for other applications.

[0078] For face recognition, eigenfaces, fisherman faces, etc. can be used. For text recognition, optical character recognition, convolutional neural networks, support vector machines, etc. can be used. For logo recognition, SIFT algorithm, color histogram, etc. can be used. For example, when using convolutional neural networks for image recognition, if the input is an n×n image, and a convolution operation is performed with an f×f convolution kernel, padding=p, and stride=s, then its output size is [(n+2p-f) / s+1]×[(n+2p-f) / s+1], [] means rounding down. For example, there is a 6×6 image that is convolved with a 3×3 convolution kernel, p=0, s=2. Then the output is (6+2×0-3) / 2+1=2.5, which is rounded to 2, and the output is a 2×2 image.

[0079] As a feasible implementation mode, if a portrait is recognized in the target video frame, this embodiment also includes: matching in a portrait library according to the portrait information of the portrait; if the match is successful, obtaining the portrait name of the matched target portrait, and adding the portrait name to the portrait information; if the match fails, generating a portrait name according to the portrait information, and adding the portrait name to the portrait information.

[0080] In a specific implementation, the portrait information can be associated with a portrait library, and the portrait library can be preset or subsequently add portrait information, which can include various attributes about the portrait, such as the portrait name, the portrait gender, the portrait age, the portrait department, and the portrait head code. If the portrait information is recognized in the target video frame, it is matched in the portrait library according to the portrait information. When there is no such portrait information in the portrait library, the portrait name can be generated according to certain rules, which can include the overall shape of the portrait (round, square, pointed, etc.), whether the gender of the recognized portrait can be identified (male, female, gender cannot be identified), etc. The serial number can be coded with numbers, letters, etc., such as encoding the recognized portrait information as male-square-A. When there is such portrait information in the portrait library, the portrait name in the portrait library is added to the portrait information so that the portrait name can be stored in the log record later.

[0081] S103: Generate a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the identified basic information of the target object.

[0082] In this step, the identified data and environmental information are organized into structured data for storage, which may include portrait information, logo information, text information, device information of the monitoring device corresponding to the target video frame, time information of the target video frame, etc.

[0083] As a preferred implementation, if a sign is recognized in the target video frame, the present embodiment further includes: determining the device manufacturer information corresponding to the sign, and adding the device manufacturer information to the log record. In a specific implementation, if a sign is recognized in the target video frame, the device manufacturer information corresponding to the sign is determined, and the device manufacturer information is added to the log record corresponding to the target video frame.

[0084] As a preferred implementation, this embodiment further includes: determining a target index item, matching a target record corresponding to the target index item in the log record, and integrating the target records in chronological order.

[0085] In a specific implementation, when multiple recording devices are in the same network or the same group, records can be retrieved and information can be organized and merged in the form of target index items. The target index items can include portraits, time, device names, etc. It supports setting integration rules for periodic information merging and users selecting custom content for real-time data merging, as well as the export of integrated information.

[0086] For example, taking portrait A as the target index item, the integrated log records are: Portrait A 2020-02-02T02:02:02 Recording device A enters, 2020-02-02T03:03:03 Recording device A exits, 2020-02-02T04:04:04 Recording device B enters, operates cabinet A, equipment A is put on the shelf, 2020-02-02T05:05:05 Recording device B exits.

[0087] The monitoring video log recording method provided in the embodiment of the present application extracts key frames from the monitoring video at a set sampling frequency, and then performs target object recognition on these video frames, including portraits, logos and texts, and records them in a log in a structured manner, thereby recording the video content of the monitoring video and facilitating users to understand the video content. By quickly searching the log, the retrieval efficiency of the monitoring video can be significantly improved.

[0088] The embodiment of the present application introduces a log recording method when a human portrait is recognized in a target video frame. Specifically:

[0089] See also Figure 2 , according to an exemplary embodiment, a flowchart of another monitoring video log recording method is shown, such as Figure 2 As shown, including:

[0090] S201: Determine a first position of a portrait in a target video frame, determine a second position of the portrait in a subsequent video frame of the target video frame in a detection video frame set, and compare the first position with the second position; if the second position is close to a center position of the target video frame relative to the first position, determine the portrait event as an entry event; if the second position is far from a center position of the target video frame relative to the first position, determine the portrait event as a departure event; add the portrait event to a log record.

[0091] In a specific implementation, in the target video frame, the initial position (first position) of the portrait is first determined, and then the new position (second position) of the portrait is tracked and determined in the continuous video frames after the target video frame. By comparing the two positions, if it is found that the second position of the portrait is closer to the center point of the video frame relative to the first position, this indicates that the portrait is moving toward the center, so this behavior is marked as an entry event; conversely, if the second position of the portrait is far away from the center point, this indicates that the portrait is moving away, and it is also marked as a departure event. Regardless of whether the portrait enters or leaves, the event is recorded in the log for subsequent monitoring and analysis.

[0092] S202: Determine a cabinet label and a position label in the cabinet in the target video frame, determine a first color block color of a position corresponding to the position label in a previous video frame of the target video frame in the detection video frame set, and a second color block color of the position label at a position corresponding to the target video frame; if the first color block color is white and the second color block color is black, determine the current operation behavior as putting the device on the shelf; if the first color block color is black and the second color block color is white, determine the current operation behavior as removing the device from the shelf;

[0093] S203: adding the current operation behavior to the log record, and adding the cabinet label and the location label as device location information of the current operation behavior to the log record;

[0094] In a specific implementation, the cabinet label and the position label in the cabinet, such as the U-position label, are identified in the target video frame, and the position of the position label in the previous frame of the target video frame and the position in the target video frame are compared. If the position of the position label in the previous frame of the target video frame is vacant or the corresponding first color block is white, and there is a device at the position of the position label in the target video frame or the corresponding second color block is black, this may mean that a device has been placed in the monitoring area, so the current operation behavior is identified as "equipment is put on the shelf". On the contrary, if there is a device at the position of the position label in the previous frame of the target video frame or the corresponding first color block is black, and the position of the position label in the target video frame is vacant or the corresponding second color block is white, this may indicate that the device has been removed, so the current operation behavior is identified as "equipment is removed from the shelf". In either case, these operation behaviors will be recorded in the log, and the cabinet label and the position label will be added to the log record as the device location information of the current operation behavior.

[0095] S204: If the current operation behavior is to remove the device from the shelf, an alarm message is generated according to the location tag.

[0096] In a specific implementation, when a device is removed from a rack or moved out of a current cabinet, it can be determined whether to generate a warning record based on user needs.

[0097] S205: Identify the color of the color blocks contained in the portrait in the target video frame and the number of color blocks corresponding to the color block colors; compare the color of the color blocks contained in the portrait in the target video frame and the number of color blocks corresponding to the color block colors in the previous video frame of the target video frame in the detection video frame set to determine the degree of change of the portrait in the target video frame relative to the previous video frame; if the degree of change of the portrait reaches a first threshold, generate an alarm message according to the degree of change of the portrait.

[0098] In a specific implementation, the color and number of color blocks contained in the portrait in the target video frame are identified, and then compared with the color and number of color blocks of the same portrait in the previous video frame to evaluate the degree of change of the portrait between two consecutive frames. If the degree of change exceeds a preset first threshold, it indicates that the portrait has changed significantly, which may be due to changes in the portrait's posture, movement or clothing. The system will generate corresponding alarm information based on the degree of change, so as to respond to possible security incidents or abnormal behaviors in a timely manner.

[0099] For example, at 2020-02-02T02:02:02, portrait A (black 100, white 100) enters recording device A, and at 2020-02-02T03:03:03, portrait A (black 130, white 110) exits. The amount of suspected content increases, and a corresponding alarm message is generated.

[0100] It can be seen that if a portrait is detected in the target video frame, the corresponding log record may include the portrait name, full name, gender, age, department, operation time, monitoring device name, portrait event, portrait color block color and the corresponding number of color blocks, cabinet label, U-position label, current operation behavior, etc.

[0101] A monitoring video log recording device provided in an embodiment of the present application is introduced below. The monitoring video log recording device described below and the monitoring video log recording method described above can be referenced to each other.

[0102] See also Figure 3 , according to an exemplary embodiment, a structural diagram of a surveillance video log recording device is shown, such as Figure 3 As shown, including:

[0103] The extraction module 301 is used to extract video frames from the monitoring video according to the current sampling frequency, and construct a detection video frame set according to the order of time information corresponding to the extracted video frames;

[0104] The recognition module 302 is used to recognize the target object in the video frame in the detection video frame set, and determine the video frame in which the target object is recognized as the target video frame; wherein the target object includes any one or a combination of a portrait, a logo, and a text;

[0105] The recording module 303 is used to generate a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the basic information of the identified target object.

[0106] The monitoring video log recording device provided in the embodiment of the present application extracts key frames from the monitoring video at a set sampling frequency, and then performs target object recognition on these video frames, including portraits, logos and texts, and records them in a log in a structured manner, thereby recording the video content of the monitoring video and facilitating users to understand the video content. By quickly retrieving the log, the retrieval efficiency of the monitoring video can be significantly improved.

[0107] Based on the above embodiment, as a preferred implementation, it also includes:

[0108] A determination module is used to determine the number of continuous video frames in the detection video frame set when the current sampling frequency needs to be updated; wherein the continuous video frame is a video frame whose degree of change relative to the previous video frame in the detection video frame set is less than a preset value; determine whether the number of continuous video frames is greater than the current video frame rate; if so, determine the current sampling frequency as the current video frame rate; if not, determine the current sampling frequency as the number of continuous video frames.

[0109] Based on the above embodiment, as a preferred implementation, it also includes:

[0110] An updating module is used to update the current sampling frequency according to a current updating frequency; wherein the current updating frequency is a preset multiple of the current sampling frequency.

[0111] Based on the above embodiments, as a preferred implementation mode, the recognition module 302 is specifically used to: extract image features of the video frames in the detection video frame set, perform feature representation on the image features to obtain image feature representation; perform feature matching on the image feature representation and standard feature representations of different types of target objects; and use classification models corresponding to different types of target objects to identify different types of target objects in the video frames and basic information of the target objects based on feature matching results.

[0112] Based on the above embodiment, as a preferred implementation manner, if a portrait is recognized in the target video frame, the method further includes:

[0113] A matching module, for matching the portrait in a portrait database according to the portrait information of the portrait; if the match is successful, obtaining the portrait name of the matched target portrait, and adding the portrait name to the portrait information; if the match fails, generating a portrait name according to the portrait information, and adding the portrait name to the portrait information;

[0114] Correspondingly, the recording module 303 is specifically used to generate a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the portrait information of the portrait.

[0115] Based on the above embodiment, as a preferred implementation, it also includes:

[0116] A portrait event recording module is used to determine the first position of the portrait in the target video frame, determine the second position of the portrait in the subsequent video frame of the target video frame in the detection video frame set, and compare the first position with the second position; if the second position is close to the center position of the target video frame relative to the first position, the portrait event is determined as an entry event; if the second position is far from the center position of the target video frame relative to the first position, the portrait event is determined as a departure event; and the portrait event is added to the log record.

[0117] Based on the above embodiment, as a preferred implementation, it also includes:

[0118] A current operation behavior recording module is used to determine the cabinet label and the position label in the cabinet in the target video frame, determine the first color block color of the position label at the corresponding position in the previous video frame of the target video frame in the detection video frame set, and the second color block color of the position label at the corresponding position in the target video frame; if the first color block color is white and the second color block color is black, the current operation behavior is determined as equipment shelving; if the first color block color is black and the second color block color is white, the current operation behavior is determined as equipment removal; the current operation behavior is added to the log record.

[0119] Based on the above embodiment, as a preferred implementation, it also includes:

[0120] The device location information recording module is used to add the cabinet label and the location label as the device location information of the portrait operation into the log record.

[0121] Based on the above embodiment, as a preferred implementation, it also includes:

[0122] The first alarm module is used to generate an alarm message according to the location tag if the current operation behavior is to remove the device from the shelf.

[0123] Based on the above embodiment, as a preferred implementation, it also includes:

[0124] The second alarm module is used to identify the color of the color blocks contained in the portrait in the target video frame and the number of color blocks corresponding to the color block colors; compare the color of the color blocks contained in the portrait in the target video frame and the number of color blocks corresponding to the color block colors in the previous video frame of the target video frame in the detection video frame set to determine the degree of change of the target video frame relative to the portrait in the previous video frame; if the degree of change of the portrait reaches a first threshold, generate an alarm message according to the degree of change of the portrait.

[0125] Based on the above embodiment, as a preferred implementation manner, if a logo is recognized in the target video frame, the method further includes:

[0126] The device manufacturer information recording module is used to determine the device manufacturer information corresponding to the flag and add the device manufacturer information to the log record.

[0127] Based on the above embodiment, as a preferred implementation, it also includes:

[0128] The integration module is used to determine a target index item, match a target record corresponding to the target index item in the log record, and integrate the target record in chronological order.

[0129] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0130] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application further provides an electronic device, Figure 4 FIG. 1 is a structural diagram of an electronic device according to an exemplary embodiment. Figure 4 As shown, the electronic equipment includes:

[0131] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0132] The processor 2 is connected to the communication interface 1 to realize information exchange with other devices, and is used to execute the monitoring video log recording method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 3.

[0133] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 4 Various buses are labeled as bus system 4 .

[0134] The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0135] It can be understood that the memory 3 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 3 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0136] The method disclosed in the above embodiment of the present application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 2 or the instruction in the form of software. The above processor 2 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 3, and the processor 2 reads the program in the memory 3 and completes the steps of the above method in combination with its hardware.

[0137] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented, which will not be repeated here for the sake of brevity.

[0138] In an exemplary embodiment, the embodiment of the present application further provides a non-volatile storage medium storing a computer program, which can be executed by the processor 2 to complete the steps of the aforementioned method.

[0139] In an exemplary embodiment, the present application also provides a computer program product, including a computer program, which is executed by the processor 2 to complete the steps of the aforementioned method.

[0140] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to computer program instructions, and the aforementioned computer program can be stored in a non-volatile storage medium. When the computer program is executed, it executes the steps of the above-mentioned method embodiment. Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product, which is stored in a non-volatile storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, a network device, etc.) to execute all or part of the methods described in each embodiment of the present application.

[0141] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A monitoring video log recording method, characterized in that: include: Extract video frames from the surveillance video according to the current sampling frequency, and construct a detection video frame set according to the order of time information corresponding to the extracted video frames; Identify the target object on the video frames in the detection video frame set, and determine the video frame in which the target object is identified as the target video frame; wherein the target object includes any one or a combination of any one of a portrait, a logo, and a text; A log record corresponding to the target video frame is generated according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the identified basic information of the target object.

2. The monitoring video log recording method according to claim 1, characterized in that: Before extracting video frames from the surveillance video according to the current sampling frequency to obtain a detection video frame set, the following steps are also included: When the current sampling frequency needs to be updated, the number of continuous video frames in the detection video frame set is determined; wherein the continuous video frame is a video frame whose degree of change relative to the previous video frame in the detection video frame set is less than a preset value; Determine whether the number of continuous video frames is greater than the current video frame rate; if so, determine the current sampling frequency as the current video frame rate; if not, determine the current sampling frequency as the number of continuous video frames.

3. The monitoring video log recording method according to claim 2, characterized in that: Also includes: The current sampling frequency is updated according to the current update frequency; wherein the current update frequency is a preset multiple of the current sampling frequency.

4. The monitoring video log recording method according to claim 1, characterized in that: Identifying the target object on the video frames in the detection video frame set includes: Extracting image features of the video frames in the detection video frame set, and performing feature representation on the image features to obtain image feature representation; performing feature matching between the image feature representation and standard feature representations of different types of target objects; Different types of target objects in the video frame and basic information of the target objects are identified based on feature matching results using classification models corresponding to different types of target objects.

5. The monitoring video log recording method according to claim 1, characterized in that: If a human portrait is recognized in the target video frame, before generating a log record corresponding to the target video frame according to time information of the target video frame, device information of a monitoring device corresponding to the target video frame, and basic information of the recognized target object, the method further includes: Matching in a portrait library according to the portrait information of the portrait; If the match is successful, the portrait name of the matched target portrait is obtained, and the portrait name is added to the portrait information; If the match fails, generating a portrait name according to the portrait information, and adding the portrait name to the portrait information; Correspondingly, generating a log record corresponding to the target video frame according to the time information of the target video frame, the device information of the monitoring device corresponding to the target video frame, and the basic information of the identified target object includes: A log record corresponding to the target video frame is generated according to time information of the target video frame, device information of a monitoring device corresponding to the target video frame, and portrait information of the portrait.

6. The monitoring video log recording method according to claim 5, characterized in that: Also includes: Determine a first position of the portrait in the target video frame, determine a second position of the portrait in a subsequent video frame of the target video frame in the detection video frame set, and compare the first position with the second position; If the second position is close to the center position of the target video frame relative to the first position, determining the portrait event as an entry event; If the second position is far away from the center position of the target video frame relative to the first position, determining the portrait event as a leaving event; The portrait event is added to the log record.

7. The monitoring video log recording method according to claim 5, characterized in that: Also includes: Determine a cabinet label and a position label in the cabinet in the target video frame, determine a first color block color of a position corresponding to the position label in a video frame preceding the target video frame in the detection video frame set, and a second color block color of the position label at a position corresponding to the target video frame; If the first color block is white and the second color block is black, the current operation behavior is determined as putting the device on the shelf; if the first color block is black and the second color block is white, the current operation behavior is determined as removing the device from the shelf; The current operation behavior is added to the log record.

8. The monitoring video log recording method according to claim 7, characterized in that: After adding the current operation behavior to the log record, it also includes: The cabinet tag and the location tag are added to the log record as the device location information of the current operation behavior.

9. The monitoring video log recording method according to claim 7, characterized in that: Also includes: If the current operation behavior is to remove the device from the shelf, an alarm message is generated according to the location tag.

10. The monitoring video log recording method according to claim 1, characterized in that: Also includes: Identify the color of the color block contained in the portrait in the target video frame and the number of color blocks corresponding to the color block color; Comparing the color of the color blocks contained in the portrait in the target video frame and the number of color blocks corresponding to the color of the color blocks in the previous video frame of the target video frame in the detection video frame set, so as to determine the degree of change of the target video frame relative to the portrait in the previous video frame; If the degree of change of the portrait reaches a first threshold, an alarm message is generated according to the degree of change of the portrait.

11. The monitoring video log recording method according to claim 1, characterized in that: If a logo is identified in the target video frame, the method further includes: Determine the device manufacturer information corresponding to the flag, and add the device manufacturer information to the log record.

12. The surveillance video log recording method according to claim 1, characterized in that: Also includes: A target index item is determined, a target record corresponding to the target index item is matched in the log record, and the target record is integrated in chronological order.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the surveillance video log recording method as claimed in any one of claims 1 to 12 when executing the computer program.

14. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed, the steps of the monitoring video log recording method according to any one of claims 1 to 12 are implemented.

15. A computer program product, characterized in that It comprises a computer program, which, when executed, implements the steps of the surveillance video log recording method according to any one of claims 1 to 12.

Images