Method for accurately identifying and identifying RDP video characters

Abstract

The invention discloses a method for accurately identifying and identifying RDP video characters, which comprises the following steps of: firstly, converting an RDP video format into an MP4 video, smoothly playing the converted MP4 video by using various playing software, then extracting a key frame image in the MP4 video, identifying characters in the key frame image by using PaddleOCR, and outputting the time and the position of the characters in the key frame. And positioning the retrieved characters, and positioning the recorded video according to the characters. According to the technical scheme, the obtained MP4-format video file is more convenient to check, the recorded video can be positioned according to the recognized characters, and auditing of the recorded video is facilitated for auditing personnel.

Description

Technical Field

[0001] The present invention relates to the technical field of screen recording security auditing, and in particular to a method for accurately identifying and marking RDP video text. Background Art

[0002] A high proportion of Windows system assets are managed through the Remote Display Protocol (RDP). Screen recording security audit is used to audit all operation behaviors of intranet and remote access users. It is a management tool that can help monitor, record, review, and replay internal personnel operation behaviors and protect server security. It can easily help managers check whether internal operators comply with operational norms and established rules and requirements.

[0003] Currently, the RDP video audit function provided by bastion host manufacturers generally only records the operation video and keyboard input. Some manufacturers extract text from the video through OCR (Optical Character Recognition). Most RDP audit systems provided by bastion host manufacturers have the following three problems: 1. Non-standard video format, which is not convenient for mainstream players to play; 2. The accuracy of extracting text from RDP videos is not high; 3. There is no text identification and positioning function, and it is impossible to locate the time and position of the text in the video. Summary of the invention

[0004] In view of this, it is necessary to provide a method for accurately identifying and marking RDP video text, which can accurately identify and extract text in the video and accurately locate its time and position in the video.

[0005] A method for accurately identifying and marking RDP video text, used for text extraction in bastion machine video audit, includes the following specific steps: Convert RDP video format to MP4 video format; Extract key frame images from MP4 videos; Use PaddleOCR to recognize text in keyframe images; Search for text to find when and where it appears in the video.

[0006] Preferably, the specific steps of converting the RDP video format to the MP4 video format include: Use the Apache Guacenc tool to open an RDP session. After the session ends, an RDP recording will be generated. Use the guacenc tool to convert RDP videos into standard MP4 videos.

[0007] Preferably, the specific steps of extracting the key frame image in the MP4 video include: Use ffmpeg to extract key frame images from MP4 video. Record the exact time of key frame images in the video.

[0008] Preferably, the specific steps of using the ffmpeg tool to extract key frame images from the MP4 video include: The specific instructions for using the ffmpeg tool to extract key frame images are as follows: ffmpeg -i rdp.m4v -vf "select='eq(pict_type,PICT_TYPE_I)',showinfo" -vsync vfr -q:v 2 -f image2 keyframes_%05d.png>&keyframes_timestamps.txt; in, -i rdp.mp4: is the input video file; -vf select='eq(pict_type,PICT_TYPE_I)': select and output only key frames; -vsync vfr: output images according to the input frame rate; -q:v 2: is the quality of the output image; the image quality ranges from 1 to 31 from low to high; -f image2: is the output image format; keyframes_%05d.png: is the name of the output file, where %05d will be replaced by the keyframe number; showinfo: Output detailed information for each keyframe.

[0009] Preferably, the specific steps of using PaddleOCR to recognize text in a key frame image include: Use PaddleOCR tool to recognize Chinese characters in keyframe images; Record the time and location of text appearance and generate a database file.

[0010] Preferably, the specific steps of recognizing characters in the key frame image by using the PaddleOCR tool include: paddleocr --image_dir rdp.jpg --use_angle_cls true --lang zh in, -image_dir rdp.jpg: is the file name of the key frame image; -use_angle_cls true: can recognize text rotated 180 degrees; -lang zh: The language type is Chinese.

[0011] Preferably, the specific steps of searching for text and finding the time and location of its appearance in the video include: Click on the text in the database file to query the database for the timestamp and coordinates of the text in the video; Call the player's seek() function to start playing from the specified timestamp; According to the coordinates of the text, mark the text with a red box on the screen.

[0012] In the above method for accurately identifying and marking RDP video text, the RDP video format is first converted to MP4 video. The converted MP4 video can be played smoothly using a variety of playback software, and then the key frame image in the MP4 video is extracted, and then PaddleOCR is used to identify the text in the key frame image, and the time and position of the text in the key frame are output, and the retrieved text is located, and the video is located based on the text. In this technical solution, the obtained MP4 format video file is easier to view, and the video can also be located according to the recognized text, which is convenient for auditors to audit the video. The technical solution of the present invention is simple, easy to implement, low cost, and easy to promote. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of a method for accurately identifying and marking RDP video text according to an embodiment of the present invention.

[0014] Figure 2 It is a schematic diagram of the output result of the method for accurately identifying and marking RDP video text according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] This embodiment takes a method for accurately identifying and marking RDP video text as an example, and the present invention will be described in detail below in conjunction with specific embodiments and drawings.

[0016] See also Figure 1 , showing a method for accurately identifying and marking RDP video text provided by an embodiment of the present invention, the specific steps include: Step S10, using Apache Guacamole to open the RDP session and generate the RDP video rdp.guac.

[0017] Step S20, using the Apache Guacenc tool, converting the video rdp.guac into the MP4 format rdp.mp4.

[0018] Use the guacenc tool to convert the RDP video into a standard MP4 video.

[0019] Specifically, the specific instructions for video format conversion are: guacenc [-s width x height] [-r bitrate] [-f] [file] For example: guacenc -s 1024x768 -r 20000 -f rdp.guac , The rdp.guac video is converted to rdp.mp4.

[0020] Step S30, using FFmpeg tool, extracting key frame images and the timestamp of each key frame from the MP4 video file rdp.mp4.

[0021] Step S31, using ffmpeg tool, extract key frame images from the MP4 video and record the exact time of the image in the video.

[0022] Specifically, the key frame image is an I frame (Intra-frame) in the video frame, which is an independently encoded complete image frame in the video sequence and does not rely on other frames for decoding. When decoding a video, the decoder can independently display the frame by decoding a key frame without any other frame information. Therefore, key frames can be used to quickly and randomly access the video, such as dragging the progress bar to any time point or quickly jumping to a part of the video.

[0023] The specific instructions for using the ffmpeg tool to extract key frame images are as follows: ffmpeg -i rdp.m4v -vf "select='eq(pict_type,PICT_TYPE_I)',showinfo" -vsync vfr -q:v 2 -f image2 keyframes_%05d.png>&keyframes_timestamps.txt; in, -i rdp.mp4: is the input video file; -vf select='eq(pict_type,PICT_TYPE_I)': select and output only key frames; -vsync vfr: output images according to the input frame rate; -q:v 2: is the quality of the output image; the image quality ranges from 1 to 31 from low to high; -f image2: is the output image format; keyframes_%05d.png: is the name of the output file, where %05d will be replaced by the keyframe number; showinfo: outputs detailed information of each key frame, including sequence number, timestamp, etc.

[0024] The names of the generated keyframe images are: keyframes_00001.png, keyframes_00002.png, keyframes_00003.png...

[0025] For example, in the keyframes_timestamps.txt file, there is the following: [Parsed_showinfo_1 @ 0x15c809ce0] n: 20 pts: 100352 pts_time:7.84duration: 512 duration_time:0.04 fmt:yuv420p cl:left sar:1 / 1 s:640x480 i:Piskey:1 type:I checksum:EBAAA7FA plane_checksum:[F801994B FBA10F60 932CFF40]mean:[64 128 128] stdev:[89.4 2.1 1.9] Among them, n:20 indicates the 21st key frame, and the corresponding image is keyframes_00021.png. pts_time is the timestamp, indicating that the time point when this key frame image appears in the rdp.mp4 video is 7.84 seconds.

[0026] Step S32, recording the exact time of the key frame image in the video.

[0027] Step S40, using the PaddleOCR tool, extract the text and coordinates appearing in the key frame image, and save them together with the key frame timestamp into the database.

[0028] Step S41, recognizing Chinese characters in the key frame image by using the PaddleOCR tool; The specific steps of recognizing characters in key frame images include: paddleocr --image_dir rdp.jpg --use_angle_cls true --lang zh in, -image_dir rdp.jpg: is the file name of the key frame image; -use_angle_cls true: can recognize text rotated 180 degrees; -lang zh: The language type is Chinese.

[0029] For example, the command output is as follows: [[[442.0, 173.0], [1169.0, 173.0], [1169.0, 225.0], [442.0, 225.0]],['China', 0.99283075]] The above command extracts text from the image rdp.jpg and extracts the word "China" with an accuracy of 0.99283075. The coordinates in the image are [[442.0, 173.0], [1169.0, 173.0], [1169.0, 225.0], [442.0, 225.0]].

[0030] Step S42, record the time and location of the text appearance, and generate a database file.

[0031] PaddleOCR can recognize text in images with an accuracy rate of over 97%. It recognizes the position of text in the image and records the time when the text appears (i.e. the time of the key frame image).

[0032] Step S50, when using a video player to play rdp.mp4, search the database according to the keyword, find out the timestamp and coordinates where the keyword appears, call the seek() function of the player, jump to the specified timestamp to play, and mark the keyword with a red border on the screen.

[0033] Step S51, click on the text in the database file to query the database for the timestamp and coordinates of the text appearing in the video; Step S52, calling the seek() function of the player to start playing from the specified timestamp; Step S53, marking the text with a red box on the screen according to the coordinates of the text.

[0034] Specifically, in this embodiment, the search results of the recorded video are as follows: Figure 2As shown, the right window of the panel displays the retrieved text content, timestamp and coordinates in the database file, and the left window is the video playback area. When you click the text in the command retrieval area in the right window, the database search is triggered. According to the search results, the player's seek() function starts playing and positioning the video from the specified timestamp. In the video playback area of ​​the left window, the corresponding text is marked with a red frame based on the coordinate position in the database file.

[0035] In the above method for accurately identifying and marking RDP video text, the RDP video format is first converted to MP4 video. The converted MP4 video can be played smoothly using a variety of playback software, and then the key frame image in the MP4 video is extracted, and then PaddleOCR is used to identify the text in the key frame image, and the time and position of the text in the key frame are output, and the retrieved text is located, and the video is located based on the text. In this technical solution, the obtained MP4 format video file is easier to view, and the video can also be located according to the recognized text, which is convenient for auditors to audit the video. The technical solution of the present invention is simple, easy to implement, low cost, and easy to promote.

[0036] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for accurately identifying and marking RDP video text, used for text extraction in bastion host video audit, characterized in that: The specific steps include: Convert RDP video format to MP4 video format; Extract key frame images from MP4 videos; Use PaddleOCR to recognize text in keyframe images; Search for text to find when and where it appears in the video.

2. The method for accurately identifying and marking RDP video text as claimed in claim 1, characterized in that: The specific steps of converting the RDP video format to the MP4 video format include: Use the Apache Guacenc tool to open an RDP session. After the session ends, an RDP recording will be generated. Use the guacenc tool to convert RDP videos into standard MP4 videos.

3. The method for accurately identifying and marking RDP video text as claimed in claim 1, characterized in that: The specific steps of extracting the key frame image in the MP4 video include: Use ffmpeg to extract key frame images from MP4 video. Record the exact time of key frame images in the video.

4. The method for accurately identifying and marking RDP video text as claimed in claim 3, characterized in that: The specific steps of using the ffmpeg tool to extract key frame images from MP4 video include: The specific instructions for using the ffmpeg tool to extract key frame images are as follows: ffmpeg -i rdp.m4v -vf "select='eq(pict_type,PICT_TYPE_I)',showinfo" -vsync vfr -q:v 2 -f image2 keyframes_%05d.png >& keyframes_timestamps.txt; in, -i rdp.mp4: is the input video file; -vf select='eq(pict_type,PICT_TYPE_I)': select and output only key frames; -vsync vfr: output images according to the input frame rate; -q:v 2: is the quality of the output image; the image quality ranges from 1 to 31 from low to high; -f image2: is the output image format; keyframes_%05d.png: is the name of the output file, where %05d will be replaced by the keyframe number; showinfo: Output detailed information for each keyframe.

5. The method for accurately identifying and marking RDP video text as claimed in claim 1, characterized in that: The specific steps of using PaddleOCR to recognize text in a key frame image include: Use PaddleOCR tool to recognize Chinese characters in keyframe images; Record the time and location of text appearance and generate a database file.

6. The method for accurately identifying and marking RDP video text as claimed in claim 5, characterized in that: The specific steps of recognizing characters in key frame images by using the PaddleOCR tool include: paddleocr --image_dir rdp.jpg --use_angle_cls true --lang zh in, -image_dir rdp.jpg: is the file name of the key frame image; -use_angle_cls true: can recognize text rotated 180 degrees; -lang zh: The language type is Chinese.

7. The method for accurately identifying and marking RDP video text as claimed in claim 5, characterized in that: The specific steps of retrieving text and finding the time and position of its appearance in the video include: Click on the text in the database file to query the database for the timestamp and coordinates of the text in the video; Call the player's seek() function to start playing from the specified timestamp; According to the coordinates of the text, mark the text with a red box on the screen.

Images