Computer bypass monitoring method and system based on HDMI (High Definition Multimedia Interface) connection

Abstract

The invention relates to the field of computer monitoring, and discloses a computer bypass monitoring method and system based on HDMI connection. The method comprises the following steps: monitoring a signal of an HDMI (High-Definition Multimedia Interface) which is used for connecting monitoring equipment and a confidential computer needing to be monitored by the monitoring equipment, starting a screen recording of the confidential computer and video shooting of a confidential computer operator to obtain screen recording data and shooting data when the HDMI signal disappears, and stopping the current screen recording and shooting operation when the HDMI signal disappears; acquiring camera data in real time, judging whether the camera data is abnormal or not, and if so, writing a wrong EDID value; and reading the EDID value of the current confidential computer in real time, and when the detected EDID value is wrong, executing a screen locking operation on the confidential computer. According to the method, the current equipment EDID value is read in real time, and when the correct value is not detected, the screen locking operation is executed, so that the technical problems that local storage is easily tampered, and when a local monitoring system is broken, all local monitoring strategies and monitoring modes are invalid can be solved.

Description

Technical Field

[0001] The present invention relates to the field of logistics, and in particular to a computer bypass monitoring method and system based on HDMI connection. Background Art

[0002] In scenarios involving the use of a large number of confidential computers, these confidential computers are mostly required to run as a single machine (not connected to the Internet, including the internal LAN), and during use, the content of the operation and the operators are required to be recorded and archived in detail.

[0003] In the prior art, the security policy is sent from the server to the client by configuring the security policy on the server and installing the security monitoring software on the client. When the server establishes a connection with the client, it can realize monitoring functions such as monitoring the screen and recording the operation. Some monitoring software can call the camera and other peripherals of the controlled computer in real time to capture data.

[0004] The above computer monitoring system relies on a network connection server or a computer local policy to monitor the computer, which has two disadvantages: 1. When leaving the network environment, real-time operations cannot be saved online, and all recorded videos are stored locally on the computer, which is easy to be tampered with.

[0005] 2. When the local monitoring system is hacked, all local monitoring strategies and monitoring methods become invalid, and the safety of use cannot be guaranteed. There are even unsafe risks such as system attacks due to loopholes in the monitoring system itself. Summary of the invention

[0006] The main purpose of the present invention is to solve the technical problem in the prior art that local storage is easily tampered with and all local monitoring strategies and monitoring modes become invalid when the local monitoring system is hacked. A computer bypass monitoring method based on HDMI connection includes the following steps: Monitor the HDMI signal, where the HDMI is used to connect the monitoring device and the confidential computer to be monitored by the monitoring device. When there is a monitoring HDMI signal, start the screen recording of the confidential computer and the video shooting of the operator of the confidential computer to obtain the screen recording data and the video data. When the HDMI signal disappears, stop the current screen recording and shooting operations; Acquire camera data in real time, determine whether the camera data is abnormal, and write an erroneous EDID value if the data is abnormal; Read the current EDID value of the confidential computer in real time. When the detected EDID value is wrong, the screen of the confidential computer will be locked; if the EDID value is normal, a storage file for monitoring will be generated.

[0007] As a preferred technical solution, the real-time acquisition of video data and determination of whether the video data is abnormal include the following steps: real-time acquisition of video data; extracting 2 frames of video images per second from the real-time acquired video data; performing human figure detection on the video images, and if no human figure is found for more than 10 seconds, determining that the data is abnormal.

[0008] As a preferred technical solution, the real-time acquisition of video data and determination of whether the video data is abnormal also include the following steps: real-time detection of the connection status between the camera and the monitoring device or the confidential computer; if the camera is offline, it is determined to be abnormal.

[0009] As a preferred technical solution, the real-time acquisition of camera data and the determination of whether the camera data is abnormal also include the following steps: configuring a face database for face recognition; acquiring camera data in real time; extracting 2 frames of camera images per second from the camera data acquired in real time; performing face detection on the camera images according to the face database, and if no human figure is found for more than 10 seconds or the image does not match the face database, it is determined to be abnormal; if a name that matches the face database is found, the name is recorded in the file name of the storage file.

[0010] As a preferred technical solution, the storage file includes a screen recording video file of a confidential computer, a video file shot by a camera, and a playback batch file. The playback batch file is used to play the screen recording video file of the confidential computer and the video file shot by the camera side by side on the left and right sides of the screen. The file name of the storage file consists of year, month, day, start time, end time, and source. The source is screen recording, shooting, and merging, which correspond to the screen recording video file of the confidential computer, the video file shot by the camera, and the playback batch file, respectively.

[0011] As a preferred technical solution, the file name of the storage file also includes a person's name. If the file name is longer than 26 characters, the generated storage file also includes a text file, and the text file is used to record the person's name.

[0012] As a preferred technical solution, the screen recording data also includes steganographic information that is invisible to the naked eye, and the steganographic information can be restored to its content through an algorithm.

[0013] Steganography - information hiding technology can be applied to the tracing of images. When a segment or an image is stolen through direct recording, copying, printing and other technical means, the original source of the image information can be obtained through the hidden information added to the image that is invisible to the naked eye and cannot be restored without a corresponding decryption method.

[0014] The overall implementation process can be divided into: image digitization - adding steganographic information - restoring the image - image transmission - obtaining the image - restoring the steganographic information through the decryption algorithm. In this process, the following points should be paid attention to: Robustness: that is, the ability to resist some operations that may cause content deformation, such as compression, cropping, rotation, brightness, printing, etc.

[0015] In order to increase the robustness of steganography, Hidden image steganography is adopted. A noise layer is added during the network training process to simulate the noise attack and compression encountered in the transmission of secret images in real scenarios. The attacked image is put into the decoding network to extract the secret information. The network takes into account the authenticity of the secret image, the accuracy of secret information extraction, and the concealment of the steganography, which further enhances the robustness of the steganography. The present invention's solution: Compared with the pixel domain, the frequency domain, especially the high frequency domain, is more suitable for image hiding. We use wavelet transform to divide the image into low- and high-frequency wavelet sub-bands before entering the reversible module, so that the network can better integrate the secret information into the cover image. Invisibility: that is, the steganographic information cannot be discovered or read through conventional means such as observation.

[0016] The present invention addresses this problem by adopting a framework HiNet based on an invertible neural network (INN). In order to improve invisibility, it is proposed to hide secret information in the wavelet domain instead of hiding it in the pixel domain.

[0017] Security: The missed detection error rate Pmd and the false alarm error rate Pfa are two basic indicators. The steganography analyzer uses the carrier X and the secret image Y set to train the classifier F(obj). Assuming that the carrier and the secret image have the same Bayesian prior probability, the average error rate Pe on the test set is used to measure the security of the steganography scheme against the steganography analyzer, where Pe is the minimum total error rate.

[0018] The response of the present invention: HiNet, a framework based on an invertible neural network (INN), adopts a new low-frequency wavelet loss to constrain the secret information to be hidden in the high-frequency wavelet sub-band, which greatly improves the security of hiding.

[0019] Capacity: Steganographic capacity. Steganographic capacity refers to the amount of information that can be carried by each embedded position in the image carrier on average.

[0020] The present invention responds to this problem by adopting a method for generating secret images based on deep learning. Compared with other image steganography methods, the steganography content is richer and the steganography capacity is larger. The HayesGAN framework is adopted to use the coding network to realize hiding secret information in images. The secret information and the carrier image are input into the coding network together to obtain the secret image; the steganography analysis network, which is also equivalent to the discriminant network, analyzes and detects the generated secret image and the original carrier image; the receiver of the information can obtain the decrypted information through the decoding network.

[0021] A second aspect of the present invention provides a computer bypass monitoring system based on HDMI connection, comprising: Monitoring equipment, HDMI, confidential computer, camera; the monitoring equipment is connected to the confidential computer that the monitoring equipment needs to monitor via HDMI, and the camera is connected to the monitoring equipment; the monitoring equipment monitors the HDMI signal in real time, and when there is a monitoring HDMI signal, starts the screen recording of the confidential computer and the video shooting of the confidential computer operator, obtains the screen recording data and the camera data, and when the HDMI signal disappears, stops the current screen recording and shooting operations; the computer bypass monitoring system also includes: an abnormality judgment unit, the abnormality judgment unit is used to obtain the camera data in real time, judge whether the camera data is abnormal, and write the wrong EDID value if it is abnormal; an execution unit, the execution unit is used to read the current EDID value of the confidential computer in real time, and when an EDID value error is detected, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

[0022] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned computer bypass monitoring method based on HDMI connection as described above.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned computer bypass monitoring method based on HDMI connection.

[0024] The present invention has the following beneficial effects: 1. The monitoring device and the monitored computer are independent of each other, there is no file transfer, and the possibility of file transfer is cut off from the architecture.

[0025] 2. All monitoring data is encrypted and stored on the monitoring device. The video cannot be played without a specific decryption player.

[0026] 3. The monitored computer must be connected to a monitoring device with a specific hardware number through an HDMI interface. Once the monitoring device is removed or damaged, the monitored device will automatically lock the screen.

[0027] 4. When using a maintenance computer to download videos stored on the monitoring device, there is no need to install a client on the maintenance computer and download through a browser (the maintenance computer will be restored to factory default settings from time to time, so the software cannot be saved). An access password is required and can be modified.

[0028] 5. The monitoring software loaded on the monitored computer reads the current device EDID value in real time. If the correct value cannot be detected, the screen lock operation is executed. This monitoring software process is a protected process and cannot be stopped. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a topological architecture diagram of the existing computer monitoring system; Figure 2 A topological architecture diagram of the computer bypass monitoring system of the present invention; Figure 3 A first flow chart of a computer bypass monitoring method based on HDMI connection provided by an embodiment of the present invention; Figure 4 A second flow chart of a computer bypass monitoring method based on HDMI connection provided by an embodiment of the present invention; Figure 5 A third flow chart of a computer bypass monitoring method based on HDMI connection provided by an embodiment of the present invention; Figure 6 A fourth flow chart of a computer bypass monitoring method based on HDMI connection provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The embodiment of the present invention provides a computer bypass monitoring method and system based on HDMI connection. The method includes: monitoring the signal of the HDMI, the HDMI is used to connect the monitoring device and the confidential computer that the monitoring device needs to monitor, when there is a monitoring HDMI signal, starting the screen recording of the confidential computer and the video shooting of the confidential computer operator, obtaining the screen recording data and the camera data, when the HDMI signal disappears, then stopping the current screen recording and shooting operation; obtaining the camera data in real time, judging whether the camera data is abnormal, if abnormal, writing the wrong EDID value; reading the current EDID value of the confidential computer in real time, when the detected EDID value is wrong, performing the screen lock operation on the confidential computer; if the EDID value detection is normal, generating a storage file for monitoring. The monitoring software loaded on the monitored computer of the present invention reads the current device EDID value in real time, and when the correct value cannot be detected, performs the screen lock operation. This monitoring software process is a protected process and cannot be stopped.

[0031] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] EDID: EDID, Extended display identification data, is a standard for display identification data developed by VESA when developing the DDC display data channel communication protocol. EDID is stored in the DDC memory of the display. When the computer host is connected to the display, the computer host will read the EDID stored in the DDC memory of the display through the DDC channel.

[0033] OSS: OSS (Object Storage Service) object storage service is an object-based mass storage service that provides customers with massive, secure, highly reliable, and low-cost data storage capabilities.

[0034] Current state of the art: like Figure 1As shown in the figure, by configuring security policies on the server and installing security monitoring software on the client, security policies are sent from the server to the client. When the server establishes a connection with the client, monitoring functions such as monitoring the screen and recording operations can be realized. Some monitoring software can call the camera and other peripherals of the controlled computer in real time to capture data.

[0035] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , a first embodiment of a computer bypass monitoring method based on HDMI connection in an embodiment of the present invention includes: like Figure 3 A first flow chart of a computer bypass monitoring method based on HDMI connection, a computer bypass monitoring method based on HDMI connection, comprising the following steps: 101. Monitor the HDMI signal, where the HDMI is used to connect the monitoring device and the confidential computer to be monitored by the monitoring device. When there is a monitoring HDMI signal, start the screen recording of the confidential computer and the video shooting of the confidential computer operator to obtain the screen recording data and the video data. When the HDMI signal disappears, stop the current screen recording and shooting operations. 102. Acquire the camera data in real time, determine whether the camera data is abnormal, and if abnormal, write an erroneous EDID value; 103. Read the current EDID value of the confidential computer in real time. When the detected EDID value is wrong, the screen of the confidential computer is locked; if the EDID value is detected normally, a storage file for monitoring is generated.

[0036] Specifically, when HDMI detects a signal, the monitoring device starts recording and shooting at the same time. When the HDMI signal disappears, the current recording operation is stopped. The monitoring software loaded on the monitored computer reads the current device EDID value in real time. When the correct value cannot be detected, the screen lock operation is executed. This monitoring software process is a protected process and cannot be stopped.

[0037] Detect lock screen: The monitoring software loaded on the monitored computer reads the current device EDID value in real time. If the correct value cannot be detected, the screen lock operation is executed. This monitoring software process is a protected process and cannot be stopped.

[0038] like Figure 4 The second flow chart of the computer bypass monitoring method based on HDMI connection is a preferred implementation mode: 201. Monitor the HDMI signal, where the HDMI is used to connect the monitoring device and the confidential computer to be monitored by the monitoring device. When there is a monitoring HDMI signal, start the screen recording of the confidential computer and the video shooting of the operator of the confidential computer to obtain the screen recording data and the video data. When the HDMI signal disappears, stop the current screen recording and shooting operations. 202. Real-time acquisition of camera data; extracting 2 frames of camera images per second from the real-time acquired camera data; performing human figure detection on the camera images, and if no human figure is found for more than 10 seconds, it is judged as abnormal. If abnormal, an incorrect EDID value is written; 203. Read the current EDID value of the confidential computer in real time. When the detected EDID value is wrong, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

[0039] Specifically, when the monitoring system starts recording, if the camera captures images showing that a person has left for more than 10 seconds (2 frames per second, i.e., no person in 20 frames); the camera is blocked, the algorithm model preset in the monitoring device is used to determine and infer the result, and the pre-recorded alarm audio is played through the camera's built-in speaker (played three times in a row for reminder). At the same time, an incorrect EDID value is written, so that the monitoring software detects the incorrect EDID value and executes the screen lock operation.

[0040] like Figure 5 A third flow chart of a computer bypass monitoring method based on HDMI connection is provided as a preferred implementation: 301. Monitor the HDMI signal, where the HDMI is used to connect the monitoring device and the confidential computer to be monitored by the monitoring device. When there is a monitoring HDMI signal, start the screen recording of the confidential computer and the video shooting of the confidential computer operator to obtain the screen recording data and the video data. When the HDMI signal disappears, stop the current screen recording and shooting operations. 302. Real-time detection of the connection status between the camera and the monitoring device or the confidential computer; if the camera is offline, it is judged as abnormal, and if abnormal, the wrong EDID value is written; 303. Read the current EDID value of the confidential computer in real time. When the detected EDID value is wrong, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

[0041] Specifically, the monitoring system detects the connection status of the USB camera in real time. When the USB camera is unplugged (the camera is offline), the pre-recorded alarm audio is played through the camera's built-in speaker (played three times in a row for reminder). At the same time, the wrong EDID value is written, so that the monitoring software detects the wrong EDID value and executes the screen lock operation.

[0042] like Figure 6 A fourth flow chart of a computer bypass monitoring method based on HDMI connection is provided as a preferred implementation manner: 401. Monitor the HDMI signal, where the HDMI is used to connect the monitoring device and the confidential computer to be monitored by the monitoring device. When there is a monitoring HDMI signal, start the screen recording of the confidential computer and the video shooting of the confidential computer operator to obtain the screen recording data and the video data. When the HDMI signal disappears, stop the current screen recording and shooting operations. 402. Configure a face database for face recognition; obtain camera data in real time; extract 2 frames of camera images per second from the camera data obtained in real time; perform face detection on the camera images according to the face database, and if no human figure is found for more than 10 seconds or the image does not match the face database, it is judged as abnormal; if a name matching the face database is found, the name is recorded in the file name of the storage file, and if it is abnormal, an incorrect EDID value is written; 403. Read the EDID value of the current confidential computer in real time. When the detected EDID value is wrong, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

[0043] Specifically, a face recognition system is added to the monitoring equipment. The face system can be configured through a maintenance computer and the web (the face database can be added, deleted, modified and checked). When the monitoring system starts recording, the images captured by the camera are used to record the corresponding names in the database in real time (the same name only appears once in a video). When a person who is not found in the database appears and it lasts for more than 10 seconds (2 frames are taken per second, that is, 20 frames of unknown person), the name that appears is displayed as "Unknown Person".

[0044] As a preferred implementation, the storage file includes a screen recording video file of a confidential computer, a video file shot by a camera, and a playback batch file. The playback batch file is used to play the screen recording video file of the confidential computer and the video file shot by the camera side by side on the left and right sides of the screen. The file name of the storage file consists of year, month, day, start time, end time, and source. The source is screen recording, shooting, and merging, which correspond to the screen recording video file of the confidential computer, the video file shot by the camera, and the playback batch file, respectively.

[0045] Specifically, for each recording, the system automatically generates three files, namely: screen recording video file (encrypted), shooting video file (encrypted), and merged playback batch file. (The merging principle is: merging if the starting time is consistent) The file name format is: year month day_start time_end time_source.XXX Here are some examples: Screen recording file: 20230808_1401_1431_screen recording.FLV Shooting file: 20230808_1401_1431_shooting.FLV Batch file: 20230808_1401_1431_merge.BAT Video Management: The stored videos and batch files are stored in a directory structure based on year, month, and day. The videos starting on the current day are stored in the directory of that day.

[0046] Directory structure example: D:\2023\08\29 The storage abandonment principle is: after exceeding the set storage capacity, the oldest video files are deleted according to the first-in-first-out principle.

[0047] When video backup is required, install the OSS service on the monitoring device, connect the monitoring device with the network interface through the maintenance notebook, enter the username and password using the browser, download the files stored on the monitoring device, and modify the password.

[0048] When you need to play the video for review, download the video-related data to the maintenance notebook, then double-click the merged batch file to play it side by side on the left and right sides of the screen, and you can adjust the window size, progress bar control, play, pause, close single window and other common operations.

[0049] As a preferred technical solution, the file name of the storage file also includes a person's name. If the file name is longer than 26 characters, the generated storage file also includes a text file, and the text file is used to record the person's name.

[0050] Specifically, the names of people that appear are reflected in the recorded file name. When too many people appear in the same video, resulting in a file name length exceeding 26 characters, a text file is generated to record the names (this text file is not generated when the file name is less than 26 characters) Example file name: Screen recording file: 20230808_1401_1431_Screen recording_Zhang San & Li Si & Wang Wu & unknown person.FLV Shooting file: 20230808_1401_1431_shooting_Zhang San & Li Si & Wang Wu & unknown person.FLV Batch file: 20230808_1401_1431_Merge_Zhang San & Li Si & Wang Wu & Unknown Person.BAT Text file: 20230808_1401_1431_personnel.TXT As a preferred technical solution, the screen recording data also includes steganographic information that is invisible to the naked eye, and the steganographic information can be restored to its content through an algorithm.

[0051] Specifically, steganographic information that is invisible to the naked eye is added to the recorded video. When a video is leaked (including file leakage, photography, copying, etc.), the content of the steganographic information can be restored through an algorithm (generally including the recording device and recording time).

[0052] The above describes the computer bypass monitoring method based on HDMI connection in the embodiment of the present invention. The following describes the computer bypass monitoring device based on HDMI connection in the embodiment of the present invention. Figure 2 , a first embodiment of a computer bypass monitoring device based on HDMI connection in an embodiment of the present invention includes: Monitoring equipment, HDMI, confidential computer, camera; the monitoring equipment is connected to the confidential computer that the monitoring equipment needs to monitor via HDMI, and the camera is connected to the monitoring equipment; the monitoring equipment monitors the HDMI signal in real time, and when there is a monitoring HDMI signal, starts the screen recording of the confidential computer and the video shooting of the confidential computer operator, obtains the screen recording data and the camera data, and when the HDMI signal disappears, stops the current screen recording and shooting operations; the computer bypass monitoring system also includes: an abnormality judgment unit, the abnormality judgment unit is used to obtain the camera data in real time, judge whether the camera data is abnormal, and write the wrong EDID value if it is abnormal; an execution unit, the execution unit is used to read the current EDID value of the confidential computer in real time, and when an EDID value error is detected, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

[0053] Specifically, it includes: a. Status monitoring module: Real-time monitoring of EDID information of devices obtained through HDMI connection; An abnormality judgment unit, the abnormality judgment unit is used to obtain camera data in real time, judge whether the camera data is abnormal, and write an erroneous EDID value if it is abnormal; an execution unit, the execution unit is used to read the EDID value of the current confidential computer in real time, when the detected EDID value is wrong, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

[0054] Real-time monitoring of camera status; Specifically, it includes a camera abnormality judgment unit, which is used to detect the connection status between the camera and the monitoring equipment or the confidential computer in real time; if the camera is offline, it is judged as abnormal.

[0055] The monitoring system detects the connection status of the USB camera in real time. When the USB camera is unplugged (the camera is offline), the pre-recorded alarm audio is played through the camera's built-in speaker (played three times in a row as a reminder). At the same time, the wrong EDID value is written, so that the monitoring software detects the wrong EDID value and executes the screen lock operation.

[0056] b. Logical rules module: Logical configuration for judging various state combinations The logic configuration here includes setting the judgment logic of the camera status, etc.

[0057] Alarm push The present invention plays the pre-recorded warning audio through the built-in speaker of the camera (plays it three times continuously for reminder).

[0058] c. Video module Video recording, including: Video capture device: This can be a hardware device such as a camera or video capture card that is used to obtain video signals from the physical world (such as live images captured by a camera) or from a digital signal source (such as a computer screen).

[0059] Video encoder: The encoder is responsible for converting analog video signals into digital formats, usually using compression algorithms to reduce storage space and network transmission bandwidth requirements. Common video encoding formats include H.264, H.265 (HEVC), and AV1.

[0060] Recording software: The recording software controls the video capture device and encoder, is responsible for starting and stopping recording, managing the storage of recorded files, and providing a user interface for user operation.

[0061] Video storage: Recorded videos are usually saved on a computer's hard drive, solid-state drive, or other storage media in the form of files such as mp4, avi, mkv, etc.

[0062] Video encryption, including: Encryption Algorithm Selection: Select an encryption algorithm, such as AES (Advanced Encryption Standard), RSA or other encryption algorithms, to encrypt video data.

[0063] Key management: Generates the keys required for encryption and ensures the secure storage and transmission of the keys. Key management usually includes the processes of key generation, storage, distribution, and destruction.

[0064] Encryption process: Use the selected encryption algorithm and key to encrypt the video file to generate an encrypted video file. The encrypted file cannot be directly accessed and played by unauthorized users.

[0065] Decryption process: Only users with the correct key and corresponding decryption algorithm can decrypt and play the video. The decryption process usually occurs in the decryption module of the video player.

[0066] Secret writing information, including: Information embedding: During the video encoding process, the cryptanalyst information that needs to be hidden (such as timestamps, device IDs, etc.) is embedded into the video data in an invisible way. The embedding method can be to modify pixel values, audio samples, or video metadata.

[0067] Information extraction: When necessary, specialized software or algorithms can be used to extract embedded steganographic information from video files. The extraction process usually needs to correspond to the embedding process to ensure that the information can be correctly extracted.

[0068] Information protection: The steganographic information should be robust enough to be extracted after the video is compressed, converted or edited. At the same time, the steganographic information should be stored in encrypted form to prevent unauthorized access.

[0069] Customized video player (decryption, dual-window side-by-side playback), including: Decryption function: The player has a built-in decryption module that can identify encrypted video files and use the corresponding decryption algorithm and key to decrypt the video for normal playback. The decryption process should be carried out with user authorization and legality.

[0070] Dual-window side-by-side playback: The player supports opening two video windows at the same time and allows users to display the two windows side by side. In this way, users can easily compare, contrast or watch two different video contents at the same time.

[0071] User interface customization: The player can customize its interface according to user needs, including window layout, control style, subtitle display, etc. The customized interface should conform to the user's usage habits and aesthetic needs.

[0072] Other functions: In addition to the basic playback function, the customized player can also add other functions according to needs, such as video screenshot, recording, subtitle editing, playlist management, etc. These functions can enhance the user experience and convenience.

[0073] d. File processing module Store video files, batch files, and text files, and provide OSS download services.

[0074] Specifically, the storage files of the present invention include screen recording video files of confidential computers, video files shot by cameras, and playback batch files. The playback batch files are used to play the screen recording video files of confidential computers and video files shot by cameras side by side on the left and right sides of the screen. The file name of the storage file consists of year, month, day, start time, end time, and source. The sources are screen recording, shooting, and merging, which correspond to the screen recording video files of confidential computers, video files shot by cameras, and playback batch files, respectively.

[0075] e. Algorithm reasoning module Face Recognition The face recognition of the present invention adopts a common algorithm, specifically, it can be the following algorithm: Eigenface: This method converts a set of face images into a set of feature vectors, or "Eigenfaces", which are the basic components of the original training image set. New images are projected into these feature subspaces and judged and recognized by their position in the subspace and the length of the projection line.

[0076] Local Binary Patterns (LBP): LBP is a visual operator used for classification in computer vision, which mainly extracts local features as the basis for discrimination.

[0077] Fisherface algorithm: This is also a face recognition algorithm.

[0078] Based on the singular value feature method: The singular value features of the face image matrix reflect the essential attributes of the image, so these features can be used for classification and recognition.

[0079] Subspace analysis method: Due to its strong descriptiveness, low computational cost, easy implementation and good separability, it is widely used in facial feature extraction and has become one of the mainstream methods of current face recognition.

[0080] Locality Preserving Projections (LPP): This is a new subspace analysis method that solves the shortcomings of traditional linear methods (such as PCA) that it is difficult to maintain the nonlinear manifold of the original data, and also solves the shortcomings of nonlinear methods that it is difficult to obtain low-dimensional projections of new sample points.

[0081] Principal Component Analysis (PCA): PCA is an important method in the field of pattern recognition and has been widely used in face recognition algorithms. However, face recognition systems based on PCA face recognition systems face an important obstacle in their applications, namely the incremental learning problem.

[0082] Humanoid Detection The human figure detection of the present invention adopts a common algorithm, specifically, the following algorithm: Methods based on traditional image processing and machine learning: Background Subtraction: A common approach is to use background subtraction for human detection. First, the system learns and models the background, and then compares the current frame with the background model to detect moving objects in the foreground, i.e., human figures.

[0083] Haar feature + cascade classifier: Haar feature is a feature descriptor in the field of computer vision and machine learning, and is often used for target detection. Combined with a cascade classifier (such as AdaBoost), it can effectively detect human figures in complex backgrounds.

[0084] HOG feature + SVM: Histogram of Oriented Gradients (HOG) is another commonly used feature descriptor, especially suitable for describing the shape of objects. Combined with the Support Vector Machine (SVM) classifier, human detection can be achieved.

[0085] Deep learning based methods: Convolutional Neural Network (CNN): In recent years, deep learning, especially convolutional neural network, has made remarkable progress in object detection tasks. Various CNN-based object detection algorithms, such as Faster R-CNN, YOLO (You Only LookOnce) and SSD (Single Shot MultiBox Detector), can be used for human detection.

[0086] Deep learning model transfer: Since deep learning models require a large amount of labeled data for training, some researchers use pre-trained models (such as models trained on ImageNet) for transfer learning to adapt to human detection tasks.

[0087] Human pose estimation: Some more advanced methods try to estimate the pose of the human body, for example using tools like OpenPose, which not only detects the human shape but also provides information about the position and pose of various parts of the human body (such as head, hands, feet, etc.).

[0088] Occlusion Detection The occlusion detection of the present invention adopts a common algorithm, specifically, the following algorithm: Occlusion detection based on geometric relationships: Z-Buffer algorithm: The occlusion situation is determined based on the distance of the ground object point on the projection light. This algorithm records the Z-Buffer matrix and visibility matrix corresponding to each image point on the original image, and determines the occlusion relationship by comparing the Z values ​​of different objects.

[0089] Contour analysis: By analyzing the contour information of an object, it is possible to determine whether the object is occluded. For example, if the contour of an object is incomplete or broken in the image, it may indicate that the object is occluded.

[0090] Occlusion detection based on optical flow field: Optical flow analysis: Optical flow is the movement pattern of pixels or feature points in an image between consecutive frames. By analyzing the optical flow field, the movement of objects can be detected to determine whether the objects are occluded. If the optical flow of a certain area suddenly changes or disappears, it may mean that the area is occluded.

[0091] Texture and color based occlusion detection: Texture analysis: By analyzing the texture information of an object, it is possible to determine whether the object is occluded. For example, if the texture of a certain area suddenly changes or disappears, it may mean that the area is occluded.

[0092] Color consistency: Analyzes the color consistency of an object. If the color of an area is significantly different from the surrounding area, it may indicate that the area is occluded.

[0093] The hardware components are as follows: a. Bypass monitoring device host: Equipped with an octa-core CPU of quad-core A76 + quad-core A55 and ARM G610MP4 GPU, built-in NPU with 6 TOPs computing power, 3TSATA hard drive, 1 x network port, 1 x USB port, 1 x HDMI In, 1 x HDMI Out.

[0094] b. Bypass monitoring device camera: Linux driver-free; Support output RGB24 format images; Not less than 2 million pixels; Support UVC format.

[0095] Figure 77 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.

[0096] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0097] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of a computer bypass monitoring method based on an HDMI connection.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0100] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computer bypass monitoring method based on HDMI connection, characterized in that: The following steps are involved: Monitor the HDMI signal, where the HDMI is used to connect the monitoring device and the confidential computer to be monitored by the monitoring device. When there is a monitoring HDMI signal, start the screen recording of the confidential computer and the video shooting of the operator of the confidential computer to obtain the screen recording data and the video data. When the HDMI signal disappears, stop the current screen recording and shooting operations; Acquire camera data in real time, determine whether the camera data is abnormal, and write an erroneous EDID value if the data is abnormal; Read the current EDID value of the confidential computer in real time. When the detected EDID value is wrong, the screen of the confidential computer will be locked; if the EDID value is normal, a storage file for monitoring will be generated.

2. A computer bypass monitoring method based on HDMI connection according to claim 1, characterized in that: The real-time acquisition of video data and determination of whether the video data is abnormal include the following steps: Acquire camera data in real time; Extract 2 frames of video images per second from the video data acquired in real time; The camera image is subjected to human figure detection, and if no human figure is found for more than 10 seconds, it is determined to be abnormal.

3. The computer bypass monitoring method based on HDMI connection according to claim 1, characterized in that: The real-time acquisition of the video data and the determination of whether the video data is abnormal further include the following steps: Real-time detection of the connection status between the camera and the monitoring equipment or the confidential computer; If the camera is offline, it is considered abnormal.

4. The computer bypass monitoring method based on HDMI connection according to claim 1, characterized in that: The real-time acquisition of the video data and the determination of whether the video data is abnormal further include the following steps: Configure the face database for face recognition; Acquire camera data in real time; Extract 2 frames of video images per second from the video data acquired in real time; Perform face detection on the camera image according to the face database. If no human figure is found for more than 10 seconds or the image does not match the face database, it is judged as abnormal. If a name that matches the face database is found, the name is recorded in the file name of the storage file.

5. The computer bypass monitoring method based on HDMI connection according to claim 4, characterized in that: The storage files include screen recording video files of confidential computers, video files shot by cameras, and playback batch files. The playback batch files are used to play the screen recording video files of confidential computers and video files shot by cameras side by side on the left and right sides of the screen. The file name of the storage file consists of year, month, day, start time, end time, and source. The sources are screen recording, shooting, and merging, which correspond to the screen recording video files of confidential computers, video files shot by cameras, and playback batch files, respectively.

6. The computer bypass monitoring method based on HDMI connection according to claim 5, characterized in that: The file name of the storage file also includes a person's name. If the file name is longer than 26 characters, the generated storage file also includes a text file, and the text file is used to record the person's name.

7. The computer bypass monitoring method based on HDMI connection according to claim 1, characterized in that: The screen recording data also includes steganographic information that is invisible to the naked eye, and the steganographic information can be restored through an algorithm to obtain the content of the steganographic information.

8. A computer bypass monitoring system based on HDMI connection, characterized in that: The computer bypass monitoring system comprises: Monitoring equipment, HDMI, confidential computer, camera; the monitoring equipment is connected to the confidential computer that the monitoring equipment needs to monitor via HDMI, and the camera is connected to the monitoring equipment; the monitoring equipment monitors the HDMI signal in real time, and when there is a monitoring HDMI signal, starts the screen recording of the confidential computer and the video shooting of the confidential computer operator, obtains the screen recording data and the video data, and stops the current screen recording and shooting operations when the HDMI signal disappears; The computer bypass monitoring system also includes: An abnormality judgment unit, the abnormality judgment unit is used to obtain the camera data in real time, judge whether the camera data is abnormal, and write an erroneous EDID value if it is abnormal; The execution unit is used to read the EDID value of the current confidential computer in real time. When an EDID value error is detected, the screen lock operation is performed on the confidential computer; if the EDID value detection is normal, a storage file for monitoring is generated.

9. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the electronic device to execute the various steps of the computer bypass monitoring method based on HDMI connection as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the computer bypass monitoring method based on HDMI connection as described in any one of claims 1 to 7 are implemented.

Images