Network information acquisition system and evidence storage control method thereof, electronic equipment and storage medium

Abstract

The invention discloses a network information acquisition system and an evidence storage control method thereof, electronic equipment and a storage medium. The system comprises a wearable controller and an audio and video recording device connected with the wearable controller. The wearable controller is used for being worn on the body of an operator and generating an evidence storage control signal based on the action of the operator, and the evidence storage control signal comprises one or more inertial data; and the audio and video recording device is also connected with the audio and video output port of the network terminal, and is used for receiving the audio and video signal output by the network terminal, processing the audio and video signal based on the evidence storage control signal to obtain an audio and video material, and storing the audio and video material in a preset time period based on the time specified by the evidence storage control signal. And evidence obtaining is realized by storing selected audio and video materials, and treatment measures are taken in time based on discovered bad information, so that healthy development of network maintenance is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and more particularly to a network information collection system, a storage evidence control method thereof, an electronic device and a storage medium. BACKGROUND

[0002] The Internet is a great invention, which connects global massive local networks into a whole, so that users can obtain massive data and materials based on the Internet. However, the Internet also has some problems while bringing convenience and benefits to people, for example, there may be some bad information, so it is necessary to monitor and handle it in time to ensure the healthy development of the Internet. SUMMARY

[0003] Therefore, the present application provides a network information collection system, a storage evidence control method thereof, an electronic device and a storage medium, which are used for collecting evidence of bad information in real time based on the monitoring of monitoring personnel on the network to maintain the healthy development of the network.

[0004] In order to achieve the above purpose, the present application provides the following scheme:

[0005] A network information collection system comprises a wearable controller and an audio and video recording device connected with the wearable controller, wherein:

[0006] The wearable controller is used to be worn on the body of an operator, and generates a storage evidence control signal based on the action of the operator, the storage evidence control signal comprising one or more inertial data;

[0007] The audio and video recording device is also connected with an audio and video output port of a network terminal, used to receive the audio and video signal output by the network terminal, and process the audio and video signal based on the storage evidence control signal to obtain audio and video materials, save the audio and video materials in a preset time period based on the time specified by the storage evidence control signal, and generate storage evidence information based on the action information of the storage evidence control signal and the audio and video materials in the preset time period.

[0008] Optionally, the wearable controller comprises a glove and a plurality of IMU sensors arranged on the glove, the IMU sensors being provided with a wireless communication module for transmitting the inertial data in a wireless manner.

[0009] Optionally, the audio and video recording device comprises a box body and a recording host arranged inside the box body, wherein:

[0010] The outer wall of the box body is provided with one or more control buttons, a display screen, a signal input port, a power input port and a master control unit connected with the recording host respectively, the master control unit is wirelessly connected with the wearable controller, and is used for receiving the evidence storage control signal;

[0011] The recording host is used for obtaining the audio and video signals output by the network terminal based on the signal input port, processing the audio and video signals based on the evidence storage control signal to obtain audio and video materials, saving the audio and video materials in a preset time period based on the time specified by the evidence storage control signal, and generating evidence storage information based on the action information of the evidence storage control signal and the audio and video materials in the preset time period.

[0012] Optionally, the recording host is configured with an audio and video decoding module, a source tracing module, a time calibration module, an integrity verification module and a forensic explanation generation module, wherein:

[0013] The audio and video decoding module is used for decoding and processing the audio and video signals to obtain video stream signals and audio signals for playing;

[0014] The source tracing module is used for adding source information to the video signals, and the source information includes operator information and equipment information;

[0015] The time calibration module is used for calibrating the time information of the recording host based on a standard time;

[0016] The integrity verification module is used for generating integrity verification information of the recorded audio and video materials based on a hash algorithm;

[0017] The forensic explanation generation module is used for generating structured forensic explanations based on the recorded audio and video materials and operation logs.

[0018] An evidence storage control method applied to the network information collection system, and the evidence storage control method comprises the following steps:

[0019] In response to a device start request of a user, the audio and video recording device is controlled to start, and the time information of the audio and video recording device is calibrated based on a standard time signal;

[0020] The audio and video signals are decoded and processed to obtain video stream signals and audio signals;

[0021] When receiving an evidence storage control signal issued by an operator, the operation type is calculated based on the evidence storage control signal;

[0022] Based on the operation type, the video stream signal and / or the audio signal are intercepted and recorded to obtain one or more audio and video materials, and the audio and video materials include traceability information.

[0023] The audio and video materials are saved.

[0024] Optionally, when the evidence storage control signal issued by the operator is received, the operation type is calculated based on the evidence storage control signal, including the steps of:

[0025] The evidence storage control signal is preprocessed to obtain multi-point inertial data that is aligned and calibrated;

[0026] The multi-point inertial data is filtered based on an infinite impulse filter to eliminate noise and gravity components therein;

[0027] The filtered multi-point inertial data is segmented using a segmentation and normalization technique to obtain multiple independent gesture segments;

[0028] The gesture segments are classified according to threshold rules to obtain the operation type.

[0029] Optionally, the saving of the audio and video materials includes the steps of:

[0030] The audio and video materials are encoded to obtain multimedia evidence storage materials;

[0031] The multimedia evidence storage materials are packaged to obtain evidence storage files;

[0032] The integrity of the evidence storage files is verified;

[0033] If it is determined through verification that the evidence storage files are incomplete, return to the encoding of the audio and video materials;

[0034] If it is determined through verification that the evidence storage files are complete, save the evidence storage files.

[0035] Optionally, it further includes the steps of:

[0036] Based on the audio and video materials and operation logs, structured forensic descriptions are generated.

[0037] An electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0038] The memory is used to store computer programs or instructions;

[0039] The processor is used to execute the computer programs or instructions to enable the electronic device to implement the evidence storage control method as described above.

[0040] A computer-readable storage medium applied to an electronic device, the storage medium carrying one or more computer programs, the one or more computer programs being executable by the electronic device, thereby enabling the electronic device to implement the evidence storage control method as described above.

[0041] From the above technical solution, the present application discloses a network information collection system and its evidence storage control method, electronic device and storage medium, the system comprises a wearable controller and an audio and video recording device connected with the wearable controller. The wearable controller is used to be worn on the body of the operator, and generates an evidence storage control signal based on the action of the operator, the evidence storage control signal comprises one or more inertial data; the audio and video recording device is also connected with the audio and video output port of the network terminal, used to receive the audio and video signal output by the network terminal, and process the audio and video signal based on the evidence storage control signal, obtain the audio and video material, and save the audio and video material in the preset time period based on the time specified by the evidence storage control signal. By saving the selected audio and video material, evidence is obtained, and disposal measures are taken in time based on the discovered bad information, so as to realize the healthy development of network maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0043] Figure 1 A schematic diagram of a network information collection system according to an embodiment of the present application;

[0044] Figure 2 A front view of an audio and video recording device according to an embodiment of the present application;

[0045] Figure 3 A side view of an audio and video recording device according to an embodiment of the present application;

[0046] Figure 4 A flowchart of an evidence storage control method according to an embodiment of the present application;

[0047] Figure 5 A flowchart of another evidence storage control method according to an embodiment of the present application;

[0048] Figure 6 A block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0050] Figure 1 A schematic diagram of a network information collection system according to an embodiment of the present application.

[0051] As shown in Figure 1 , the network information collection system according to the present embodiment is used for collecting evidence or target information on a network, for example, for collecting evidence materials for litigation activities for network security monitoring personnel or for lawyers or parties. The network information collection system comprises a wearable controller and an audio and video recording device 200 connected to the wearable controller 100.

[0052] The wearable controller is used to be worn on the body of an operator, such as on the hand. When the operator makes a certain action, the wearable controller can generate an evidence storage control signal based on the action and send the evidence storage control signal to the audio and video recording device.

[0053] The wearable controller in the present application is a glove-type controller, which comprises a glove 101. A plurality of IMU sensors 102 are arranged on the glove. When the glove is worn on the hand of the operator, the IMU sensors will generate and output inertial data when the operator makes a hand action. Each IUM sensor has a wireless transmission module such as Bluetooth arranged on the top. The IMU sensor transmits the inertial data to the audio and video recording device through the wireless transmission module after generating the inertial data. In view of the above description, the evidence storage control signal comprises one or more inertial data. Here, the inertial data refers to the three-axis acceleration value and the three-axis angular velocity value.

[0054] The IMU sensor is a microelectronic device integrating an accelerometer and a gyroscope, which is used to measure and report the linear acceleration and angular velocity of an object in three-dimensional space. Its main input signals come from multi-axis accelerometers (measuring specific forces acting on the sensor) and multi-axis gyroscopes (measuring angular velocity). The output is inertial data, including three-axis acceleration value, three-axis angular velocity value, etc. These data often need to be processed by real-time filtering and fusion algorithms (such as Kalman filtering) to estimate attitude, velocity and position. With its self-contained characteristics, IMU is widely used in inertial navigation systems, unmanned aerial vehicle and robot attitude control, virtual reality headsets and motion tracking of smartphones, as well as automotive electronic stability systems and wearable fitness devices, to realize positioning compensation in indoor / tunnel environments, real-time attitude detection and accurate motion analysis.

[0055] The audio-video recording device is connected with an audio-video output port of a network terminal, such as an HDMI port, for receiving an audio-video signal output by the network output port, and processing the audio-video signal based on a storage control signal to obtain audio-video material, where the audio-video material refers to a video signal and an audio signal; and saving the audio-video material in a preset time period based on a time specified by the storage control signal, and processing the audio-video material in the preset time period based on action information of the storage control signal to obtain storage evidence information.

[0056] As shown in Figure 2 The audio-video recording device includes a box body 201 and a recording host arranged in the box body, where the recording host refers to an electronic device for realizing the recording function, and at least includes a circuit board, a plurality of electronic elements arranged on the circuit board, a power input port connected with the circuit board, an operation port, etc. One or more control buttons 202, a display screen, an indicator light and a master control unit are arranged on the box body. A signal input port and a power input port are further arranged on the side of the box body, as shown in Figure 3 The signal input port can be a USB port and an HDMI port, etc. The master control unit is wirelessly connected with a wearable controller for receiving a storage control signal.

[0057] The recording host is configured to obtain an audio-video signal based on the signal input port, process the audio-video signal based on the storage control signal to obtain audio-video material, save the audio-video material in a preset time period based on a time specified by the storage control signal, and generate storage evidence information based on the action information of the storage control signal and the audio-video material in the preset time period.

[0058] The recording host is configured with an audio-video decoding module, a source tracing module, a time correction module, an integrity verification module and a forensic explanation generation module. The modules can be hardware modules or functional modules based on program running.

[0059] The audio-video decoding module is configured to decode and process the audio-video signal to obtain a video stream signal and an audio signal for playing. The module aims to decode and process the audio-video signal transmitted by the investigated device to provide a basis for subsequent display and further processing. The specific implementation steps are as follows:

[0060] Firstly, the recording host converts the audio-video signal into a digital stream format through an HDMI receiver chip. For video data, a dedicated video decoding chip (for example, Realtek RTD1296) is used for decompression processing. The decoding chip first separates the video frame data from the received video data stream, and then performs decoding operation on the video frame data to restore the compressed format (such as H.264) to the standard format (such as YUV).

[0061] The decoded video frame data is stored in the cache to ensure the efficiency and smoothness of the subsequent output or display process. For audio data, a dedicated audio decoding chip (such as Cirrus Logic CS43131) is used for decompression processing. Audio data is usually transmitted in compressed format (such as AAC), and the audio decoding chip restores it to uncompressed PCM format, thereby recovering the original audio waveform. To achieve audio-video synchronization, the decoding chip coordinates the synchronized output of the decoded audio and video signals by comparing the timestamps of the audio and video signals. Finally, the decoded audio signal is transmitted to the output module for playback or further processing.

[0062] The source tracing module is used to add tracing information to the video signal, including operator information and device information. Tracing is a core capability of forensics, and its main role is to ensure the credibility and traceability of collected data. In law enforcement activities, through the source tracing function, the collection subject and device information of the forensic data can be effectively verified, preventing evidence from being falsified, tampered with, or improperly used, thereby improving public credibility and improving data management efficiency.

[0063] The present application uses invisible watermark as tracing information, and realizes the source tracing function by embedding invisible watermark. Invisible watermark is a digital identifier that cannot be detected by the naked eye but can be extracted and verified by technical means, and specific information can be embedded without affecting data quality. Specifically, the invisible watermark content of the device includes the number of the operator, so that each collected data can be accurately associated to a specific operator.

[0064] The present application uses the open source model RivaGAN to realize the embedding of invisible watermark. The RivaGAN model is composed of an attention module (Attention Module), an encoder (Encoder), a decoder (Decoder), a critic network (Critic Network), and an adversary network (Adversary Network), and its workflow is as follows:

[0065] 1. Select the embedding area: The RivaGAN model uses a convolution-based attention module to generate an attention distribution for each pixel in the video frame, from which the area suitable for embedding the watermark is selected. This selection method hides the watermark data in complex scenes while enhancing the robustness of the watermark.

[0066] 2. Watermark embedding: The encoder of the model embeds a fixed-length data vector into the video frame sequence. In this application scenario, the encoder embeds the number of the law enforcement officer on patrol into the video frame to generate a video with a watermark. The encoding process uses residual mask technology to ensure that the disturbance to the original frame is less than 0.01, thereby maintaining the video quality.

[0067] 3、Watermark extraction and verification: The decoder is responsible for extracting the embedded data from the video with watermark. By combining the weighted calculation of pixel contribution of the attention module, the decoder can efficiently recover the watermark information.

[0068] 4、Quality assessment and adversarial training: The critic network is used to evaluate the visual quality of the watermark video, and the adversary network tries to remove the watermark. The feedback of these two networks is used to optimize the model, ensuring the concealment and stability of the watermark. In addition, RivaGAN introduces a noise layer during training, allowing the model to learn to resist common video transformations such as scaling, cropping, compression, etc., further improving the robustness of the watermark.

[0069] According to relevant experiments, even if the video is cropped or scaled by 80%-100%, the decoding accuracy of the watermark can still approach 99%. This technology provides reliable protection for provenance tracing, and provides strong technical support for data credibility and traceability in digital forensics activities.

[0070] The time calibration module is used to calibrate the time information of the recording host based on standard time. In network information collection, such as digital forensics, time information is a key factor in evaluating the legality and usability of evidence. Time calibration aims to ensure that all data recorded have accurate and reliable time stamps, so that the time information of the data is true and reliable, and is not affected by device power failure or environmental changes. This function is directly related to the integrity of the evidence chain, and can effectively avoid the risk of evidence effectiveness being questioned due to time errors. In addition, time calibration not only ensures the time continuity of recorded data, but also provides a unified time reference for cross-device or cross-system data comparison and analysis. Accurate time information is crucial for restoring the process of a case, analyzing the sequence of events, and providing strong evidence in court or administrative investigations.

[0071] The time calibration of the present application is based on a real-time clock (RTC) chip. The RTC chip is a standalone hardware module that can continuously track time and date. Even if the forensic device is powered off, the RTC chip can still operate normally through the built-in battery, ensuring the continuity of time recording. Its core working mechanism includes: using a crystal oscillator with a low-power frequency of 32.768 kHz for timekeeping to achieve high-precision time management; the built-in battery provides long-term independent power supply to ensure that the device does not lose time information when powered off; provides an I2C standard interface to communicate with the host chip of the forensic device for time data reading or updating.

[0072] When the recording host is first started or restored to factory settings, it establishes communication with a national time center (such as the National Time Service Center of the Chinese Academy of Sciences) to obtain accurate standard time. This time is then synchronized and set to the device's RTC chip, ensuring that the device has an accurate time reference from the start. In addition, each time the forensic device connects to the background management system, the system obtains the latest standard time from the national time center and synchronizes it to the device's RTC chip, ensuring that the device clock always matches the national time center's time, thereby avoiding time drift or errors after a long time.

[0073] At each start of the recording host, the forensic device reads the current time from the RTC chip and updates it to the system time. Then, during audio and video data collection, the forensic device uses the system time as a timestamp and embeds it in the data file, ensuring accurate time information for data recording. In this way, the forensic device can synchronize accurate time at each start and maintain time consistency during data collection, providing reliable time basis for subsequent forensics and legal procedures.

[0074] The integrity verification module is used to generate integrity verification information based on the recorded audio and video materials. Integrity verification aims to ensure that the data collected by the system is not tampered with or lost during storage and transmission, to protect the authenticity and reliability of network patrol law enforcement records. This function is important in law enforcement, not only helping to improve law enforcement transparency and prevent illegal behavior, but also providing credible evidence for subsequent judicial procedures or internal review, ensuring the fairness of the forensics process and the non-tamperability of the data.

[0075] The integrity verification function of the present application is implemented through a hash algorithm. Whenever new data (such as screen recording or screenshots) is collected and stored, the device generates a SHA-256 hash value and stores it together with the new data in the device's built-in storage medium. The hash value, as a "fingerprint" of the data, will change significantly even if the data content changes slightly later. Therefore, the hash value can accurately reflect the integrity of the data.

[0076] When the data is uploaded or transmitted to the background management system, the system recalculates the hash value of the data and compares it with the original stored hash value. If they match, it means that the data has not been tampered with or damaged during transmission; if the hash values do not match, it indicates that the data may have been modified or damaged, prompting that the integrity of the data is threatened. In this way, the integrity verification function can effectively protect the security and credibility of the forensic data.

[0077] The forensic statement generation module is used to generate structured forensic statements based on the recorded audio and video materials and operation logs.

[0078] The forensic report generation function automatically generates a structured forensic report based on the recorded video, screenshots, and operation logs of the operator, which will describe in detail when and how the operator obtained the electronic evidence. Generally, this function is triggered after the recording host completes the forensic data upload to the background management system.

[0079] The input of the forensic report generation module has three contents, which are: a video composed of several frames , a screenshot containing target information , and an operation log composed of three-axis acceleration and three-axis angular velocity at each time point (see formula 1 for definition). The data of the operation log comes from multiple IMU sensors.

[0080] (1)

[0081] Where:

[0082] : the th timestamp (usually in seconds)

[0083] : the th IMU sensor's acceleration vector at time (unit )

[0084] : the th IMU sensor's angular velocity vector at time (unit ).

[0085] The forensic report generation module finally outputs a structured forensic report, which contains five elements: forensic time, forensic location (which platform), forensic steps, and the main content of electronic evidence.

[0086] The specific workflow of the forensic report automatic generation function includes two stages: data processing and analysis and forensic report automatic generation. The main goal of the data processing and analysis stage is to determine the operator's action type at each time, such as clicking, sliding, or stationary, and the operation object (which UI component in the screen), and generate the visual input (screenshots before and after operation) and part of the text input (operation record) of the VLM based on these information.

[0087] The operation type is determined according to the data of the operation log, that is, the inertial data generated by the IMU sensor. First, the inertial data output by the IMU sensor is preprocessed, that is, the coordinate system between the sensors is aligned and calibrated to ensure the consistency of the fusion of multiple point data; then, the noise in the three-axis acceleration value and the three-axis angular velocity value is removed and the gravity component is eliminated through an infinite impulse response filter; then, the continuous inertial data is segmented into independent gesture segments using the segmentation and normalization technique, that is, the start and end positions of the gesture are determined by finding the extreme points of the three-axis acceleration value, thereby realizing segmentation; finally, each gesture segment is classified through threshold rules, thereby obtaining the operation type. When the three-axis acceleration value and the three-axis angular velocity value of all IMU sensors are in a low amplitude range (acceleration less than 0.5 m / s², angular velocity less than 5 ), the action type of this segment is considered to be stationary; when a short-time high-amplitude acceleration mutation (acceleration greater than 0.5 m / s² and duration more than 100 ms) occurs, the action type of this segment is considered to be clicking; when the acceleration changes (greater than 1.5 m / s²) have consistent directions (included angle less than 25 ) and long durations (more than 200 ms), the action type of this segment is considered to be sliding.

[0088] To analyze the operation object, the approximate position of the operation needs to be analyzed first, and then the UI component near the position that supports the corresponding action type is matched. Specifically, first, the displacement of the IMU sensor on the thumb tip is calculated. Through continuous coordinate transformation and double integration of the three-axis acceleration value and the three-axis angular velocity value of the IMU sensor, and combined with the zero velocity update correction technique, the displacement of the thumb tip relative to the starting position is finally calculated and the approximate position of the operation position from the right lower corner of the electronic device is inferred, that is, the position of the operation. Then, identify which UI components are on the interface of the electronic device during the operation. Specifically, first, extract the frame during the operation. Then input the picture of this frame into the trained YOLOv8 model, which will output which UI components are in this frame and the position of the UI components relative to the right lower corner of the device. Finally, determine the operation object based on the position of the operation, the action type, and the UI component information. Specifically, first, list the UI components near the operation position according to the operation position, and sort them from near to far. Then, see if the UI component supports the operation type, if not, check the next one in order, and directly find the supported UI component. Thus, the operation object is analyzed.

[0089] Finally, based on the information analyzed previously, extract information from the video and process it as input for the subsequent forensic evidence automatic generation stage. Specifically, assume that there are dynamic actions in the entire collection process, and for the action executed at , the action type is , operation object is , first extract the pictures in , and , then frame and in a conspicuous rectangle in , and generate an operation record , , The format is {“time“: ,“action_type“: , “object“: }. Finally output groups and , and records .

[0090] The main task of the forensic statement generation module is to make prompt words to guide the VLM to understand the forensic process and generate structured forensic statements. Specifically, assuming that the entire forensic process involves N dynamic actions, then input pictures and two texts to the VLM. Among them pictures include screenshots before and after each action operation, and screenshots obtained by police officers discovering key information and clicking the screenshot button. Two texts are prompt words guiding the VLM to understand the task and operation records containing records of data. The output of the VLM is the forensic statement. The prompt words here are as follows:

[0091] “You are an experienced electronic forensic expert. I will provide you with:

[0092] 1. A set of 2N+1 screenshots labeled Image_0, Image_1…Image_2N in order.

[0093] 2. N operation records, each record contains:

[0094] -time: operation time

[0095] -action_type: operation type, taking values “click” or “slide”

[0096] -object: operation object, interactive UI component (e.g. “button”)

[0097] 3. The last screenshot Image_2N contains the target content I want to find

[0098] Your task is to generate a structured forensic report, which must include the following sections:

[0099] A. Identity and Environment

[0100] - Describe the identity by which the police officer logged into the platform (e.g., account type, username, and password).

[0101] - Specify the platform being investigated (e.g., application or website name and version information).

[0102] B. Operating Procedures

[0103] Explain each operation i (i ranges from 1 to N):

[0104] 1. Referencing Image_{2i-2} (a screenshot before the operation with visual cues), briefly describe the content of the current interface.

[0105] 2. Operation type and operation object

[0106] 3. Referencing Image_{2i-1} (the screenshot after the operation), briefly describe the operation result interface.

[0107] C. Target Content Discovery

[0108] - Referencing Image_2N, a detailed description of the content found in the screenshot.

[0109] Precautions:

[0110] 1. All descriptions are based on the provided screenshots and operation logs and should not be based on conjecture.

[0111] 2. Use concise and formal reporting language; all sections must be clearly labeled.

[0112] As can be seen from the above technical solution, this application provides a network information collection system, which includes a wearable controller and an audio / video recording device connected to the wearable controller. The wearable controller is worn on the operator's body and generates an evidence storage control signal based on the operator's movements. The evidence storage control signal includes one or more inertial data. The audio / video recording device is also connected to the audio / video output port of a network terminal, used to receive the audio / video signals output by the network terminal, process the audio / video signals based on the evidence storage control signal to obtain audio / video materials, and save the audio / video materials within a preset time period based on the time specified by the evidence storage control signal. Evidence collection is achieved by saving selected audio / video materials, and timely measures are taken based on the discovery of harmful information, thereby maintaining the healthy development of the network.

[0113] Figure 4 This is a flowchart illustrating an evidence preservation control method according to an embodiment of this application.

[0114] As Figure 4 shown, the evidence storage control method provided by the embodiment is applied to the network information collection system of the above embodiment, and is used to control the network information collection system to implement collection of network information based on the operation of a user. The evidence storage control method specifically includes the following steps:

[0115] S1, calibrate time information when the audio and video recording device starts.

[0116] That is, when the user issues a device start request, the audio and video recording device of the network information collection system is started, and after the hardware is started, the time information of the audio and video recording device is calibrated based on a standard time signal. The time calibration here is realized based on the time calibration module described above, and the operation process has been described in detail above, and will not be repeated here.

[0117] S2, decode the audio and video signals to obtain video stream signals and audio signals.

[0118] That is, after the device completes the start and is connected with the network terminal, the audio and video signals output by the audio and video output port are decoded to obtain video stream signals and audio signals. The decoding of the audio and video signals is realized based on the audio and video decoding module described above, and the detailed description is suggested above, and will not be repeated here.

[0119] S3, analyze the operation type of the evidence storage control signal issued by the operator.

[0120] The operation type is determined according to the data of the operation log (i.e., the inertial data generated by the IMU sensor). First, the inertial data output by the IMU sensor is preprocessed, that is, the coordinate systems between the sensors are aligned and calibrated to ensure the fusion consistency of the multi-point data; then, the noise in the three-axis acceleration values and three-axis angular velocity values is removed and the gravity component is eliminated through an infinite impulse response filter; then, the continuous inertial data is segmented into independent gesture segments through segmentation and normalization technology, that is, the start and end positions of the gesture are determined by finding the extreme points of the three-axis acceleration values, so as to realize segmentation; finally, each gesture segment is classified through threshold rules, so as to obtain the operation type. When the three-axis acceleration values and three-axis angular velocity values of all IMU sensors are in a low amplitude range (acceleration less than 0.5 m / s², angular velocity less than 5 ), it is considered that the action type of this segment is stationary; when there is a short-time high-amplitude acceleration mutation (acceleration greater than 0.5 m / s², and duration more than 100 ms), it is considered that the action type of this segment is click; when there is a consistent direction (included angle less than 25 ) and long duration (more than 200 ms) acceleration change (greater than 1.5 m / s²), it is considered that the action type of this segment is sliding.

[0121] S4. Capture and record video stream signals and / or audio signals based on the operation type.

[0122] After determining the type of operation input by the operator, the video stream signal or audio signal is recorded, or both are recorded simultaneously, to obtain audio and video materials, which include added source information. The addition of source information to the audio and video materials is implemented based on the aforementioned source tracing module, whose tracing function has been described in detail above and will not be repeated here.

[0123] Analyzing the target of an operation requires first determining its approximate location, then matching nearby UI components that support the corresponding action type. Specifically, the displacement of the IMU sensor on the thumb's tip is calculated first. Through continuous coordinate transformation and double integration of the IMU sensor's three-axis acceleration and angular velocity values, combined with zero-velocity update correction technology, the displacement of the thumb's tip relative to its initial position is calculated, inferring the approximate location of the operation from the lower right corner of the electronic device—the operation's location. Next, the UI components on the electronic device's interface during the operation are identified. Specifically, the frame in question during the operation is extracted. This frame is then input into a trained YOLOv8 model, which outputs the UI components in that frame and their positions relative to the lower right corner of the device. Finally, the target of the operation is determined based on its location, action type, and UI component information. Specifically, UI components near the operation's location are listed, sorted from nearest to farthest. Then, each UI component is checked to see if it supports the operation type; if not, the next component is checked sequentially until a supporting UI component is found. This completes the analysis of the target of the operation.

[0124] Finally, based on the information analyzed previously, information is extracted and processed from the video, serving as input for the subsequent automatic generation stage of evidence collection documentation. Specifically, assuming the entire acquisition process has... A dynamic action, for in Actions performed at all times Its action type is The object of operation is First, extract the video from the video. Time and The scene and Then Enclosed in a conspicuous rectangular frame And generate an operation record. , The format is {"time": "action_type": "object": Finally output individual Group And And Article .

[0125] S5, save the audio and video material.

[0126] Specifically, the audio and video material can be saved in a local storage medium, or the audio and video material can be saved remotely by uploading to a host computer or server.

[0127] In the saving process, first, the audio and video material is encoded to obtain multimedia evidence material; then the multimedia evidence material is packaged to obtain an evidence file; then, the integrity of the evidence file is verified, the integrity verification function is provided by the above-mentioned integrity verification module, which has been described in detail above, and will not be repeated here; if it is determined that the evidence file is not complete through verification, the audio and video material is encoded; if it is determined that the evidence file is complete through verification, the evidence file is saved.

[0128] From the above technical solution, it can be seen that the present application provides a kind of evidence control method, the method is applied to the above network information collection system, specifically, in response to the device start request of user, control audio and video recording device starts, and based on standard time signal, the time information of audio and video recording device is calibrated; video stream signal and audio signal are obtained by decoding the audio and video signal; when receiving the evidence control signal issued by the operator, the operation type is calculated based on the evidence control signal; video stream signal and / or audio signal are intercepted and recorded based on operation type, to obtain one or more audio and video materials, audio and video material includes trace information; save the audio and video material. Through the above operation steps, the network information collection system realizes the function of evidence.

[0129] In addition, in one specific embodiment of the present application, the following steps are included, as shown in Figure 5 .

[0130] S6, generate structured forensic explanation based on audio and video material and operation log.

[0131] Here the generation of forensic explanation is realized based on the above-mentioned forensic explanation generation module, and since the function of the module has been described in detail above, it will not be repeated here.

[0132] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0133] Although the operations are depicted in a particular, sequential order, this should not be understood as requiring or

[0134] It is to be understood that the steps of the methods recited in the method embodiments of the present disclosure can be carried out in a different order and / or concurrently with each other. Further, the method embodiments can include more steps or fewer steps than those illustrated in the diagrams. The scope of the present disclosure should not be limited to the described embodiments.

[0135] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0136] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure.

[0137] Reference will now be made to Figure 6FIG. 1 shows a structural diagram of an electronic device suitable for use in implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), and the like, as well as a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0138] The electronic device can include a processing device (e.g., a central processor, a graphic processor, etc.) 601 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded into a random access memory (RAM) 603 from an input device 606. In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, the ROM, and the RAM are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0139] Generally, the following devices can be connected to the I / O interface: input devices including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 608 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 609. The communication devices 609 can allow the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the electronic device is shown as having various devices, it should be understood that all of the shown devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0140] The present application also provides a computer-readable storage medium embodiment.

[0141] The above computer-readable storage medium is applied to an electronic device and carries one or more computer programs, when the one or more computer programs are executed by the electronic device, the electronic device controls the audio and video recording device to start in response to a device start request of a user, and calibrates time information of the audio and video recording device based on a standard time signal; decodes the audio and video signal to obtain a video stream signal and an audio signal; when receiving a storage evidence control signal issued by an operator, calculates an operation type based on the storage evidence control signal; intercepts and records the video stream signal and / or the audio signal based on the operation type to obtain one or more audio and video materials, the audio and video material includes traceability information; saves the audio and video material. Through the above operation steps, the network information collection system realizes the function of storing evidence.

[0142] Note that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example and without limitation, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0143] In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave that bears computer readable program code. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. Computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, apparatus or device. Program code contained on a computer readable medium can be transmitted by any suitable medium, including but not limited to wire, cable, RF, etc., or any suitable combination of the foregoing.

[0144] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be mutually referred to.

[0145] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0146] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be used to distinguish one entity or action from another entity or action, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by an "comprising" statement serves as a means plus function alternative.

[0147] The above detailed description of the technical solutions provided by the present application has been given, and the principles and implementation manners of the present application are described by applying specific examples in this document. The above description of the examples is only for helping to understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will have changes, and the above description of the content of the specification should not be understood as a limitation of the present application.

Claims

1. A network information collection system, characterized in that, It includes a wearable controller and an audio / video recording device connected to the wearable controller, wherein: The wearable controller is worn on the operator's body and generates evidence storage control signals based on the operator's movements. The evidence storage control signals include one or more inertial data. The audio and video recording device is also connected to the audio and video output port of the network terminal, for receiving the audio and video signals output by the network terminal, processing the audio and video signals based on the evidence storage control signal to obtain audio and video materials, storing the audio and video materials within a preset time period based on the time specified by the evidence storage control signal, and generating evidence storage information based on the action information of the evidence storage control signal and the audio and video materials within the preset time period.

2. The network information collection system as described in claim 1, characterized in that, The wearable controller includes a glove and multiple IMU sensors mounted on the glove. The IMU sensors are equipped with wireless communication modules for transmitting the inertial data wirelessly.

3. The network information acquisition system as described in claim 1, characterized in that, The audio and video recording device includes a housing and a recording host disposed inside the housing, wherein: The outer wall of the box is provided with one or more control buttons, a display screen, a signal input port, a power input port and a main control unit respectively connected to the recording host. The main control unit is wirelessly connected to the wearable controller and is used to receive the evidence storage control signal. The recording host is used to acquire the audio and video signals output by the network terminal based on the signal input port, process the audio and video signals based on the evidence preservation control signal to obtain audio and video materials, save the audio and video materials within a preset time period based on the time specified by the evidence preservation control signal, and generate evidence preservation information based on the action information of the evidence preservation control signal and the audio and video materials within the preset time period.

4. The network information acquisition system as described in claim 3, characterized in that, The recording host is equipped with an audio / video decoding module, a source tracing module, a time synchronization module, an integrity verification module, and an evidence generation module, wherein: The audio and video decoding module is used to decode the audio and video signals to obtain video stream signals and audio signals for playback; The source tracing module is used to add source tracing information to the video signal, and the source tracing information includes operator information and equipment information; The time calibration module is used to calibrate the time information of the recording host based on the standard time; The integrity verification module is used to generate integrity verification information for the recorded audio and video materials based on a hash algorithm; The evidence collection description generation module is used to generate structured evidence collection descriptions based on the recorded audio and video materials and operation logs.

5. A method for controlling evidence storage, applied to the network information collection system as described in any one of claims 1 to 4, characterized in that, The evidence preservation control method includes the following steps: In response to the user's device startup request, the system controls the audio and video recording device to start, and performs time information calibration processing on the audio and video recording device based on a standard time signal; The audio and video signals are decoded to obtain video stream signals and audio signals; When a storage control signal is received from the operator, the operation type is calculated based on the storage control signal; Based on the operation type, the video stream signal and / or the audio signal are extracted and recorded to obtain one or more audio and video materials, the audio and video materials including source information; The audio and video materials are saved.

6. The evidence storage control method as described in claim 5, characterized in that, When a data storage control signal is received from an operator, the operation type is calculated based on the data storage control signal, including the following steps: The evidence storage control signal is preprocessed to obtain aligned and calibrated multi-point inertial data; The multi-point inertial data is filtered using an infinite impact filter to eliminate noise and gravity components. By using segmentation and normalization techniques, the filtered multi-point inertial data is segmented to obtain multiple independent gesture segments; The gesture fragments are classified according to threshold rules to obtain the operation type.

7. The evidence preservation control method as described in claim 5, characterized in that, The process of saving the audio and video materials includes the following steps: The audio and video materials are encoded to obtain multimedia evidence materials; The multimedia evidence materials are packaged to obtain evidence files; The integrity of the stored evidence documents is verified. If the verification determines that the evidence file is incomplete, then return to the step of encoding the audio and video materials; If the evidence file is verified to be complete, then the evidence file is saved.

8. The evidence preservation control method according to any one of claims 5 to 7, characterized in that, It also includes the following steps: A structured forensic documentation is generated based on the audio and video materials and operation logs.

9. An electronic device, characterized in that, The electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs or instructions; The processor is used to execute the computer program or instructions to enable the electronic device to implement the evidence storage control method as described in any one of claims 5 to 8.

10. A computer-readable storage medium for use in electronic devices, characterized in that, The storage medium carries one or more computer programs that can be executed by the electronic device, thereby enabling the electronic device to implement the evidence control method as described in any one of claims 5 to 8.

Images