Method and system based on multi-device collaborative evidence collection

Abstract

The invention discloses a multi-device collaborative evidence obtaining method and system, and relates to the technical field of evidence obtaining. Comprising the steps that a main device shoots an evidence obtaining target to obtain a target material evidence video stream, and an auxiliary device shoots an evidence obtaining behavior process to obtain an evidence obtaining process video stream; detecting the synchronism of the picture motion state and the equipment motion state of the target material evidence video stream corresponding to the same timestamp, and comparing and analyzing the synchronism of the physical motion trail and the image motion trail of the main equipment to obtain a synchronous verification result; combining the multiple video streams corresponding to the timestamp and the sensor data according to the time sequence to obtain an original evidence collection data packet, and comparing the initial hash value corresponding to each video stream with the verification hash value to obtain a hash verification result; and writing the synchronous verification result and the Hash verification result into the original evidence obtaining data packet. According to the invention, the credibility and reliability of the electronic evidence can be enhanced.

Description

Technical Field

[0001] This application relates to the field of forensics technology, and in particular to a method and system for collaborative forensics based on multiple devices. Background Technology

[0002] In today's digital age, electronic data plays a crucial role in various legal cases, commercial disputes, and security investigations.

[0003] However, the ease with which electronic data can be modified and forged poses a significant threat to the authenticity and reliability of evidence. Traditional electronic data forensics methods often rely on single devices or limited data sources, making it difficult to comprehensively and in real-time record various information during the process. This hinders the formation of an effective system of mutual corroboration of evidence, allowing crucial evidence such as videos to be altered before the intervention of a Trusted Timestamp Authority (TSA). This reduces the authenticity and reliability of evidence during electronic data forensics, thereby affecting the fairness of the entire case and the judgment. These problems not only complicate judicial work but also diminish the credibility and authority of electronic data as evidence.

[0004] Therefore, how to improve the authenticity and reliability of evidence in the process of electronic data forensics has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method, system, device, medium, and product for collaborative evidence collection based on multiple devices, which can improve the authenticity and reliability of evidence during electronic data forensics.

[0006] In a first aspect, embodiments of this application provide a method for multi-device collaborative evidence collection, applied to an evidence collection system. The evidence collection system includes at least two terminal devices, a server, and a timestamp authentication system. The method includes: responding to an evidence collection command, synchronously acquiring video data through at least two terminal devices and obtaining sensor data from a master device in real time, wherein at least one master device captures the evidence target to obtain a target evidence video stream, and at least one auxiliary device captures the evidence collection process to obtain an evidence collection process video stream, the target evidence video stream and the evidence collection process video stream forming a multi-channel video stream; during the video acquisition process, the terminal devices embed timestamps generated by the same time reference source into the video frames and sensor data in the multi-channel video streams, and upload the multi-channel video streams and sensor data to the server; the server identifies the device motion state of the master device based on the sensor data and detects the image of the target evidence video stream corresponding to the same timestamp. The synchronization of motion state with device motion state, and the synchronization of the main device's image motion trajectory in the video stream during the evidence collection process with its physical motion trajectory determined by the sensor data, are analyzed to perform video synchronization verification on multiple video streams and obtain synchronization verification results. After the multiple video streams are uploaded, the server combines the timestamp-corresponding multiple video streams and sensor data according to the time sequence to obtain the original evidence collection data packet. The initial hash value corresponding to each video stream is compared with the verification hash value to obtain the hash verification result. The initial hash value is the hash value generated by the terminal device based on the original acquired video stream, and the verification hash value is the hash value generated by the server based on the received video stream. The synchronization verification result and hash verification result are written into the original evidence collection data packet to generate the target evidence collection data packet, and the target evidence collection data packet is uploaded to the timestamp authentication system for credible timestamp evidence solidification to generate tamper-proof electronic evidence files.

[0007] Secondly, embodiments of this application provide a system for collaborative evidence collection based on multiple devices. This system includes: at least two terminal devices, a server, and a timestamp authentication system. The at least two terminal devices are used to respond to evidence collection instructions, synchronously capture video, and acquire sensor data from at least one master device in real time. At least one master device captures the target evidence to obtain a target evidence video stream, and at least one auxiliary device captures the evidence collection process to obtain an evidence collection process video stream. The target evidence video stream and the evidence collection process video stream constitute multiple video streams. The terminal devices are used to embed timestamps generated by the same time reference source into the video frames and sensor data in the multiple video streams during video acquisition, and upload the multiple video streams and sensor data to the server. The server is used to identify the device motion state of the master device based on the sensor data, and detect the image motion state and device of the target evidence video stream corresponding to the same timestamp. The synchronization of motion states, and the comparison and analysis of the synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and the physical motion trajectory determined by the sensor data, are used to perform video synchronization verification on multiple video streams to obtain synchronization verification results; the server is also used to combine the multiple video streams and sensor data corresponding to the timestamps according to the time sequence after the multiple video streams are uploaded to obtain the original evidence collection data packet, and compare the initial hash value corresponding to each video stream with the verification hash value to obtain the hash verification result. The initial hash value is the hash value generated by the terminal device based on the original video stream, and the verification hash value is the hash value generated by the server based on the received video stream; the server is also used to write the synchronization verification result and the hash verification result into the original evidence collection data packet to generate the target evidence collection data packet, and upload the target evidence collection data packet to the timestamp authentication system for credible timestamp evidence solidification to generate tamper-proof electronic evidence files.

[0008] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the steps of the method for multi-device collaborative forensics as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the method for multi-device collaborative forensics as described in the first aspect.

[0010] Fifthly, embodiments of this application provide a computer program product stored in a non-volatile storage medium, which, when executed by a processor, implements the steps of the method for multi-device collaborative forensics as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps of the method for multi-device collaborative forensics as described in the first aspect.

[0012] This application provides a method, system, device, medium, and product for multi-device collaborative evidence collection. It records the evidence target and the evidence collection process targeting the target through collaborative recording by at least two terminal devices, while simultaneously acquiring sensor data from the main device in real time. The combination of these three data not only records the state of the physical evidence but also presents the operational trajectory of the evidence-collecting entity, constructing a chain of evidence-collecting behaviors related to the evidence collection process, thus enhancing the integrity and relevance of the evidence. Furthermore, multiple video streams and sensor data can form mutually corroborating evidence chains. Therefore, this application can accurately and effectively verify the authenticity of electronic evidence formed by multiple video streams based on sensor data. Specifically, it identifies the actual motion state of the device based on sensor data and simultaneously verifies whether the motion state (such as shaking or rotation) of the target physical evidence video stream is consistent with the physical motion of the device. By comparing the image motion trajectory of the main device in the auxiliary device's captured image with the physical motion trajectory determined by the sensors, it verifies the logical consistency of motion across multiple perspectives. The terminal generates an initial hash value, the server verifies the received stream hash value, and combined with the timestamp embedded in the same time reference source, it ensures the integrity and timeliness of the data from acquisition to transmission. In this way, when an attacker tampers with one of the video streams, the tampering behavior targeting a single video stream can be accurately identified. Considering the high difficulty for attackers to simultaneously tamper with the video streams of the evidence collection process and the target physical evidence, this method can increase the difficulty of tampering from the source, reduce the risk of video tampering, and enhance the credibility and reliability of electronic evidence. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings of the embodiments of this application will be briefly described below.

[0014] Figure 1 This is a flowchart illustrating a method for multi-device collaborative forensics provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a system for collaborative evidence collection based on multiple devices provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The principles and spirit of this application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided to make the principles and spirit of this application clearer and more thorough, enabling those skilled in the art to better understand and implement the principles and spirit of this application. The exemplary embodiments provided herein are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described herein without inventive effort are within the scope of protection of this application.

[0016] In this document, terms such as first, second, and third are used only to distinguish one entity (or operation) from another entity (or operation), and are not intended to require or imply any order or relationship between these entities (or operations).

[0017] The following is a brief description of the concepts and technical terms that may be involved in the embodiments of this application.

[0018] A Trusted Timestamp (TSA) is an electronic certificate issued by an authoritative organization in accordance with international standards to prove the existence, integrity, and immutability of electronic data.

[0019] Before describing the technical solutions provided in the embodiments of this application, in order to facilitate understanding of the embodiments of this application, this application first specifically explains the problems existing in the related technologies: Traditional electronic data forensics often lacks effective real-time verification mechanisms, making it possible for critical evidence such as videos to be tampered with before TSA intervention, thus affecting the fairness of the entire case and the judgment. Specifically, traditional electronic data forensics often relies on recording with a single device, which has a limited perspective and cannot simultaneously capture the details of the evidence target and the behavior of the evidence-gathering subject. This results in a gap in the "process integrity" of the evidence chain, making it difficult to comprehensively and in real-time record various information during the evidence gathering process, and failing to form an effective evidence verification system, thus limiting the methods of evidence gathering. Furthermore, the credibility of traditional electronic evidence is highly dependent on external, delayed certification by public authorities (such as notarized certificates issued by notary offices), and cannot prove that there was no tampering, interruption, or human intervention during the recording.

[0020] In view of the inventors’ above-mentioned research findings, this application provides a method, system, device, medium and product for collaborative evidence collection based on multiple devices, which aims to solve the problem of low authenticity and reliability of evidence in traditional electronic data forensics.

[0021] The method for multi-device collaborative forensics provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0022] Figure 1 This is a flowchart illustrating a method for multi-device collaborative forensics based on an embodiment of this application. The executing entity of this method can be an forensics system, which may include... Figure 2 The document shows at least two terminal devices 210, a server 220, and a timestamp authentication system 230. The server can be a server for a trusted authentication application. Terminal device 210 can include at least one of the following: smartphone, tablet, wearable device, etc. It should be noted that the terminal device can include at least one camera, allowing users to hold the terminal device for evidence collection or to fix the terminal device to a stand and hold the stand for evidence collection, etc. This application does not specifically limit the evidence collection application scenario.

[0023] The following example, using the evidence collection system as the executing entity of the multi-device collaborative evidence collection method, illustrates the multi-device collaborative evidence collection method of this application. It should be noted that the aforementioned executing entity and application scenario do not constitute a limitation on this application.

[0024] like Figure 1 As shown, the method for multi-device collaborative forensics provided in this application embodiment may include steps 110-150.

[0025] Step 110: In response to the evidence collection command, synchronous video acquisition is performed through at least two terminal devices and sensor data from at least one master device is acquired in real time. Step 120: During the video acquisition process, the terminal device embeds timestamps generated by the same time reference source into the video frames and sensor data in the multiple video streams, and uploads the multiple video streams and sensor data to the server. Step 130: The server identifies the device motion state of the main device based on sensor data, detects the synchronization between the image motion state of the target evidence video stream corresponding to the same timestamp and the device motion state, and compares and analyzes the synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and the physical motion trajectory determined by the sensor data. Video synchronization verification is performed on multiple video streams to obtain the synchronization verification result. Step 140: After the multiple video streams are uploaded, the server combines the timestamp-corresponding multiple video streams and sensor data according to the time sequence to obtain the original evidence data packet, and compares the initial hash value corresponding to each video stream with the verification hash value to obtain the hash verification result. Step 150: Write the synchronization verification result and hash verification result into the original evidence data packet to generate the target evidence data packet, and upload the target evidence data packet to the timestamp authentication system for trusted timestamp evidence solidification to generate tamper-proof electronic evidence files.

[0026] The multi-device collaborative evidence collection method provided in this application records the evidence target and the evidence collection behavior process targeting the target through collaborative recording by at least two terminal devices. Simultaneously, it acquires sensor data from the main device in real time. The combination of these three data sources not only records the state of the physical evidence but also presents the operational behavior trajectory of the evidence collection entity, constructing a chain of evidence collection behaviors related to the evidence collection process. This enhances the integrity and relevance of the evidence and systematically solves the fundamental deficiency in proving the integrity of the process from a traditional single-perspective viewpoint. Furthermore, by deeply integrating and analyzing different types of data, it can display the links and details of the evidence collection process in detail, providing judicial personnel with richer and more accurate evidentiary information.

[0027] Meanwhile, multiple video streams and sensor data can form a mutually corroborating chain of evidence. Therefore, this application can accurately and effectively verify the authenticity of electronic evidence formed by multiple video streams based on sensor data. Specifically, it identifies the actual motion state of the device based on sensor data and simultaneously verifies whether the motion state of the target evidence video stream (such as shaking or rotation) is consistent with the physical motion of the device; by comparing the image motion trajectory of the main device in the auxiliary device's captured image with the physical motion trajectory determined by the sensor, it verifies the logical consistency of motion between multiple perspectives; the terminal generates an initial hash value, the server verifies the received stream hash value, and combined with the timestamp embedded in the same time reference source, it ensures the integrity and timeliness of the data from acquisition to transmission.

[0028] These three verifications form an internal verification scheme based on the physical correlation of multi-source information. This verification scheme does not rely on external intervention and can autonomously discover logical contradictions in the real-time stage of evidence generation, enabling the evidence to have the technical ability to "prove its innocence". The authenticity of the evidence is jointly guaranteed by mathematical algorithms (hash), physical laws (motion consistency) and tamper-proof timestamps (timestamp authentication system), thus constructing an objective technical trust system that does not rely on the endorsement of traditional authoritative institutions.

[0029] Thus, when an attacker tampers with one of the video streams, the tampering can be accurately identified for that single stream. Considering the difficulty for an attacker to simultaneously tamper with both the video stream used in the evidence collection process and the target evidence video stream, this approach increases the difficulty of tampering at the source, reducing the risk of video tampering and enhancing the credibility and reliability of electronic evidence. Furthermore, the verification process of this internal verification scheme is synchronized with video acquisition and uploading, eliminating the need for post-audit. The credibility authentication of evidence shifts from traditional "external institution verification" to "real-time self-verification by technical means." Its credibility no longer solely relies on external, lagging authentication methods, effectively improving the strength and efficiency of proving the original authenticity and process integrity of electronic evidence.

[0030] The specific implementation of the above steps will be described in detail below with reference to specific embodiments.

[0031] In step 110, in response to the evidence collection instruction, synchronous video acquisition is performed through at least two terminal devices and sensor data from at least one master device is acquired in real time.

[0032] In step 110, the at least two terminal devices include at least one main device and at least one auxiliary device. The at least one main device is used to capture the evidence target to obtain a video stream of the target evidence. The terminal devices other than the main device are auxiliary devices. The auxiliary devices can be fixed or handheld. The at least one auxiliary device is used to capture the evidence collection process to obtain a video stream of the evidence collection process. The elements in the evidence collection process can include at least the evidence collection execution subject and the evidence collection target, and may also include the evidence collection environment.

[0033] In some embodiments, at least two terminal devices may both be rear-facing cameras.

[0034] The evidence collection target can be a physical object that needs to be secured as evidence (such as contract texts, certificates, physical evidence), and the main device will focus on capturing details of its changes in state, such as contract details and the signing process. The evidence collection execution entity can be a natural person operating the terminal device to carry out the evidence collection behavior (such as judicial personnel or contracting parties). The evidence collection environment can be the physical space scene surrounding the evidence collection execution entity (such as a signing room or an accident scene), including environmental elements such as background objects and lighting conditions.

[0035] During the evidence collection process, the main device is configured to focus on the core target evidence. Through functions such as autofocus and optical zoom, it can clearly and accurately capture the detailed features of the evidence. For example, for a product suspected of infringement, the main device can capture key information such as the product's appearance, trademark, model, and serial number, ensuring accurate identification and authentication of the evidence in subsequent evidence analysis.

[0036] The auxiliary equipment simultaneously records the evidence collection process, specifically recording the evidence collector's actions towards the target, such as the collector's movements and language, as well as changes in the target's state. In addition, it can record dynamic changes in the surrounding environment, such as on-site personnel activities, lighting changes, and sounds. In this way, the target evidence is closely linked to the evidence collection process, forming a mutually corroborating chain of evidence.

[0037] Sensor data includes at least spatial pose data and position data, with the spatial pose data including at least angular velocity and acceleration. During evidence collection, GPS data records the location changes of the person collecting evidence in high-precision latitude and longitude coordinates, providing spatial location information for the evidence collection process. For example, in investigating a case involving geographical location information, GPS data can accurately record the location of the person collecting evidence at different times. Combined with video evidence, this can clearly show the person's movement trajectory and the location of the evidence collection, enhancing the relevance and credibility of the evidence.

[0038] An IMU (Inertial Measurement Unit) contains components such as accelerometers and gyroscopes, enabling real-time monitoring of the actions, postures, and motion states of the person being investigated. Accelerometers measure changes in acceleration in various directions, while gyroscopes detect rotation angles and angular velocities. Analysis of this data allows for the precise reconstruction of the person's movement trajectory and behavioral patterns. For example, in cases requiring evidence of the person's actions, IMU data can record the acceleration and rotation information of actions such as picking up, putting down, or moving equipment, corroborating video evidence and providing more detailed and accurate evidence for the trial.

[0039] By deeply integrating and analyzing GPS and IMU data, a chain of actions for evidence collection can be constructed. Specifically, data processing algorithms are used to synchronize GPS and IMU data in time and extract features, and then this data is correlated with video evidence. For example, GPS data is used to determine the time points of the person collecting evidence at different locations, and IMU data is used to analyze the person's actions and postures at those locations. This data is then matched with videos recorded by front and rear cameras to form a complete "chain of actions for evidence collection." This chain clearly demonstrates the individual actions, location changes, and interactions between the person collecting evidence and the physical evidence and environment during the evidence collection process, further enhancing the completeness and relevance of the evidence.

[0040] In scenarios where there is only one main device and one auxiliary device, in order to achieve synchronous and collaborative recording between the two terminal devices, the evidence collection system also includes: a trusted evidence collection application capable of communicating with the server. Multiple video streams are uploaded to the server through the trusted evidence collection application. In some embodiments of this application, before step 110 above, the method may further include the following steps: Establish communication and interaction between the master device and the auxiliary device, and calibrate the time base of the master device and the auxiliary device based on the time synchronization protocol to achieve clock synchronization between devices; When both the main device and the auxiliary device are equipped with trusted forensics applications, an end-to-end data transmission channel is established between the main device and the server to form a first transmission channel, and an end-to-end data transmission channel is established between the auxiliary device and the server to form a second transmission channel; and / or, When the main device has a trusted forensics application installed, but the auxiliary device does not have a trusted forensics application installed, an end-to-end data transmission channel is established between the main device and the server to obtain the first transmission channel, and an end-to-end data transmission channel is established between the main device and the auxiliary device to obtain the third transmission channel.

[0041] Specifically, the first transmission channel is used for real-time transmission of the target evidence video stream and sensor data from the main device; the second transmission channel is used for real-time transmission of the evidence collection process video stream from the auxiliary device; the third transmission channel is used to transmit the evidence collection process video stream from the auxiliary device to the main device in real-time; and the first transmission channel is used for real-time transmission of the target evidence video stream and sensor data from the main device, and forwarding the evidence collection process video stream from the auxiliary device to the server. All of the above transmission channels are encrypted link channels established using encryption protocols.

[0042] Time synchronization protocols are used to calibrate a device's time with a standard time source, calculating clock offsets through timestamp exchange. Time synchronization protocols can be broadly categorized into hardware-based and software-based protocols. Hardware-based protocols include those defined by the Global Positioning System (GPS) and the Precision Time Protocol (PTP) as defined by the IEEE 1588 standard, offering high accuracy. Software-based protocols include those such as the Network Time Protocol (NTP) and the Simple Network Time Protocol (SNTP), which are simpler to deploy.

[0043] A trusted forensics application refers to a dedicated program that is digitally signed and has tamper-proof capabilities. It assists in executing the forensics process and encrypts data transmission. When both the primary and secondary devices have a trusted forensics application installed, the application can monitor their operational status in real time and process and integrate the recorded video stream data. For example, through timestamp marking and video stream synchronization algorithms, it can accurately align video stream data recorded by multiple devices, enabling accurate association of physical evidence information with the forensics process during subsequent evidence viewing and analysis. If only the primary device has a trusted forensics application installed, the primary device accesses the video stream of the forensics process recorded by the secondary device through a third transmission channel and uploads this video stream and the target physical evidence video stream to the server in real time through a first transmission channel.

[0044] In this embodiment, the time base of the master and slave devices is calibrated based on a time synchronization protocol to achieve clock synchronization between devices, ensuring that multiple video streams are aligned in the time dimension. This ensures that two terminal devices can start and stop recording at the same time and maintain precise synchronization of recording time. Through trusted application-enforced encryption and channel isolation design, it not only guarantees resistance to tampering attacks during sensitive data transmission but also accommodates the access needs of slave devices without trusted applications installed, reducing deployment costs. Based on this, by ensuring the spatiotemporal consistency of data recorded by multiple terminals, a verifiable chain of synchronization evidence is provided for subsequent verification of multiple video streams. In some embodiments of this application, calibrating the time base of the master device and the auxiliary device to achieve clock synchronization between the devices may include: the master device acting as a time server broadcasting Precise Time Protocol (PTP) messages through the local area network, and the auxiliary device receiving the messages calculating the network delay and adjusting its local clock. In this way, the time error between the devices can be controlled within 1ms, achieving precise time synchronization.

[0045] In step 120, during the video acquisition process, the terminal device embeds timestamps generated by the same time reference source into the video frames and sensor data in the multiple video streams, and uploads the multiple video streams and sensor data to the server.

[0046] In step 120, the terminal device synchronously activates the IMU (Inertial Measurement Unit) and positioning module during video acquisition. Using the same time reference source as a reference, it adds a microsecond-level timestamp to each frame of video and sensor data, and can upload the sensor data to the server in real time through an independent thread.

[0047] Among them, the same time reference source can be an atomic clock chip trained by Beidou / GPS satellites, which can provide microsecond-level accurate time reference. Terminal devices can directly write time data during the raw RAW frame output stage of the image sensor, avoiding operating system scheduling delay.

[0048] In some embodiments of this application, the upload protocol for multiple video streams can be optimized. When a terminal device uploads multiple video streams to a server, it can use the Secure Reliable Transport (SRT) protocol to ensure transmission reliability.

[0049] In step 130, the server identifies the device motion state of the main device based on sensor data, detects the synchronization between the image motion state of the target evidence video stream corresponding to the same timestamp and the device motion state, and compares and analyzes the synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and the physical motion trajectory determined by the sensor data, performs video synchronization verification on multiple video streams, and obtains the synchronization verification result.

[0050] In step 130, the synchronization between the image motion state (e.g., pixel displacement vector calculated by optical flow) of the target evidence video stream at the same timestamp and the device motion state (e.g., device displacement vector obtained by accelerometer integration) is detected. The synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process (obtained by tracking the image position of the main device in the auxiliary device video) and its physical motion trajectory (calculated from sensor data) is compared and analyzed.

[0051] Both tests are based on the principle of rigid physical motion (the movement of the device will inevitably cause the image it captures and the device itself to produce regular motion in the image captured by other devices). If both tests pass, a "physical verification result" representing physical synchronization is generated.

[0052] The following explains the synchronization verification process between the target evidence video stream and the main device: Specifically, the device motion state refers to the physical motion pattern analyzed in real time by the nine-axis sensor built into the main device. This can include events such as being stationary, moving, rotating, and falling. The nine-axis sensor can include a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. When the main device undergoes physical movement, the video stream of the target evidence captured by the main device will inevitably follow the laws of rigid motion, optical imaging principles, and relative motion laws. Therefore, the video stream of the target evidence will present a picture motion state synchronized with the device motion state.

[0053] For example, when an evidence collector runs and chases a target while holding the main device, the device will sway horizontally and bounce vertically as the arm swings. The motion state of the video stream of the target evidence should be as follows: the video screen will sway left and right in sync with the horizontal swaying, and shake up and down in sync with the vertical bounce. If the device suddenly turns (such as a sharp 90° turn), the screen will rotate rapidly horizontally, and the edges of the scene will be blurred, which is consistent with rotational inertia.

[0054] This application can detect the synchronization between the motion state of the target evidence video stream and the motion state of the device based on the following rules: the actual movement direction of the main device is opposite to the displacement direction of the screen; the screen motion state shakes synchronously when the main device shakes; the screen motion state rotates synchronously when the main device rotates.

[0055] In some embodiments of this application, step 130 above, which identifies the device motion state of the main device based on sensor data and detects the synchronization between the motion state of the target evidence video stream corresponding to the same timestamp and the device motion state, may specifically include the following steps 1301-1303: Step 1301: When a device movement event is identified based on the accelerometer data in the sensor data, the first matching result is obtained by verifying the matching between the image displacement vector of the target evidence video stream at the same timestamp and the device displacement vector corresponding to the accelerometer data.

[0056] Specifically, the accelerometer data is triaxial acceleration data, and the image displacement vector is the average image displacement vector generated using a dense optical flow algorithm with the frame center region as a reference. The image displacement vector can include at least the image displacement direction and the image displacement amplitude. The server can determine the device displacement vector based on the accelerometer data and gyroscope data. The device displacement vector can include at least the device displacement direction and the device displacement amplitude.

[0057] When the device displacement direction is opposite to the screen displacement direction, and the screen displacement amplitude in the same coordinate system matches the device displacement amplitude, the screen displacement vector and the device displacement vector are matched to obtain the first matching result used to characterize the synchronization between the screen motion state and the device motion state when the device moves.

[0058] Matching the image displacement amplitude with the device displacement amplitude can be achieved by the amplitude deviation between the image displacement amplitude and the device displacement amplitude being less than a preset amplitude deviation threshold, or by the amplitude ratio between the image displacement amplitude and the device displacement amplitude approaching 1 and falling within a preset numerical range, such as [0.95, 1.05], or by the correlation coefficient between the changing trends of the image displacement amplitude and the device displacement amplitude within the same time window being higher than a preset correlation threshold, such as higher than 0.9 or 0.95. Both the preset amplitude deviation threshold and the preset numerical range can be set according to specific needs, and this application does not impose specific limitations on them.

[0059] Device displacement amplitude is the physical space displacement directly measured by sensors. It is based on the sensor coordinate system (such as the XYZ axis of an IMU), and the unit is millimeters or meters. It belongs to the object space. On the other hand, image displacement amplitude is the pixel displacement extracted through image processing (such as optical flow or template matching). It is based on the image coordinate system (such as the UV axis), and the unit is pixels. It belongs to the image space.

[0060] Therefore, before determining whether the image displacement amplitude and the device displacement amplitude match, they need to be unified to the same coordinate system.

[0061] In some embodiments, the device displacement direction and the screen displacement direction can be verified first. If the device displacement direction and the screen displacement direction are opposite, the device displacement amplitude and the screen displacement amplitude can be converted to the same coordinate system based on the calibration coefficient, and then the amplitude deviation value between the screen displacement amplitude and the device displacement amplitude can be calculated.

[0062] The calibration coefficient is a conversion factor between pixel and physical size determined by a calibration board or a reference object of known size. Coordinate system transformation is performed by multiplying the calibration coefficient by the device displacement amplitude or the image displacement amplitude. When the calibration coefficient is the object-space size / image-space size, the image displacement amplitude of the pixel size is multiplied by the calibration coefficient and transformed to the sensor coordinate system to obtain the image displacement amplitude of the physical size. When the calibration coefficient is the image-space size / object-space size, the device displacement amplitude of the object-space size is multiplied by the calibration coefficient and transformed to the image coordinate system to obtain the device displacement amplitude of the pixel size.

[0063] Step 1302: When a device rotation event is detected based on gyroscope data in the sensor data, a second matching result is obtained by comparing the angle deviation between the screen rotation angle of the target evidence video stream at the same timestamp and the device rotation angle corresponding to the gyroscope data.

[0064] Specifically, optical flow methods, such as Farneback dense optical flow or sparse optical flow, are used to extract inter-frame motion vector fields from the target evidence video stream. Then, the average rotation angle is calculated based on the center region of the image to obtain the image rotation angle. After the server performs low-pass filtering on the gyroscope data to remove high-frequency noise, it performs time integration on the filtered angular velocity data to generate an absolute rotation angle value around the Z-axis, thus obtaining the device rotation angle. If the angular deviation between the image rotation angle and the device rotation angle in the same coordinate system is less than a preset angular deviation threshold, a second matching result is obtained to characterize the synchronization between the image motion state and the device motion state during device rotation. This preset angular deviation threshold can be set according to requirements, such as 10°, 20°, or other values. This application does not specifically limit this value.

[0065] Before comparing the screen rotation angle of the target evidence video stream at the same timestamp with the corresponding device rotation angle of the gyroscope data, it is necessary to convert the screen rotation angle and the device rotation angle to the same coordinate system.

[0066] Specifically, the device rotation angle in the sensor coordinate system can be converted to the image coordinate system using formula (1).

[0067] θ2=(360°-θ1)mod360° (1) Where θ1 is the device rotation angle in the sensor coordinate system and θ2 is the device rotation angle in the image coordinate system.

[0068] It should be noted that, theoretically, the angle deviation between the screen rotation angle and the device rotation angle is zero, but there are gyroscope noise and optical flow calculation errors. Therefore, the preset angle deviation threshold can be set to a value greater than zero.

[0069] Step 1303: Based on the first matching result and the second matching result, determine the synchronization between the motion state of the image and the motion state of the device in the target evidence video stream.

[0070] Specifically, when the first matching result indicates that the motion state of the screen is synchronized with the motion state of the device when the device moves, and when the second matching result indicates that the motion state of the screen is synchronized with the motion state of the device when the device rotates, it is determined that the motion state of the target evidence video stream is synchronized with the motion state of the device.

[0071] In this embodiment, a synchronous verification mechanism based on the principle of physical immutability is constructed by dual motion consistency verification of accelerometer and gyroscope data and video footage. This scheme is based on the principle of rigid motion constraints (i.e., the physical motion of the device inevitably causes its camera footage to show a reversed and proportionally coordinated motion trajectory). It requires that the device displacement vector generated by the accelerometer data and the image displacement vector extracted by the video optical flow method at the same timestamp strictly match in amplitude error and direction consistency. Simultaneously, it requires that the deviation between the device rotation angle calculated by the gyroscope data and the image rotation angle meet certain conditions. This collaborative verification of multimodal sensors and video data ensures that any tampering with a single video stream (such as inserting static frames, forging displacement or rotation effects) will disrupt the consistency of physical motion, thus being accurately identified and located. Attackers would need to simultaneously forge sensor data chains, coordinate modifications to multiple video streams, and accurately simulate physical motion laws to bypass detection, exponentially increasing the difficulty of tampering. This design transforms the verification of the authenticity of electronic evidence from the traditional reliance on external agency certification to an internal verification scheme based on the physical correlation of multi-source information, thereby improving the original authenticity, process integrity, and final acceptance of evidence in judicial scenarios.

[0072] In other embodiments of this application, in order to further improve the accuracy of verifying the authenticity of the target evidence video stream, the above-mentioned verification of the matching between the image displacement vector of the target evidence video stream at the same timestamp and the device displacement vector corresponding to the accelerometer data may include the following steps: Using the rate of change of kinetic energy of the device displacement vector and the amplitude variance of the image displacement vector as energy indicators, the kinetic energy change rate sequence of the main device and the amplitude variance sequence of the target evidence video stream are obtained. Calculate the correlation coefficient between the kinetic energy change rate sequence and the amplitude variance sequence. If the correlation coefficient exceeds the preset correlation threshold, then the image displacement vector and the device displacement vector are matched.

[0073] Among them, the rate of change of kinetic energy is used to characterize the acceleration / deceleration trend of the motion state, the amplitude variance is used to characterize the fluctuation intensity of the image amplitude sequence around the average value. The larger the variance, the stronger the data dispersion and the worse the stability. The correlation coefficient ranges from negative one to positive one.

[0074] In some embodiments, the correlation coefficient can be the Pearson correlation coefficient. The above calculation of the correlation coefficient between the kinetic energy change rate sequence and the amplitude variance sequence may include: calculating the covariance of the kinetic energy change rate sequence and the amplitude variance sequence respectively; dividing the covariance value by the product of the standard deviations of the two sequences to obtain the correlation coefficient with a value range from negative one to positive one.

[0075] Thus, the rate of change of kinetic energy reflects the change of motion state from the perspective of energy dynamics, while the amplitude variance quantifies the fluctuation characteristics from the perspective of data stability. Combining the two can effectively detect the consistency of motion state between the screen motion state and the device motion state in the physical vision dimension, so as to accurately identify the motion inconsistency caused by video tampering.

[0076] The following describes the synchronization verification process between the video stream and the main device during the evidence collection process: Specifically, the physical motion trajectory refers to the actual movement path of the main device in three-dimensional physical space. This trajectory is calculated from data from sensors built into the main device (such as an inertial measurement unit, IMU) and is usually represented by a sequence of displacement, velocity, or acceleration changes over time. The image motion trajectory refers to the two-dimensional visual motion path of the main device as the target in the video stream of the evidence collection process captured by auxiliary equipment. This trajectory is obtained by analyzing the changes in the image position of the main device in consecutive video frames (such as using optical flow methods or target tracking algorithms). The physical verification result refers to the conclusion on whether multiple video streams are real and synchronized by comparing the synchronization between the physical motion trajectory and the image motion trajectory (such as the consistency of direction, velocity, and acceleration), and comparing the synchronization between the motion state of the target evidence video stream and the motion state of the device.

[0077] The process of extracting the image motion trajectory may include: presetting high-contrast marker points on the surface of the main device as reference features for motion tracking in video frames; performing feature point matching on consecutive video frames and calculating the pixel displacement and rotation angle of the marker points between adjacent frames; recording the pixel displacement and rotation angle of the marker points in all frames of the video stream during the evidence collection process in chronological order, and converting the pixel displacement and rotation angle to the sensor coordinate system (or physical coordinate system) through calibration parameters to generate the image motion trajectory of the main device.

[0078] The video stream of the evidence collection process records the entire process of the evidence collection entity using the main device to collect evidence from the target. Therefore, the actual movement trajectory of the main device will be presented in the video stream of the evidence collection process, and there will be similarity with the image movement trajectory in the video stream of the evidence collection process. For example, when the main device moves to the right relative to the evidence collection entity, if the video stream of the evidence collection process is not tampered with, the video stream of the evidence collection process at the same timestamp will show the image of "the main device moving to the right relative to the evidence collection entity".

[0079] Based on this, the comparative analysis of the synchronization between the image motion trajectory and the physical motion trajectory of the main device in the video stream during the evidence collection process can include: comparing the consistency of the geometric features of the corresponding trajectories of the image motion trajectory and the physical motion trajectory of the main device; and comparing the temporal consistency of all key motion nodes in the image motion trajectory and the physical motion trajectory of the main device; if the consistency of both exceeds a preset threshold, a result representing the synchronization between the image motion trajectory and the physical motion trajectory of the main device is generated.

[0080] The trajectory geometric features may include, but are not limited to: trajectory morphology features, velocity-acceleration curve morphology features, and key motion nodes being events that cause sudden changes in the motion state of the main equipment, such as sudden rotation, abrupt stop, sudden acceleration, or sudden angular velocity. This preset threshold can be set according to specific needs, for example, to 0.92, 0.93, or other values; this application does not impose specific limitations on it.

[0081] In this embodiment, by comparing the geometric features of the image motion trajectory and the physical motion trajectory of the main device, the similarity of their motion trajectory shapes can be accurately determined. By comparing the temporal consistency of key motion nodes in the image motion trajectory and the physical motion trajectory of the main device, temporal alignment verification can be performed. Combining the two similarity judgment results, the similarity between the image motion trajectory and the physical motion trajectory of the main device can be effectively identified. This similarity judgment result can then be used to accurately verify the synchronization between the video stream and the main device during the evidence collection process, thereby improving the accuracy of identifying video tampering behavior in the video stream during the evidence collection process.

[0082] In some embodiments, comparing the consistency of the geometric features of the image motion trajectory and the physical motion trajectory of the main device may include: extracting key points of the motion trajectory of the main device (such as start point, end point, inflection point) from the image motion trajectory; extracting the device displacement vector and device rotation angle at the corresponding time point from the physical motion trajectory; calculating the geometric overlap of the two trajectories (such as the deviation of the trajectory inflection point position ≤ 5%); calculating the consistency of the motion direction (such as the angle deviation ≤ 10°); if both the overlap and the direction consistency meet the threshold, the trajectory is determined to be consistent in geometric features.

[0083] In some embodiments, comparing the temporal consistency of all key motion nodes in the image motion trajectory and the physical motion trajectory of the main device may include: for the same key motion node, extracting the first start timestamp and the second start timestamp corresponding to the key motion node from the image motion trajectory and the physical motion trajectory, respectively, as well as the first duration and the second duration corresponding to the key motion node; determining the start time difference based on the first start timestamp and the second start timestamp, and determining the duration difference based on the first duration and the second duration; determining the timing verification result corresponding to the key motion node based on the start time difference and / or the duration difference; and determining the target timing verification result of the image motion trajectory and the physical motion trajectory based on the timing verification results of all key motion nodes.

[0084] Wherein, the first start timestamp is the starting time point of the key motion node in the image motion trajectory, and the second start timestamp is the starting time point of the key motion node in the physical motion trajectory. The first duration is the duration of the key motion node in the image motion trajectory, and the second duration is the duration of the key motion node in the physical motion trajectory. The time difference between the first start timestamp and the second start timestamp is the start time difference, and the difference between the first duration and the second duration is the duration difference. If the start time difference is less than a preset time difference threshold, and / or the duration difference is less than a preset duration threshold, then the timing verification result is determined to be that the key motion node is time-aligned in the image motion trajectory and the physical motion trajectory.

[0085] Based on the timing verification results of all key motion nodes, the timing verification results of the image motion trajectory and the physical motion trajectory are determined. This can be done by determining the target timing result as the sequential timing alignment of the image motion trajectory and the physical motion trajectory when the timing verification results of all key motion nodes indicate that the event timing is aligned; or by counting the number of key motion nodes whose timing verification results indicate that the event timing is not aligned, obtaining the number of outliers, comparing the ratio of the number of outliers to the total number of key motion nodes with a preset error ratio, and determining the target timing result as the sequential timing alignment of the image motion trajectory and the physical motion trajectory when the ratio does not exceed the preset error ratio. The preset error ratio can be set according to specific needs, such as 0.1, 0.12, or 0.15, etc., and this application does not impose any specific limitations on it.

[0086] In this embodiment of the application, by verifying the time synchronization between the main device and the video stream of the evidence collection process for the same key motion node, the similarity results between the physical motion trajectory and the image motion trajectory can be obtained.

[0087] By comparing and analyzing the time synchronization of the video stream of the evidence collection process and the video stream of the target physical evidence corresponding to the same timestamp, it is possible to perform time alignment verification on multiple video streams to determine whether the two video streams are synchronized. For example, if the audio of multiple video streams with the same timestamp is not synchronized, it means that the two video streams are not synchronized in time.

[0088] Since multiple video streams are recorded in the same environment and scene, multiple audio streams share the same real physical sound field, and their audio stream data should be synchronized and consistent. If a tamperer modifies a video frame, the corresponding audio must be modified simultaneously to maintain audio-visual synchronization within that video stream, but this modification will inevitably disrupt the synchronization of the multiple audio streams.

[0089] Based on this, in order to effectively identify the audio synchronization of multiple video streams, in some embodiments of this application, step 130 may specifically include: Based on sensor data, the device motion state of the main device is identified, and the synchronization between the motion state of the target evidence video stream corresponding to the same timestamp and the device motion state is detected. The synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and its physical motion trajectory is compared and analyzed to obtain the physical verification result. By comparing and analyzing the time synchronization between the video stream of the evidence collection process and the video stream of the target physical evidence corresponding to the same timestamp, the time synchronization results are obtained. If both the physical verification result and the time verification result pass, a synchronization verification result is obtained, which indicates that the synchronization verification of the multi-channel video streams has passed.

[0090] In the above embodiments, comparing and analyzing the time synchronization between the video stream of the evidence collection process and the video stream of the target physical evidence corresponding to the same timestamp to obtain the time synchronization result can specifically include the following steps: The audio signal of the video stream during the evidence collection process is extracted from the server as the first audio stream, and the audio signal of the video stream of the target physical evidence is extracted as the second audio stream. The cross-correlation function of the first audio stream and the second audio stream is determined, and if the peak position and peak amplitude in the cross-correlation function both meet the preset audio synchronization conditions, the audio signals of the multiple video streams are determined to be consistent.

[0091] The cross-correlation function is used to quantify the temporal similarity between the first and second audio streams. The peak position represents the time difference after alignment of the two audio streams, and the peak amplitude reflects the signal homology. The peak position, representing the time difference between the two audio streams under optimal alignment, can be converted to time (seconds) using the sampling rate. The smaller the time difference represented by the peak position, the higher the signal synchronization. The closer the peak amplitude is to 1, the higher the signal synchronization and the stronger the homology; a value close to 0 indicates no correlation. Based on this, this application can set preset audio synchronization conditions including: the time difference represented by the peak position is less than a preset time difference, and the peak amplitude is greater than a preset amplitude. Both the preset time difference and the preset amplitude can be set according to specific needs, and this application does not impose specific limitations on them. When the audio signals of multiple video streams are determined to be consistent, it indicates that the multiple video streams have time synchronization.

[0092] Thus, by determining whether the video stream of the evidence collection process is synchronized with the video stream of the target physical evidence through the peak position, and by determining whether the video stream of the evidence collection process and the video stream of the target physical evidence are from the same source and can be synchronized through the peak amplitude, combining the two can accurately perform audio synchronization verification on multiple video streams, thereby precisely identifying cases where local video frames have been tampered with. For example, when an attacker tampers with a specific frame in the target physical evidence video stream and simultaneously modifies the corresponding audio segment, replacing key physical evidence video frames and ambient sounds, the abnormal time period of audio asynchrony between multiple video streams can be accurately identified, and the tampering of local video frames can be detected. In this way, based on the mutual verification process between the corresponding audio signals of multiple video streams, the tampered video frames can be accurately located, providing a quantifiable anti-tampering evidence chain for forensic identification.

[0093] In some embodiments of this application, to further improve the accuracy of synchronization verification of multiple video streams, step 130 may specifically include: Based on sensor data, the device motion state of the main device is identified, and the synchronization between the motion state of the target evidence video stream corresponding to the same timestamp and the device motion state is detected. The synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and its physical motion trajectory is compared and analyzed to obtain the physical verification result. The visual verification result is obtained by detecting the matching of visual events in the video stream of the target physical evidence corresponding to the same timestamp and the video stream of the evidence collection process. If both the physical verification result and the visual verification result pass, a synchronization verification result is obtained, which indicates that the synchronization verification of the multi-channel video streams has passed.

[0094] Specifically, based on the physical consistency check, further detection is performed to check whether there are matching visual events in the target evidence video stream and the evidence collection process video stream corresponding to the same timestamp (or adjacent time windows). For example, at the moment when handwriting begins to appear in the target evidence video stream, the action of the evidence collector writing should be observable in the evidence collection process video stream. This check focuses on the visual logical correlation between the evidence collection behavior and the changes in the state of the evidence. Only when both the physical check result (proving that the video stream acquisition is synchronized and has not been tampered with in terms of motion patterns) and the visual check result (proving that the video stream content is consistent in the evidence collection logic) pass, a synchronization check result is generated to characterize the synchronization of multiple video streams.

[0095] In this way, the server performs cross-verification from two dimensions: physical motion laws and visual content logic. By combining physical and visual verification results, it can identify the synchronization of multiple video streams in multi-dimensional scenarios, significantly reducing the risk of misjudgment that may exist with a single verification mechanism (such as physical synchronization with content tampering by advanced techniques), making the synchronization verification results more reliable. For an attacker to successfully tamper with the video stream undetected, they not only need to simultaneously forge the physical motion trajectories of multiple video streams, but also ensure that the forged visual content perfectly matches the event logic (such as the forged handwriting appearing at the exact same time as the forged writing action). This further greatly increases the technical threshold and cost of tampering, enhancing the tamper resistance of electronic evidence. If any type of verification fails after the attacker tampers with the video stream, the server can effectively identify the tampering behavior. This mechanism not only verifies the temporal synchronization of the video streams but also verifies the correlation between the forensic behavior and the result in visual evidence, providing stronger technical proof for the authenticity and completeness of the evidence collection process.

[0096] In some embodiments of this application, the above-mentioned detection of the matching of visual events in the target evidence video stream and the evidence collection process video stream corresponding to the same timestamp may include: For multiple video streams corresponding to the same timestamp, the physical evidence area where the evidence target is located is located by the target detection model, and the evidence state feature vector is extracted from the physical evidence area. The similarity of the physical evidence state feature vectors of multiple video frames is calculated. If the similarity exceeds the preset similarity threshold, the visual event matching verification of the multiple video streams is determined to be successful.

[0097] In this context, the evidence region refers to the local area in a video frame image that contains the target to be examined (such as a contract, document, or item), identified through object detection technology. It is typically represented by a bounding box or pixel mask. The evidence state feature vector refers to the set of feature values ​​extracted from the evidence region to quantify the state of the evidence at a specific moment. This vector usually contains various visual features that characterize key state information such as the appearance, shape, and textual content of the evidence. These features may include, but are not limited to: contour features: features describing the outline of the evidence (such as Fourier descriptors and Hu moments); texture features: features describing the surface texture of the evidence (such as Local Binary Pattern (LBP) and Gray-Level Co-occurrence Matrix (GLCM) features); color features: features describing the color distribution of the evidence (such as color histograms); and depth features: high-level semantic features extracted from the evidence region using deep learning models such as Convolutional Neural Networks (CNNs).

[0098] Specifically, for the target evidence video stream and the evidence collection process video stream corresponding to the same timestamp, pre-trained target detection models (such as YOLO, SSD, Faster R-CNN) are used to process the video frames to accurately locate the evidence area where the evidence target (evidence) is located. Traditional image processing algorithms or deep learning feature extractors are used to extract evidence state feature vectors from the evidence area located in each video frame. The similarity between the two evidence state feature vectors extracted from the target evidence video stream and the evidence collection process video stream at the current timestamp is calculated (commonly cosine similarity or the reciprocal of Euclidean distance, etc.). If the similarity value exceeds a preset similarity threshold (e.g., 0.85 or higher), it is considered that the evidence state presented by the two video streams at the current moment is highly consistent, and the visual event matching verification passes.

[0099] Thus, the feature vector of the tampered evidence area will be significantly different from the feature vector in the real video stream. Therefore, by calculating the similarity between the feature vectors of the evidence state in the evidence area, it is possible to effectively identify tampering methods such as the replacement of evidence with static images or the malicious modification of evidence content (such as PS), thereby enhancing the detection capability of content replacement and tampering.

[0100] In other embodiments of this application, the above-mentioned detection of the matching of visual events in the target evidence video stream and the evidence collection process video stream corresponding to the same timestamp may include: Based on the real-time received target physical evidence video stream, detect physical evidence status change events in the target physical evidence video stream and record event timestamps; The system detects whether there is a visual event in the target evidence video stream under the same event timestamp that matches the event of change in the state of the evidence. If such an event exists, the visual event matching verification of the multiple video streams passes.

[0101] Among them, the event of a change in the state of physical evidence refers to the moment or time period during which the state of the target evidence (physical evidence) changes significantly in the video stream of the target evidence. Such changes may include, but are not limited to: the physical evidence appearing in the frame for the first time, the physical evidence being picked up / put down, new handwriting appearing on the physical evidence, the physical evidence being stamped, the physical evidence being folded / unfolded, and changes in the content of the electronic physical evidence screen. This event is usually accompanied by a significant change in the image pixel values.

[0102] Specifically, the server can analyze the received target evidence video stream in real time, use specific algorithms (such as inter-frame difference method, background subtraction method, optical flow method combined with threshold judgment, or more complex event detection neural network) to detect the occurrence of evidence state change events, and accurately record the event timestamp t_event of the event.

[0103] Then, cross-video stream event correlation verification is performed: In the video stream of the evidence collection process captured by the auxiliary equipment, it is searched for a visual event that logically matches the aforementioned evidence status change event at the same time t_event (or within a very small time tolerance window around t_event, considering transmission and processing delays). For example, if a signature is detected on the contract at time t_event in the target evidence video stream (evidence status change event), then the action of the evidence collector writing a signature should be detectable near time t_event in the evidence collection process video stream (matching visual event).

[0104] If a visual event is found in the video stream of the evidence collection process that is temporally aligned and logically matched with the event of change in the state of the target physical evidence in the video stream, then the visual event matching verification is deemed to have passed.

[0105] Thus, comprehensive feature extraction and comparison of every single video frame is unnecessary. Cross-video stream matching verification is triggered only when key event points are detected, reducing computational overhead and facilitating real-time or near-real-time verification. This verification scheme directly focuses on critical moments when the state of physical evidence changes. These moments are often the most important and verifiable stages in the evidence collection process, making the verification work more targeted and effective. By verifying the high temporal synchronicity and logical causal relationship between "changes in the state of physical evidence" and "operations of the evidence collectors," the correlation between the evidence collection actions and the resulting changes in physical evidence is strengthened, making the chain of evidence more complete and credible. Furthermore, this scheme can effectively detect tampering by inserting or deleting key event frames (such as deleting several frames of the signing process, leaving only the static image of the completed signature), as this would disrupt the temporal correspondence between the two video streams.

[0106] In some embodiments of this application, the server can automatically switch the error tolerance between macro / normal / long-range modes according to the object distance, and intelligently adjust the rotation angle threshold according to the device's movement speed.

[0107] In some embodiments of this application, after synchronous video capture is performed via at least two terminal devices in response to an evidence collection instruction, the method may further include: During video acquisition, the terminal device monitors the evidence collection process video stream in real time and outputs a prompt message when the distance between the main device or the evidence collection target and the reference point of the evidence collection screen reaches a set value, so that the evidence collection process video stream records the evidence collection behavior of the evidence collection subject using the main device to collect evidence from the evidence collection target throughout the process.

[0108] The evidence-gathering screen refers to the real-time image stream captured by auxiliary equipment (such as another mobile phone or a fixed camera) during the evidence-gathering process. The core purpose of this screen is to record the entire process of the evidence-gathering entity (such as law enforcement personnel) using the main device (such as the mobile phone recording the evidence) to operate the evidence-gathering target (such as a contract or electronic screen). Reference points refer to one or more predefined reference points in the evidence-gathering screen used for spatial location determination; these can be the center point or boundary points of the screen. The center point typically refers to the geometric center of the screen or the visual center point calculated based on the content weight of the screen. Boundary points refer to points on the four edges of the screen, often used to determine whether the target is about to move out of the frame. In practice, to simplify calculations, the midpoint of each edge can be selected as a representative boundary point.

[0109] Specifically, when the distance between the main device or the evidence collection target and the reference point of the center point is greater than the first preset distance, and / or the distance between the main device or the evidence collection target and the reference point of the boundary point is less than the second preset distance, it is determined that there is a risk of going out of the frame, and a prompt message is output. This prompt message is used to guide the evidence collection execution subject to adjust the position of the device or target in real time to ensure that the entire evidence collection behavior is recorded completely and clearly.

[0110] In this way, by calculating the distance between key targets and the frame reference point in real time, an early warning can be issued before the target moves out of the frame or deviates from the core area. This effectively avoids the problem of key evidence being missed due to improper operation, ensuring the integrity and usability of the video stream of the "evidence collection process" and guaranteeing the integrity of the evidence collection process record. This mechanism provides real-time, visual feedback to the evidence collection entity, guiding them to follow standardized shooting angles and distances, thereby directly improving the image quality and evidentiary value of the final "evidence collection process video stream" and reducing evidentiary flaws caused by poor image composition.

[0111] In step 140, after the multiple video streams are uploaded, the server combines the timestamps of the multiple video streams and sensor data according to the time sequence to obtain the original evidence data packet, and compares the initial hash value corresponding to each video stream with the verification hash value to obtain the hash verification result.

[0112] In step 140, the initial hash value is the hash value generated by the terminal device based on the originally acquired video stream. This refers to the hash value calculated immediately after the terminal device finishes acquiring the video stream but before uploading (e.g., target evidence video stream, evidence collection process video stream). This value represents the "original fingerprint" of the video stream at its initial generation, before any transmission or processing, and serves as the benchmark reference value for subsequent verification. The verification hash value is the hash value generated by the server based on the received video stream. This refers to the hash value recalculated by the server after successfully receiving each video stream data uploaded by the terminal device, using the exact same hash algorithm as the terminal device. This value is compared with the initial hash value sent by the terminal device to verify whether the data remained consistent during transmission. The server merges multiple video frames and sensor data into a structured data packet according to the timestamp.

[0113] In some embodiments of this application, the initial hash value corresponding to each video stream is compared with the verification hash value to obtain a hash verification result, including: if the initial hash value corresponding to each video stream in the multiple video streams is consistent with the verification hash value, a hash verification result indicating successful hash verification is obtained.

[0114] Specifically, the server performs a string-level precise comparison between the initial hash value (obtained from the terminal) and the verification hash value (calculated by itself) corresponding to each video stream. Only when the initial hash value and verification hash value of all video streams (such as the target evidence video stream and the evidence collection process video stream) are completely consistent does the server determine that the overall hash verification is successful and generate a hash verification result to indicate that it has passed. If the two hash values ​​of any video stream do not match, the hash verification is immediately determined to have failed.

[0115] Thus, compared to verifying only the hash value of a single video stream, dual hash verification based on multiple video streams further increases the difficulty of tampering with the original evidence data packets. This mechanism can sensitively detect unexpected errors (such as network packet loss, data corruption) or malicious tampering (such as man-in-the-middle attacks, data packet tampering) that may occur during the transmission of video stream data from the terminal device to the server. Even the slightest change in data will result in a completely different hash value, thus ensuring the integrity of electronic evidence during transmission. Furthermore, each video stream must pass its own hash verification, providing independent and robust technical proof of the integrity of each video evidence stream. This avoids the situation where a single damaged or tampered video stream "contaminates" the overall evidence packet, meeting the requirement that each link in the judicial evidence chain must be reliable. The "hash verification result," combined with physical verification results such as "time synchronization" and "consistency of image motion state," constitutes a multi-dimensional and comprehensive anti-tampering protection system for electronic evidence. Hash verification focuses on ensuring the integrity of data during static storage and transmission, while physical verification focuses on verifying the authenticity and logic of video content during dynamic acquisition. The two complement each other, effectively enhancing the overall credibility and judicial acceptance of electronic evidence.

[0116] In step 150, the synchronization verification result and hash verification result are written into the original evidence data packet to generate the target evidence data packet, and the target evidence data packet is uploaded to the timestamp authentication system for trusted timestamp evidence solidification to generate tamper-proof electronic evidence files.

[0117] In step 150, the target forensic data packet is a structured data packet that integrates multiple video streams, sensor data, synchronization verification results, and hash verification results. The trusted timestamp evidence is solidified by a timestamp authentication system digitally signing the hash value of the target forensic data packet and binding it with a timestamp. The tamper-proof electronic evidence file is an unmodifiable file containing TSA credentials, a digital signature, and complete evidence data. Only when both the synchronization verification result and the hash verification result pass can the authenticity and validity of the multiple video streams in the target forensic data packet be proven.

[0118] In some embodiments of this application, uncompressed images of the evidence collection process video stream and the target physical evidence video stream can be simultaneously displayed using picture-in-picture technology. The main image is the target physical evidence video stream, showing the evidence collector's actions regarding the evidence. The picture-in-picture image is the evidence collection process video stream, which displays the scene environment and the activities of other personnel. In this way, when reviewing video evidence, one can simultaneously see video content from two different perspectives, providing a more intuitive understanding of various information during the evidence collection process. For example, when reviewing video evidence involving human activities and the display of physical evidence, the main screen can show the evidence collector's filming of the physical evidence, while the picture-in-picture shows the environment at the scene and the activities of other people. This presentation method helps judicial personnel to understand the content of the evidence more comprehensively and accurately, thereby enhancing the probative value of the evidence.

[0119] Corresponding to the method embodiments of this application, this application also provides a system for collaborative evidence collection based on multiple devices, which is an evidence collection system.

[0120] Figure 2 This is a schematic diagram of the structure of a system for collaborative forensics based on multiple devices, as provided in an embodiment of this application. Figure 2 As shown, the system 200 based on multi-device collaborative evidence collection may include: at least two terminal devices 210, a server 220, and a timestamp authentication system 230.

[0121] Among them, at least two terminal devices 210 are used to respond to evidence collection instructions, perform synchronous video acquisition, and acquire sensor data of the main device in real time. At least one main device captures the evidence target to obtain a target evidence video stream, and at least one auxiliary device captures the evidence collection process to obtain an evidence collection process video stream. The target evidence video stream and the evidence collection process video stream constitute a multi-channel video stream. During video acquisition, the terminal devices 210 are used to embed timestamps generated by the same time reference source into the video frames and sensor data in the multi-channel video streams, and upload the multi-channel video streams and sensor data to a server. The server 220 is used to identify the device motion state of the main device based on sensor data, detect the synchronization between the image motion state of the target evidence video stream corresponding to the same timestamp and the device motion state, and compare and analyze the main device's motion in the evidence collection process video stream. The synchronization between the image motion trajectory and the physical motion trajectory determined by the sensor data is used to perform video synchronization verification on multiple video streams to obtain a synchronization verification result. After the multiple video streams are uploaded, the server 220 is also used to combine the multiple video streams and sensor data corresponding to the timestamps according to the time sequence to obtain the original evidence data packet, and compare the initial hash value corresponding to each video stream with the verification hash value to obtain the hash verification result. The initial hash value is the hash value generated by the terminal device based on the original acquired video stream, and the verification hash value is the hash value generated by the server based on the received video stream. The server 220 is also used to write the synchronization verification result and the hash verification result into the original evidence data packet to generate the target evidence data packet, and upload the target evidence data packet to the timestamp authentication system for credible timestamp evidence solidification to generate tamper-proof electronic evidence files.

[0122] The system for collaborative evidence collection based on multiple devices provided in this application embodiment records the evidence collection target and the evidence collection behavior process targeting the target using at least two terminal devices. Through synchronous and collaborative recording by at least two terminal devices and the combination with sensor data, the system can present the operational behavior trajectory of the evidence collection subject, constructing an evidence collection behavior chain associated with the evidence collection process, thereby enhancing the integrity and relevance of the evidence. Furthermore, by deeply integrating and analyzing different types of data, the system can display the links and details in the evidence collection process in detail, providing judicial personnel with richer and more accurate evidence information. Simultaneously, multiple video streams and sensor data can form mutually corroborating evidence chains. Therefore, this application can accurately and effectively verify the authenticity of electronic evidence formed by multiple video streams based on sensor data. Specifically, if the motion state of the target physical evidence video stream is synchronized with the device motion state, and the image motion trajectory of the main device in the evidence collection process video stream is synchronized with its physical motion trajectory, it indicates that both individual video streams match the sensor data, and the multiple video streams are synchronized. In this way, when an attacker tampers with one of the video streams, the tampering behavior targeting a single video stream can be accurately identified. Considering the high difficulty for attackers to simultaneously tamper with the video streams of the evidence collection process and the target physical evidence, this method can increase the difficulty of tampering from the source, reduce the risk of video tampering, and enhance the credibility and reliability of electronic evidence.

[0123] In some embodiments of this application, the terminal device is further configured to: after synchronous video acquisition through at least two terminal devices in response to an evidence acquisition instruction, during the video acquisition process, monitor the evidence acquisition screen corresponding to the video stream of the evidence acquisition process in real time through the terminal device, and output a prompt message when the distance between the main device or the evidence acquisition target and the reference point of the evidence acquisition screen reaches a set value, so that the video stream of the evidence acquisition process records the evidence acquisition behavior of the evidence acquisition execution subject using the main device to acquire evidence of the evidence acquisition target throughout the process.

[0124] In some embodiments of this application, there is one main device and one auxiliary device. The forensic system further includes: a trusted forensic application capable of communicating with a server; multiple video streams are uploaded to the server through the trusted forensic application; at least two terminal devices; and is further configured to: establish a communication connection between the main device and the auxiliary device before synchronous video acquisition through at least two terminal devices, and calibrate the time base of the main device and the auxiliary device based on a time synchronization protocol to achieve clock synchronization between the devices; and, when both the main device and the auxiliary device have the trusted forensic application installed, establish an end-to-end data transmission channel between the main device and the server to obtain a first transmission channel, and an end-to-end data transmission channel between the auxiliary device and the server to obtain a second transmission channel. The system uses a second transmission channel. The first transmission channel is used for real-time transmission of the target evidence video stream and sensor data from the main device, while the second transmission channel is used for real-time transmission of the evidence collection process video stream from the auxiliary device. If the main device has a trusted evidence collection application installed, but the auxiliary device does not, an end-to-end data transmission channel is established between the main device and the server to obtain the first transmission channel, and an end-to-end data transmission channel is established between the main device and the auxiliary device to obtain the third transmission channel. The third transmission channel is used to transmit the evidence collection process video stream from the auxiliary device to the main device in real-time, while the first transmission channel is used to transmit the target evidence video stream and sensor data from the main device in real-time and forward the evidence collection process video stream from the auxiliary device to the server.

[0125] In some embodiments of this application, the server is specifically used to: when a device movement event is identified based on accelerometer data in sensor data, obtain a first matching result by verifying the matching between the image displacement vector of the target evidence video stream at the same timestamp and the device displacement vector corresponding to the accelerometer data; when a device rotation event is identified based on gyroscope data in sensor data, obtain a second matching result by comparing the angular deviation between the image rotation angle of the target evidence video stream at the same timestamp and the device rotation angle corresponding to the gyroscope data; and determine the synchronization between the image motion state and the device motion state in the target evidence video stream based on the first matching result and the second matching result.

[0126] In some embodiments of this application, the server is specifically used to: compare the consistency of the geometric features of the corresponding trajectory between the image motion trajectory and the physical motion trajectory of the main device, and compare the temporal consistency of all key motion nodes in the image motion trajectory and the physical motion trajectory of the main device; if the consistency of both exceeds a preset threshold, a result representing the similarity between the image motion trajectory and the physical motion trajectory of the main device is generated.

[0127] In some embodiments of this application, the server is specifically used to: identify the device motion state of the main device based on sensor data, detect the synchronization between the image motion state and the device motion state of the target evidence video stream corresponding to the same timestamp, and compare and analyze the synchronization between the image motion trajectory and the physical motion trajectory of the main device in the evidence collection process video stream to obtain a physical verification result; detect the matching of visual events in the target evidence video stream and the evidence collection process video stream corresponding to the same timestamp to obtain a visual verification result; and obtain a synchronization verification result that represents the success of the multi-channel video stream synchronization verification when both the physical verification result and the visual verification result pass.

[0128] In some embodiments of this application, the server is specifically used to: locate the physical evidence area where the evidence target is located by using a target detection model for multiple video streams corresponding to the same timestamp, and extract the physical evidence state feature vector from the physical evidence area; calculate the similarity of the physical evidence state feature vectors of the multiple video frames, and determine that the visual event matching verification of the multiple video streams is passed if the similarity exceeds a preset similarity threshold.

[0129] In some embodiments of this application, the server is specifically used to: detect events of change in the state of physical evidence in the target physical evidence video stream based on the real-time received target physical evidence video stream, and record the event timestamp; detect whether there is a visual event in the target physical evidence video stream under the same event timestamp that matches the event of change in the state of physical evidence, and if so, the visual event matching verification of multiple video streams passes.

[0130] In some embodiments of this application, the server is specifically used to: if the initial hash value corresponding to each video stream in the multi-channel video stream is consistent with the verification hash value, obtain a hash verification result indicating that the hash verification is successful.

[0131] The system for multi-device collaborative forensics provided in this application embodiment can achieve... Figure 1 The methods described in this embodiment are the various processes implemented by the forensic system and can achieve the same technical effects. To avoid repetition, they will not be described again here.

[0132] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0133] like Figure 3 As shown, the electronic device 300 includes a memory 301, a processor 302, and a computer program stored in the memory 301 and capable of running on the processor 302.

[0134] In one example, the processor 302 described above may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0135] Memory 301 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method of multi-device collaborative forensics based on the embodiments of the first aspect of this application.

[0136] The processor 302 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 301, in order to implement the multi-device collaborative forensics method in the first aspect embodiment described above.

[0137] In some examples, the electronic device 300 may also include a communication interface 303 and a bus 310. For example, Figure 3 As shown, the memory 301, processor 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0138] The communication interface 303 is mainly used to enable communication between various modules, systems, units, and devices in the embodiments of this application. Input devices and output devices can also be connected through the communication interface 303.

[0139] Bus 310 includes hardware, software, or both, that couples components of electronic device 300 together. For example, and not limitingly, bus 310 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0140] The electronic device provided in this application embodiment is capable of achieving Figure 1 The various processes implemented by the electronic device in the method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.

[0141] In conjunction with the multi-device collaborative forensics method described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement the steps of any of the multi-device collaborative forensics methods described in the above embodiments.

[0142] In conjunction with the multi-device collaborative forensics method described in the above embodiments, this application embodiment can provide a computer program product to implement it. This (computer) program product is stored in a non-volatile storage medium, and when executed by at least one processor, it implements the steps of any of the multi-device collaborative forensics methods described in the above embodiments.

[0143] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described method embodiment based on multi-device collaborative forensics, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0144] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0145] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0146] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0147] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or systems. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0148] The aspects of this disclosure have been described above with reference to flowchart illustrations and block diagrams of methods, systems, and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and block diagrams, and combinations of blocks in the flowchart illustrations and block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing system to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing system, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0149] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for collaborative evidence collection based on multiple devices, characterized in that, Applied to an evidence collection system, the evidence collection system including at least two terminal devices, a server, and a timestamp authentication system, the method includes: In response to an evidence collection command, synchronous video acquisition is performed through at least two terminal devices, and sensor data of at least one main device is acquired in real time. The sensor data includes at least data characterizing the device motion state of the main device. The at least one main device captures the evidence target to obtain a target evidence video stream, and at least one auxiliary device captures the evidence collection process to obtain an evidence collection process video stream. The target evidence video stream and the evidence collection process video stream form a multi-channel video stream. During video acquisition, the terminal device embeds timestamps generated by the same time reference source into the video frames and sensor data in the multiple video streams, and uploads the multiple video streams and the sensor data to the server. The server identifies the device motion state of the main device based on sensor data, detects the synchronization between the image motion state of the target evidence video stream corresponding to the same timestamp and the device motion state, and compares and analyzes the synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and the physical motion trajectory determined by the sensor data. The multi-channel video stream is then subjected to video synchronization verification to obtain the synchronization verification result. After the multiple video streams are uploaded, the server combines the timestamp-corresponding video streams and sensor data according to the time sequence to obtain the original evidence data packet. The initial hash value corresponding to each video stream is compared with the verification hash value to obtain the hash verification result. The initial hash value is the hash value generated by the terminal device based on the original video stream, and the verification hash value is the hash value generated by the server based on the received video stream. The synchronization verification result and hash verification result are written into the original evidence collection data packet to generate the target evidence collection data packet. The target evidence collection data packet is then uploaded to the timestamp authentication system for evidence solidification, generating an tamper-proof electronic evidence file.

2. The method according to claim 1, characterized in that, After synchronous video capture is performed via the at least two terminal devices in response to the evidence collection instruction, the method further includes: During video acquisition, the terminal device monitors the evidence collection screen corresponding to the video stream of the evidence collection process in real time, and outputs a prompt message when the distance between the main device or the evidence collection target and the reference point of the evidence collection screen reaches a set value, so that the video stream of the evidence collection process records the evidence collection behavior of the evidence collection subject using the main device to collect evidence from the evidence collection target throughout the process.

3. The method according to claim 1, characterized in that, The main device and the auxiliary device are both one unit. The forensic system further includes: a trusted forensic application capable of communicating with the server; the multiple video streams are uploaded to the server through the trusted forensic application; and before synchronous video capture is performed through the at least two terminal devices, the system further includes: Establish communication and interaction between the master device and the auxiliary device, and calibrate the time base of the master device and the auxiliary device based on the time synchronization protocol to achieve clock synchronization between devices; When both the main device and the auxiliary device are equipped with a trusted forensic application, an end-to-end data transmission channel is established between the main device and the server to form a first transmission channel, and an end-to-end data transmission channel is established between the auxiliary device and the server to form a second transmission channel. The first transmission channel is used for real-time transmission of the target evidence video stream and sensor data from the main device, and the second transmission channel is used for real-time transmission of the forensic process video stream from the auxiliary device; and / or, When the main device has a trusted forensic application installed, and the auxiliary device does not have a trusted forensic application installed, an end-to-end data transmission channel is established between the main device and the server to obtain a first transmission channel, and an end-to-end data transmission channel is established between the main device and the auxiliary device to obtain a third transmission channel. The third transmission channel is used to transmit the video stream of the evidence collection process of the auxiliary device to the main device in real time, and the first transmission channel is used to transmit the target physical evidence video stream and sensor data of the main device in real time, and forward the video stream of the evidence collection process of the auxiliary device to the server.

4. The method according to claim 1, characterized in that, Based on sensor data, the device motion state of the main equipment is identified, and the synchronization between the motion state of the target evidence video stream corresponding to the same timestamp and the device motion state is detected, including: When a device movement event is identified based on accelerometer data in sensor data, the first matching result is obtained by verifying the matching between the image displacement vector of the target evidence video stream at the same timestamp and the device displacement vector corresponding to the accelerometer data. When a device rotation event is detected based on gyroscope data from sensor data, a second matching result is obtained by comparing the rotation angle of the target evidence video stream at the same timestamp with the angular deviation of the device rotation angle corresponding to the gyroscope data. Based on the first and second matching results, the synchronization between the motion state of the image and the motion state of the device in the target evidence video stream is determined.

5. The method according to claim 1, characterized in that, Comparative analysis of the synchronization between the main device's image motion trajectory and its physical motion trajectory in the video stream during the evidence collection process includes: Compare the consistency of the geometric features of the corresponding trajectories of the image motion trajectory and the physical motion trajectory of the main device, and compare the temporal consistency of all key motion nodes in the image motion trajectory and the physical motion trajectory of the main device. If the consistency between the two exceeds a preset threshold, a result is generated that represents the synchronization between the image motion trajectory and the physical motion trajectory of the main device.

6. The method according to claim 1, characterized in that, Based on sensor data, the device motion state of the main equipment is identified, and the synchronization between the image motion state and the device motion state of the target evidence video stream corresponding to the same timestamp is detected. The synchronization between the image motion trajectory and the physical motion trajectory of the main equipment in the video stream during the evidence collection process is compared and analyzed. Video synchronization verification is performed on the multiple video streams to obtain the synchronization verification results, including: Based on sensor data, the device motion state of the main device is identified, and the synchronization between the motion state of the target evidence video stream corresponding to the same timestamp and the device motion state is detected. The synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and its physical motion trajectory is compared and analyzed to obtain the physical verification result. The visual verification result is obtained by detecting the matching of visual events in the video stream of the target physical evidence corresponding to the same timestamp and the video stream of the evidence collection process. If both the physical verification result and the visual verification result pass, a synchronization verification result is obtained, which indicates that the synchronization verification of the multi-channel video streams has passed.

7. The method according to claim 6, characterized in that, Detecting the matching of visual events in the video stream of the target evidence corresponding to the same timestamp and the video stream of the evidence collection process, including: For multiple video streams corresponding to the same timestamp, the physical evidence area where the evidence target is located is located by the target detection model, and the evidence state feature vector is extracted from the physical evidence area. The similarity of the physical evidence state feature vectors of multiple video frames is calculated. If the similarity exceeds a preset similarity threshold, the visual event matching verification of the multiple video streams is determined to be successful.

8. The method according to claim 6, characterized in that, Detecting the matching of visual events in the video stream of the target evidence corresponding to the same timestamp and the video stream of the evidence collection process, including: Based on the real-time received target physical evidence video stream, detect physical evidence status change events in the target physical evidence video stream and record event timestamps; The system detects whether there is a visual event in the target evidence video stream under the same event timestamp that matches the event of the change in the state of the evidence. If there is, the visual event matching verification of the multiple video streams passes.

9. The method according to claim 1, characterized in that, The initial hash value corresponding to each video stream is compared with the verification hash value to obtain the hash verification result, including: If the initial hash value and the verification hash value of each video stream in the multi-channel video stream are consistent, a hash verification result indicating successful hash verification is obtained.

10. An evidence collection system based on multi-device collaborative evidence collection, characterized in that, The system includes at least two terminal devices, a server, and a timestamp authentication system, wherein: The at least two terminal devices are used to respond to the evidence collection command, perform synchronous video acquisition and acquire sensor data of at least one main device in real time, wherein the at least one main device captures the evidence target to obtain a target evidence video stream, and at least one auxiliary device captures the evidence collection process to obtain an evidence collection process video stream, and the target evidence video stream and the evidence collection process video stream form a multi-channel video stream. The terminal device is used to embed timestamps generated by the same time reference source into the video frames and sensor data in the multiple video streams during the video acquisition process, and upload the multiple video streams and the sensor data to the server. The server is used to identify the device motion state of the main device based on sensor data, detect the synchronization between the image motion state and the device motion state of the target evidence video stream corresponding to the same timestamp, compare and analyze the synchronization between the image motion trajectory of the main device in the video stream during the evidence collection process and the physical motion trajectory determined by the sensor data, perform video synchronization verification on the multiple video streams, and obtain the synchronization verification result. The server is also configured to, after the multiple video streams are uploaded, combine the multiple video streams and sensor data corresponding to the timestamps according to the time sequence to obtain the original evidence data packet, and compare the initial hash value corresponding to each video stream with the verification hash value to obtain the hash verification result. The initial hash value is the hash value generated by the terminal device based on the original video stream, and the verification hash value is the hash value generated by the server based on the received video stream. The server is also used to write the synchronization verification result and hash verification result into the original evidence data packet, generate the target evidence data packet, and upload the target evidence data packet to the timestamp authentication system for evidence solidification, generating tamper-proof electronic evidence files.

Images