Intelligent scrolling screenshot consolidation system for iOS equipment based on cooperation of Wi-Fi and Bluetooth

Abstract

The invention discloses an intelligent rolling screenshot evidence fixing system for iOS equipment based on cooperation of Wi-Fi and Bluetooth. The system comprises an iOS device, a PC running a consolidation program, and a wireless communication module. The consolidation program comprises a protocol analysis module used for establishing a screen projection connection through a reverse AirPlay protocol and receiving a screen coding stream; the audio and video processing module is used for decoding the coded stream and generating a screenshot and a video; the AI image processing module is used for analyzing the screen image frame and generating a rolling control instruction; and the Bluetooth control module is used for simulating man-machine interface equipment through Bluetooth so as to drive the iOS equipment screen to automatically roll. The system circularly executes the processes of image analysis, decision rolling, execution control and screenshot storage through cooperation of all the modules until complete screenshot of long content is completed. According to the invention, automatic, high-precision and whole-process rolling screenshot and evidence solidification of iOS equipment screen content are realized.

Description

Technical Field

[0001] This invention relates to the field of computer software and mobile device collaboration technology, and in particular to an intelligent scrolling screenshot authentication system for iOS devices based on Wi-Fi and Bluetooth collaboration. Background Technology

[0002] With the widespread adoption of mobile devices and the continuous evolution of the mobile application ecosystem, iOS devices, with their massive user base, have become a core source of evidence carrying crucial clues in digital forensic investigations. In various investigative scenarios, such as case investigation and electronic data preservation, investigators often need to capture application content that extends beyond the visible screen, such as long bills, chat logs, emails, or social media updates. This content is often directly related to the core facts of the case.

[0003] However, the lack of native scrolling screenshot functionality in the iOS system has created a significant technological gap that poses a major obstacle to digital forensics. The traditional solution for scrolling screenshots on iOS devices using computers involves taking a single-screen screenshot followed by manual scrolling, repeating this process until the screenshot is complete. This not only results in low efficiency due to the cumbersome operation, but also risks human error that could lead to omissions or damage to the integrity of the evidence, thus affecting its legal admissibility in judicial proceedings. Summary of the Invention

[0004] The purpose of this invention is to achieve automated, high-precision scrolling screenshots and evidence collection of long pages, chat logs, and other content without requiring any software installation or system settings on the iOS device (such as enabling developer mode).

[0005] In a first aspect, embodiments of the present invention provide an intelligent scrolling screenshot authentication system for iOS devices based on Wi-Fi and Bluetooth collaboration, comprising: an iOS device, a PC running an authentication program, and a wireless communication module connecting the two; the wireless communication module includes a Wi-Fi module and a Bluetooth module, wherein the Wi-Fi module is used to establish a high-bandwidth data channel, and the Bluetooth module is used to establish a reverse control channel between the PC and the iOS device; The evidence-gathering program running within the PC includes: The protocol parsing module is used to establish an encrypted screen mirroring connection with iOS devices via the reverse AirPlay protocol, and to receive and decrypt the real-time screen encoding stream from iOS devices. The audio and video processing module is used to decode and extract frames from the real-time screen encoding stream, output screen image frames, and generate single-frame screenshots or screen recording files based on the screen image frames. The AI ​​image processing module is used to analyze screen image frames, identify target content areas, interface elements and scrolling status, determine the starting point, ending point and precise distance of each scroll based on image similarity algorithm, and generate screen scrolling control commands. The Bluetooth control module is used to simulate a human-machine interface device via the Bluetooth Low Energy protocol, convert control commands into mouse control events that can be recognized by the iOS device, and drive the iOS device screen to scroll automatically. The system achieves intelligent scrolling screenshots through the collaboration of various modules, following the process below: the protocol parsing module obtains the screen encoding stream, and the audio and video processing module decodes it into screen image frames; the AI ​​image processing module analyzes the image frames and generates a command to trigger the Bluetooth control module to drive the iOS device to scroll when scrolling is required; after scrolling stops and the image stabilizes, the audio and video processing module performs screenshot saving; the AI ​​image processing module continuously analyzes new image frames and repeatedly executes the above scrolling and screenshot process until the screenshot is determined to be complete.

[0006] Optionally, the protocol parsing module is specifically used for: The PC is simulated as an AirPlay receiver, and the simulated AirPlay receiver device information is broadcast within the local area network via the Bonjour service; Perform a TLS handshake with the iOS device that initiated the screen mirroring to establish an encrypted communication channel; Receives real-time screen-encoded streams sent by iOS devices via the UDP protocol, encapsulated as RTP packets; The RTP data packets are decapsulated to separate the video encoded stream and the audio encoded stream, wherein the video stream is encoded in H.264 or H.265 format.

[0007] Optionally, the audio and video processing module includes: The decoding unit is used to decode the separated video encoded stream and convert H.264 or H.265 format video data into raw image frames in YUV or RGB format. The screenshot unit is used to encode the original image frame into a JPG or PNG format image file for saving; The encoding and encapsulation unit is used to encode the decoded video frames, convert the synchronously received audio encoded stream into AAC format, and finally encapsulate the encoded video data and audio data into an MP4 container to form an MP4 video file.

[0008] Optionally, the AI ​​image processing module includes: The element detection unit uses a deep learning-based target detection model or image segmentation model to identify specific interface elements and their position coordinates in the screen image frame. The specific interface elements include, but are not limited to, chat bubbles, list item boundaries, application navigation bars, or page scroll bars. The scrolling decision unit, connected to the element detection unit, is used to calculate the precise displacement required to scroll the interface area where the specific interface element is located to the next area to be captured, based on the identified element position and a preset screenshot strategy. The completion determination unit is used to compare the screen images before and after the scrolling operation triggered by the Bluetooth control module using an image similarity algorithm; when the similarity between the scrolled image and the image before scrolling is higher than a preset threshold, it is determined that no new valid content has been generated during the current scrolling, thereby terminating the screenshot process; or when a preset termination mark is detected, the screenshot process is terminated.

[0009] Optionally, the AI ​​image processing module is configured to execute a targeted screenshot strategy, the screenshot strategy including: The context-based continuous screenshot strategy is used to control screen scrolling when a chat session interface is detected to ensure that continuous chat bubbles are captured in sequence. The list full screenshot strategy is used to control screen scrolling to capture the complete list content from the first item to the last item when a list or table interface is detected. A key region priority strategy is used to prioritize ensuring that regions containing predetermined key element types are fully captured based on the identification results of the element detection unit.

[0010] Optionally, the preset termination marker includes at least one of the following: The specific application interface bottom mark or end marker identified by the element detection unit; After multiple consecutive scrolling, the number of times the completion determination unit determines that there is no new valid content reaches a preset threshold. The screen content has been restored to the starting screen or homepage of this screenshot task.

[0011] Optionally, the Bluetooth control module is specifically used for: On the PC, a human-machine interface device conforming to the Bluetooth Low Energy specification is configured through the Bluetooth protocol stack provided by the operating system, and the report descriptor of the human-machine interface device is configured as a mouse type. Broadcasts simulated human-machine interface devices for iOS devices to discover and pair with; Once paired successfully, control data conforming to the human-computer interface device mouse report format is sent to the iOS device through the notification channel of the general property configuration file to simulate mouse scrolling, clicking or dragging events.

[0012] Optionally, the Bluetooth control module is further configured to: After executing the scrolling operation according to the instructions of the AI ​​image processing module, a delay is triggered to wait for the interface rendering to stabilize. Once the interface stabilizes, the audio and video processing module is notified to take a screenshot. The AI ​​image processing module is also configured to: when the completion determination unit determines that no valid new content has been generated in a single scroll, control the Bluetooth control module to perform a reverse small-amplitude scroll verification to confirm whether the scroll has actually reached the bottom.

[0013] Optionally, the evidence preservation procedure further includes an evidence chain management module, used for: Automatically generate metadata files containing timestamps, device identifiers, and operation logs for a complete scrolling screenshot task; All screenshot files, screen recording files, and metadata files generated in this task are associated and stored, and their hash values ​​are calculated. The hash value is stored in the blockchain or digitally signed to form an immutable evidence package.

[0014] Secondly, embodiments of the present invention provide a method for securing evidence through intelligent scrolling screenshots on iOS devices based on Wi-Fi and Bluetooth collaboration, comprising the following steps: A high-bandwidth data channel based on Wi-Fi and a reverse control channel based on Bluetooth, whereby the PC controls the iOS device, are simultaneously established between the PC running the authentication program and the iOS device. An encrypted screen mirroring connection is established with the iOS device via the reverse AirPlay protocol, and the real-time screen encoding stream from the iOS device is received and decrypted. The real-time screen encoded stream is decoded and frame-stripped to output continuous screen image frames. Real-time analysis of screen image frames identifies target content areas, interface elements, and scrolling status. Based on image similarity algorithms, it determines the starting point, ending point, and precise distance of each scroll in the scrolling screenshot. When scrolling is required, it generates control commands containing scrolling parameters. The control commands are converted into simulated mouse control events that can be recognized by the iOS device by simulating a human-machine interface device via Bluetooth Low Energy protocol, and then sent to the iOS device to drive its screen to scroll automatically. After a scrolling operation is completed and the content of the screen image frame stabilizes, the current image frame is captured and saved as a screenshot file; The process of analyzing, making decisions, controlling scrolling, and taking screenshots based on new screen image frames is executed repeatedly until it is determined that the screenshot of the target content is complete.

[0015] This embodiment innovatively integrates Wi-Fi reverse projection and Bluetooth HID (Bluetooth Human Interface Device) reverse control technology. Without jailbreaking, installing any applications, or maintaining the native iOS system, it achieves fully automatic, high-precision scrolling screenshots and screen recording of long content on the device screen. Its AI intelligent module can accurately identify content boundaries and determine scrolling parameters, ensuring the integrity and continuity of the screenshot content. At the same time, the dual-channel collaborative operation ensures high efficiency and stability of the operation, ultimately generating a complete and credible evidence chain including chronological screenshots and process recordings. This effectively solves the technical challenge of automating electronic evidence preservation in the closed iOS ecosystem. Attached Figure Description

[0016] Figure 1 A schematic diagram illustrating the overall technical solution provided in the embodiments of the present invention; Figure 2 A schematic diagram of the architecture of the intelligent scrolling screenshot authentication system for iOS devices based on Wi-Fi and Bluetooth collaboration provided in an embodiment of the present invention; Figure 3 The flowchart illustrates the intelligent scrolling screenshot verification method for iOS devices based on Wi-Fi and Bluetooth collaboration, as provided in this embodiment of the invention. Detailed Implementation

[0017] The present invention will be described in detail below through embodiments.

[0018] With the widespread adoption of mobile devices and the continuous evolution of the mobile application ecosystem, iOS devices, with their massive user base, have become a core source of evidence carrying crucial clues in digital forensic investigations. In various investigative scenarios, such as case investigation and electronic data preservation, investigators often need to capture application content that extends beyond the visible screen, such as long bills, chat logs, emails, or social media updates. This content is often directly related to the core facts of the case.

[0019] However, the lack of a native scrolling screenshot function in iOS has long been a significant obstacle to digital forensics. Traditional methods for taking scrolling screenshots on iOS devices using computers involve taking a single-screen screenshot followed by manual scrolling, repeating this process until the screenshot is complete. This cumbersome process not only leads to low efficiency but also risks human error that could compromise the integrity of evidence, thus affecting its legal admissibility in judicial proceedings. This invention provides a non-intrusive, wireless reverse control method for taking scrolling screenshots on iOS devices, posing no risk and supporting all iOS devices.

[0020] To address the technical challenge of achieving automated scrolling screenshots on iOS devices without jailbreaking or installation using existing technologies, this invention proposes an intelligent scrolling screenshot solution based on Wi-Fi and Bluetooth collaboration. The core technology lies in the following: First, by reverse engineering the AirPlay protocol, an encrypted screen mirroring connection is established between the computer and the iOS device. The screen encoding stream is acquired and decrypted via the Wi-Fi channel, and frames are extracted to obtain screenshots. Second, a low-latency reverse control channel is established on the computer to simulate a standard human-machine interface device using the Bluetooth Low Energy protocol, simulating mouse events to drive screen scrolling. Finally, an AI image analysis module is introduced to process the screen image in real time, intelligently identifying content boundaries and scrolling status, and precisely controlling the start and end points and displacement of the scrolling operation. This solution achieves automated, high-precision scrolling screenshots and evidence capture of long pages, chat logs, and other content without requiring any software installation or system settings on the iOS device (such as enabling developer mode).

[0021] To ensure the plan is clearly described, the following will first combine... Figure 1 The overall technical solution of the embodiments of the present invention will be described in detail.

[0022] like Figure 1 As shown, the system provided in this embodiment mainly consists of two parts: an iOS device and a PC device (personal computer). The two communicate with each other through wireless Bluetooth and wireless Wi-Fi connections.

[0023] Step 1: Screen mirroring and audio / video stream acquisition.

[0024] Users can pull down the device menu on their iOS devices and select the "Screen Mirroring" feature. The iOS device will then send screen-encoded data to a receiving device on the same local network via Wi-Fi using the AirPlay protocol.

[0025] On the PC, the AirPlay protocol parsing module begins operation. It simulates an Apple TV receiver, receiving encrypted data streams from iOS devices and performing protocol parsing and decryption. After parsing, it separates the H.264 / H.265 format video stream and the PCM format audio stream, completing the initial acquisition of audio and video data.

[0026] Step 2: Processing and presentation of audio and video streams.

[0027] The parsed raw audio and video data is sent to the audio and video decoding module. This module decodes the video stream, restoring it to continuous image frames; and performs synchronous decoding on the audio stream.

[0028] The decoded data flows in three directions: The first flow is to send the video to an audio / video player for real-time playback, allowing users to view the iOS device's screen on their PC screen in real time.

[0029] The second flow is to the video frame extraction module, which extracts a single frame from the continuous image frames and encodes it into an image file in JPG or PNG format for saving as a screenshot.

[0030] The third flow is to the audio and video decoding and synthesis MP4 module, which re-encodes the decoded video frames and encapsulates them together with the synchronized audio data into an MP4 container to form a complete video file for storage.

[0031] Step 3: AI-powered intelligent analysis and rolling decision-making.

[0032] Images extracted from frames of the video stream are transmitted in real time to the AI ​​image processing module. This module analyzes the current screen content, identifying target areas (such as chat lists or webpage text), determining content boundaries, and tracking scrolling status. Based on preset screenshot strategies and image analysis results (such as using similarity calculations to determine if the bottom has been reached), the AI ​​image processing module intelligently and precisely controls the position and range of movement of the scrolling screenshot, and generates control commands for the next scrolling action.

[0033] Step 4: Bluetooth reverse control and automated scrolling.

[0034] Control commands generated by the AI ​​image processing module (such as "scroll up N units") are sent to the Bluetooth control module. This Bluetooth control module connects wirelessly via Bluetooth to simulate a Bluetooth Low Energy compliant human-machine interface device on the PC, specifically a mouse.

[0035] The Bluetooth control module converts received control commands into control data packets conforming to the Bluetooth HID mouse report format (e.g., setting the scroll wheel displacement field in the report). This data packet is sent to the paired iOS device via the Bluetooth channel. The iOS device's operating system recognizes this data packet as an input report from a real Bluetooth mouse, thereby performing the corresponding screen scrolling operation to achieve automated control.

[0036] The complete workflow loop is as follows: After the system starts, it enters a complete automated closed-loop process of "perception-decision-execution-recording": First, the AI ​​image processing module analyzes the current screen image obtained and decoded by the AirPlay protocol in real time, intelligently determines the scrolling parameters, and triggers the Bluetooth control module to execute the scrolling operation; after scrolling is completed, the new screen content of the iOS device is then wirelessly transmitted to the PC via AirPlay; the audio and video processing module on the PC decodes the new data stream, and the AI ​​image processing module simultaneously analyzes the decoded new image and saves the current frame as a valid screenshot; subsequently, the system again determines whether to continue scrolling to obtain more content based on the latest image. This core loop of "analyzing the image - deciding on scrolling - transmitting the new image - saving the screenshot" will continue to iterate until the AI ​​image processing module determines that all the preset target content has been captured, and finally automatically generates a series of screenshot files arranged in chronological or logical order, and optionally synthesizes a complete screen recording video, thus completing an end-to-end intelligent scrolling screenshot process.

[0037] This embodiment offers at least the following beneficial effects: 1. Implemented non-intrusive, fully automatic scrolling screenshots for iOS devices.

[0038] This solution creatively integrates two major technical approaches: Wi-Fi screen mirroring data acquisition and Bluetooth reverse control. The entire process requires no third-party applications to be installed on the iOS device, no jailbreaking is needed, and developer mode needs to be enabled. It achieves fully automated operation without the user's awareness, utilizing native system functions (AirPlay mirroring and Bluetooth HID support), thus solving the automation tool development challenges posed by the closed nature of the iOS system. Furthermore, the system replaces the traditional manual scrolling, visual judgment, and manual screenshotting processes through a closed loop of "analysis-decision-control-recording," achieving completely unattended operation from start to finish.

[0039] 2. It ensured the integrity of the screenshot content and the reliability of the evidence chain.

[0040] Through real-time analysis of screen content via an AI image processing module, it can accurately identify scroll boundaries (such as the bottom of chat sessions or the end of lists), intelligently determine the timing and magnitude of scrolling, and avoid content duplication or omissions caused by fixed scrolling distances. It directly extracts frames from the video encoding stream natively projected from the device, and can optionally synthesize high-definition MP4 recordings, resulting in high-fidelity images and videos that faithfully reproduce the original appearance of the device screen. Furthermore, the complete automated process generates sequentially clear screenshots and / or coherent recordings, which, combined with optional metadata records, form an immutable, complete, and continuous chain of electronic evidence, greatly enhancing its probative value in legal or auditing scenarios.

[0041] 3. Improved the accuracy, efficiency and stability of operation.

[0042] Compared to image recognition-based simulated clicks (susceptible to interface changes) or coordinate-based blind swiping (prone to misalignment), this solution uses Bluetooth to simulate the standard HID protocol for control. This is recognized by the iOS system as real hardware input, resulting in precise, stable control commands and excellent compatibility. Furthermore, the automated process frees users from tedious repetitive operations, significantly improving efficiency when processing long documents and chat logs. Wi-Fi handles high-bandwidth data transmission, while Bluetooth provides low-latency, precise control; the two channels operate independently without interference, ensuring smooth and stable operation of the entire automated process.

[0043] Fourth, it provides a good user experience and real-time monitoring capabilities.

[0044] Using a PC-based audio and video player, users can view the screen mirroring from their iOS device in real time and monitor the entire screenshot process intuitively. The system simultaneously outputs both single-frame screenshots and composite video recordings, allowing users to choose the appropriate format for archiving, review, or presentation in various scenarios.

[0045] In summary, the embodiments of the present invention have successfully overcome the technical barriers to achieving installation-free, high-precision, and fully automatic scrolling screenshots on the iOS system through innovative hardware and software collaborative architecture. It has extremely high practical value and broad application prospects in fields such as judicial evidence collection, audit verification, content archiving, and automated testing.

[0046] After a detailed description of the overall technical solution of this invention, the following section will elaborate on the intelligent scrolling screenshot authentication system for iOS devices based on Wi-Fi and Bluetooth collaboration, as provided in this embodiment of the invention. Figure 2 A schematic diagram of the architecture of an iOS device intelligent scrolling screenshot verification system based on Wi-Fi and Bluetooth collaboration according to an embodiment of the present invention is shown.

[0047] like Figure 2 As shown, the system provided in this embodiment mainly includes three parts: an iOS device 10, a PC 20 running a certificate program, and a wireless communication module 30 connecting the two.

[0048] The wireless communication module 30 includes: Wi-Fi module 31 is used to establish a high-bandwidth, low-latency data transmission channel between iOS device 10 and PC 20, mainly carrying audio and video data streams of screen mirroring.

[0049] Bluetooth module 32 is used to establish a low-latency, high-reliability reverse control channel between PC 20 and iOS device 10, enabling PC to simulate input devices to precisely control iOS devices.

[0050] On PC 20, a verification program that implements the core functions of this invention is running. This verification program includes the following logical modules: Protocol parsing module 21: This module is the starting point for system data input. Its core function is to reverse engineer and simulate the Apple TV AirPlay receiver service via software. When iOS device 10 initiates screen mirroring, this module establishes a TLS-based encrypted connection with iOS device 10, receiving and decrypting the real-time screen encoding stream from the iOS device. This process utilizes native iOS system functions entirely, without requiring any applications to be installed on the iOS device.

[0051] Audio / video processing module 22: This module is connected to the protocol parsing module 21 and is responsible for processing the received encoded stream. It mainly performs two key tasks: 1. Decoding and frame extraction: The video stream (usually in H.264 / H.265 format) is decoded in real time to output continuous screen image frames, providing raw materials for subsequent analysis.

[0052] 2. Content Confirmation Output: Based on the decoded image frames, a confirmation result is generated. This includes generating single-frame screenshots (saved as JPG / PNG, etc.) and synthesizing screen recording files (encapsulating video frames and synchronized audio into MP4, etc.), thereby achieving synchronized recording of the video and the process.

[0053] AI Image Processing Module 23: This module is the system's "intelligent brain," connected to the audio / video processing module 22 to acquire real-time screen image frames. It utilizes computer vision and deep learning technologies to perform the following analysis and decisions: 1. Content awareness: Identify target content areas (such as chat windows and document lists) and key interface elements (such as scroll bars and list item boundaries) in the current screen.

[0054] 2. Status Judgment: Analyze the layout and scrolling status of the content, and accurately calculate the starting point and ending point of the scrolling screenshot, as well as the precise pixel distance required for each scroll, based on the image similarity algorithm.

[0055] 3. Command generation: Based on the above analysis results, determine whether scrolling needs to be triggered, and generate screen scrolling control commands containing specific scrolling parameters when necessary.

[0056] Bluetooth Control Module 24: This module acts as the system's "actuator," connecting to the AI ​​Image Processing Module 23 to receive control commands. Its working principle is as follows: On the PC, a standard human-machine interface device (such as a mouse) is simulated via Bluetooth Low Energy protocol. Upon receiving a command, the Bluetooth control module converts it into a standard HID mouse control event (such as simulating mouse wheel scrolling) recognizable by the iOS system and sends it to the paired iOS device 10 via Bluetooth channel 32. The iOS system treats this event as real hardware input, thereby driving the screen to scroll automatically and precisely.

[0057] The system workflow is as follows: The entire system achieves intelligent scrolling screenshots through the close collaboration of the aforementioned modules. Its workflow forms an automated closed loop: the protocol parsing module 21 continuously acquires the screen encoding stream, which is decoded into image frames by the audio / video processing module 22. The AI ​​image processing module 23 analyzes the current image frame; if scrolling is required, it generates a command to trigger the Bluetooth control module 24 to drive the iOS device screen to scroll. Once scrolling stops and the image rendering stabilizes, the audio / video processing module 22 immediately takes a screenshot and saves it. Subsequently, the AI ​​image processing module 23 continues to analyze and make decisions based on new image frames. The system continuously executes this core process of "analysis and decision → Bluetooth scrolling → screenshot saving" until the AI ​​image processing module 23 determines that all target content has been captured, at which point the process automatically terminates.

[0058] This embodiment innovatively integrates Wi-Fi reverse projection and Bluetooth HID reverse control technology, achieving fully automatic, high-precision scrolling screenshots and screen recording of long content on the device screen without jailbreaking, installing any applications, or maintaining the native iOS system. Its AI intelligent module can accurately identify content boundaries and determine scrolling parameters, ensuring the integrity and continuity of the screenshot content. At the same time, the dual-channel collaborative operation ensures high efficiency and stability of the operation, ultimately generating a complete and credible evidence chain including chronological screenshots and process recordings, effectively solving the technical challenge of automated electronic evidence preservation in the closed iOS ecosystem.

[0059] exist Figure 2 Based on the illustrated embodiment, as one implementation of this invention, the protocol parsing module is specifically used to perform the following steps: Step a1: Simulate the PC as an AirPlay receiver and broadcast the simulated AirPlay receiver device information within the local area network via the Bonjour service.

[0060] The protocol parsing module 21 first runs a simulated AirPlay receiver service on the PC. This module actively broadcasts device information conforming to the Apple AirPlay specification within the local area network via the Bonjour service, including the device name, service type, and capability parameters. This allows iOS devices 10 on the same network to automatically discover and identify this PC as an available projection target in the screen mirroring device list, thus completing receiver discovery at the system level.

[0061] Step a2: Perform a TLS handshake with the iOS device that initiated the screen mirroring to establish an encrypted communication channel.

[0062] When the user selects the simulated receiver and initiates a screen mirroring request from the iOS device 10, the protocol parsing module 21 immediately initiates a secure handshake process with the iOS device 10. Both parties perform mutual authentication and key negotiation based on the TLS / SSL (Transport Layer Security) protocol, establishing an end-to-end encrypted communication channel. This step is crucial, ensuring that the subsequently transmitted screen encoded stream cannot be eavesdropped on or tampered with by a third party, thus meeting the data security and originality requirements of the authentication process.

[0063] Step a3: Receive the real-time screen encoded stream sent by the iOS device via the UDP protocol, which is encapsulated as RTP (Real-Time Transport Protocol) data packets.

[0064] After the encrypted channel is established, iOS device 10 begins to encode and package the screen content in real time and send it. Protocol parsing module 21 receives data packets from the iOS device via UDP (User Datagram Protocol). These data packets are encapsulated according to the RTP (Real-Time Protocol) and contain both compressed video frames (corresponding to screen images) and synchronized audio data (corresponding to device sound), together forming a complete real-time screen encoding stream. The use of UDP and RTP ensures low latency and real-time performance of streaming media transmission.

[0065] Step a4: Decapsulate the RTP data packets to separate the video encoded stream and the audio encoded stream. The video stream is encoded in H.264 or H.265 format.

[0066] The protocol parsing module 21 parses and decapsulates the received RTP data packets. It strips the RTP header and extracts the valid media data from the payload. Subsequently, the module separates the composite stream into independent video and audio encoded streams according to the format defined by the AirPlay protocol. The video stream typically uses the efficient H.264 or H.265 video encoding format to minimize bandwidth usage while ensuring image quality; the audio stream is typically in PCM or AAC format. The separated raw encoded streams are output to the downstream audio and video processing module 22 for further decoding and processing.

[0067] Through the steps a1 to a4 described above, the protocol parsing module 21 successfully implemented the complete screen projection link from "device discovery" to "secure connection establishment" and then to "streaming media reception and parsing", providing a stable, secure and real-time source of raw data for the entire intelligent scrolling screenshot system.

[0068] Based on the above embodiments, as one implementation of the present invention, the audio and video processing module includes: The decoding unit is used to decode the separated video encoded stream, converting H.264 or H.265 format video data into raw image frames in YUV or RGB format.

[0069] The screenshot unit is used to encode the original image frame into a JPG or PNG image file for saving.

[0070] The encoding and encapsulation unit is used to encode the decoded video frames, convert the synchronously received audio encoded stream into AAC format, and finally encapsulate the encoded video data and audio data into an MP4 container to form an MP4 video file.

[0071] Specifically, based on the audio and video stream separation completed by the protocol parsing module as described above, the audio and video processing module 22 further processes the received raw encoded stream to generate a certificate file that can be directly used and archived. This module specifically includes the following functional units working together: Decoding Unit: This unit receives the video encoded stream separated from the protocol parsing module 21. Given that iOS device screen mirroring typically uses high-efficiency video encoding standards, the decoding unit has built-in support for hardware or software decoding of H.264 (AVC) and H.265 (HEVC) formats. Its core function is to decode these compressed video data streams in real time, restoring them to a series of continuous, uncompressed original image frames. These image frames are typically converted to YUV color space (such as YUV420p) or RGB color space formats for subsequent display, analysis, or re-encoding operations. The decoding process maintains the resolution and frame rate of the original image, providing a high-quality image source for subsequent processing.

[0072] Screenshot Unit: This unit is directly connected to the decoding unit and receives the decoded raw image frames in real time. Its core task is to capture the current frame and generate a static image file at a specific moment in the system decision (e.g., after each screen scrolling stops and the content stabilizes). The screenshot unit performs secondary encoding on the selected raw image frame (YUV or RGB format) using an image encoding library (such as libjpeg, libpng), converting it into a widely supported JPG (lossy compression) or PNG (lossless compression) image file, and saves it to local storage according to preset naming rules (such as timestamp, serial number). This process generates static evidence of a series of key states in the scrolling screenshot task.

[0073] Encoding and Packaging Unit: This unit is responsible for generating dynamic process log files, i.e., screen recording video. It synchronously processes two inputs: one is the continuous raw video frames output by the decoding unit; the other is the raw audio encoded stream (such as PCM) separated by the protocol parsing module 21. This unit performs the following pipeline operations: 1. Video re-encoding: The original video frames are recompressed and encoded. A suitable encoding format that balances quality and versatility, such as H.264, is typically chosen to generate a video basic stream suitable for storage and playback.

[0074] 2. Audio transcoding: Converts synchronized raw audio streams (such as PCM) into more efficient and universal AAC format audio basic streams.

[0075] 3. Multiplexing and Encapsulation: The encoded video primary stream and AAC audio primary stream are synchronized according to strict timestamps and encapsulated together into a standard MP4 container file. The MP4 format offers good compatibility, compression ratio, and support for streaming playback. The resulting video file completely records the entire screen operation process from start to finish, including sound, constituting dynamic evidence of the process.

[0076] Through the collaborative work of the decoding unit, screenshot unit, and encoding and encapsulation unit, the audio and video processing module 22 realizes a complete production chain from the original encoded stream to the final evidence results (single frame screenshots and complete screen recordings), providing the entire system with key result output capabilities.

[0077] Based on the continuous screen image frames provided by the aforementioned audio and video processing module, the AI ​​image processing module 23, acting as the system's "intelligent decision-making center," is responsible for understanding screen content, planning scrolling paths, and determining task completion. In this embodiment, this module specifically consists of the following three logical units, working together to complete the closed loop from perception to decision-making: 1. Element Detection Unit: This unit is responsible for performing high-level semantic understanding on each input screen image frame. It deploys trained deep learning-based object detection models (such as YOLO, SSD series) or image segmentation models (such as U-Net, Mask R-CNN). These models can accurately identify and locate specific interface elements in the image that are highly relevant to the scrolling screenshot task, including but not limited to: (1) Chat bubble: In instant messaging applications, identify the outline and bounding box of a single message bubble.

[0078] (2) List item boundaries: In a list or article view, identify the dividing line or the entire area of ​​each item (such as a news headline or contact item). (3) Application navigation bar / page scrollbar: Identify the position and length of UI components that mark the top / bottom of the page, or dynamic scrollbars, to infer the current browsing progress.

[0079] The output of this unit is the category label of these elements and their precise location coordinates in the image (usually bounding box coordinates [x1, y1, x2, y2]), providing structured information for subsequent decision-making.

[0080] 2. Scrolling Decision Unit: This unit is closely connected to the element detection unit. It receives the results of element detection and performs logical calculations based on a preset screenshot strategy. Its core function is to intelligently calculate the precise pixel displacement required for the iOS device screen to scroll completely into view for the target content (such as an uncaptured chat bubble or the next list item) based on the positional distribution of the currently identified elements. For example, under the "contextual continuous screenshot strategy," if the chat bubble at the bottom of the current frame has been captured, this unit will calculate the displacement required to completely scroll the bubble off the screen and move the next uncaptured bubble to the appropriate position. This calculated displacement value, along with the scrolling direction, is encapsulated into specific screen scrolling control instructions and sent to the Bluetooth control module for execution.

[0081] 3. Completion Decision Unit: This unit is responsible for monitoring the task progress and determining when to terminate. It employs the following two complementary decision mechanisms.

[0082] The first determination mechanism is based on content change: it uses image similarity algorithms (such as Structural Similarity Index (SSIM) and Perceptual Hash (pHash)) to quantitatively compare two key screen images before and after a scrolling operation triggered by the Bluetooth control module. If the calculated similarity is higher than a preset threshold (e.g., 0.95), it indicates that the scrolling has not brought enough new screen content and the end of the page may have been reached. When multiple consecutive scrolls are determined to have "not generated any new valid content," the unit will issue a termination signal.

[0083] The second determination mechanism is based on termination markers: Simultaneously, the unit also monitors the output of the element detection unit, searching for preset termination markers. For example, it might detect specific "No more content" messages, the copyright information bar at the bottom of the application, or a visual state indicating that the page scrollbar has reached the bottom. Once these markers are identified, the unit will immediately determine that the screenshot process is complete.

[0084] Through the precise perception of the element detection unit, the intelligent planning of the rolling decision unit, and the reliable monitoring of the completion judgment unit, the AI ​​image processing module 23 enables the entire system to "understand" the screen like a human and make efficient automated operation decisions until the task is completed.

[0085] Building upon the basic content awareness and decision-making capabilities of the AI ​​image processing module 23, this module can be flexibly configured to execute various targeted and intelligent screenshot strategies to adapt to the needs of different application scenarios. These strategies are implemented by the rolling decision unit, achieving adaptive processing for different interface types through preset logical rules and parameters. The following are examples of three core screenshot strategies: 1. Contextual continuous screenshot strategy.

[0086] This strategy is specifically designed for instant messaging applications (such as WeChat, QQ, and iMessage) or linear document reading scenarios. When the element detection unit identifies characteristic elements such as chat bubbles, continuous message entries, or paragraph text blocks in the current interface, the system automatically activates this strategy.

[0087] The working logic is as follows: The scrolling decision unit calculates the scroll displacement based on the detected chat bubble bounding boxes. Its goal is to ensure that after each scroll, the bottommost bubble of the previous screenshot can be completely moved out of the visible screen area, while the topmost bubble of the next screenshot that is not in the historical screenshots is completely pulled into the field of view.

[0088] In this way, the system will strictly follow the time or space order, capturing each chat bubble or message segment one after another without overlap or omission, and finally generating a complete and coherent chat history or document screenshot sequence, perfectly preserving the contextual logic of the conversation or article.

[0089] 2. Strategy for taking full screenshots of the list.

[0090] This strategy applies to news lists, address books, file managers, product lists, or any tabular interface with a clearly defined item structure. The system automatically enables this strategy when the element detection unit identifies features such as list item boundaries or table row / column separators.

[0091] The working logic is as follows: The scrolling decision unit treats the list as a whole. It may first locate the top of the list (e.g., by recognizing the navigation bar or title), and then control the screen to scroll in units of list item height. Each scroll aims to move the last item currently captured out of the frame and bring the next item or the next few items fully into the center of the screen for a final screenshot.

[0092] In this way, the system can automatically scroll from the first item in the list until the last item is captured, ensuring that the entire list content is fully retrieved. This is suitable for scenarios where all list items need to be archived or checked.

[0093] 3. Prioritize key regions strategy.

[0094] This strategy is suitable for scenarios where the interface elements are mixed, but only specific types of information need to be focused on. For example, in a complex application interface, you only need to capture all "error message pop-ups" or all areas of a webpage that contain "price" information.

[0095] The working logic is as follows: This strategy heavily relies on the recognition capabilities of the element detection unit. Users or the system can pre-define a set of key element types (such as "buttons," "pop-ups," "specific icons," and "text boxes containing numbers"). After receiving the detection results, the scrolling decision unit prioritizes calculation and control of scrolling to ensure that all areas marked as "critical" within the current field of view are fully captured. If a single scroll cannot capture all critical areas, it will plan multiple scrolls to cover all targets.

[0096] In this way, the system can intelligently filter irrelevant information, actively seek out and ensure that all specific interface elements that users care about are clearly and completely captured, greatly improving the targeting and efficiency of information collection in complex interfaces.

[0097] By configuring and switching the above strategies, this system can flexibly handle various screenshot tasks, ranging from simple linear content to complex structured interfaces, achieving a balance between versatility and specialization, and significantly enhancing the system's practical value.

[0098] Furthermore, to ensure that the intelligent scrolling screenshot process can end accurately and reliably at the appropriate time, avoiding invalid loops or resource waste, the completion determination unit of the AI ​​image processing module 23 in this system is configured to monitor a series of preset termination conditions. When any of the following conditions are met, the system will determine that the screenshot task is complete and automatically terminate the process: Condition 1: Identify the bottom marker or end marker of a specific application interface.

[0099] This is one of the most direct and reliable termination conditions, relying on the high-precision recognition capability of the element detection unit. Many application interfaces display clear visual markers as scrolling nears the end of the content, for example... Text prompts such as "No more," "END," and "All content loaded." Specific icons or visual dividers, such as a centered horizontal line or a special bottom bar. Or, the consistent appearance of a "back to top" floating button commonly found in social applications.

[0100] The completion judgment unit continuously analyzes the recognition results of the element detection unit. Once a specific interface element of this type, which is predefined in the "termination tag library", is detected and its position is stably located in the bottom area of ​​the screen, it is immediately determined that the content endpoint has been reached, triggering the process termination.

[0101] Condition 2: No valid new content is generated during continuous scrolling.

[0102] This condition applies to interfaces that do not have a clear bottom marker, or whose markers are difficult to detect consistently. It is based on a core assumption: if multiple consecutive scrolls fail to bring up substantial new content, then it is very likely that the end has been reached.

[0103] Each time the Bluetooth control module triggers a scrolling operation, the completion determination unit uses an image similarity algorithm (such as SSIM) to rigorously compare the similarity between the last frame screenshot before scrolling and the first newly acquired frame image after scrolling stabilizes. If the similarity is higher than a preset threshold (e.g., SSIM > 0.98), it is determined that this scrolling "did not generate any new valid content," and the counter is incremented by 1. When the number of consecutive occurrences of this "invalid scrolling" reaches a preset threshold (e.g., 3 or 5 consecutive times), the completion determination unit will infer that there is no more content to load, thus determining that the screenshot is complete and terminating the process. This mechanism effectively addresses misjudgments caused by the "loading" state, network latency, or minor interface fluctuations.

[0104] Condition 3: The screen content is detected to have returned to the starting point or home page.

[0105] This condition is primarily used to handle situations involving task anomalies or user intervention, ensuring system controllability.

[0106] Scenario 1 (Unexpected Return): During the screenshot process, if the iOS device screen jumps back to the starting application interface (or even the system home screen) of the current screenshot task due to a global notification pop-up from the application, a missed call, or accidental touch by the user, the completion judgment unit will determine that the task has been unexpectedly interrupted by comparing the current screen with the baseline screen saved at the start of the task, or by detecting the icon layout unique to the home screen, thereby safely terminating the process and avoiding infinite scrolling on the error page.

[0107] Scenario 2 (Looping Content): For content that inherently has looping characteristics (such as "recommended reading" in some news apps), this condition can be used in conjunction with Condition 2. If the system detects that the current screen is highly similar to a screenshot from a long time ago (not the previous one), it may also determine that the content loop has started and thus intelligently terminate the loop.

[0108] Through the aforementioned multi-dimensional and complementary termination condition determination mechanism, this system can achieve accurate, reliable, and automated termination of the screenshot process in various complex real-world application scenarios, greatly improving the system's robustness and user experience.

[0109] In this invention system, the Bluetooth control module 24 is the key execution unit for enabling precise reverse control of the iOS device from the PC. This module utilizes the support of modern operating systems for Bluetooth HID devices to virtualize a standard Bluetooth input device on the PC via software, thereby achieving automated operation of the iOS device. The specific implementation steps are as follows: Step 1: Creating and configuring a virtual HID device.

[0110] The Bluetooth control module 24 first calls the underlying Bluetooth protocol stack interface provided by the operating system (such as Windows' WinRT API, macOS's IOBluetooth framework, or Linux's BlueZ) on the PC. It programmatically creates a virtual device that conforms to the Bluetooth Low Energy specification. During creation, the core operation is defining and setting the device's reporting descriptor. The module explicitly configures this descriptor as a "mouse" type. This definition includes standard mouse device characteristics, such as the number of available buttons, the resolution range of X / Y axis movement, and the vertical / horizontal scroll wheel's displacement reporting capability. This configuration ensures that the virtual device is recognized as a standard Bluetooth mouse at the protocol level.

[0111] Step 2: Device broadcasting and pairing connection.

[0112] After the virtual HID device is configured, the Bluetooth control module 24 controls it to enter discoverable mode and begins broadcasting its presence via Bluetooth. The broadcast packet contains information such as the device name and HID device type. At this time, the user can search for and discover this PC-simulated virtual mouse device in the "Settings" → "Bluetooth" list on their iOS device. After the user clicks to connect, or through the module's preset simplified pairing process (such as using a fixed pairing code), the iOS device and the virtual HID device on the PC complete the standard Bluetooth pairing and connection process, establishing a stable communication link.

[0113] Step 3: Sending control commands and simulating events.

[0114] After successful pairing, the virtual HID device and the iOS device establish communication based on the Common Attribute Profile (GATT). The Bluetooth control module 24 sends control data through a specific notification channel in the GATT. When the AI ​​image processing module 23 sends a control command (such as "vertical scroll -120 units"), the core function of this module is: (1) Data conversion: Convert high-level logic instructions into binary data packets that conform to the HID mouse report format. For a scroll instruction, the field representing the scroll wheel displacement in the data packet will be set to the corresponding value (for example, 0xFFF4 represents -12, and the specific conversion depends on the resolution defined in the report descriptor).

[0115] (2) Data transmission: Send the correctly formatted HID report data packet to the connected iOS device through the established GATT notification channel.

[0116] (3) Event Execution: After receiving this HID report, the operating system kernel of the iOS device will treat it as an input signal from a real physical Bluetooth mouse and generate corresponding input events at the system level. As a result, operations such as scrolling the mouse wheel, moving (dragging) the cursor, or clicking the mouse button will be executed precisely on the screen, ultimately driving the target application interface to scroll or interact according to the predetermined instructions, thus achieving precise programmatic control.

[0117] Through the standard “create-pair-send” process described above, the Bluetooth control module 24 successfully converts the software commands from the PC into real hardware input events that are acceptable to the iOS device, thus completing a key link in the automated control process.

[0118] To further improve the reliability, accuracy, and final output quality of the intelligent scrolling screenshot system, in an optimized embodiment of the present invention, the collaborative working logic between the Bluetooth control module 24 and the AI ​​image processing module 23 has been enhanced, introducing two key optimization strategies: Strategy 1: Optimize scrolling delay and screenshot trigger.

[0119] This strategy aims to address the reality that mobile application interfaces require a certain amount of time to redraw content, load data, or complete animations after scrolling, ensuring that the screenshot captures the final, fully stabilized image.

[0120] The implementation is as follows: After receiving the scrolling command from the AI ​​image processing module 23 and successfully sending the HID report and driving the screen to scroll, the Bluetooth control module 24 does not immediately notify the audio / video processing module 22 to take a screenshot. Instead, it starts a built-in configurable delay timer (e.g., a delay of 300 milliseconds to 1 second). This delay time can be preset or adaptively learned according to the response characteristics of the target application.

[0121] During the delay, the system continues to receive and decode the video stream normally via the AirPlay link, but the AI ​​image processing module 23 and the screenshot unit are in a "waiting" state. After the delay ends, the Bluetooth control module 24 or a central coordinator sends a "frame stabilized, screenshot can be taken" trigger signal to the audio / video processing module 22. The audio / video processing module 22 then captures and saves the current frame.

[0122] This mechanism effectively avoids capturing blurry, incomplete, or loading images due to incomplete interface rendering, ensuring that every screenshot is a clear and complete final state, greatly improving the usability and evidentiary value of the screenshot results.

[0123] Strategy 2: Reverse scrolling verification mechanism.

[0124] This strategy aims to address potential issues such as elasticity, dynamic loading failures, or minor positioning deviations that may occur when scrolling to the bottom of certain applications or web pages. It provides a proactive verification method to prevent premature termination or content omissions due to misjudgment.

[0125] The trigger condition is: when the completion judgment unit in the AI ​​image processing module 23 determines for the first time that a certain scroll "did not generate valid new content" through image similarity comparison (i.e., it is suspected to have reached the bottom), the system will not immediately terminate the process.

[0126] The scrolling decision unit of the AI ​​image processing module 23 will immediately generate a reverse, small-amplitude scrolling instruction (e.g., "scroll up 20 pixels") and send it to the Bluetooth control module 24 for execution.

[0127] After performing this reverse scroll, the system captures a new image again, and the completion determination unit re-analyzes the image.

[0128] If the image changes after scrolling backwards (similarity below the threshold), it indicates that the initial judgment may have encountered a "flexible pause" or stopped precisely at the edge of a content-free area, and the system has not truly reached the bottom. The AI ​​image processing module will determine that the process is incomplete and continue the normal downward scrolling screenshot process.

[0129] If the image remains unchanged after scrolling backwards (the similarity is still above the threshold), it strongly confirms that the screen content cannot be scrolled upwards, meaning it has truly reached the absolute bottom or content boundary. At this point, the completion determination unit finally confirms that the screenshot task is complete and the process safely terminates.

[0130] This "proactive probing-secondary confirmation" verification mechanism significantly enhances the system's robustness and fault tolerance in determining the scroll termination point, almost completely avoiding premature termination caused by interface characteristics or instantaneous states, and achieving truly complete screenshot coverage.

[0131] To meet the stringent requirements for the integrity, traceability, and immutability of electronic evidence in judicial, auditing, and other scenarios, in one embodiment of the present invention, the evidence preservation procedure further integrates an evidence chain management module. This module is not a necessary part of the core scrolling screenshot process, but rather a key component for post-event enhancement and evidence preservation, responsible for the systematic management, correlation, and credibility processing of all data generated during the automated process. It may include the following steps: Step 1: Automatic Metadata Generation. Each time a complete scrolling screenshot task is initiated, the evidence chain management module automatically creates a metadata file. This file records the complete context information of the task in a structured format (such as JSON or XML), including but not limited to: Timestamps: Task start time, end time, and the precise time of each screenshot and video clip.

[0132] Device Identifier: The target iOS device's model, system version, unique device identifier, and the PC device information for performing authentication.

[0133] Operation Log: A detailed, time-series operation record that documents every "scrolling decision" made by the AI ​​image processing module (such as scrolling direction and pixel distance), every HID command sent by the Bluetooth control module, every screenshot event triggered by the screenshot unit, and all judgments made by the completion judgment unit and the final termination reason.

[0134] Step Two, File Association and Storage: After the task is completed, the evidence chain management module logically packages and physically associates all output files. It creates a dedicated directory to store all screenshot files (numbered sequentially), complete screen recording files, and the aforementioned metadata files together. Simultaneously, it generates an index list file that clearly lists all files and their relationships.

[0135] Step 3: Hash value calculation and solidification. The evidence chain management module performs a cryptographic hash operation (such as SHA-256) on the complete evidence package (including all screenshots, videos, metadata, and index files) to generate a unique, fixed-length digital fingerprint.

[0136] Step 4, Trusted Evidence Storage. To place this digital fingerprint under public and tamper-proof supervision, the module provides at least one of the following evidence storage methods: 1. Blockchain Evidence Storage. The calculated hash value, along with a summary of the task's key metadata, is submitted to one or more trusted blockchain networks for evidence storage via API calls. The characteristics of blockchain ensure that once the hash value is on the chain, its existence and time point are permanently and publicly recorded, and cannot be modified afterwards.

[0137] 2. Digital Signature Evidence Preservation: Using a pre-set digital certificate private key, the hash value (or the entire index list) of the evidence package is digitally signed. This signature can be independently verified, proving the integrity of the evidence package at the time of signing and that the signing was performed by a specific entity (certificate holder).

[0138] 3. Combined notarization: Hash values ​​can be simultaneously notarized on the blockchain and digitally signed locally, forming a double layer of protection.

[0139] Ultimately, the evidence chain management module outputs a complete, self-contained evidence package. This package includes not only screenshots and video recordings reflecting the facts themselves, but also all information proving the generation process, relationships, and tamper-proof nature of this content. Any third party can independently verify the integrity and credibility of the entire evidence package by recalculating the file hash value and comparing it with the on-chain record or verifying the digital signature.

[0140] The introduction of this embodiment elevates the technical solution of the present invention from a highly efficient automated tool to a complete evidence-building system that conforms to modern electronic evidence rules and can generate digital evidence with strong legal force.

[0141] This invention also provides a method for intelligent scrolling screenshot authentication on iOS devices based on Wi-Fi and Bluetooth collaboration, such as... Figure 3 As shown, it includes the following steps: S310 establishes both a high-bandwidth Wi-Fi-based data channel and a Bluetooth-based reverse control channel between a PC running a certificate program and an iOS device.

[0142] The S320 establishes an encrypted screen mirroring connection with iOS devices via reverse AirPlay protocol, receiving and decrypting real-time screen encoding streams from iOS devices.

[0143] The S330 decodes and extracts frames from the real-time screen encoded stream, outputting continuous screen image frames.

[0144] The S340 analyzes screen image frames in real time, identifies target content areas, interface elements, and scrolling status, and determines the starting point, ending point, and precise distance of each scroll based on image similarity algorithms. When scrolling is required, it generates control commands containing scrolling parameters.

[0145] The S350 uses Bluetooth Low Energy to simulate a human-machine interface device, converting control commands into simulated mouse control events that can be recognized by iOS devices, and sending them to iOS devices to drive their screens to scroll automatically.

[0146] S360 captures a screenshot from the current image frame and saves it as a screenshot file after a scrolling operation is completed and the screen image frame content has stabilized.

[0147] S370 continuously executes the process of analyzing, making decisions, controlling scrolling and taking screenshots based on new screen image frames until it is determined that the screenshot of the target content is complete.

[0148] It should be noted that the steps, technical features, and implementation principles involved in this method embodiment are the same as those described above. Figure 1 The system implementation described in the corresponding text (including the iOS device, PC, wireless communication module, and the collaborative workflow of each functional module) is completely identical and substantially the same. Specifically, each step (S310 to S360 and its loop) in this method embodiment is a concrete manifestation of the functions performed by the verification procedure and each functional module in the aforementioned system embodiment in the method flow.

[0149] Therefore, to avoid repetition, this specification will not elaborate on the specific implementation details of the method embodiments. Those skilled in the art, having fully understood the foregoing system embodiments, will undoubtedly recognize and implement all the technical solutions of the method embodiments.

[0150] This invention also provides an electronic device, comprising: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method described above.

[0151] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method.

Claims

1. A smart scrolling screenshot authentication system for iOS devices based on Wi-Fi and Bluetooth collaboration, characterized in that, include: An iOS device, a PC running a certificate program, and a wireless communication module connecting the two; the wireless communication module includes a Wi-Fi module and a Bluetooth module, the Wi-Fi module is used to establish a high-bandwidth data channel, and the Bluetooth module is used to establish a reverse control channel between the PC and the iOS device. The evidence-gathering program running within the PC includes: The protocol parsing module is used to establish an encrypted screen mirroring connection with iOS devices via the reverse AirPlay protocol, and to receive and decrypt the real-time screen encoding stream from iOS devices. The audio and video processing module is used to decode and extract frames from the real-time screen encoding stream, output screen image frames, and generate single-frame screenshots or screen recording files based on the screen image frames. The AI ​​image processing module is used to analyze screen image frames, identify target content areas, interface elements and scrolling status, determine the starting point, ending point and precise distance of each scroll based on image similarity algorithm, and generate screen scrolling control commands. The Bluetooth control module is used to simulate a human-machine interface device via the Bluetooth Low Energy protocol, convert control commands into mouse control events that can be recognized by the iOS device, and drive the iOS device screen to scroll automatically. The system achieves intelligent scrolling screenshots through the collaboration of various modules, following the process below: the protocol parsing module obtains the screen encoding stream, and the audio and video processing module decodes it into screen image frames; the AI ​​image processing module analyzes the image frames and generates a command to trigger the Bluetooth control module to drive the iOS device to scroll when scrolling is required; after scrolling stops and the image stabilizes, the audio and video processing module performs screenshot saving; the AI ​​image processing module continuously analyzes new image frames and repeatedly executes the above scrolling and screenshot process until the screenshot is determined to be complete.

2. The method according to claim 1, characterized in that, The protocol parsing module is specifically used for: The PC is simulated as an AirPlay receiver, and the simulated AirPlay receiver device information is broadcast within the local area network via the Bonjour service; Perform a TLS handshake with the iOS device that initiated the screen mirroring to establish an encrypted communication channel; Receives real-time screen-encoded streams sent by iOS devices via the UDP protocol, encapsulated as RTP packets; The RTP data packets are decapsulated to separate the video encoded stream and the audio encoded stream, wherein the video stream is encoded in H.264 or H.265 format.

3. The system according to claim 2, characterized in that, The audio and video processing module includes: The decoding unit is used to decode the separated video encoded stream and convert H.264 or H.265 format video data into raw image frames in YUV or RGB format. The screenshot unit is used to encode the original image frame into a JPG or PNG format image file for saving; The encoding and encapsulation unit is used to encode the decoded video frames, convert the synchronously received audio encoded stream into AAC format, and finally encapsulate the encoded video data and audio data into an MP4 container to form an MP4 video file.

4. The method according to claim 1, characterized in that, The AI ​​image processing module includes: The element detection unit uses a deep learning-based target detection model or image segmentation model to identify specific interface elements and their position coordinates in the screen image frame. The specific interface elements include, but are not limited to, chat bubbles, list item boundaries, application navigation bars, or page scroll bars. The scrolling decision unit, connected to the element detection unit, is used to calculate the precise displacement required to scroll the interface area where the specific interface element is located to the next area to be captured, based on the identified element position and a preset screenshot strategy. The completion determination unit is used to compare the screen images before and after the scrolling operation triggered by the Bluetooth control module using an image similarity algorithm; when the similarity between the scrolled image and the image before scrolling is higher than a preset threshold, it is determined that no new valid content has been generated during the current scrolling, thereby terminating the screenshot process; or when a preset termination mark is detected, the screenshot process is terminated.

5. The system according to claim 4, characterized in that, The AI ​​image processing module is configured to execute a targeted screenshot strategy, which includes: The context-based continuous screenshot strategy is used to control screen scrolling when a chat session interface is detected to ensure that continuous chat bubbles are captured in sequence. The list full screenshot strategy is used to control screen scrolling to capture the complete list content from the first item to the last item when a list or table interface is detected. A key region priority strategy is used to prioritize ensuring that regions containing predetermined key element types are fully captured based on the identification results of the element detection unit.

6. The system according to claim 4, characterized in that, The preset termination marker includes at least one of the following: The specific application interface bottom mark or end marker identified by the element detection unit; After multiple consecutive scrolling, the number of times the completion determination unit determines that there is no new valid content reaches a preset threshold. The screen content has been restored to the starting screen or homepage of this screenshot task.

7. The system according to claim 1, characterized in that, The Bluetooth control module is specifically used for: On the PC, a human-machine interface device conforming to the Bluetooth Low Energy specification is configured through the Bluetooth protocol stack provided by the operating system, and the report descriptor of the human-machine interface device is configured as a mouse type. Broadcasts simulated human-machine interface devices for iOS devices to discover and pair with; Once paired successfully, control data conforming to the human-computer interface device mouse report format is sent to the iOS device through the notification channel of the general property configuration file to simulate mouse scrolling, clicking or dragging events.

8. The system according to claim 4, characterized in that, The Bluetooth control module is also configured to: After executing the scrolling operation according to the instructions of the AI ​​image processing module, a delay is triggered to wait for the interface rendering to stabilize. Once the interface stabilizes, the audio and video processing module is notified to take a screenshot. The AI ​​image processing module is also configured to: when the completion determination unit determines that no valid new content has been generated in a single scroll, control the Bluetooth control module to perform a reverse small-amplitude scroll verification to confirm whether the scroll has actually reached the bottom.

9. The system according to claim 1, characterized in that, The evidence preservation procedure also includes an evidence chain management module, used for: Automatically generate metadata files containing timestamps, device identifiers, and operation logs for a complete scrolling screenshot task; All screenshot files, screen recording files, and metadata files generated in this task are associated and stored, and their hash values ​​are calculated. The hash value is stored in the blockchain or digitally signed to form an immutable evidence package.

10. A method for intelligent scrolling screenshot authentication on iOS devices based on Wi-Fi and Bluetooth collaboration, characterized in that, Includes the following steps: A high-bandwidth data channel based on Wi-Fi and a reverse control channel based on Bluetooth, whereby the PC controls the iOS device, are simultaneously established between the PC running the authentication program and the iOS device. An encrypted screen mirroring connection is established with the iOS device via the reverse AirPlay protocol, and the real-time screen encoding stream from the iOS device is received and decrypted. The real-time screen encoded stream is decoded and frame-stripped to output continuous screen image frames. Real-time analysis of screen image frames identifies target content areas, interface elements, and scrolling status. Based on image similarity algorithms, it determines the starting point, ending point, and precise distance of each scroll in the scrolling screenshot. When scrolling is required, it generates control commands containing scrolling parameters. The control commands are converted into simulated mouse control events that can be recognized by the iOS device by simulating a human-machine interface device via Bluetooth Low Energy protocol, and then sent to the iOS device to drive its screen to scroll automatically. After a scrolling operation is completed and the content of the screen image frame stabilizes, the current image frame is captured and saved as a screenshot file; The process of analyzing, making decisions, controlling scrolling, and taking screenshots based on new screen image frames is executed repeatedly until it is determined that the screenshot of the target content is complete.

Images