An inter-frame parallel processing and collaborative frame rate control system and method

Abstract

This application discloses an inter-frame parallel processing and cooperative frame rate control system and method, relating to the field of optoelectronic navigation equipment technology. The method includes: an image acquisition module for acquiring continuous multi-frame sequence image data; a parallel processing module for preprocessing the continuous multi-frame sequence image data to obtain image preprocessing result data, and performing displacement vector calculation to obtain displacement vector results; a data scheduling and storage module for cache management and read / write scheduling of the image preprocessing result data; a cooperative frame rate control module for determining the target running frame rate based on the coarse displacement vector results, and dynamically switching the working mode, storage unit configuration information, and oscillator control strategy; and a displacement output module for converting the high-precision displacement vector results into output data conforming to the peripheral protocol. This application can balance high performance and low power consumption, achieving high-efficiency, low-power displacement detection.

Description

Technical Field

[0001] This application relates to the field of optoelectronic navigation equipment technology, and in particular to an inter-frame parallel processing and cooperative frame rate control system and method. Background Technology

[0002] Existing photoelectric navigation sensors typically employ a serial processing flow, meaning that the current frame image must sequentially complete exposure, image acquisition, image preprocessing, and inter-frame matching calculations to output the displacement result before the next frame acquisition can begin. In this architecture, each processing module executes sequentially, with other modules idle while the current module is working. Therefore, existing technologies suffer from a fundamental technical contradiction: improving frame rate and maximum tracking IPS (Inches Per Second) performance necessitates a high-performance parallel processing architecture, leading to a significant increase in power consumption; conversely, reducing power consumption requires a low-performance serial architecture, limiting the upper limits of frame rate and IPS performance, thus failing to achieve a balance between high performance and low power consumption.

[0003] Therefore, it is urgent to design an inter-frame parallel processing and collaborative frame rate control scheme to solve the problems of low frame rate and limited maximum tracking IPS value, frame rate and parallel processing disconnect, and insufficient power management of oscillator (OSC) in traditional photoelectric navigation sensors, so as to balance high performance and low power consumption and achieve high-efficiency and low-power displacement detection. Summary of the Invention

[0004] The purpose of this application is to provide an inter-frame parallel processing and cooperative frame rate control system and method that can balance high performance and low power consumption, and achieve high-efficiency, low-power displacement detection.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides an inter-frame parallel processing and cooperative frame rate control system, which includes: an image acquisition module, a data scheduling and storage module, a parallel processing module, a cooperative frame rate control module, and a displacement output module.

[0006] The output of the image acquisition module is connected to the input of the parallel processing module. The controlled end of the image acquisition module is connected to the output of the collaborative frame rate control module. The output of the data scheduling and storage module is connected to the input of the parallel processing module. The controlled end of the data scheduling and storage module is connected to the output of the collaborative frame rate control module. The output of the parallel processing module is connected to the input of the displacement output module and the input of the collaborative frame rate control module. The controlled end of the parallel processing module is connected to the output of the collaborative frame rate control module.

[0007] The image acquisition module is used to acquire continuous multi-frame sequence image data; the continuous multi-frame sequence image data includes: historical reference frame image data, previous frame image data and current frame image data.

[0008] The parallel processing module is used to preprocess the continuous multi-frame sequence image data to obtain image preprocessing result data, and to calculate the displacement vector based on the image preprocessing result data to obtain the displacement vector result; the image preprocessing result data includes: image preprocessing result data of historical reference frames, image preprocessing result data of the previous frame, and image preprocessing result data of the current frame; the displacement vector result includes: high-precision displacement vector result and coarse displacement vector result.

[0009] The data scheduling and storage module is used to manage the cache and schedule the read and write operations of the image preprocessing result data.

[0010] The collaborative frame rate control module is used to determine the target running frame rate based on the coarse displacement vector result, and dynamically switch the working mode, storage unit configuration information and oscillator control strategy based on the target running frame rate.

[0011] The displacement output module is used to convert the high-precision displacement vector result into output data that conforms to the peripheral protocol.

[0012] Secondly, this application provides an inter-frame parallel processing and cooperative frame rate control method, which is implemented based on the inter-frame parallel processing and cooperative frame rate control system described in the first aspect. The inter-frame parallel processing and cooperative frame rate control method includes: Acquire continuous multi-frame sequence image data; the continuous multi-frame sequence image data includes: historical reference frame image data, previous frame image data and current frame image data.

[0013] The continuous multi-frame sequence image data is preprocessed to obtain image preprocessing result data, and displacement vector is calculated based on the image preprocessing result data to obtain displacement vector results; the image preprocessing result data includes: image preprocessing result data of historical reference frames, image preprocessing result data of the previous frame, and image preprocessing result data of the current frame; the displacement vector results include: high-precision displacement vector results and coarse displacement vector results.

[0014] Based on the coarse displacement vector results, the target running frame rate is determined, and the working mode, storage unit configuration information, and oscillator control strategy are dynamically switched based on the target running frame rate.

[0015] The high-precision displacement vector result is converted into output data that conforms to the peripheral protocol.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides an inter-frame parallel processing and collaborative frame rate control system and method. It acquires continuous multi-frame sequence image data through an image acquisition module, and coordinates with a data scheduling and storage module to cache and schedule the preprocessing results of the continuous multi-frame sequence image data. This solves the technical problems of data access conflicts and strong inter-frame dependencies in traditional serial architectures, providing stable data support for parallel computing and avoiding module idle waiting issues. The parallel processing module synchronously completes the preprocessing and displacement vector calculation of continuous multi-frame sequence image data, overcoming the shortcomings of traditional single-pipeline processing, such as long processing time and low tracking IPS limit. This reduces single-frame processing time, ensures sub-pixel detection accuracy, and improves frame loss and positioning offset problems under high-speed motion. Through collaborative... The frame rate control module adaptively switches working modes based on displacement vector results, storage unit configuration information, and oscillator control strategies, solving the problems of traditional frame rate adjustment methods being singular and the mismatch between computing power and load. It achieves intelligent adaptation of architecture and storage resources under high and low frame rate conditions, reducing unnecessary computing power consumption. The displacement output module standardizes and converts displacement vector results, ensuring stable data output. Through the synergistic effect of five major functional modules, this application solves the core contradiction of traditional photoelectric sensors being unable to balance performance and power consumption. While ensuring high frame rate and high tracking performance, it optimizes power consumption management, balancing high performance and low power consumption, improving device endurance, and achieving high-efficiency, low-power displacement detection to meet the needs of high-performance navigation detection. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The working architecture of existing traditional photoelectric navigation sensors; Figure 2 This is a schematic diagram of the functional modules of an inter-frame parallel processing and cooperative frame rate control system according to an embodiment of this application; Figure 3 This is a timing diagram for a traditional serial architecture. Figure 4 A timing diagram of a parallel pipeline architecture provided in one embodiment of this application; Figure 5 A timing diagram of an ultra-high frame rate mode architecture provided in an embodiment of this application; Figure 6 A schematic diagram of a triple-buffered parallel access principle provided in one embodiment of this application; Figure 7 A schematic block diagram of a collaborative frame rate control logic provided in an embodiment of this application; Figure 8 This is a block diagram of the dual-memory unit control mechanism; Figure 9 This is a flowchart illustrating an inter-frame parallel processing and collaborative frame rate control method according to an embodiment of this application. Figure 10 This is a flowchart illustrating an inter-frame parallel processing and collaborative frame rate control method according to another embodiment of this application. Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0019] Reference numerals: 101-Image acquisition module; 102-Data scheduling and storage module; 103-Parallel processing module; 104-Cooperative frame rate control module; 105-Displacement output module. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Explanation of technical terms: Photoelectric navigation equipment: An LED lighting system illuminates a target plane, the target plane reflects light into the device's sensor to form an electronic image, the device processes the electronic image (such as image matching), and outputs the relative displacement between the device and the target plane. This type of equipment is called photoelectric navigation equipment.

[0022] Grayscale image: The grayscale level is determined by dividing the white and black regions into several levels according to a logarithmic relationship. There are 256 grayscale levels. Image pixel values ​​are assumed to be 8-bit unsigned binary numbers, corresponding to pixel values ​​from 0 to 255.

[0023] Inter-frame Parallel Processing: A processing architecture for photoelectric navigation sensors, characterized by the simultaneous preprocessing of the current frame image data and the calculation of the displacement vector of the previous frame image preprocessing data, thereby achieving parallel processing between frames and improving system throughput.

[0024] Cooperative Frame Rate Control: A frame rate control method characterized by deep coordination between frame rate adjustment and processing architecture, storage unit configuration, and oscillator operating state. In high or ultra-high frame rate mode, an inter-frame fully parallel processing architecture is enabled, while in low frame rate mode, an inter-frame serial processing architecture is switched and power consumption is optimized to achieve an intelligent balance between processing efficiency and power consumption.

[0025] Two-stage displacement matching: a displacement calculation method that first obtains a coarse displacement vector result through fast matching, and then obtains a high-precision displacement vector result through high-precision matching, thereby reducing calculation delay while ensuring accuracy.

[0026] Inter-frame matching is a technique that calculates the relative displacement between two or more consecutive frames of images by comparing feature points or pixel regions. Displacement detection is typically achieved using block matching algorithms or optical flow methods.

[0027] Throughput: The number of frames or the amount of data a system can process per unit of time; it is an important indicator of processing efficiency. In photoelectric navigation sensors, throughput directly affects the frame rate, real-time performance, and IPS performance of displacement detection.

[0028] Oscillator: An oscillator that provides the operating clock for a digital signal processor (DSP). The oscillator requires a certain settling time (typically hundreds of microseconds to milliseconds) to turn on and off. Turning off the oscillator in low frame rate mode can significantly reduce power consumption, but in high frame rate mode, the oscillator must be kept on to avoid the start-stop time affecting the frame rate.

[0029] Dynamic Oscillator Control: A power optimization technique that dynamically controls the oscillator's operating state based on the frame rate mode: in low frame rate mode, the oscillator is turned off after DSP processing to save power; in high frame rate mode, the oscillator is conditionally switched on and off based on the inter-frame idle time; in ultra-high frame rate mode, the oscillator is kept on to meet the requirements of high frame rate and high IPS, achieving an intelligent balance between power consumption and performance.

[0030] IPS: Inches per second, a core technical parameter characterizing the maximum tracking speed in the fields of mouse input devices and optical sensing modules.

[0031] In existing technologies, the working architecture of traditional photoelectric navigation sensors, such as Figure 1As shown, the working module of the traditional photoelectric navigation sensor includes an image acquisition module, an image preprocessing module, an image matching module, and a displacement vector output module. The working method of the traditional photoelectric navigation sensor includes the following sequential steps: Step S101: Expose the current frame (Nth frame) image data to obtain the exposed image data.

[0032] Step S102: Convert the exposed image data to grayscale to obtain a grayscale image.

[0033] In step S103, the grayscale image undergoes preliminary processing by the image preprocessing module.

[0034] Step S104: Perform pixel matching between the preprocessed Nth frame image data and the (N-1)th frame image data.

[0035] Step S105: Output the displacement detection result.

[0036] Traditional photoelectric navigation sensors operate in a serial execution mode. This mode requires that the processing interval T between adjacent frames must be greater than or equal to the sum of the serial processing times described above, making the system unable to meet the real-time requirements of high frame rate applications.

[0037] Existing inter-frame parallel processing and cooperative frame rate control systems have the following shortcomings: (1) Low frame rate and limited maximum tracking IPS: The traditional mouse sensor processing architecture adopts inter-frame serial operation logic. The current image frame must wait for the previous frame to complete image parsing, displacement calculation and data output before the acquisition and processing process can be started. There is a strong time sequence dependency between frames, which can easily cause data processing delay and low running frame rate. At the same time, it greatly limits the maximum tracking motion IPS that the sensor can support. Under high-speed movement conditions, it is easy to have frame loss, positioning offset and trajectory recognition failure.

[0038] (2) Frame rate and parallel processing are disconnected: In the traditional mouse sensor architecture, the frame rate adjustment and the running state of the parallel processing module are independent of each other and lack coordinated control. There are technical problems such as the parallel processing module not being fully activated in high frame rate scenarios, resulting in processing bottlenecks, i.e., low parallelism processing is still used, wasting processing power, or the parallel module is still running at full load in low frame rate scenarios, resulting in power consumption waste. It is impossible to achieve the optimal match between frame rate and parallel processing efficiency.

[0039] (3) Insufficient power management of oscillators: The oscillators of traditional mouse sensors adopt a fixed start-stop mode and cannot dynamically adjust their working state according to the frame rate. In low frame rate scenarios, the oscillator remains in the always-on state, causing unnecessary power consumption. In high frame rate or high IPS demand scenarios, if the oscillator adopts intermittent start-stop control, its start-stop delay will affect the timing synchronization of image acquisition and data processing, which will lead to frame rate fluctuations and reduced positioning accuracy, and cannot take into account both power consumption control and high performance requirements.

[0040] Therefore, this application aims to address the problems of low frame rate and limited maximum tracking IPS value in traditional photoelectric navigation sensors, the disconnect between frame rate and parallel processing, and insufficient power management of oscillators, in order to achieve high-efficiency and low-power displacement detection.

[0041] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] like Figure 2 As shown, this application provides an inter-frame parallel processing and cooperative frame rate control system, which includes: an image acquisition module 101, a data scheduling and storage module 102, a parallel processing module 103, a cooperative frame rate control module 104, and a displacement output module 105.

[0043] The output of the image acquisition module 101 is connected to the input of the parallel processing module 103. The controlled end of the image acquisition module 101 is connected to the output of the collaborative frame rate control module 104. The output of the data scheduling and storage module 102 is connected to the input of the parallel processing module 103. The controlled end of the data scheduling and storage module 102 is connected to the output of the collaborative frame rate control module 104. The output of the parallel processing module 103 is connected to the input of the displacement output module 105 and the input of the collaborative frame rate control module 104. The controlled end of the parallel processing module 103 is connected to the output of the collaborative frame rate control module 104.

[0044] The image acquisition module 101 is used to acquire continuous multi-frame sequence image data; the continuous multi-frame sequence image data includes: historical reference frame image data, previous frame image data and current frame image data.

[0045] Specifically, the image acquisition module 101 provides the system with time-aligned raw image data.

[0046] The parallel processing module 103 is used to preprocess the continuous multi-frame sequence image data to obtain image preprocessing result data, and to perform displacement vector calculation based on the image preprocessing result data to obtain displacement vector results; the image preprocessing result data includes: image preprocessing result data of historical reference frames, image preprocessing result data of the previous frame, and image preprocessing result data of the current frame; the displacement vector results include: high-precision displacement vector results and coarse displacement vector results.

[0047] Specifically, the parallel processing module 103 implements switchable processing of inter-frame parallel / serial architecture to complete image preprocessing and displacement vector calculation.

[0048] The data scheduling and storage module 102 is used to perform cache management and read / write scheduling of the image preprocessing result data.

[0049] Specifically, the data scheduling and storage module 102 provides a conflict-free data access channel for inter-frame parallel processing.

[0050] The collaborative frame rate control module 104 is used to determine the target running frame rate based on the coarse displacement vector result, and dynamically switch the working mode, storage unit configuration information and oscillator control strategy based on the target running frame rate.

[0051] Specifically, the collaborative frame rate control module 104 realizes deep collaborative control of frame rate adjustment and processing architecture, as well as dynamic management of system power consumption.

[0052] The displacement output module 105 is used to convert the high-precision displacement vector result into output data that conforms to the peripheral protocol.

[0053] In an exemplary embodiment, the data scheduling and storage module 102 includes: a first storage unit 1021, a second storage unit 1022, a third storage unit 1023, and a scheduling controller 1024.

[0054] The first storage unit 1021 is used to store the image preprocessing result data of the historical reference frame.

[0055] The second storage unit 1022 is used to store the image preprocessing result data of the previous frame (the N-1th frame).

[0056] The third storage unit 1023 is used to store the image preprocessing result data of the current frame (the Nth frame).

[0057] The scheduling controller 1024 is used to manage the read and write address allocation of the first storage unit 1021, the second storage unit 1022 and the third storage unit 1023, and dynamically reallocate the storage mapping relationship of the first storage unit 1021, the second storage unit and the third storage unit 1023 according to the frame processing progress, so as to realize the cyclic reuse of the frame buffer.

[0058] It should be noted that the first storage unit 1021, the second storage unit 1022, and the third storage unit 1023 are all configured with independent full-duplex read / write channels, supporting the image preprocessing unit's write operation on the current frame image data. This is performed asynchronously and in parallel with the displacement calculation unit's read operation on the image preprocessing result data of the previous frame and the image preprocessing result data of the historical reference frame, without bus access conflicts. Furthermore, each storage unit can serve as a storage area for the image preprocessing data of the historical reference frame, the image preprocessing result data of the previous frame (N-1th frame), or the image preprocessing result data of the current frame (Nth frame).

[0059] In an exemplary embodiment, the parallel processing module 103 includes an image preprocessing unit 1031 and a displacement calculation unit 1032. Furthermore, the image preprocessing unit 1031 and the displacement calculation unit 1032 are two processing sub-units operating in parallel.

[0060] The image preprocessing unit 1031 is used to preprocess the continuous multi-frame sequence image data to obtain image preprocessing result data.

[0061] The image preprocessing unit 1031 is also used to preprocess the current frame image data and output the preprocessing result to the data scheduling and storage module 102.

[0062] The displacement calculation unit 1032 includes a displacement calculation subunit with switchable timing, namely a fast displacement calculation subunit 10321 and a high-precision displacement calculation subunit 10322.

[0063] The high-precision displacement calculation subunit 10322 is used to perform sub-pixel-level fine matching calculation on the image preprocessing result data of the previous frame and the image preprocessing result data of the historical reference frame to obtain a high-precision displacement vector result, and transmit the high-precision displacement vector result to the displacement output module 105.

[0064] The fast displacement calculation subunit 10321 is used to perform fast block matching calculation on the image preprocessing result data of the current frame and the image preprocessing result data of the historical reference frame to obtain a coarse displacement vector result, and transmit the coarse displacement vector result to the cooperative frame rate control module 104.

[0065] It should be noted that the core processing timing of the parallel processing module 103 is as follows: In the high frame rate parallel architecture mode, the image preprocessing unit 1031 of the current frame and the high-precision displacement calculation subunit 10322 of the previous frame are executed synchronously and in parallel; the processing of the fast displacement calculation subunit 10321 of the current frame can only be started after the image data preprocessing of the current frame is completed; in the low frame rate serial architecture mode, the parallel processing link is closed, and the serial timing execution of the displacement vector calculation of the image preprocessing data of the current frame starts only after the image data preprocessing of the current frame is completed. That is, after the image data of the current frame is acquired, the image data preprocessing of the current frame is performed, and after the preprocessing is completed, fast displacement matching calculation and sub-pixel level fine matching calculation are performed in sequence.

[0066] In an exemplary embodiment, the collaborative frame rate control module 104 includes: a motion state detection unit 1041, a frame rate decision unit 1042, an architecture switching control unit 1043, and an oscillator dynamic control unit 1044.

[0067] The movement state detection unit 1041 is used to receive the coarse displacement vector result and, based on the coarse displacement vector result, obtain the current movement speed and operating condition information of the device.

[0068] The frame rate decision unit 1042 is used to determine the target operating frame rate, working mode and storage unit configuration information based on the current device's moving speed and operating condition information.

[0069] The operating condition information includes conventional parameters such as movement status and image matching quality.

[0070] The oscillator dynamic control unit 1044 is used to determine the oscillator control strategy based on the target running frame rate.

[0071] In low frame rate mode, the oscillator is turned off after the digital signal processor completes the processing of the current frame image data to reduce static power consumption; in high frame rate mode, the oscillator is kept on to ensure the timing stability requirements of high frame rate and high effective tracking IPS, and to avoid the impact of oscillator start-stop delay on frame rate.

[0072] The architecture switching control unit 1043 is used to switch the working mode, the storage unit configuration information and the oscillator control strategy according to the target running frame rate.

[0073] Specifically, the working architecture enables a fully parallel inter-frame processing architecture in high frame rate mode and switches to an inter-frame serial processing architecture in low frame rate mode, while simultaneously shutting down unnecessary computing links.

[0074] In an exemplary embodiment, the frame rate decision unit 1042 includes: a data acquisition subunit 10421, a target frame rate determination subunit 10422, a frame rate difference judgment subunit 10423, an operating parameter configuration subunit 10424, and a closed-loop monitoring and control subunit 10425.

[0075] The data acquisition subunit 10421 is used to acquire the current moving speed and operating condition information of the device.

[0076] The target frame rate determination subunit 10422 is used to determine the target operating frame rate of the system based on the current device moving speed and operating condition information.

[0077] The frame rate difference judgment subunit 10423 is used to determine whether there is a difference between the target running frame rate and the current running frame rate, and obtain a first judgment result.

[0078] The running parameter configuration subunit 10424 is used to determine the corresponding working mode and storage unit configuration information according to the target running frame rate when the first judgment result is yes.

[0079] The closed-loop monitoring and control subunit 10425 is used to call and activate the data acquisition subunit 10421 to continuously acquire the current device moving speed and operating condition information when the first judgment result is negative, so as to realize dynamic closed-loop control.

[0080] In an exemplary embodiment, the frame rate mode range corresponding to the target running frame rate includes, from low to high, a low frame rate mode range, a high frame rate mode range, and an ultra-high frame rate mode range.

[0081] The operation parameter configuration subunit 10424 includes: a low frame rate adaptation unit 104241, a high frame rate adaptation unit 104242, and an ultra-high frame rate adaptation unit 104243.

[0082] The low frame rate adaptation unit 104241 is used to switch the system to serial working mode when the target running frame rate is within the range of the low frame rate mode. The serial working mode is configured as a dual storage unit mode. The dual storage unit mode enables the first storage unit 1021 and the second storage unit 1022 and disables the third storage unit 1023.

[0083] The high frame rate adaptation unit 104242 is used to switch the system to a parallel working mode when the target running frame rate is within the range of the high frame rate mode. The parallel working mode is configured as a three-storage-unit mode. The three-storage-unit mode enables the first storage unit 1021, the second storage unit 1022 and the third storage unit 1023.

[0084] The ultra-high frame rate adaptation unit 104243 is used to switch the system to the parallel working mode and configure it to the three-storage-unit mode when the target running frame rate is within the range of the ultra-high frame rate mode.

[0085] In one exemplary embodiment, the oscillator dynamic control unit 1044 includes: a first control subunit 10441, a second control subunit 10442, and a third control subunit 10443.

[0086] The first control subunit 10441 is used to employ an unconditional dynamic switching strategy for the oscillator when the target running frame rate is within the low frame rate mode range.

[0087] The second control subunit 10442 is used to employ an oscillator conditional switching strategy when the target running frame rate is within the high frame rate mode range.

[0088] The third control subunit 10443 is used to employ a strategy of keeping the oscillator always on when the target running frame rate is within the range of the ultra-high frame rate mode.

[0089] In one exemplary embodiment, this application provides an inter-frame parallel processing architecture and a triple-buffered parallel access mechanism.

[0090] This mechanism addresses the core technical problems of low frame rate and limited IPS values ​​in traditional serial architectures, such as... Figure 3 As shown, this is a timing diagram of a traditional serial architecture. In a traditional serial architecture, after the current frame image is acquired through exposure, image preprocessing, fast inter-frame matching, and high-precision inter-frame matching must be completed sequentially. Only after calculating and outputting the displacement result can the acquisition of the next frame be started. In order to save power consumption, the power supply is intermittently turned off between two frames.

[0091] like Figure 4 As shown in the figure, this application provides a timing diagram of a parallel pipeline architecture. In order to shorten the frame processing time and improve the frame rate, the preprocessing of the current frame image data and the high-precision matching unit of the previous frame image data are carried out in parallel through the parallel pipeline architecture.

[0092] In ultra-high frame rate mode, to minimize frame processing time, the oscillator is constantly on, and idle time is eliminated, thus meeting the requirements of ultra-high frame rate and ultra-high IPS. The architecture is as follows: Figure 5 As shown, this application provides a timing diagram of an ultra-high frame rate mode architecture.

[0093] To break the strong serial dependency between frames in the traditional architecture, the preprocessing of the current frame image data and the high-precision displacement calculation of the previous frame are executed synchronously and in parallel. This transforms the original serial superposition processing time into parallel processing time, eliminates inter-frame waiting redundancy, and significantly improves the system throughput and the maximum tracking IPS limit.

[0094] In one exemplary embodiment, this application provides a triple-buffered parallel access implementation mechanism, which achieves conflict-free data connection for parallel processing through independent read / write channels of three storage units. The principle of triple-buffered parallel access is as follows: Figure 6 As shown.

[0095] The image preprocessing unit 1031 writes the image preprocessing result of the current frame into the third storage unit 1023; the high-precision displacement calculation unit 1032 simultaneously reads the preprocessed data of the previous frame from the second storage unit 1022 and the preprocessed data of the historical reference frame from the first storage unit 1021, and performs inter-frame matching and displacement vector calculation. The above two operations are executed in parallel through independent read and write channels, without interference or bus access conflicts; after a single frame is processed, the scheduling controller 1024 updates the address pointer to realize the cyclic switching of the roles of the three storage unit buffers, completing the seamless flow of frame data.

[0096] To ensure the continuity and accuracy of inter-frame matching and to avoid matching failures caused by insufficient overlap between the image preprocessing results of historical reference frames and the image preprocessing results of the current frame after continuous device movement, a dynamic update sub-mechanism for the image preprocessing results of historical reference frames is set up.

[0097] The trigger condition for this mechanism is whether the image preprocessing result data update trigger condition of the historical reference frame is met, that is, whether the common range between the image preprocessing result data of the historical reference frame and the image preprocessing result data of the current frame is small enough. The mechanism is executed by seamlessly replacing the image preprocessing result data of the historical reference frame through address pointer remapping, requiring no additional data transfer, only modifying the logical role mapping of the storage unit, and without adding extra processing latency. Specifically, the common range between the image preprocessing result data of the historical reference frame and the image preprocessing result data of the current frame refers to the total area of ​​the effective pixel regions in the image preprocessing result data of the two frames that have spatially overlapping areas and whose sum of absolute differences (SAD) of corresponding pixels is lower than a preset matching threshold.

[0098] When the area of ​​the common interval is less than the proportion of the total effective pixel area of ​​the preprocessing result data of a single frame image to a preset update threshold, it is determined that the common interval is small enough, and the system triggers dynamic updates of the image preprocessing result data of the historical reference frame.

[0099] When the area of ​​the common interval is greater than or equal to the proportion of the total effective pixel area of ​​the preprocessed image data of a single frame, it is determined that the common interval is not small enough, and the system performs a regular pointer loop operation.

[0100] In an exemplary embodiment, the standard three-buffered data transfer process of this application includes the following steps: Step S201: The image acquisition module 101 acquires the original grayscale image data (i.e., the current frame image data) of the current frame (the Nth frame).

[0101] In step S202, the image preprocessing unit 1031 reads the current frame image data and performs preprocessing operations.

[0102] Step S203: Write the preprocessed current frame data (i.e., the image preprocessing result data of the current frame) into the third storage unit 1023.

[0103] Step S204 is performed synchronously with step S202. The high-precision displacement calculation subunit 10322 reads the preprocessed data of the previous frame (the N-1th frame) from the second storage unit 1022 (i.e., the image preprocessing result data of the previous frame), and reads the preprocessed data of the historical reference frame (the N-2th frame or an earlier reference frame) from the first storage unit 1021 (i.e., the image preprocessing result data of the historical reference frame), and performs subpixel-level displacement matching calculation.

[0104] Step S205: After the preprocessing of the current frame image data and the displacement vector calculation of the image preprocessing result data of the previous frame are completed, it is determined whether the triggering condition for updating the image preprocessing result data of the historical reference frame is met, that is, whether the common interval between the image preprocessing result data of the historical reference frame and the image preprocessing result data of the current frame is small enough; if the above triggering condition is met, proceed to step S206 (execute the image preprocessing result data update of the historical reference frame); if not, proceed to step S207 (execute the normal pointer loop).

[0105] Step S206: Perform the image preprocessing result data update operation of the historical reference frame: The scheduling controller 1024 remaps the address pointers of the three storage units to complete the replacement and role reset of the image preprocessing result data of the historical reference frame: The third storage unit 1023 (image preprocessing result data of the current frame): remapped to the image preprocessing result data of the new historical reference frame; The second storage unit 1022 (image preprocessing result data of the previous frame): remapped to the storage area of ​​the image preprocessing result data of the next frame (frame N+1); The first storage unit 1021 (image preprocessing result data of the original historical reference frame): remapped to the image preprocessing result data of the new previous frame (i.e., the image preprocessing result data of the original current frame N); The cumulative displacement is synchronously cleared, the matching similarity benchmark value is updated, and the process proceeds to step S208.

[0106] Step S207, perform regular pointer loop operation (i.e., the original step S205, keeping the image preprocessing result data of the historical reference frame unchanged): the scheduling controller 1024 updates the storage unit address pointer and completes the loop switching of the buffer role: the first storage unit 1021: keeps the mapping relationship of the image preprocessing result data of the historical reference frame unchanged; the second storage unit 1022: is mapped to the storage area of ​​the preprocessing result of the next frame (the N+1th frame) (i.e., the image preprocessing result data of the next frame); the third storage unit 1023: is mapped to the storage area of ​​the preprocessing result of the Nth frame (i.e., the image preprocessing result data of the current frame), and is used as the image preprocessing result data of the previous frame for the next frame displacement vector calculation; proceed to step S208.

[0107] In step S208, the high-precision displacement calculation result is output to the displacement output module 105, completing the single-frame parallel processing flow.

[0108] In one exemplary embodiment, this application provides a collaborative frame rate control mechanism, which addresses the core technical problems of frame rate disconnection from parallel processing and insufficient power management in traditional architectures. Figure 7 This is a schematic block diagram of the collaborative frame rate control logic of this application. Figure 7 The frame rate range in the logic block diagram is for illustrative purposes only. The actual definition is affected by multiple factors (oscillator clock frequency, sensor pixel array size, etc.).

[0109] In one exemplary embodiment, to overcome the limitations of traditional technologies that adjust the frame rate solely based on movement speed, this application achieves end-to-end coordinated control of frame rate adjustment, operating mode, storage unit configuration information, and oscillator control strategy. It dynamically matches the optimal processing architecture and power consumption strategy based on the target operating frame rate, achieving an intelligent balance between high performance and low power consumption. This application defines a hierarchical operating mode, dividing the system into three levels based on the frame rate range. The configuration rules for each mode are as follows: Low frame rate mode: The frame period is sufficient to meet the timing requirements of serial processing. It adopts a "serial operation mode + dual memory unit configuration + oscillator unconditional dynamic switching strategy" to simplify control logic and minimize power consumption. The operating rules of the dual memory units in low frame rate mode are as follows: Figure 8 As shown.

[0110] The dual storage unit configuration information is configured as follows: the first storage unit 1021 stores the image preprocessing result data of the previous frame, the second storage unit 1022 stores the image preprocessing result data of the current frame, and the third storage unit 1023 is turned off to reduce power consumption.

[0111] Serial processing logic: In the serial processing flow, the preprocessing of the current frame image data is completed first, and the image preprocessing result data of the current frame is written into the second storage unit 1022. Then, the image preprocessing result data of the previous frame is read from the first storage unit 1021 and the image preprocessing result data of the current frame is read from the second storage unit 1022. Frame matching and displacement vector calculation are then performed.

[0112] Pointer switching rule: After a single frame is processed, the scheduling controller 1024 remaps the second storage unit 1022 to the new storage area of ​​the previous frame through address pointer remapping, and remaps the first storage unit 1021 to the water level storage area of ​​the image preprocessing result data of the next frame, so as to realize the cyclic reuse of the dual storage units.

[0113] High frame rate mode: The frame period is short and parallel processing is required to meet timing requirements. It adopts "parallel working mode + three memory unit configuration + oscillator conditional switching strategy" to balance processing performance and power consumption.

[0114] Ultra-high frame rate mode: The frame period is extremely short, and the oscillator start-stop stabilization time exceeds the frame period margin. It adopts "parallel working mode + three memory unit configuration + oscillator always-on strategy" to ensure timing stability and tracking accuracy under high frame rate and high IPS.

[0115] In an exemplary embodiment, the collaborative frame rate control standard process of this application is as follows: In step S301, the movement status detection unit 1041 obtains the current movement speed V and operating condition information of the device.

[0116] In step S302, the frame rate decision unit 1042 determines the target operating frame rate F_target of the system based on the current device movement speed and operating condition information.

[0117] Step S303: Determine whether there is a difference between the target running frame rate F_target and the current running frame rate. If the frame rate switching condition is met (i.e., there is a difference between the target running frame rate and the current running frame rate), proceed to step S304; if not, maintain the current working state and return to step S301 for continuous monitoring: if the current device moving speed is detected to be relatively fast, increase the frame rate; if the current device moving speed is detected to be relatively slow, decrease the frame rate.

[0118] Step S304: Determine the corresponding working mode and storage unit configuration information based on the target running frame rate F_target.

[0119] If F_target falls within the low frame rate mode range: the system switches to serial working mode, is configured as a dual storage unit mode, and disables the third storage unit 1023.

[0120] If F_target falls within the high frame rate mode range: the system switches to parallel working mode and is configured as a three-storage-unit mode, that is, all storage units are enabled.

[0121] If F_target falls within the ultra-high frame rate mode range: the system switches to parallel working mode and is configured as a three-memory unit mode.

[0122] Step S305: Determine the corresponding oscillator control strategy based on the target frame rate F_target. The execution rules and applicable scenarios of each strategy are as follows: If F_target belongs to the low frame rate mode range: adopt the oscillator unconditional dynamic switching strategy; the execution rule of the oscillator unconditional dynamic switching strategy is that after the DSP completes the full process of the current frame image data, it immediately shuts down the oscillator to cut off static power consumption; before the next frame image data acquisition starts, the oscillator is started in advance according to the preset oscillator start-stop stabilization time to ensure that the clock is stable before starting the next frame processing.

[0123] If F_target belongs to the high frame rate mode range: the oscillator conditional switching strategy is adopted; the execution rule of the oscillator conditional switching strategy is that after the DSP completes the full process of the current frame image data, it first calculates the inter-frame idle time T_idle from the end of the current frame image data processing to the start of the next frame image data acquisition, compares T_idle with the preset oscillator start-stop stabilization time T_stable, and makes a judgment.

[0124] If T_idle≥n×T_stable, that is, the safe start-stop window is met, then the oscillator is turned off, and the oscillator is started in advance according to the preset oscillator start-stop stabilization time before the next frame of image data is acquired.

[0125] If T_idle < n × T_stable, meaning the safe start / stop window is not met, then the oscillator is kept on to avoid start / stop delays affecting the timing and frame rate stability of the next frame.

[0126] Wherein, n is the safety margin coefficient for starting and stopping the oscillator, and n is a real number greater than 1. Its value is preset in the system register and can be adjusted according to hardware characteristics.

[0127] The oscillator start-stop stabilization time T_stable is a fixed time required for the oscillator to output a stable clock signal from startup, which is preset in the system register; the n times T_stable safety threshold is a reserved timing protection margin to avoid the risk of oscillator stabilization timeout caused by operating condition fluctuations.

[0128] The inter-frame idle time T_idle is the time interval between the end of the DSP process for the current frame image data and the start of the acquisition of the next frame image data, which is determined by the system's target frame rate and the processing time per frame.

[0129] If F_target falls within the ultra-high frame rate mode range: adopt the oscillator always-on strategy; the execution rule of the oscillator always-on strategy is to keep the oscillator continuously on throughout the entire frame period, completely avoiding the timing impact of oscillator start-stop delay on high frame rate and high IPS performance, and ensuring the timing synchronization of image acquisition and displacement vector calculation.

[0130] Step S306: Synchronize the switching of the collaborative execution architecture, storage unit, and oscillator to avoid timing conflicts.

[0131] Step S307: Update the frame processing parameters, including exposure time, processing timeout time, and parallel synchronization parameters.

[0132] Step S308, return to step S301, continuously monitor the movement status of the equipment, and realize dynamic closed-loop control.

[0133] This mechanism achieves the optimal balance between power consumption and performance across the entire frame rate range through hierarchical adaptation of a three-stage oscillator control strategy: extreme power reduction at low frame rates, balance between power consumption and timing stability at high frame rates, and absolute high-performance output at ultra-high frame rates, thus completely solving the technical defect in traditional architectures where power consumption control and frame rate stability cannot be balanced.

[0134] Based on the same inventive concept, this application also provides an inter-frame parallel processing and cooperative frame rate control method based on the aforementioned inter-frame parallel processing and cooperative frame rate control system. The solution provided by this method is similar to the implementation scheme described in the above system. Therefore, the specific limitations of one or more inter-frame parallel processing and cooperative frame rate control method embodiments provided below can be found in the above-described limitations of an inter-frame parallel processing and cooperative frame rate control system, and will not be repeated here.

[0135] In one exemplary embodiment, such as Figure 9 As shown, a method for inter-frame parallel processing and cooperative frame rate control is provided, including: Step S401: Acquire continuous multi-frame sequence image data; the continuous multi-frame sequence image data includes: historical reference frame image data, previous frame image data and current frame image data.

[0136] Step S402: Preprocess the continuous multi-frame sequence image data to obtain image preprocessing result data, and calculate the displacement vector based on the image preprocessing result data to obtain the displacement vector result; the image preprocessing result data includes: image preprocessing result data of historical reference frames, image preprocessing result data of the previous frame, and image preprocessing result data of the current frame; the displacement vector result includes: high-precision displacement vector result and coarse displacement vector result.

[0137] Step S403: Based on the coarse displacement vector result, determine the target running frame rate, and dynamically switch the working mode, storage unit configuration information and oscillator control strategy based on the target running frame rate.

[0138] Step S404: Convert the high-precision displacement vector result into output data that conforms to the peripheral protocol.

[0139] As an optional implementation, before step S403 above, the method may further include the following steps S501 to S505.

[0140] Step S501: Determine whether the current working mode of the system is low frame rate mode, and obtain the second determination result.

[0141] In step S502, when the second judgment result is yes, the system executes the serial working mode, is configured as a dual storage unit mode, and adopts the oscillator unconditional dynamic switching strategy.

[0142] Step S503: When the second judgment result is negative, determine whether the current working mode of the system is high frame rate mode, and obtain the third judgment result.

[0143] In step S504, when the third judgment result is yes, the system executes the parallel working mode, is configured as a three-memory-cell mode, and adopts the oscillator conditional switching strategy.

[0144] In step S505, when the third judgment result is negative, the system executes the parallel working mode, is configured as a three-memory-cell mode, and adopts the oscillator always-on strategy.

[0145] As an optional implementation, step S403 can be replaced by steps S601 to S603.

[0146] Step S601: Based on the coarse displacement vector result of the current frame, determine the target running frame rate, and determine whether the target running frame rate needs to be adjusted to obtain the fourth determination result.

[0147] In step S602, when the fourth judgment result is yes, the system synchronously completes the pre-configuration of parameters for the working mode, storage unit configuration, and oscillator control strategy, and dynamically executes the switching when the next frame is processed.

[0148] In step S603, when the fourth judgment result is negative, the system executes the oscillator control strategy of the current frame to complete the dynamic switching of the clock state.

[0149] As another alternative implementation method, such as Figure 10 As shown, another method for inter-frame parallel processing and collaborative frame rate control is provided, which specifically includes the following steps.

[0150] In step S701, the image acquisition module 101 starts and completes the grayscale image acquisition of the current frame operation surface (i.e., acquires the current frame image data) according to the current configuration parameters, and outputs the raw image data to the parallel processing module 103.

[0151] Step S702: Determine whether the current working mode of the system is low frame rate mode. If yes, proceed to step S703; otherwise, proceed to step S704.

[0152] Step S703: Execute the low frame rate mode configuration: Enable dual storage unit configuration, disable unnecessary third storage unit 1023 to reduce power consumption; Start the serial processing flow: Image preprocessing unit 1031 completes preprocessing of the current frame's original image (current frame image data), and then displacement calculation unit 1032 sequentially completes fast block matching calculation and sub-pixel-level high-precision displacement calculation of the current frame's image preprocessing result data, without parallel operation; Synchronously execute the oscillator unconditional dynamic switching strategy: After the DSP completes the full process of the current frame's operation, immediately turn off the oscillator, and before the next frame's image data acquisition starts, turn on the oscillator after a pre-set oscillator start-stop stabilization time to ensure clock stability; After this step is completed, proceed to step S707.

[0153] Step S704: Determine whether the current working mode of the system is high frame rate mode. If yes, proceed to step S705; otherwise, proceed to step S706.

[0154] Step S705: Execute the high frame rate mode configuration: Enable the three-storage-unit configuration to support parallel conflict-free read and write; Start the parallel processing flow and execute inter-frame parallel operations: While the image preprocessing unit 1031 preprocesses the original image of the current frame (current frame image data), the high-precision displacement calculation unit 1032 performs sub-pixel-level high-precision displacement calculation on the image preprocessing result data of the previous frame. The two operations are executed synchronously and in parallel through independent read and write channels without bus access conflicts; After the parallel processing is completed, the fast block matching calculation of the image preprocessing result data of the current frame is started to obtain a rough displacement result, which is output to the collaborative frame rate control module 104; Synchronously execute the oscillator conditional switching strategy: After the DSP completes the full-process operation of the current frame image data, it first determines whether the inter-frame idle time meets the safe start-stop window. If it does, the oscillator is turned off; otherwise, the oscillator is kept on. After this step is completed, proceed to step S707.

[0155] Step S706: Execute the configuration for the ultra-high frame rate mode: Enable the three-storage-unit configuration to support parallel conflict-free read and write; Start the parallel processing flow and perform inter-frame parallel operations: While the image preprocessing unit 1031 preprocesses the original image of the current frame, the high-precision displacement calculation unit 1032 performs sub-pixel-level high-precision displacement calculation on the image preprocessing result data of the previous frame. The two operations are executed synchronously and in parallel through independent read and write channels without bus access conflicts; After the parallel processing is completed, the fast block matching calculation of the image preprocessing result data of the current frame is started to obtain the coarse displacement vector result and output it to the collaborative frame rate control module 104; The oscillator always-on strategy is executed synchronously to keep the oscillator continuously on throughout the entire frame cycle, avoiding the timing impact of start-stop delay on high frame rate and high IPS performance; After this step is completed, proceed to step S707.

[0156] In step S707, the displacement output module 105 converts the data format and encapsulates the communication protocol of the high-precision displacement vector result obtained in this calculation, and outputs the displacement result to the outside world.

[0157] In step S708, the collaborative frame rate control module 104 performs closed-loop control: based on the image preprocessing result data, displacement vector result and image matching quality of the current frame, it determines whether the target running frame rate needs to be adjusted. If adjustment is required, it synchronously completes the pre-configuration of parameters for the working mode, storage unit configuration information and oscillator control strategy, and waits for the next frame to perform the switch.

[0158] Step S709: Execute the current frame oscillator control strategy to complete the clock state switch.

[0159] Step S710: Determine whether the system needs to continue acquiring the next frame of image data. If it is determined to continue acquiring, return to step S701 and start the next frame of image data acquisition and processing loop; if it is determined to stop working, then end.

[0160] This application addresses the issues of low frame rate and limited maximum tracking IPS: by constructing an inter-frame parallel processing architecture, it breaks the bottleneck of strong inter-frame timing dependency in traditional serial architectures, enabling the image preprocessing process of the current frame and the high-precision displacement calculation process of the previous frame to be executed synchronously and in parallel, directly eliminating the redundant waiting time between frames, significantly compressing the total processing time of a single frame, and improving the system's processing throughput; coupled with a three-storage-unit independent full-duplex read / write channel design, it completely solves the data read / write access conflict problem of traditional architectures, further releasing parallel computing power, significantly increasing the system's frame rate limit and maximum effective tracking IPS limit, fundamentally avoiding frame loss, positioning offset, and trajectory recognition failure problems under high-speed movement conditions, maintaining sub-pixel-level displacement detection accuracy throughout, and fully meeting the stringent requirements of high-performance scenarios.

[0161] This application solves the problem of the disconnect between frame rate and parallel processing: by constructing a full-link collaborative frame rate control mechanism, it breaks through the limitation of the traditional solution where frame rate adjustment and processing architecture are independent of each other, and realizes deep binding and synchronous linkage between frame rate adjustment and working mode, storage unit configuration, and oscillator working state: when the sensor is in a high frame rate / ultra-high frame rate operating condition, the inter-frame fully parallel processing architecture is automatically enabled, and the three storage units are simultaneously enabled to maximize the release of parallel computing power, match the real-time processing requirements under high frame rate, and avoid performance bottlenecks caused by insufficient computing power; when the sensor switches to a low frame rate operating condition, it automatically switches to an inter-frame serial processing architecture, and the unnecessary third storage unit 1023 is simultaneously shut down, retaining only the core computing link. With the corresponding power consumption optimization strategy, the problem of ineffective power consumption waste in low load scenarios is completely solved, and the dynamic optimal matching of processing efficiency, timing stability and power consumption is achieved under all operating conditions.

[0162] This application addresses the problem of insufficient oscillator power management by designing a hierarchical oscillator dynamic control mechanism. This overcomes the limitations of traditional fixed start-stop modes and resolves the core contradiction of balancing power control and frame rate stability. Based on frame rate and IPS requirements, it achieves adaptive adjustment of the oscillator's operating state: When the sensor operates at a low frame rate, an unconditional dynamic switching strategy is implemented, immediately shutting down the oscillator after the digital signal processor completes the full-process calculation of the current frame, cutting off static power consumption and achieving extreme power management; when the sensor switches to a high frame rate, a conditional switching strategy is implemented, using a safety threshold of inter-frame idle time and oscillator start-stop stability time to further reduce operating power consumption while ensuring absolute timing stability and no frame rate fluctuations; when the sensor switches to ultra-high frame rate and high IPS operating conditions, a always-on oscillator strategy is implemented, completely avoiding the timing impact of oscillator start-stop delay on high frame rate and high IPS performance, achieving a balance between power control and high performance requirements across the entire frame rate range.

[0163] Therefore, through the synergistic effect of the above three technical solutions, this application can achieve high-performance and stable output of e-sports-grade IPS with high response speed and high effective tracking IPS while ensuring high accuracy of displacement detection throughout the process. This fully adapts to high-performance application scenarios such as e-sports mice, which have stringent requirements for real-time positioning and high-speed tracking reliability. At the same time, through dynamic architecture switching and hierarchical power consumption optimization strategies, it significantly reduces the ineffective power consumption in low frame rate operation scenarios, achieving a balance between high-performance device indicators and low power consumption requirements.

[0164] Furthermore, the inter-frame parallel processing architecture and triple-buffered parallel access mechanism of this application break the traditional inter-frame serial dependency, realize the parallel execution of the preprocessing of the current frame image data and the bit vector shift calculation of the image preprocessing data of the previous frame. With the independent read and write channels of the three storage units, data access conflicts and inter-frame waiting time are eliminated, fundamentally improving the system frame rate and the maximum tracking IPS limit. At the same time, it supports dynamic switching between serial and parallel dual modes, taking into account both performance and power consumption.

[0165] The end-to-end collaborative frame rate control mechanism of this application breaks through the limitations of traditional single frame rate adjustment, realizes end-to-end collaboration between frame rate adjustment and working mode, storage unit configuration, and oscillator working state, dynamically matches the optimal system configuration according to real-time working conditions, completely solves the problem of frame rate being disconnected from processing architecture, and achieves intelligent balance between performance and power consumption.

[0166] The hierarchical dynamic power consumption management mechanism of this application includes two core technologies: dynamic oscillator control and dynamic configuration of memory units. It adjusts the oscillator switching strategy and the number of memory units enabled according to the frame rate mode. Under the premise of ensuring high frame rate and high IPS performance, it maximizes the reduction of static power consumption and operating power consumption in low load scenarios, thus solving the problem of insufficient power consumption management in traditional architectures.

[0167] This application addresses the core technological shortcomings of traditional photoelectric navigation sensors by achieving a deep balance between high performance and low power consumption through three core innovations: inter-frame parallel processing architecture, end-to-end collaborative frame rate control, and hierarchical dynamic power consumption management. Compared to existing technologies, its core advantages are as follows: (1) Significantly improved core processing performance, with a substantial increase in frame rate and tracking IPS limit: This application breaks through the strong inter-frame dependency bottleneck of the traditional serial architecture. Through the inter-frame parallel processing architecture, it achieves synchronous parallel execution of image preprocessing of the current frame and high-precision displacement calculation of the previous frame, directly eliminating the redundant waiting time between frames, significantly compressing the total processing time of a single frame, and improving the system's processing throughput. Combined with the three-buffered independent read and write channel design, it completely solves the data access conflict problem of the traditional architecture, significantly increases the system frame rate limit, and simultaneously improves the sensor's maximum effective tracking IPS limit. This fundamentally avoids frame loss, positioning offset, and trajectory recognition failure problems under high-speed movement conditions, maintaining sub-pixel level displacement detection accuracy throughout the process, and fully meeting the stringent requirements of high-performance application scenarios such as e-sports mice.

[0168] (2) Intelligent collaborative adaptation across the entire link, completely solving the problem of the disconnect between frame rate and processing capacity: This application breaks through the limitations of the traditional solution of single frame rate adjustment, and constructs a collaborative control mechanism for the entire link of frame rate, architecture, storage, and clock. It can dynamically determine the target running frame rate according to the real-time moving speed of the device and the image matching quality, and synchronously switch the working mode, storage unit configuration and oscillator control strategy. Under low frame rate conditions, it automatically switches between serial processing architecture and dual storage unit configuration, simplifying the control logic and avoiding computing power redundancy; under high / ultra-high frame rate conditions, it automatically switches between parallel processing architecture and full configuration of three storage units, maximizing the release of computing power, realizing accurate matching of processing efficiency and load requirements under all conditions, without performance bottlenecks and ineffective computing power waste.

[0169] (3) Hierarchical and refined power consumption management, significantly optimizing power consumption levels across all scenarios: This application achieves precise power consumption management across the entire frame rate range through two core technologies: hierarchical dynamic control of the oscillator and dynamic configuration of the storage unit. Under the premise of ensuring high-performance and stable output, it significantly reduces the operating power consumption of the device. In low frame rate mode, the static power consumption of the system and the static power consumption of the memory are significantly reduced by unconditionally switching the oscillator and shutting down unnecessary storage units. In high frame rate mode, the operating power consumption is further reduced by the conditional switching strategy of the oscillator with a safety margin, under the premise of absolutely ensuring timing stability and no frame rate fluctuation. In ultra-high frame rate mode, the oscillator is kept on all the time, completely avoiding the impact of start-stop delay on high frame rate and high IPS performance, achieving a two-way balance between extreme power reduction under low load and absolute performance protection under high load, and significantly improving the device's battery life.

[0170] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 11 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores consecutive multi-frame sequence image data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an inter-frame parallel processing and cooperative frame rate control method.

[0171] Figure 11 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0172] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0173] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0174] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations and be authorized by the owner of the corresponding device.

[0175] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0176] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0177] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A frame-to-frame parallel processing and cooperative frame rate control system, characterized in that, The inter-frame parallel processing and cooperative frame rate control system includes: an image acquisition module, a data scheduling and storage module, a parallel processing module, a cooperative frame rate control module, and a displacement output module. The output of the image acquisition module is connected to the input of the parallel processing module; the controlled end of the image acquisition module is connected to the output of the collaborative frame rate control module; the output of the data scheduling and storage module is connected to the input of the parallel processing module; the controlled end of the data scheduling and storage module is connected to the output of the collaborative frame rate control module; the output of the parallel processing module is connected to the input of the displacement output module and the input of the collaborative frame rate control module; and the controlled end of the parallel processing module is connected to the output of the collaborative frame rate control module. The image acquisition module is used to acquire continuous multi-frame sequence image data; the continuous multi-frame sequence image data includes: historical reference frame image data, previous frame image data, and current frame image data; The parallel processing module is used to preprocess the continuous multi-frame sequence image data to obtain image preprocessing result data, and to calculate the displacement vector based on the image preprocessing result data to obtain the displacement vector result; the image preprocessing result data includes: image preprocessing result data of historical reference frames, image preprocessing result data of the previous frame, and image preprocessing result data of the current frame; the displacement vector result includes: high-precision displacement vector result and coarse displacement vector result; The data scheduling and storage module is used to perform cache management and read / write scheduling of the image preprocessing result data; The collaborative frame rate control module is used to determine the target running frame rate based on the coarse displacement vector result, and dynamically switch the working mode, storage unit configuration information and oscillator control strategy based on the target running frame rate; The displacement output module is used to convert the high-precision displacement vector result into output data that conforms to the peripheral protocol.

2. The inter-frame parallel processing and cooperative frame rate control system according to claim 1, characterized in that, The parallel processing module includes: an image preprocessing unit and a displacement calculation unit; The image preprocessing unit is used to preprocess the continuous multi-frame sequence image data to obtain image preprocessing result data; The displacement calculation unit includes: a fast displacement calculation subunit and a high-precision displacement calculation subunit; The high-precision displacement calculation subunit is used to perform sub-pixel-level fine matching calculation on the image preprocessing result data of the previous frame and the image preprocessing result data of the historical reference frame to obtain a high-precision displacement vector result, and transmit the high-precision displacement vector result to the displacement output module; The fast displacement calculation subunit is used to perform fast block matching calculation on the image preprocessing result data of the current frame and the image preprocessing result data of the historical reference frame to obtain a coarse displacement vector result, and transmit the coarse displacement vector result to the cooperative frame rate control module.

3. The inter-frame parallel processing and cooperative frame rate control system according to claim 2, characterized in that, The collaborative frame rate control module includes: a motion state detection unit, a frame rate decision unit, an architecture switching control unit, and an oscillator dynamic control unit; The movement state detection unit is used to receive the coarse displacement vector result and, based on the coarse displacement vector result, obtain the current movement speed and operating condition information of the device; The frame rate decision unit is used to determine the target operating frame rate, working mode, and storage unit configuration information based on the current device's moving speed and operating condition information. The oscillator dynamic control unit is used to determine the oscillator control strategy based on the target running frame rate; The architecture switching control unit is used to switch the operating mode, the storage unit configuration information, and the oscillator control strategy according to the target operating frame rate.

4. The inter-frame parallel processing and cooperative frame rate control system according to claim 3, characterized in that, The frame rate decision unit includes: a data acquisition subunit, a target frame rate determination subunit, a frame rate difference judgment subunit, an operating parameter configuration subunit, and a closed-loop monitoring and control subunit. The data acquisition subunit is used to acquire the current moving speed and operating condition information of the device; The target frame rate determination subunit is used to determine the target operating frame rate of the system based on the current device movement speed and operating condition information; The frame rate difference judgment subunit is used to determine whether there is a difference between the target running frame rate and the current running frame rate, and to obtain a first judgment result; The running parameter configuration subunit is used to determine the corresponding working mode and storage unit configuration information according to the target running frame rate when the first judgment result is yes. The closed-loop monitoring and control subunit is used to call and activate the data acquisition subunit to continuously acquire the current device movement speed and operating condition information when the first judgment result is negative, so as to realize dynamic closed-loop control.

5. The inter-frame parallel processing and cooperative frame rate control system according to claim 1, characterized in that, The data scheduling and storage module includes: a first storage unit, a second storage unit, a third storage unit, and a scheduling controller; The first storage unit is used to store the image preprocessing result data of the historical reference frame; The second storage unit is used to store the image preprocessing result data of the previous frame; The third storage unit is used to store the image preprocessing result data of the current frame; The scheduling controller is used to manage the read and write address allocation of the first storage unit, the second storage unit, and the third storage unit, and dynamically reallocates the storage mapping relationship of the first storage unit, the second storage unit, and the third storage unit according to the frame processing progress, so as to realize the cyclic reuse of the frame buffer.

6. The inter-frame parallel processing and cooperative frame rate control system according to claim 5, characterized in that, The frame rate mode range corresponding to the target running frame rate, from low to high, includes: low frame rate mode range, high frame rate mode range, and ultra-high frame rate mode range. The operation parameter configuration subunit includes: a low frame rate adaptation unit, a high frame rate adaptation unit, and an ultra-high frame rate adaptation unit. The low frame rate adaptation unit is used to switch the system to serial working mode when the target running frame rate is within the range of the low frame rate mode. The serial working mode is configured as a dual storage unit mode. The dual storage unit mode enables the first storage unit and the second storage unit and disables the third storage unit. The high frame rate adaptation unit is used to switch the system to a parallel working mode when the target running frame rate is within the range of the high frame rate mode. The parallel working mode is configured as a three-storage-unit mode. The three-storage-unit mode enables the first storage unit, the second storage unit, and the third storage unit. The ultra-high frame rate adaptation unit is used to switch the system to the parallel working mode and configure it to the three-storage-unit mode when the target running frame rate is within the range of the ultra-high frame rate mode.

7. The inter-frame parallel processing and cooperative frame rate control system according to claim 6, characterized in that, The oscillator dynamic control unit includes: a first control subunit, a second control subunit, and a third control subunit; The first control subunit is used to employ an unconditional dynamic switching strategy for the oscillator when the target running frame rate is within the low frame rate mode range. The second control subunit is used to employ an oscillator conditional switching strategy when the target running frame rate is within the high frame rate mode range; The third control subunit is used to employ a strategy of keeping the oscillator always on when the target frame rate is within the range of the ultra-high frame rate mode.

8. A method for inter-frame parallel processing and cooperative frame rate control, characterized in that, The inter-frame parallel processing and cooperative frame rate control method is implemented based on the inter-frame parallel processing and cooperative frame rate control system according to any one of claims 1-7, and the inter-frame parallel processing and cooperative frame rate control method includes: Acquire continuous multi-frame sequence image data; the continuous multi-frame sequence image data includes: historical reference frame image data, previous frame image data, and current frame image data; The continuous multi-frame sequence of image data is preprocessed to obtain image preprocessing result data, and displacement vector is calculated based on the image preprocessing result data to obtain displacement vector results; the image preprocessing result data includes: image preprocessing result data of historical reference frames, image preprocessing result data of the previous frame, and image preprocessing result data of the current frame; the displacement vector results include: high-precision displacement vector results and coarse displacement vector results; Based on the coarse displacement vector results, the target running frame rate is determined, and the working mode, storage unit configuration information and oscillator control strategy are dynamically switched based on the target running frame rate. The high-precision displacement vector result is converted into output data that conforms to the peripheral protocol.

9. The inter-frame parallel processing and cooperative frame rate control method according to claim 8, characterized in that, Before the steps of preprocessing the continuous multi-frame sequence image data to obtain image preprocessing result data, and calculating the displacement vector based on the image preprocessing result data to obtain the displacement vector result, the method further includes: Determine whether the system's current operating mode is low frame rate mode, and obtain the second determination result; When the second judgment result is yes, the system executes the serial working mode, is configured as a dual memory unit mode, and adopts the oscillator unconditional dynamic switching strategy. If the second judgment result is negative, determine whether the current working mode of the system is high frame rate mode, and obtain the third judgment result; When the third judgment result is yes, the system executes the parallel working mode, is configured as a three-memory cell mode, and adopts the oscillator conditional switching strategy; When the third judgment result is negative, the system executes the parallel working mode, is configured as a three-memory unit mode, and adopts the oscillator always-on strategy.

10. The inter-frame parallel processing and cooperative frame rate control method according to claim 8, characterized in that, Based on the coarse displacement vector results, the target running frame rate is determined, and the working mode, storage unit configuration information, and oscillator control strategy are dynamically switched based on the target running frame rate, specifically including: Based on the coarse displacement vector result of the current frame, the target running frame rate is determined, and it is determined whether the target running frame rate needs to be adjusted, thus obtaining a fourth determination result; When the fourth judgment result is yes, the system synchronously completes the pre-configuration of parameters for working mode, storage unit configuration, and oscillator control strategy, and dynamically executes the switching when the next frame is processed; When the result of the fourth judgment is negative, the system executes the oscillator control strategy of the current frame to complete the dynamic switching of the clock state.

Images