Image acquisition method, system, device and readable storage medium

Abstract

The application discloses an image acquisition method, system, device and readable storage medium, relates to the technical field of wearable devices, and the method is applied to a wearable device. The wearable device comprises an MCU and an ISP, and the MCU and the ISP are connected through GPIO pins. The method comprises the following steps: if the MCU detects an image acquisition event trigger, an image acquisition start signal is sent to the ISP through the GPIO pin of the MCU; and if the ISP receives the image acquisition start signal, the ISP starts to acquire image data. The application improves the starting speed of ISP image acquisition and improves the real-time response of ISP image acquisition.

Description

Technical Field

[0001] This application relates to the field of wearable device technology, and in particular to an image acquisition method, system, device and readable storage medium. Background Technology

[0002] With the rapid development of technology, wearable devices have gradually become an indispensable part of people's lives. From smartwatches to augmented reality (AR) glasses, these devices, with their portability and versatility, have greatly enriched the user's interactive experience. Among these devices, the microcontroller unit (MCU) and the image signal processor (ISP) are two key components. The MCU is primarily responsible for the acquisition and processing of sensor data, such as data from gyroscopes and accelerometers, which is crucial for the device's spatial positioning and motion monitoring. The ISP, on the other hand, focuses on processing image data captured by cameras, providing users with clear, high-quality visual content. The collaborative work between the two ensures the efficient operation of wearable devices in various application scenarios.

[0003] Currently, the image acquisition initiation method of wearable devices mainly relies on software command triggering mechanisms. Specifically, the MCU initiates image acquisition through I... 2 Bus protocols such as C (Inter-Integrated Circuit) and SPI (Serial Peripheral Interface) send control commands to the ISP. For example, when a user makes a specific gesture, the MCU parses the sensor data and determines that a photo-taking operation needs to be performed. Then, it needs to send a command to the ISP through bus communication, and the ISP then completes the image acquisition initialization and data processing process.

[0004] However, this software-instruction-based ISP startup method suffers from high startup latency and insufficient real-time performance. Specifically: First, bus communication involves protocol encapsulation, transmission, and parsing. In high-speed action scenarios (such as rapid image capture), communication delays can easily cause inaccurate timestamps between sensor data and image frames, leading to motion blur or image stitching errors. Second, bus resources may be occupied by other peripherals, causing ISP trigger commands to queue and further exacerbating response lag.

[0005] Therefore, improving the startup speed of ISP image acquisition to enhance the real-time response of ISP image acquisition has become an urgent technical problem to be solved. Summary of the Invention

[0006] The main objective of this application is to provide an image acquisition method, system, device, and readable storage medium, aiming to solve the technical problem of how to improve the startup speed of ISP image acquisition in order to improve the real-time response of ISP image acquisition.

[0007] To achieve the above objectives, this application provides an image acquisition method applied to a wearable device. The wearable device includes an MCU and an ISP, and the MCU and the ISP are interconnected by GPIO pins. The image acquisition method includes:

[0008] If the MCU detects that an image acquisition event has been triggered, it sends an image acquisition start signal to the ISP through the MCU's GPIO pin;

[0009] If the ISP receives the image acquisition start signal, it will begin acquiring image data.

[0010] In one embodiment, there are multiple pairs of interconnected GPIO pins between the MCU and the ISP. The step of sending an image acquisition start signal to the ISP through the GPIO pins of the MCU includes:

[0011] Obtain the imaging mode corresponding to the image acquisition event;

[0012] Determine the target GPIO pin among the GPIO pins of the MCU that corresponds to the imaging mode, wherein different imaging modes correspond to different GPIO pins;

[0013] The image acquisition start signal is sent to the ISP via the target GPIO pin.

[0014] In one embodiment, after the step of sending an image acquisition start signal to the ISP via the target GPIO pin, the method further includes:

[0015] The parameter transmission pins in each of the GPIO pins of the ISP are determined, wherein the parameter transmission pins are the GPIO pins in all the GPIO pins of the ISP used for transmitting image parameters;

[0016] The image parameters of the image data acquired by the ISP are transmitted to the MCU through the parameter transmission pin.

[0017] In one embodiment, the step of starting image data acquisition if the ISP receives the image acquisition start signal includes:

[0018] If the ISP receives the image acquisition start signal, it will start the small core system in the ISP;

[0019] Image data is acquired based on the small-core system after startup, and the large-core system in the ISP is started after the image data is acquired;

[0020] The large-core system, after startup, performs image processing on the acquired image data and sends the processed image data to the MCU.

[0021] In one embodiment, the step of performing image processing on the acquired image data based on the large-core system after startup includes:

[0022] Obtain the timestamp recorded when the image data was acquired;

[0023] The timestamp and the image data are subjected to image encoding processing to obtain the processed image data.

[0024] In one embodiment, before the step of obtaining the timestamp recorded when the image data was acquired, the method further includes:

[0025] Obtain the time difference observation value of the ISP;

[0026] The time difference observations are input into the Kalman filter algorithm to obtain the drift rate prediction.

[0027] The frequency division coefficient of the timer in the ISP is adjusted and / or the auto-reload register value of the timer is reset based on the drift rate prediction value, so as to record the timestamp when the ISP acquires image data based on the adjusted timer;

[0028] The adjusted frequency division coefficient is negatively correlated with the predicted drift rate, while the automatic reload register value is positively correlated with the predicted drift rate.

[0029] In one embodiment, after the step of sending an image acquisition start signal to the ISP via the GPIO pin of the MCU, the method further includes:

[0030] If the MCU detects that a stop image acquisition event has been triggered, it sends an image acquisition stop signal to the ISP through the GPIO pin to cause the ISP to stop image acquisition.

[0031] Furthermore, to achieve the above objectives, this application also provides an image acquisition system deployed in a wearable device. The wearable device includes an MCU and an ISP, with GPIO pins interconnected between the MCU and the ISP. The image acquisition system includes:

[0032] The transmitting module is used to send an image acquisition start signal to the ISP through the GPIO pin of the MCU if the MCU detects that an image acquisition event has been triggered;

[0033] The acquisition module is used to start acquiring image data if the ISP receives the image acquisition start signal.

[0034] In addition, to achieve the above objectives, this application also provides a wearable device, the wearable device including a processor, the processor including an MCU and an ISP, the MCU and the ISP having interconnected GPIO pins, the processor being used to execute the steps of the image acquisition method described above.

[0035] In addition, to achieve the above objectives, this application also provides a readable storage medium, which is a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of the image acquisition method described above.

[0036] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image acquisition method described above.

[0037] One or more technical solutions proposed in this application have at least the following technical effects:

[0038] This application interconnects the MCU and ISP of a wearable device via GPIO pins, meaning the MCU and ISP share GPIO pins. If the MCU detects an image acquisition event, it sends an image acquisition start signal to the ISP through the MCU's GPIO pin. If the ISP receives the image acquisition start signal, it begins acquiring image data. Thus, this embodiment upgrades the image acquisition start signal transmission mechanism from software protocol-driven to direct hardware triggering by sharing GPIO pins between the MCU and ISP, effectively overcoming the latency and resource contention defects of traditional bus communication. Specifically, when the MCU detects an image acquisition event and needs to acquire images, it sends an image acquisition start signal to the ISP through the GPIO pin to directly start the ISP's image acquisition process. This hardware triggering mechanism eliminates the need for I... 2The protocol encapsulation, transmission, and parsing required by the C / SPI bus can reduce instruction transmission latency from milliseconds to microseconds, significantly shortening system response time. Furthermore, GPIO, as a physical channel independent of the bus, avoids instruction queuing issues that occur when the bus is occupied by peripherals, ensuring that trigger signals can be transmitted to the ISP without blocking when an image acquisition event occurs. Therefore, through the above methods, this solution improves the startup speed of ISP image acquisition, achieving high real-time performance and low latency image acquisition triggering. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating the first embodiment of the image acquisition method of this application;

[0042] Figure 2 This is a schematic diagram of the image acquisition process according to an embodiment of the image acquisition method of this application;

[0043] Figure 3 This is a flowchart illustrating the fourth embodiment of the image acquisition method of this application;

[0044] Figure 4 This is a schematic diagram of the system structure of the image acquisition system of this application;

[0045] Figure 5 This is a schematic diagram of the hardware operating environment of the image acquisition method apparatus in the embodiments of this application.

[0046] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] In wearable devices such as AI glasses, the MCU is responsible for sensor data acquisition (such as gyroscopes and accelerometers), while the ISP processes camera image data. Data synchronization and communication between the two are crucial, especially during rapid image capture. How to quickly activate the ISP for image capture is a pressing technical problem that needs to be solved.

[0049] Based on this, the main solution of this application is: the MCU and ISP of the wearable device are interconnected through GPIO pins, that is, the MCU and ISP share GPIO pins. If the MCU detects that an image acquisition event is triggered, it sends an image acquisition start signal to the ISP through the GPIO pin of the MCU; if the ISP receives the image acquisition start signal, it starts to acquire image data.

[0050] This application upgrades the image acquisition start signal transmission mechanism from software protocol-driven to direct hardware triggering by sharing GPIO pins between the MCU and the ISP, effectively overcoming the latency and resource contention defects of traditional bus communication. Specifically, when the MCU detects an image acquisition event trigger and needs to perform image acquisition, it sends an image acquisition start signal to the ISP through the GPIO pin to directly start the ISP's image acquisition process. This hardware triggering mechanism eliminates the protocol encapsulation, transmission, and parsing required by the I2C / SPI bus, reducing instruction transmission latency from milliseconds to microseconds, significantly shortening system response time. Furthermore, as a physical channel independent of the bus, GPIO avoids instruction queuing problems caused by the bus being occupied by peripherals, ensuring that the trigger signal can be transmitted to the ISP without blocking when an image acquisition event occurs. Therefore, this solution improves the ISP's image acquisition startup speed and achieves high real-time performance and low latency image acquisition triggering.

[0051] It should be noted that the execution subject of the image acquisition method embodiments of this application can be a wearable device capable of realizing the above functions, such as smart glasses or smartwatches. The image acquisition method embodiments of this application do not impose specific limitations on this.

[0052] Based on this, this application proposes an image acquisition method according to a first embodiment. The image acquisition method is applied to a wearable device, which includes an MCU and an ISP. The MCU and the ISP are interconnected by GPIO pins. (Refer to...) Figure 1 As shown, the image acquisition method includes:

[0053] Step S10: If the MCU detects that an image acquisition event has been triggered, it sends an image acquisition start signal to the ISP through the GPIO pin of the MCU;

[0054] The GPIO pin refers to a General Purpose Input / Output (GPIO) pin. As a programmable digital interface, it supports bidirectional level signal transmission and enables direct interaction between hardware modules without relying on complex communication protocols. There is at least one pair of interconnected GPIO pins between the MCU and the ISP. These interconnected GPIO pins communicate directly via hardware circuitry, enabling protocol-free level signal transmission. Furthermore, the GPIO pins of both devices can be connected through pre-defined traces on a printed circuit board (PCB) or physical wiring on a flexible printed circuit board (FPC), or in other ways, provided that hardware communication between the two devices is possible. This embodiment does not impose specific limitations on this.

[0055] The image acquisition event refers to a pre-configured trigger condition used to initiate image acquisition. In specific implementations, the trigger condition includes, but is not limited to, the following operational forms: user pressing of physical buttons on mobile terminals (such as the side shutter button on a smartphone), trigger responses of physical controls on wearable devices (such as pressing the crown of a smartwatch or operating the touch area on the bezel of smart glasses), and other predefined human-computer interaction behaviors. This embodiment is not specifically limited, and those skilled in the art can adjust the implementation according to the specific product form.

[0056] After the MCU detects an image acquisition event, it sends an image acquisition start signal to the ISP via its GPIO pin. This signal can be any signal that the GPIO pin can send to notify the ISP to initiate image acquisition. For example, when the MCU determines that a photo-taking operation needs to be performed, it can configure the GPIO pin to output a specific level signal, such as switching the pin level from low to high. This type of signal directly acts on the corresponding GPIO input pin of the ISP, triggering it to immediately start the image sensor to acquire image data, thereby avoiding delays introduced by software protocol interaction.

[0057] Step S20: If the ISP receives the image acquisition start signal, it begins to acquire image data.

[0058] After the MCU detects an image acquisition event, it configures the output mode of its GPIO pins to directly transmit the image acquisition start signal to the corresponding GPIO input pin of the ISP via the MCU's GPIO output pin. Since the GPIO pins of the MCU and the ISP are interconnected, the signal transmission path is independent of the I / O pins. 2The C / SPI bus eliminates the need for any software protocol stack encapsulation and parsing process. When the ISP's GPIO input pin receives a level transition signal, its hardware logic circuitry can immediately respond to the image acquisition start signal and initiate image acquisition.

[0059] Furthermore, after receiving the image acquisition start signal, the ISP can also send a status confirmation signal (such as a low level to a high level) back to the MCU through the same GPIO pin or another independent GPIO pin, forming a hardware handshake mechanism, thereby achieving bidirectional synchronization between the MCU and the ISP without the need for bus interaction.

[0060] This embodiment upgrades the image acquisition start signal transmission mechanism from software protocol-driven to direct hardware triggering by sharing GPIO pins between the MCU and the ISP, effectively overcoming the latency and resource contention defects of traditional bus communication. Specifically, when the MCU detects an image acquisition event triggering and needs to perform image acquisition, it sends an image acquisition start signal to the ISP through the GPIO pin to directly start the ISP's image acquisition process. This hardware triggering mechanism eliminates the need for I... 2 The protocol encapsulation, transmission, and parsing required by the C / SPI bus can reduce instruction transmission latency from milliseconds to microseconds, significantly shortening system response time. Furthermore, GPIO, as a physical channel independent of the bus, avoids instruction queuing issues that occur when the bus is occupied by peripherals, ensuring that trigger signals can be transmitted to the ISP without blocking when an image acquisition event occurs. Therefore, through the above methods, this solution improves the startup speed of ISP image acquisition, achieving high real-time performance and low latency image acquisition triggering.

[0061] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. On this basis, there are multiple pairs of interconnected GPIO pins between the MCU and the ISP. The step of sending an image acquisition start signal to the ISP through the GPIO pins of the MCU includes:

[0062] Step A10: Obtain the imaging mode corresponding to the image acquisition event;

[0063] It should be noted that multiple pairs of GPIO pins can be connected in a one-to-one correspondence, such as the GPIO0 pin of the MCU being connected to the GPIO0 pin of the ISP, and the GPIO1 pin of the MCU being connected to the GPIO pin of the ISP.

[0064] The imaging mode refers to the preset working mode of image acquisition, which may include photo mode (single frame image capture), video mode (continuous multi-frame image streaming acquisition), live photo mode (combination of short video and still image) or live streaming mode (real-time encoding and streaming media transmission), etc. Different modes correspond to different image processing procedures and parameter configurations.

[0065] Step A20: Determine the target GPIO pin corresponding to the imaging mode among the GPIO pins of the MCU, wherein different imaging modes correspond to different GPIO pins;

[0066] After acquiring the imaging mode, the MCU can determine the target GPIO pin bound to that imaging mode from among its various GPIO pins according to a preset mode-pin mapping table. For example, if the imaging mode is a photo mode, it is mapped to the MCU's GPIO0 pin; if it is a video recording mode, it is mapped to the GPIO1 pin. By assigning independent GPIO pins to different modes, the MCU does not need to transmit mode parameters through software protocols. It only needs to select the corresponding pin to send a trigger signal, thus embedding the mode information in the physical channel selection, thereby achieving hardware-level differentiation and notification of imaging modes.

[0067] Step A30: Send an image acquisition start signal to the ISP through the target GPIO pin.

[0068] When the image acquisition start signal is sent to the ISP via the target GPIO pin, the MCU drives the target pin to output an image acquisition start signal (such as a high-level pulse signal). This signal is transmitted to the corresponding GPIO pin of the ISP via a physical direct connection path. The ISP detects the GPIO pin number of the input signal; for example, receiving a GPIO0 signal indicates photo mode, and a GPIO1 signal indicates video mode. It directly determines the current imaging mode and calls the preset acquisition parameters (such as frame rate, resolution, and encoding format) corresponding to that imaging mode to start image data acquisition. This mechanism requires no additional communication protocol parsing; mode configuration is completed solely through hardware pin selection, significantly shortening mode switching time and supporting flexible expansion to multiple modes.

[0069] In one possible implementation, after the step of sending an image acquisition start signal to the ISP via the target GPIO pin, the method further includes:

[0070] Step B10: Determine the parameter transmission pin among the GPIO pins of the ISP, wherein the parameter transmission pin is the GPIO pin used to transmit image parameters among all the GPIO pins of the ISP;

[0071] At least one pair of GPIO pins can be predefined as a dedicated parameter transmission channel between the ISP and the MCU. For example, the GPIO2 pin of the ISP can be connected to the GPIO2 pin of the MCU to form a parameter transmission path independent of the image acquisition trigger pin. The function of the parameter transmission pin can be fixed by hardware configuration and is used to transmit dynamic parameters during the image acquisition process, such as the exposure time, gain value, or image resolution of the current frame, thereby achieving functional isolation from the trigger command pin through physical channels.

[0072] Step B20: The image parameters of the image data acquired by the ISP are transmitted to the MCU through the parameter transmission pin.

[0073] After acquiring image data, the ISP sends image parameters to the MCU via parameter transmission pins, for example, through parallel transmission of binary encoded parameters via multiple pins. The MCU monitors the level changes of these parameter transmission pins in real time to parse the corresponding image parameter values. This direct hardware transmission mechanism does not occupy bus resources, and the parameter transmission delay and trigger signal can be synchronously controlled at the microsecond level. This ensures that the MCU can dynamically adjust sensor acquisition strategies based on real-time image parameters, such as the gyroscope data fusion frequency, thereby further optimizing the synchronization accuracy between the MCU and the ISP and improving system energy efficiency.

[0074] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, the step of starting image data acquisition if the ISP receives the image acquisition start signal includes:

[0075] Step C10: If the ISP receives the image acquisition start signal, then the small core system in the ISP is started;

[0076] When the ISP receives an image acquisition start signal (such as a high-level pulse) through the target GPIO pin, its hardware logic circuit immediately activates the low-power small core system. This small core system is a lightweight processor core, such as the FreeRTOS system, dedicated to sensor control and basic data acquisition. The startup process of the small core system is entirely driven by hardware trigger signals, without relying on software initialization procedures. This allows it to complete basic operations such as powering and enabling the image sensor, loading exposure parameters, and reading raw data within microseconds, ensuring rapid response and maintaining a low-power state for the acquisition task, further improving the image acquisition response speed.

[0077] Step C20: Based on the small-core system after startup, image data is acquired, and after the image data is acquired, the large-core system in the ISP is started;

[0078] After acquiring image data, the small-core system starts the large-core system in the ISP. Specifically, the startup of the high-performance large-core system can be triggered by the ISP's internal hardware interrupt signal or a specific status register flag. The large-core system is a multi-core computing unit dedicated to complex image processing, such as the Linux system.

[0079] The big core system starts up after the small core system, so its initialization process can be executed in parallel with the data acquisition phase of the small core system. For example, while the small core is transmitting the first frame of image data to the buffer, the big core completes the loading of the algorithm library and pipeline configuration, thereby avoiding the power waste caused by the continuous operation of the full-function core in the traditional solution.

[0080] Step C30: Based on the big core system after startup, perform image processing on the acquired image data, and send the processed image data to the MCU.

[0081] The large-core system performs image processing on the raw image data, such as mosaicking, noise reduction, dynamic range adjustment, and encoding compression, and then sends the processed image data to the MCU. Specifically, the processed image data can be transmitted to the MCU through a preset communication interface, such as a GPIO pin that reuses the parameter transmission in step B20 or an independent physical channel.

[0082] During transmission, the MCU can synchronously adjust the timestamp alignment strategy of the sensor data based on the real-time received image parameters to ensure accurate matching between motion compensation and image data. Through time-sharing collaboration between the small core system and the large core system and the on-demand startup mechanism, the overall power consumption of the ISP is reduced while maintaining high processing performance, and the startup speed of ISP image acquisition is further improved.

[0083] In one possible implementation, after the step of sending an image acquisition start signal to the ISP via the GPIO pin of the MCU, the method further includes:

[0084] Step D10: If the MCU detects a stop image acquisition event, it sends an image acquisition stop signal to the ISP through the GPIO pin to cause the ISP to stop image acquisition.

[0085] The "stop image acquisition" event refers to a pre-configured stop condition used to halt image acquisition. In specific implementations, triggering conditions include, but are not limited to, the following operational states: detecting that a user has released a physical button, the image acquisition duration reaching a preset threshold, the gyroscope detecting that the device remains stationary for a duration exceeding a preset duration, and other predefined human-computer interaction behaviors. This embodiment is not specifically limited, and those skilled in the art can adjust the implementation according to the specific product form.

[0086] When the MCU detects a stop image acquisition event, it sends an image acquisition stop signal to the ISP via the MCU's GPIO pin. The signal form of this image acquisition stop signal can form complementary logic with the signal form of the start command. For example, if the start signal is a high-level pulse signal on the GPIO pin, then the stop signal can be a high-level pulse on the same pin that transitions to a falling edge pulse, i.e., a low-level pulse signal. This distinguishes between start and stop operations at the hardware level, ensuring the clarity of the control timing.

[0087] The MCU can send a stop command through a GPIO pin directly connected to the ISP, such as the same pin used to trigger the start in step A30, or a separate GPIO pin dedicated to stop control.

[0088] Furthermore, upon receiving the image acquisition stop signal, the ISP's hardware logic circuit immediately terminates the image sensor's data reading process and triggers a collaborative shutdown mechanism between the small and large core systems: the small core system stops acquiring raw data and enters a low-power standby state, while the large core system completes processing the current frame, sends the result back to the MCU, and releases computing resources. During this process, the transmission of the stop command relies entirely on the level transition of the hardware signal; its delay is in the microsecond range, similar to the start command, and it is independent of bus communication, thus maintaining high real-time performance of global control even at the end stage. Through hardware-triggered end-to-end closed-loop control, the system meets the requirements for rapid start and stop while further reducing power waste caused by invalid acquisition.

[0089] For example, to aid in understanding the technical concept or principle of the image acquisition method combined with the first and second embodiments described above, a specific embodiment is provided below. In this specific embodiment, the image acquisition method is applied to smart glasses. The MCU and ISP in the smart glasses share GPIO pins to send pulse signals. Based on this, referring to... Figure 2 As shown, the image acquisition process includes:

[0090] 1. When the ISP side of the smart glasses is powered on, the GPIO0 pin on the MCU side is pulled high, that is, the GPIO0 pin sends a rising edge pulse signal to the ISP side, which is also the image acquisition start signal.

[0091] 2. After receiving the rising edge pulse signal, the ISP first starts the FreeRTOS system to take a picture, that is, to acquire the image. After taking the picture, it starts the Linux system.

[0092] 3. The FreeRTOS system on the ISP side sends the photo data (i.e., image data) and photo capture commands to the Linux system.

[0093] 4. The Linux system on the ISP side processes the captured data frame by frame and transmits the processed captured data to the MCU according to the captured command.

[0094] 5. The MCU saves the images transmitted from the ISP.

[0095] It should be noted that the above examples are only used to help understand this embodiment and do not constitute a limitation on the image acquisition process of this embodiment. Any simple modifications based on this technical concept are within the protection scope of this application.

[0096] Based on the first, second, and / or third embodiments of this application, in the fourth embodiment of this application, the content that is the same as or similar to the above-described embodiments one, two, and three can be referred to the above description and will not be repeated hereafter. Based on this, the step of performing image processing on the acquired image data based on the large-core system after startup includes:

[0097] Step E10: Obtain the timestamp recorded when the image data was acquired;

[0098] It should be noted that when the ISP collects image data, it can record the timestamp of each frame of the image when it is collected, so that the timestamp of each frame of the image can be obtained, that is, the timestamp of the time when the frame of the image is collected.

[0099] Specifically, the precise timestamp of the image sensor exposure completion at the time of each frame image acquisition can be obtained from the hardware clock module of the ISP or the global timer synchronized with the MCU, and the timestamp can be recorded.

[0100] Step E20: Perform image encoding processing on the timestamp and the image data to obtain the processed image data.

[0101] Encoding timestamps and image data can be performed by embedding timestamps as metadata into the header or footer of image frames and executing an encoding process that includes timestamp information. For example, Supplemental Enhancement Information (SEI) can be inserted into H.264 / H.265 video streams, or EXIF ​​(Exchangeable Image File Format) timestamps can be written into JPEG / HEIF still image files.

[0102] In one possible implementation, refer to Figure 3 As shown, before the step of obtaining the timestamp recorded when the image data was acquired, the method further includes:

[0103] Step F10: Obtain the time difference observation value of the ISP;

[0104] Specifically, the time difference observation can be the difference between the local timestamp recorded by the ISP and the reference timestamp recorded by the reference clock at the same time. The reference clock refers to a high-reliability clock source inside or outside the system, such as a global system timer driven by a high-precision crystal oscillator of the MCU, or a stable clock module synchronized by a phase-locked loop inside the ISP. Such clock sources maintain periodic synchronization with the ISP timer through direct hardware connection. Other examples include the 1PPS pulse signal of the GPS module, the NTP (Network Time Protocol) calibration clock transmitted by a smartphone via Bluetooth or Wi-Fi, or the global clock output by an independent temperature-compensated crystal oscillator module.

[0105] Furthermore, a reference time interval can be obtained. This reference time interval refers to the standard time length between two adjacent time difference observations, such as the 1-second pulse interval of GPS or the millisecond-level timer trigger interval of a mobile phone clock. This reference time interval, along with the time difference observations, is input into the Kalman filter algorithm. The inherent stability of the reference clock source ensures the reference time interval. As an input parameter of the Kalman filter algorithm, it models the dynamic change of the clock drift rate over time, thereby improving the accuracy of the drift rate prediction and the system's ability to suppress long-term drift.

[0106] Step F20: Input the time difference observation into the Kalman filter algorithm to obtain the drift rate prediction value;

[0107] The Kalman filter algorithm is a state estimation model where the state equation models the dynamic process of clock drift rate (the change in time difference per unit time). The observation equation correlates the measured time difference with the drift rate linearly. By iteratively updating the state covariance matrix and Kalman gain, it outputs the optimal estimate of the ISP timer drift rate, i.e., the drift rate prediction, which is used to quantify the deviation trend of the ISP timer frequency, such as the number of microseconds faster / slower per second.

[0108] Step F30: Adjust the division coefficient of the timer in the ISP and / or reset the auto-reload register value of the timer according to the drift rate prediction value, so as to record the timestamp when the ISP acquires image data based on the adjusted timer;

[0109] The adjusted frequency division coefficient is negatively correlated with the predicted drift rate, while the automatic reload register value is positively correlated with the predicted drift rate.

[0110] The frequency division factor of the ISP timer is dynamically adjusted based on the predicted drift rate. For example, if the predicted ISP timer frequency is too high, the frequency division factor is increased to reduce the counting frequency, and / or the auto-reload register value is adjusted. For example, if the timing period needs to be extended, the reload value is increased to prolong the single-cycle duration. After adjustment, the actual frequency of the ISP timer approaches the MCU's global timing reference, thereby suppressing clock drift and improving the accuracy of the timestamp. Specifically, the adjusted frequency division factor is negatively correlated with the predicted drift rate (the larger the drift rate, the larger the adjustment of the frequency division factor to compensate for high-frequency deviations), and the auto-reload register value is positively correlated with the predicted drift rate (when the drift rate is positive, the reload value is increased to prolong the cycle).

[0111] In addition, it should be noted that this method can also be used to adjust the divider coefficient of the timer in the MCU and / or reset the auto-reload register value of the timer, which will not be elaborated here.

[0112] By constructing an adaptive clock synchronization mechanism based on Kalman filtering, the problem of timestamp inaccuracy caused by clock drift between the ISP and MCU in wearable devices is effectively solved. By reusing multiple reliable clock references such as the MCU's high-precision crystal oscillator, the ISP's phase-locked loop synchronization module, or an external GPS / mobile phone clock, the system can dynamically acquire the observed difference between the ISP's local time and the reference time. Combined with the stability input of the reference time interval, the system uses the Kalman filtering algorithm to model the dynamic changes in clock drift in real time, accurately predicting the frequency deviation trend per unit time. Based on the predicted drift rate, the system automatically adjusts the division factor and automatic reload register value of the ISP timer in reverse. For example, when a high timer frequency is detected, the division factor is increased to reduce the counting rate, or the reload value is extended to prolong the counting period when a positive deviation accumulates. This brings the actual frequency of the ISP timer closer to the reference clock reference, suppressing microsecond-level timestamp errors caused by long-term drift. This closed-loop calibration process achieves high-precision time synchronization between the ISP and the sensor data acquisition end by dynamically tuning existing timer parameters without requiring additional hardware modules. Its adaptability is compatible with various scenarios, such as indoor reliance on the MCU's internal clock and outdoor switching to GPS / mobile phone clocks, while avoiding the additional power consumption caused by frequent bus communication or complex protocol parsing in traditional solutions. Through the coordinated optimization of hardware-level timer calibration and software algorithms, the system's clock stability and spatiotemporal consistency of multimodal data are significantly improved in complex environments, providing reliable underlying time reference support for high-speed image acquisition and motion compensation.

[0113] Furthermore, this application also proposes a startup device, wherein the image acquisition system is deployed in a wearable device, the wearable device including an MCU and an ISP, and there are interconnected GPIO pins between the MCU and the ISP, as shown in the reference. Figure 4As shown, the image acquisition system includes:

[0114] The transmitting module 10 is used to send an image acquisition start signal to the ISP through the GPIO pin of the MCU if the MCU detects that an image acquisition event has been triggered.

[0115] The acquisition module 20 is used to start acquiring image data if the ISP receives the image acquisition start signal.

[0116] In one embodiment, there are multiple pairs of interconnected GPIO pins between the MCU and the ISP, and the transmitting module 10 is further configured to:

[0117] Obtain the imaging mode corresponding to the image acquisition event;

[0118] Determine the target GPIO pin among the GPIO pins of the MCU that corresponds to the imaging mode, wherein different imaging modes correspond to different GPIO pins;

[0119] The image acquisition start signal is sent to the ISP via the target GPIO pin.

[0120] In one embodiment, the sending module 10 is further configured to:

[0121] The parameter passing pins in each of the GPIO pins of the ISP are determined, wherein the parameter passing pins are the GPIO pins in all the GPIO pins of the ISP used to pass image parameters;

[0122] The image parameters of the image data acquired by the ISP are transmitted to the MCU through the parameter transmission pin.

[0123] In one embodiment, the acquisition module 20 is further configured to:

[0124] If the ISP receives the image acquisition start signal, it will start the small core system in the ISP;

[0125] Image data is acquired based on the small-core system after startup, and the large-core system in the ISP is started after the image data is acquired;

[0126] The large-core system, after startup, performs image processing on the acquired image data and sends the processed image data to the MCU.

[0127] In one embodiment, the processing module is further configured to:

[0128] Obtain the timestamp recorded when the image data was acquired;

[0129] The timestamp and the image data are subjected to image encoding processing to obtain the processed image data.

[0130] In one embodiment, the acquisition module 20 is further configured to:

[0131] Obtain the time difference observation value of the ISP;

[0132] The time difference observations are input into the Kalman filter algorithm to obtain the drift rate prediction.

[0133] The frequency division coefficient of the timer in the ISP is adjusted and / or the auto-reload register value of the timer is reset based on the drift rate prediction value, so as to record the timestamp when the ISP acquires image data based on the adjusted timer;

[0134] The adjusted frequency division coefficient is negatively correlated with the predicted drift rate, while the automatic reload register value is positively correlated with the predicted drift rate.

[0135] In one embodiment, the sending module 20 is further configured to:

[0136] If the MCU detects that a stop image acquisition event has been triggered, it sends an image acquisition stop signal to the ISP through the GPIO pin to cause the ISP to stop image acquisition.

[0137] Furthermore, this application also proposes a wearable device, which includes a processor, comprising an MCU and an ISP, wherein the MCU and the ISP are interconnected by GPIO pins, and the processor is used to execute the steps of the image acquisition method described above.

[0138] refer to Figure 5 The diagram illustrates a structural schematic suitable for implementing wearable devices according to embodiments of this application. Wearable devices in embodiments of this application may also include, but are not limited to, smart glasses, smartwatches, virtual reality glasses, virtual reality helmets, extended reality glasses, extended reality helmets, headphones, etc. Figure 5 The wearable device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0139] like Figure 5As shown, the wearable device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the wearable device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the wearable device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show wearable devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0140] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0141] The wearable device provided in this application, employing the image acquisition method described above, solves the technical problem of how to improve the startup speed of ISP image acquisition, thereby enhancing the real-time response of ISP image acquisition. Compared with the prior art, the beneficial effects of the wearable device provided in this application are the same as those of the image acquisition method described above, and other technical features of this wearable device are the same as those disclosed in the previous embodiment, and will not be repeated here.

[0142] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0143] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0144] In addition, to achieve the above objectives, embodiments of this application also provide a readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the image acquisition method described above.

[0145] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0146] The aforementioned computer-readable storage medium may be included in a wearable device; or it may exist independently and not incorporated into a wearable device.

[0147] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the wearable device, the wearable device: if the MCU detects that an image acquisition event has been triggered, it sends an image acquisition start signal to the ISP through the MCU's GPIO pin; if the ISP receives the image acquisition start signal, it begins to acquire image data.

[0148] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the modules themselves.

[0151] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image acquisition method. This solves the technical problem of how to improve the startup speed of ISP image acquisition, thereby improving the real-time response of ISP image acquisition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image acquisition method provided in the above embodiments, and will not be repeated here.

[0152] Furthermore, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the steps of the image acquisition method described above.

[0153] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-described image acquisition method, and will not be repeated here.

[0154] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0155] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software sensor. This computer software sensor is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0157] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image acquisition method, characterized in that, The image acquisition method is applied to a wearable device, which includes an MCU and an ISP, with GPIO pins interconnected between the MCU and the ISP. The image acquisition method includes: If the MCU detects that an image acquisition event has been triggered, it sends an image acquisition start signal to the ISP through the MCU's GPIO pin; If the ISP receives the image acquisition start signal, it will begin acquiring image data.

2. The image acquisition method as described in claim 1, characterized in that, There are multiple pairs of interconnected GPIO pins between the MCU and the ISP. The step of sending an image acquisition start signal to the ISP through the GPIO pins of the MCU includes: Obtain the imaging mode corresponding to the image acquisition event; Determine the target GPIO pin among the GPIO pins of the MCU that corresponds to the imaging mode, wherein different imaging modes correspond to different GPIO pins; The image acquisition start signal is sent to the ISP via the target GPIO pin.

3. The image acquisition method as described in claim 2, characterized in that, After the step of sending the image acquisition start signal to the ISP via the target GPIO pin, the method further includes: The parameter transmission pins in each of the GPIO pins of the ISP are determined, wherein the parameter transmission pins are the GPIO pins in all the GPIO pins of the ISP used for transmitting image parameters; The image parameters of the image data acquired by the ISP are transmitted to the MCU through the parameter transmission pin.

4. The image acquisition method as described in claim 1, characterized in that, The step of starting image data acquisition if the ISP receives the image acquisition start signal includes: If the ISP receives the image acquisition start signal, it will start the small core system in the ISP; Image data is acquired based on the small-core system after startup, and the large-core system in the ISP is started after the image data is acquired; The large-core system, after startup, performs image processing on the acquired image data and sends the processed image data to the MCU.

5. The image acquisition method as described in claim 4, characterized in that, The step of performing image processing on the acquired image data based on the large-core system after startup includes: Obtain the timestamp recorded when the image data was acquired; The timestamp and the image data are subjected to image encoding processing to obtain the processed image data.

6. The image acquisition method as described in claim 5, characterized in that, Before the step of obtaining the timestamp recorded when the image data was acquired, the method further includes: Obtain the time difference observation value of the ISP; The time difference observations are input into the Kalman filter algorithm to obtain the drift rate prediction. The frequency division coefficient of the timer in the ISP is adjusted and / or the auto-reload register value of the timer is reset based on the drift rate prediction value, so as to record the timestamp when the ISP acquires image data based on the adjusted timer; The adjusted frequency division coefficient is negatively correlated with the predicted drift rate, while the automatic reload register value is positively correlated with the predicted drift rate.

7. The image acquisition method according to any one of claims 1 to 6, characterized in that, After the step of sending the image acquisition start signal to the ISP via the GPIO pin of the MCU, the method further includes: If the MCU detects that a stop image acquisition event has been triggered, it sends an image acquisition stop signal to the ISP through the GPIO pin to cause the ISP to stop image acquisition.

8. An image acquisition system, characterized in that, The image acquisition system is deployed in a wearable device, which includes an MCU and an ISP. The MCU and the ISP are interconnected via GPIO pins. The image acquisition system includes: The transmitting module is used to send an image acquisition start signal to the ISP through the GPIO pin of the MCU if the MCU detects that an image acquisition event has been triggered; The acquisition module is used to start acquiring image data if the ISP receives the image acquisition start signal.

9. A wearable device, characterized in that, The wearable device includes a processor, which includes an MCU and an ISP, with GPIO pins interconnected between the MCU and the ISP, and the processor is used to execute the steps of the image acquisition method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium includes a computer-readable storage medium storing an image acquisition program, which, when executed by a processor, implements the steps of the image acquisition method as described in any one of claims 1 to 7.

Images