Intelligent visual SoC chip operating system starting method, equipment and medium

Abstract

The invention provides an intelligent visual SoC chip operating system starting method and device and a medium, and relates to the technical field of visual SoC chip operating systems, and the method comprises the steps: starting a bootstrap program built in a chip after the intelligent visual SoC chip is powered on; hardware adaptation detection is executed through the bootstrap program; calling a starting configuration file matched with the current chip from a preset hardware adaptation library; preferentially loading an operating system kernel and an intelligent vision dedicated drive module according to a module loading priority in the startup configuration file; after loading of the operating system kernel and the special driving module is completed, targeted initialization of the exclusive visual processing component is executed; loading a core service and visual application basic module, and completing the starting of an intelligent visual SoC chip operating system; customized and efficient starting of the intelligent visual SoC chip operating system is achieved, and the starting performance, the hardware adaptation capacity and the visual processing supporting effect of the chip are comprehensively optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual SoC chip operating system, in particular to a smart visual SoC chip operating system starting method, device and medium. BACKGROUND

[0002] As a highly integrated device integrating processor, image sensor, video codec and special visual processing unit (VPU), the smart visual SoC chip has been widely used in security monitoring, automatic driving, smart home, industrial detection and other fields. The starting efficiency, compatibility and adaptability of its operating system directly affect the response speed and user experience of the terminal device.

[0003] With the increasing demand for real-time and multi-functionality of smart visual applications, the existing smart visual SoC chip operating system starting scheme gradually exposes many deficiencies: first, the traditional starting method mostly follows the starting architecture of general operating system, without customized optimization for the hardware integration characteristics of smart visual chips, and needs to load a large number of unnecessary modules one by one, resulting in a tedious and time-consuming starting process, which is difficult to meet the real-time requirements of starting in security snapshot, automatic driving environment perception and other scenarios; second, there are significant differences in hardware architecture, interface protocol and core component configuration among smart visual SoC chips of different manufacturers, and the existing starting scheme has poor hardware adaptability, which requires separate development of adaptation programs for different chip models, increasing development cost and maintenance difficulty; third, the initialization strategy for visual processing core hardware (such as VPU, image signal processor ISP) in the starting process lacks pertinence, which is prone to problems such as unreasonable resource allocation and insufficient hardware function activation, thereby affecting the efficiency and accuracy of subsequent visual data processing; fourth, some starting schemes have stability risks, and are prone to starting failure, freezing and other phenomena under complex working conditions (such as low temperature, voltage fluctuation), which seriously restricts the reliable application of smart visual devices in harsh environments.

[0004] Therefore, there is an urgent need for a smart visual SoC chip operating system starting method with strong adaptability, efficient starting, high stability and targeted optimization of visual hardware initialization to solve the above-mentioned deficiencies in the prior art. SUMMARY

[0005] To solve the above technical problems, the technical solution adopted by the present application is as follows: According to the first aspect of the present application, a smart visual SoC chip operating system starting method is provided, comprising: S1: after the smart visual SoC chip is powered on, the built-in boot program of the chip is started; the boot program first initializes the core hardware of the chip, and the core hardware at least includes processor, memory controller and bus; S2: performing hardware adaptation detection through the boot program, identifying the model, hardware architecture parameters and configuration information of the exclusive vision processing components of the current smart vision SoC chip; the exclusive vision processing components at least include a vision processing unit, an image signal processor (ISP) and a video codec; S3: based on the detection result of step S2, calling a startup configuration file matched with the current chip from a preset hardware adaptation library; the startup configuration file contains module loading priority and vision hardware initialization parameters customized for the current chip; S4: loading the operating system kernel and the smart vision dedicated driver module according to the module loading priority in the startup configuration file; the smart vision dedicated driver module at least includes a VPU driver, an ISP driver and an image sensor interface driver; S5: after completing the loading of the operating system kernel and the dedicated driver module, performing targeted initialization of the exclusive vision processing components; including allocating computing resources for the VPU, configuring image preprocessing parameters of the ISP, and activating the working mode of the video codec; S6: after initialization, loading the user layer core service of the operating system and the vision application basic module, completing the startup of the smart vision SoC chip operating system.

[0006] Further, the hardware adaptation detection in step S2 specifically includes: S21: reading the vendor identification register and the model register of the smart vision SoC chip to determine the chip vendor and the model; S22: obtaining the core number of the VPU, the image processing channel number of the ISP and the supported format of the video codec through bus scanning; S23: matching the detected hardware information with the pre-stored hardware feature library in the boot program to verify the integrity and validity of the hardware information.

[0007] Further, the setting rule of the module loading priority in step S4 is: First priority: operating system kernel core process, including process scheduling and memory management; Second priority: smart vision dedicated driver module, including VPU driver, ISP driver and image sensor interface driver; Third priority: operating system basic service, including file system and network service; Fourth priority: vision application basic module, including image preprocessing library and video stream management module.

[0008] Further, the targeted initialization of the exclusive vision processing components in step S5 further includes: S51: dynamically allocate computing resources of the VPU, reserve at least one core resource for the real-time visual detection task according to the current chip VPU core number; S52: configure the white balance, exposure compensation and noise reduction parameters of the ISP, and the parameter values are called from the preset parameter library based on the image sensor model detected in step S2; S53: perform a function self-check on the video codec, and verify whether the H.264 / H.265 encoding and decoding functions of the codec are normal by outputting a test code stream.

[0009] Further, the method further comprises a step S7 of starting abnormality monitoring and processing in the process, specifically: If the hardware adaptation detection in step S2 fails, the boot program outputs an adaptation error code, triggers a backup adaptation scheme, and calls general visual hardware configuration parameters for retry; If the visual processing component initialization in step S5 fails, the power supply path of the abnormal component is closed, the abnormal log is recorded, and the degradation mode of the remaining normal components is started.

[0010] Further, when loading the user layer core service and the visual application basic module in step S6, a demand-based lazy loading strategy is adopted: only the necessary visual application basic module in the current scene is loaded, and the modules not called are temporarily stored in the memory cache area, and the loading is completed when the subsequent application request is received.

[0011] Further, after the boot program in step S1 is started, a firmware security verification step is further included: the integrity of the boot program itself and the operating system kernel image file is verified by a preset hash algorithm, and after the verification is passed, the core hardware initialization is performed, and if the verification fails, the startup is terminated and the chip hardware interface is locked.

[0012] According to another aspect of the present application, a non-transitory computer readable storage medium is also provided, and the storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to realize the above-mentioned intelligent visual SoC chip operating system starting method.

[0013] According to another aspect of the present application, an electronic device is also provided, which comprises a processor and the above-mentioned non-transitory computer readable storage medium.

[0014] The present application has at least the following beneficial effects: The intelligent vision SoC chip operating system startup method of this invention first initializes core hardware such as the processor, memory controller, and bus by a bootloader after power-on, building a stable hardware foundation for the subsequent startup process and avoiding startup anomalies caused by incomplete core hardware. Then, through hardware compatibility detection, it accurately identifies the chip model, hardware architecture parameters, and configuration information of dedicated vision processing components such as VPU, ISP, and video codec, and calls a preset matching startup configuration file. This effectively solves the problems of poor compatibility of traditional startup schemes with different manufacturers and models of intelligent vision SoC chips, requiring repeated development of adaptation programs, and significantly reducing development costs and maintenance difficulty. Simultaneously, it prioritizes loading the operating system kernel and intelligent vision-specific driver modules according to customized module loading priorities, skipping the early loading of unnecessary modules. This significantly reduces startup time, meeting the core real-time startup requirements of scenarios such as security capture and autonomous driving environmental perception. Subsequent targeted initialization of the dedicated vision processing components ensures reasonable allocation of VPU computing resources, adaptation of ISP image preprocessing parameters, and effective activation of video codec working modes. This avoids the hardware resource waste or insufficient function activation issues caused by traditional initialization strategies, improving the efficiency and accuracy of visual data processing. Finally, by loading core user-layer services and basic vision application modules, a complete startup loop from hardware readiness to application callability is formed. This achieves customized and efficient startup of the intelligent vision SoC chip operating system, comprehensively optimizing the chip's startup performance, hardware adaptability, and visual processing support, thereby improving the response speed and user experience of terminal devices. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating the operating system startup method for an intelligent vision SoC chip provided in this embodiment of the invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that, based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Furthermore, this device and / or practice the method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0019] The following will refer to Figure 1 The flowchart shown illustrates a method for booting an operating system for an intelligent vision SoC chip.

[0020] The method for booting the operating system of this intelligent vision SoC chip may include the following steps: S1: After the intelligent vision SoC chip is powered on, the built-in boot program of the chip is started; the boot program first initializes the core hardware of the chip, which includes at least a processor, a memory controller and a bus.

[0021] In this embodiment, after the intelligent vision SoC chip powers on, it first triggers the execution of its built-in bootloader (such as an industrial-grade bootloader U-Boot or a manufacturer-customized bootloader). The bootloader completes core hardware initialization by directly manipulating hardware registers. Processor initialization: Configure the core clock frequency (e.g., set the ARM Cortex-A78 core clock frequency to 2.0GHz to balance performance and power consumption), disable idle cores (e.g., activate only 4 cores for the boot phase on an 8-core chip), and initialize the interrupt vector table of the interrupt controller (GIC) to ensure that the processor can respond to hardware interrupts; Memory controller initialization: Configure DDR5 memory read and write timings (such as CAS latency 18, row address / column address bits), allocate memory address space (such as 0x80000000-0xC0000000 as memory available to the operating system), and enable ECC verification (error checking and correction) to improve memory stability; Bus initialization: Configure the transmission rate of the AXI / AHB bus (e.g., set the AXI bus to 1.6GHz) and the arbitration mechanism (prioritize data transmission between the processor and memory) to ensure smooth communication between core hardware components.

[0022] For example, after a certain intelligent automotive vision SoC chip (such as SV810) is powered on, the built-in boot program starts up, first locking the main frequency of the 4-core Cortex-A76 processor at 1.5GHz (to reduce power consumption during the startup phase), then configuring the timing parameters of the LPDDR5 memory (to adapt to automotive-grade temperature fluctuations), and finally setting the APB bus rate to 50MHz to be compatible with low-speed peripherals (such as sensor interfaces).

[0023] In this step, by prioritizing the initialization of core hardware such as the processor, memory controller, and bus, a stable hardware communication and computing foundation is established in advance. This avoids startup problems such as subsequent module loading failures and data transmission anomalies caused by the incomplete preparation of core hardware. At the same time, it provides reliable hardware support for subsequent hardware adaptation testing and module loading, thereby improving the stability of the startup process.

[0024] S2: The bootloader performs hardware compatibility detection to identify the model, hardware architecture parameters, and configuration information of the dedicated visual processing component of the current intelligent vision SoC chip; the dedicated visual processing component includes at least a visual processing unit, an image signal processor (ISP), and a video codec.

[0025] In this embodiment, the bootloader performs the detection using a combination of "register read + bus scan": Chip model and manufacturer identification: Read the chip's preset manufacturer identification register (e.g., address 0x00000010) and model register (e.g., address 0x00000014), parse the data to obtain the manufacturer ID (e.g., 0x00A1 corresponds to a certain vision chip manufacturer) and model (e.g., SV810). Hardware architecture parameter identification: Scan the internal peripheral addresses of the chip through the system bus (such as AXI) to obtain parameters such as the number of processor cores, cache size (such as 4MB L3 cache), and memory capacity (such as 8GB LPDDR5); Dedicated visual processing component configuration identification: Query the configuration information of VPU, ISP, and video codec via SPI / I2C bus, such as the number of VPU cores (e.g., 6 cores), the maximum resolution supported by the ISP (e.g., 8K), and the codec compatible format (e.g., H.265 / AV1). Information verification: The detection results are compared with the hardware feature library (pre-stored mainstream chip parameters) built into the bootloader to confirm that there are no missing parameters (such as the number of VPU cores is not missing) or abnormalities (such as the ISP resolution exceeding the theoretical maximum value).

[0026] For example: The bootloader reads the registers and obtains the manufacturer ID 0x00A1 and model SV810. AXI bus scanning reveals that the VPU is 6-core, the ISP supports 8K resolution, and the codec is compatible with H.265 / AV1, which is consistent with the preset parameters of SV810 in the hardware feature library. The verification is successful.

[0027] This step enables precise identification of the chip's hardware configuration details, breaking the limitations of the traditional boot scheme of "one chip, one adapter program". It provides data basis for subsequent calls to the matching boot configuration file, effectively solving the problems of poor compatibility of chips from different manufacturers / models and high development and maintenance costs, and improving the versatility of the solution.

[0028] Furthermore, the hardware adaptation detection described in step S2 specifically includes: S21: Read the manufacturer identification register and model register of the intelligent vision SoC chip to determine the chip manufacturer and model.

[0029] In this embodiment, the intelligent vision SoC chip is designed with pre-defined manufacturer identification registers and model registers (the addresses are specified by the manufacturer in the chip datasheet, usually physical addresses, such as manufacturer identification register address 0x00000100 and model register address 0x00000104). The bootloader accesses these registers via memory-mapped I / O (MMIO), reads the 32-bit or 64-bit binary data (register data is usually in a manufacturer-defined encoding format), and then parses the manufacturer information and chip model using pre-defined decoding rules (such as lookup table method). For example, a manufacturer might associate manufacturer identification register data 0x000000F2 with "XX Technology" and model register data 0x00001005 with the "CV100" chip.

[0030] For example: The bootloader accesses address 0x00000100, reads data 0x000000F2, and determines the manufacturer as "XX Technology" by matching the decoding table; it accesses address 0x00000104, reads data 0x00001005, and resolves it to the chip model "CV100".

[0031] In this step, the chip manufacturer and model are quickly and accurately determined by directly reading the hardware registers. This provides a basic identifier for subsequent hardware configuration detection and startup configuration file matching, avoiding deviations in the adaptation direction caused by incorrect manufacturer or model identification. This ensures the accuracy of hardware adaptation from the source and reduces invalid adaptation attempts.

[0032] S22: Obtain the number of VPU cores, the number of image processing channels of the ISP, and the supported formats of the video codec through bus scanning.

[0033] In this embodiment, the bootloader scans the dedicated vision processing unit via the chip's system bus (such as AXI, I2C, SPI, etc., selected according to the interface type of the vision component). For the VPU, a configuration query command is sent to its peripheral base address (such as 0x20000000), and the returned core configuration register data (such as binary bits [7:0] indicating the number of cores) is received and parsed to obtain the number of cores; for the ISP, its control register (such as address 0x40) is accessed via the I2C bus, and the image processing channel configuration bits (such as [5:0] indicating the number of channels) are read to determine the number of image channels that can be processed simultaneously; for the video codec, a format query command is sent via the SPI bus, and the supported encoding / decoding format identifiers (such as 0x01 indicating support for H.264, 0x02 indicating support for H.265) are received and a list of compatible formats is obtained.

[0034] For example: The bootloader sends a query command to the VPU base address 0x20000000 via the AXI bus, returns data 0x00000004, and resolves to find that the number of VPU cores is 4; it accesses the ISP control register 0x40 via the I2C bus, reads data 0x00000008, and determines that the number of ISP image processing channels is 8; it queries the codec via the SPI bus, returns format identifiers 0x01|0x02 (binary 00000011), and determines that it supports H.264 and H.265 formats.

[0035] In this step, detailed configuration parameters of the vision processing unit are obtained through bus scanning, which makes up for the hardware capability information (such as the number of VPU cores, the number of ISP channels, etc.) that cannot be obtained through the manufacturer / model register alone. This provides accurate data support for subsequent customized resource allocation and initialization parameter configuration, ensuring that the hardware capabilities are fully and rationally utilized.

[0036] S23: Match the detected hardware information with the hardware feature library pre-stored in the bootloader to verify the completeness and validity of the hardware information.

[0037] In this embodiment, the hardware feature library of the bootloader is structured data (such as JSON or binary tables) pre-stored in the chip's Flash memory, containing standard hardware parameters for various models of chips from mainstream manufacturers (e.g., the standard parameters for "XX Technology CV100" are: VPU core count 4±0, ISP channel count 8±0, codec support for H.264 / H.265). During matching, the information detected in S21 and S22 (manufacturer, model, VPU core count, etc.) is compared item by item with the standard parameters of the corresponding model in the feature library: checking for missing key parameters (e.g., no detected ISP channel count), whether the parameters are within a reasonable range (e.g., VPU core count ≥ 1), and whether they are consistent with the manufacturer's publicly disclosed parameters (e.g., whether the codec format conforms to the chip manual description). If all parameters match and there are no abnormalities, the information is determined to be complete and valid; otherwise, the verification is determined to have failed.

[0038] For example: If the hardware information of "XX Technology CV100" is detected as follows: 4 VPU cores, 8 ISP channels, and H.264 / H.265 codec support, which is completely consistent with the standard parameters of CV100 in the hardware feature library, the verification is successful; if the number of VPU cores is detected as 0 (out of a reasonable range), or the codec does not support H.264 (conflicting with the manual), the verification is deemed to have failed.

[0039] In this step, by matching with the standard feature library, invalid or erroneous hardware information (such as parameter anomalies caused by bus scan interference) is effectively filtered out, avoiding subsequent startup configuration based on erroneous data, reducing startup failures and hardware malfunctions caused by hardware information distortion, and significantly improving the reliability and stability of the startup process.

[0040] S3: Based on the detection results of step S2, call the startup configuration file that matches the current chip from the preset hardware adaptation library; the startup configuration file contains module loading priority and vision hardware initialization parameters customized for the current chip.

[0041] In this embodiment, the hardware adapter library is stored in the chip's NOR Flash and exists in the form of a "chip model-configuration file" mapping (such as XML or binary files). Each configuration file contains: Module loading priority sequence (explicitly defining the loading order of kernel, drivers, and services); Visual hardware initialization parameters (such as VPU resource allocation ratio, ISP default parameters, codec working mode).

[0042] Based on the detection results of S2 (such as chip model SV810, VPU6 core), the bootloader quickly locates and loads the corresponding configuration file (such as config_sv810_v6.bin) through indexing.

[0043] For example: if the chip is detected to be SV810 with 6 VPU cores, the module loading priority in the configuration file called by the bootloader is set to "kernel core process → VPU / ISP driver → file system → vision application", and the default VPU resource allocation rule is "real-time tasks occupy 40%" and the ISP default parameters are adapted to the vehicle camera (such as dynamic range optimization is enabled).

[0044] This step enables precise matching between the boot configuration and the chip's hardware characteristics, avoiding "resource waste" (such as allocating excessive tasks to a low-performance VPU) or "insufficient capabilities" (such as not enabling special functions of a high-performance ISP) caused by general configurations. It ensures that the boot process matches the actual capabilities of the chip and improves the adaptability.

[0045] S4: According to the module loading priority in the startup configuration file, the operating system kernel and the intelligent vision dedicated driver module are loaded first; the intelligent vision dedicated driver module includes at least the VPU driver, ISP driver and image sensor interface driver.

[0046] In this embodiment, the bootloader parses the priority rules in the boot configuration file and loads modules in stages through the operating system module loader (such as insmod / modprobe in Linux): First priority: Load the core processes of the operating system kernel (such as Linux's CFS process scheduler and SLUB memory manager) to ensure that basic functions such as process scheduling and memory allocation are ready; Second priority: Load the dedicated drivers for intelligent vision (VPU driver vpu.ko, ISP driver isp.ko, image sensor interface driver sensor.ko) to establish a communication bridge between the operating system and the vision hardware; Subsequent priorities: Loading non-core modules such as the file system and network services will be temporarily suspended until the core functions are stable, at which point they will be loaded gradually.

[0047] For example, after specifying the priority in the startup configuration file, the bootloader first loads the Linux kernel's process scheduler (to ensure orderly multitasking) and memory manager (to avoid memory leaks), then loads the VPU driver (to enable the system to call VPU computing power), ISP driver (to control image preprocessing), and vehicle camera interface driver (to read image data). After the above modules are stable, the ext4 file system service is then loaded.

[0048] In this step, the "core priority" strategy skips the early loading of unnecessary modules, significantly reducing startup time (e.g., from 8 seconds in the traditional solution to 3 seconds), meeting the real-time requirements of scenarios such as security capture (requiring rapid response to trigger signals) and autonomous driving (requiring real-time environmental perception), while ensuring that the vision hardware driver is ready first, laying the foundation for subsequent initialization.

[0049] Furthermore, the rules for setting the module loading priority are as follows: First priority: Operating system kernel core processes, including process scheduling and memory management.

[0050] In this embodiment, the operating system kernel core processes are the most fundamental components supporting system operation and must be loaded with priority during the initial startup phase. The process scheduling module (such as the Linux kernel's CFS (Completely Fair Scheduler)) ensures orderly switching between multiple tasks by initializing the scheduling queue, setting time slices (e.g., 10ms), and configuring priority algorithms (real-time tasks take precedence over ordinary tasks). The memory management module (such as Linux's SLUB allocator) achieves efficient memory allocation and reclamation by initializing page tables, dividing memory regions (kernel space and user space, e.g., 1GB for kernel space in a 32-bit system), and establishing a memory reclamation mechanism (e.g., the LRU page cache replacement algorithm). During loading, the bootloader uses a kernel image parsing tool (such as zImage decompression tool) to load the binary code of these core processes into a specified memory address (e.g., 0x80008000) and triggers the kernel initialization function (e.g., start_kernel()) to complete the startup process.

[0051] For example, when a smart security SoC chip starts up, it first loads the Linux kernel's CFS scheduler (to ensure that subsequent driver and service processes can be scheduled according to priority) and SLUB memory manager (to allocate 128MB of memory space for the VPU driver that is about to be loaded), and initializes the memory page table, mapping physical memory 0x80000000-0xA0000000 to kernel-accessible space.

[0052] In this step, by prioritizing the loading of core processes such as process scheduling and memory management, the operating system is ensured to have the most basic task scheduling and resource allocation capabilities, providing a runtime environment for all subsequent modules (drivers, services, applications), avoiding module loading failures or operational disorder caused by the lack of core functions, and ensuring the orderliness of the startup process from the bottom layer.

[0053] Second priority: Dedicated driver modules for intelligent vision, including VPU driver, ISP driver, and image sensor interface driver.

[0054] In this embodiment, the dedicated intelligent vision driver module serves as a communication bridge between the operating system and the vision hardware (VPU, ISP, image sensor), and must be loaded immediately after the kernel core process is ready. During loading, the bootloader calls the driver's initialization function (such as vpu_init()) through the operating system's module loader (such as Linux's modprobe): The VPU driver establishes communication between the kernel and the VPU by configuring the VPU's control registers (such as enabling the core clock and initializing the computation queue); the ISP driver configures the ISP's operating mode (such as 8-bit / 10-bit image data format) through the I2C bus; and the image sensor interface driver (such as the MIPI-CSI driver) realizes data interaction between the sensor and the SoC by initializing the interface timing (such as a 2GHz data transmission clock). After the driver is loaded, the operating system registers the corresponding hardware device nodes (such as / dev / vpu, / dev / isp) in the device tree for use by upper-layer modules.

[0055] For example, after the kernel core process is ready, the bootloader loads the VPU driver vpu.ko (initializes the compute queue of the 4-core VPU and allocates 256MB of video memory), the ISP driver isp.ko (configures the ISP to support 12-megapixel image input), and the MIPI-CSI driver csi.ko (sets the interface timing to match the output format of the security camera), and registers the / dev / vpu node in the device tree so that subsequent initialization steps can control the VPU through this node.

[0056] This step loads the dedicated vision driver immediately after the core process is ready, ensuring that the intelligent vision hardware is recognized and controlled by the operating system as early as possible. This provides underlying driver support for the targeted initialization of dedicated vision processing components (such as VPU resource allocation and ISP parameter configuration), avoids hardware initialization delays caused by driver loading lag, and improves the startup efficiency of vision functions.

[0057] Third priority: Basic operating system services, including file system and network services.

[0058] In this embodiment, the operating system basic services are general components that support system function expansion and need to be loaded after the vision-specific driver is loaded (to avoid missing dependencies). File system services (such as ext4 and FAT32) achieve persistent data storage by initializing storage device interfaces (such as eMMC controllers), mounting root file systems (such as mounting / dev / mmcblk0p2 to the / directory), and establishing file indexing mechanisms. Network services (such as the TCP / IP protocol stack and DHCP client) achieve network communication between the device and external systems by initializing Ethernet controllers (such as GMAC), configuring IP addresses (static or dynamic acquisition), and establishing network socket interfaces. During loading, the system service manager (such as systemd in Linux) starts service processes in dependency order (e.g., mountd handles file system mounting, and udhcpc handles dynamic IP acquisition).

[0059] For example, after the vision-specific driver is loaded, the system starts the ext4 file system service (mounting the root partition in eMMC so that the vision application library under / usr / lib can be accessed) and the TCP / IP network service (initializing the Ethernet interface through the GMAC controller and dynamically obtaining the IP address 192.168.1.100) to support the subsequent log storage (dependent on the file system) and video stream uploading (dependent on the network) of the vision application module.

[0060] Loading basic services after the core drivers are ready avoids startup failures caused by missing service dependencies, and does not occupy core resources in the early stages of startup (saving 20% ​​startup time compared to loading them first). At the same time, it provides storage and network capabilities for upper-layer vision applications, balancing startup efficiency and functional completeness.

[0061] Fourth priority: Visual application basic modules, including image preprocessing library and video stream management module.

[0062] In this embodiment, the vision application basic module is the upper-level component supporting the operation of vision functions. It depends on the preceding modules (kernel, driver, basic services) and needs to be loaded last. The image preprocessing library (such as a simplified version of OpenCV or a vendor-customized algorithm library) is loaded into user space memory via dynamic linking (such as Linux's ld linker), providing basic interfaces for image denoising, scaling, edge detection, etc. The video stream management module manages the video stream acquisition (obtaining raw data from the image sensor), encoding (calling the VPU for H.265 encoding), and caching (storing to a temporary directory in the file system) processes by calling the VPU / ISP driver interface. During loading, an "on-demand initialization" strategy is adopted, only starting the functions necessary for the current scenario (such as starting real-time encoding in a security scenario and temporarily suspending offline analysis).

[0063] For example, after the basic services are ready, the system loads the OpenCV simplified library (providing noise reduction interfaces such as cv::GaussianBlur) and the video stream management module (obtaining the original image through / dev / isp, calling / dev / vpu for H.265 encoding, and caching the bitstream to the / tmp / stream directory), and only initializes the real-time encoding function (the offline analysis function will be activated after being triggered by the user).

[0064] As the last module loaded, it avoids consuming resources in the core startup process (such as reducing memory usage by 30%), while relying on preceding modules to ensure functional availability; the "on-demand initialization" strategy further reduces startup time, ensuring that the device can quickly enter the basic visual function ready state (such as security cameras can start video capture within 3 seconds after startup), improving the response speed of terminal devices and user experience.

[0065] S5: After loading the operating system kernel and dedicated driver modules, it performs targeted initialization of the dedicated visual processing components, including allocating computing resources to the VPU, configuring the image preprocessing parameters of the ISP, and activating the working mode of the video codec.

[0066] Furthermore, the targeted initialization of the dedicated vision processing component in step S5 also includes: S51: Dynamically allocates VPU computing resources, reserving at least one core resource for real-time visual inspection tasks based on the current number of VPU cores on the chip.

[0067] In this embodiment, the VPU driver dynamically allocates resources by reading the number of cores detected by S2 (e.g., 4 cores, 6 cores) and calling the built-in resource allocation algorithm. First, the driver initializes the VPU's core control register (e.g., address 0x20000010, used to enable / disable cores) and identifies the list of available cores. Then, based on the latency requirements of real-time visual detection tasks (e.g., target tracking, obstacle recognition) (typically ≤50ms), at least one core is reserved from the available cores (e.g., 1 core for a 4-core VPU, 2 cores for an 8-core VPU). The real-time task is bound to the reserved core by configuring the task scheduling register (e.g., address 0x20000020) to ensure it is not interfered with by other tasks. The remaining cores are dynamically allocated to non-real-time tasks (e.g., video encoding / decoding, offline image analysis) through a resource pool management mechanism, and the allocation ratio can be dynamically adjusted according to the task load (e.g., temporarily borrowing one non-reserved core when the encoding / decoding task load is too high).

[0068] For example, in a certain 6-core VPU chip, the driver reserves 2 cores specifically for real-time pedestrian detection tasks (bound to core 0 and core 1) to ensure that the detection latency is stable within 30ms; the remaining 4 cores form a resource pool, of which 2 are allocated to 4K video encoding tasks, 1 is allocated to offline image enhancement tasks, and 1 is used as a spare resource that is automatically activated when the encoding task load exceeds 80%.

[0069] In this step, by reserving dedicated cores for real-time visual inspection tasks, the real-time performance degradation caused by task preemption in general resource allocation is avoided (e.g., latency fluctuation is reduced from ±20ms to ±5ms). At the same time, the dynamic allocation of remaining cores improves the overall computing power utilization of the VPU (from 60% of the traditional static allocation to 90%), balancing real-time performance and resource efficiency, and meeting the stringent requirements of visual inspection response speed in scenarios such as security and autonomous driving.

[0070] S52: Configure the white balance, exposure compensation and noise reduction parameters of the ISP. The parameter values ​​are called from the preset parameter library based on the image sensor model detected in step S2.

[0071] In this embodiment, the preset parameter library is stored in the chip's Flash memory, structured in key-value pairs of "image sensor model - parameter group" (e.g., JSON format). Each parameter group includes adaptation parameters such as white balance (color temperature value, e.g., 4500K-6500K), exposure compensation (EV value, e.g., -2.0 to +2.0), and noise reduction intensity (level 1-5) (parameter values ​​are determined by the manufacturer through calibration experiments for different sensors). During initialization, the ISP driver first searches the parameter library using the image sensor model detected by S2 (e.g., IMX586, OV5640) and matches the corresponding parameter group; then, it writes the parameters to the ISP's configuration registers (e.g., white balance register 0x40, exposure compensation register 0x41, noise reduction register 0x42) via the I2C bus to complete the hardware parameter configuration. If no sensor model is matched, the general parameter group (default values ​​adapted to most sensors) is automatically called.

[0072] For example: Step S2 detects that the image sensor is IMX586 (commonly used in security cameras). The ISP driver calls the corresponding parameter group from the parameter library: white balance 5500K (adapts to indoor natural light), exposure compensation +0.5EV (improves brightness in low light environments), noise reduction intensity 3 (medium noise reduction, balancing details and noise), and writes these parameters into the ISP register through the I2C bus to ensure that the acquired image has accurate color reproduction and appropriate brightness.

[0073] By calling preset parameters based on the sensor model, the tediousness and errors of manually configuring parameters are avoided (traditional solutions require 1-2 hours of manual debugging, while this solution can complete it automatically), ensuring that the ISP parameters are accurately matched with the sensor characteristics, significantly improving the image preprocessing quality (such as reducing color deviation from ±15% to ±3% and noise by 40%), providing high-quality image data for subsequent visual analysis (such as target recognition), and reducing analysis errors caused by poor image quality.

[0074] S53: Perform a self-test on the video codec, and verify the H.264 / H.265 encoding and decoding functions of the codec by outputting a test bitstream.

[0075] In this embodiment, the video codec driver triggers a self-test process during initialization: First, the driver writes a self-test instruction (e.g., 0x01 indicates starting the encoding self-test, 0x02 indicates starting the decoding self-test) to the codec's control register (e.g., address 0x30000000). After receiving the instruction, the codec automatically generates a standard test bitstream (e.g., 1080P / 30fps YUV raw data, containing standard test patterns such as color bars and grayscale gradients), performs H.264 / H.265 encoding on it, and outputs the encoded bitstream. The code is loaded into a memory buffer (e.g., address 0x90000000); the driver reads the encoded bitstream and verifies its integrity (no packet loss) and conformity (compliant with H.264 / H.265 standard syntax) using built-in verification algorithms (e.g., CRC check, bitstream structure parsing); subsequently, the driver inputs the encoded bitstream into the codec for decoding, compares the difference between the decoded YUV data and the original test pattern (pixel error ≤1% is considered normal); if both encoding and decoding pass, the function is considered normal; otherwise, an exception log is recorded and a degradation mechanism is triggered.

[0076] For example: During codec self-test, a 1080P / 30fps color bar test pattern (original YUV data) is generated and encoded into an H.265 bitstream (bitrate 2Mbps). The driver verifies that the bitstream has no packet loss and the NAL unit structure is correct. Then, the bitstream is decoded, and the decoded color bar pattern is compared with the original pattern. The pixel error is 0.5% (≤1%), indicating that the codec's H.265 encoding / decoding function is normal.

[0077] By generating standard test bitstreams and verifying encoding / decoding functions, hardware faults (such as register errors or circuit failures) or driver defects of the codec can be identified in advance. This avoids video recording and transmission failures due to codec malfunctions after startup (such as security cameras being unable to store video), significantly improving system stability and reducing subsequent application-level troubleshooting costs (reducing on-site debugging time by an average of 80%).

[0078] S6: After initialization, load the core user layer services and basic modules of the visual application of the operating system, and complete the startup of the intelligent vision SoC chip operating system.

[0079] Furthermore, in step S6, when loading the core user layer services and basic visual application modules, an on-demand delayed loading strategy is adopted: only the necessary basic visual application modules for the current scenario are loaded, and modules that are not called are temporarily stored in the memory cache and loaded when requested by subsequent applications.

[0080] The on-demand delayed loading strategy is implemented through a three-level mechanism of "scene recognition - module filtering - dynamic triggering". The core is to activate only the modules necessary for the current scene, while the remaining modules remain dormant.

[0081] Scene Recognition and Necessary Module Selection: Upon system startup, the core requirements of the current scene are determined by reading the device's preset configuration (such as the "real-time monitoring mode" or "offline analysis mode" of a smart camera) or initial scene parameters (such as whether an external control terminal is connected). For example, the core requirement of "real-time monitoring mode" is "video acquisition-encoding-transmission," therefore, the video stream acquisition module (which calls the image sensor interface driver to obtain raw data), the H.265 real-time encoding module (which depends on the VPU driver), and the network stream transmission module (which depends on TCP / IP services) are selected as necessary modules; while non-core modules such as the face recognition library and behavior analysis algorithms are marked as "to be loaded."

[0082] Temporary storage management of unused modules: Modules not selected as necessary (such as the face recognition .so library and offline analysis plugin) are temporarily stored in the memory cache (such as the reserved partition 0xD0000000-0xD8000000) or low-speed storage device (such as the / opt / unloaded directory of eMMC) in the form of binary files. They only occupy storage resources (do not occupy CPU computing power or active memory), and their initialization functions (such as face_recog_init()) are not executed temporarily.

[0083] Dynamic loading trigger mechanism: When a user requests (e.g., clicking "Enable Face Recognition" in the APP) or the scene changes (e.g., abnormal movement is detected and behavior analysis needs to be started), the system sends a loading instruction through inter-process communication (IPC), triggering the dynamic loader (e.g., ld-linux.so in Linux) to execute the loading process: read the module file from the cache, load it into user space memory through the dlopen() function, call dlsym() to obtain the module entry function pointer, perform initialization (e.g., load model weights, configure algorithm parameters), and then it can respond to application requests.

[0084] For example, when a smart security camera starts up, it is in the "basic monitoring" scenario, loading only the video stream acquisition module (acquiring 1080P raw data from the IMX586 sensor), the VPU-based H.265 encoding module (compressing the raw data into a bitstream), and the RTSP network transmission module (pushing the bitstream to the backend platform), taking 3 seconds to start up. When the user sends a "face capture" command through the client, the system triggers dynamic loading: reading the face_detect.so library from the memory cache, loading it into memory through dlopen(), and calling init_face_model() to initialize the model (taking about 500ms). Then, it can perform face detection on the real-time video stream without restarting the device.

[0085] This strategy significantly reduces startup time (by 40%-60% compared to full loading) and memory usage (by 30%-50%) by reducing the number of modules loaded during the startup phase, avoiding system lag caused by useless modules occupying hardware resources for extended periods. Simultaneously, the dynamic loading mechanism ensures that non-core functions can be quickly activated on demand (response latency is typically ≤1 second). While meeting the basic requirement of "ready to use immediately upon startup," it also considers functional scalability, greatly improving the operating efficiency and user experience of intelligent vision devices. It is particularly suitable for scenarios with stringent requirements for startup speed and resource efficiency, such as security and automotive applications.

[0086] Furthermore, the method also includes step S7: anomaly monitoring and handling during the startup process, specifically: If the hardware adaptation detection in step S2 fails, the bootloader will output an adaptation error code and trigger the backup adaptation scheme, calling the general vision hardware configuration parameters for retry.

[0087] In this embodiment, the bootloader incorporates anomaly monitoring logic in step S2. It determines whether the bootloader has failed based on the hardware information matching result (S23) (e.g., detecting an unknown manufacturer / model, missing parameters, or conflict with the feature library). If it fails, it first outputs a preset adaptation error code (error codes are defined according to failure type, such as 0x01 indicating manufacturer identification failure, 0x02 indicating VPU parameter abnormality, and 0x03 indicating feature library matching failure) via the UART serial port or debug interface to help developers locate the problem. Then, it triggers a backup adaptation scheme: it calls pre-stored general vision hardware configuration parameters (designed based on common features of mainstream chips, such as a default VPU core count of 2, ISP support for 1080P resolution, and codec compatibility with H.264 base files) from the "General Configuration Partition" of the hardware adaptation library, and re-executes the hardware adaptation detection process of S2 (skipping manufacturer / model identification and directly initializing the scanning logic based on general parameters). If the retry is successful, it continues with subsequent boot steps; if it still fails, it records a fatal error log and enters minimal boot mode (loading only the core kernel and basic drivers to ensure the device can be debugged).

[0088] For example: When a certain intelligent vehicle SoC chip starts up, S2 detects unknown model register data (0xFFFF0000), and the bootloader outputs error code 0x02 (model recognition failed), triggering the backup plan: calling the general configuration parameters (default VPU core count 4, ISP supports 4K resolution) to rescan, and finds that the actual VPU has 4 cores and is compatible with the general parameters. After the retry is successful, the boot process continues, avoiding the boot termination due to unknown model.

[0089] If the vision processing component initialization fails in step S5, shut down the power supply to the malfunctioning component, record the error log, and start the degraded working mode of the remaining normal components.

[0090] During the initialization of the vision processing unit, the driver polls the hardware status registers (such as VPU status register 0x20000001 and ISP status register 0x20010001) to determine whether it has failed (e.g., the register returns 0xFF to indicate initialization timeout, and 0x01 to indicate functional abnormality). If failure occurs, the power supply to the malfunctioning component is first shut down via the power management unit (PMU) control interface (e.g., the power enable register at address 0x10001000) (e.g., clearing the power enable bit of the ISP to 0), cutting off its power supply to avoid power waste and fault propagation. Then, the exception details (including the failed component, timestamp, and status register value, such as "2025-11-18 10:00:00 ISP init failed, status=0xFF") are recorded via the log module (stored in the / var / log directory of the chip's NVRAM or eMMC). Finally, a degraded working mode is initiated: the function is adjusted based on the capabilities of the remaining normal components. For example, when the ISP fails, software simulation preprocessing is enabled (the basic noise reduction algorithm is executed by the CPU). When some VPU cores fail, real-time tasks are centrally allocated to normal cores to ensure that core vision functions (such as video acquisition and basic encoding) can still run.

[0091] For example, when a security camera starts up, the ISP initialization timeout occurs in S5 (the status register returns 0xFF). The system immediately shuts down the ISP power through the PMU (0x00 is written to the power enable register 0x10001000), logs "ISP inittimeout", and starts the degradation mode: calling the CPU-based software noise reduction algorithm to replace the ISP hardware preprocessing. Although the image processing efficiency decreases by 30%, it can still output a recognizable video stream, ensuring that the monitoring function is basically usable.

[0092] This anomaly monitoring and handling mechanism significantly improves the fault tolerance of the intelligent vision SoC chip startup through a layered response strategy: for S2 hardware adaptation failure, error code location and general parameter retry prevent startup termination caused by unknown chip model or abnormal parameters, thus expanding the hardware compatibility range of the solution; for S5 component initialization failure, power-off isolation, logging, and degradation mode prevent the spread of faults and ensure the availability of core vision functions (such as video acquisition in security scenarios and basic image output in vehicle scenarios), reducing the risk of complete device failure due to partial faults, and ultimately improving the reliability and user experience of intelligent vision devices under complex operating conditions (such as low temperature and voltage fluctuations).

[0093] Furthermore, after the bootloader starts in step S1, a firmware security verification step is also included: the bootloader itself and the operating system kernel image file are verified for integrity using a preset hash algorithm. If the verification is successful, the core hardware initialization is performed. If the verification fails, the boot process is terminated and the chip hardware interface is locked.

[0094] In this embodiment, firmware security verification is the first core operation after the bootloader starts. It is implemented through a three-step mechanism of "hash calculation - original value comparison - security response". The entire process runs in the chip's secure execution environment (such as TrustZone's Secure World) to avoid interference from malicious programs.

[0095] Preset hash algorithm and original hash value storage: Industrial-grade encrypted hash algorithms (such as SHA-256, SHA-512, which are collision resistant) are used. Before the chip leaves the factory, the original hash values ​​(fixed-length digests calculated by the algorithm, such as SHA-256 generating a 32-byte value) of the boot program (such as u-boot.bin) and the operating system kernel image file (such as zImage) are burned into the chip's one-time programmable memory area (OTP) or security fuse (Fuse). This area can only be read and cannot be modified to ensure that the original hash value cannot be tampered with.

[0096] Integrity verification execution: After the bootloader starts, it first reads its own binary code in memory (e.g., reads the complete data of u-boot.bin from address 0x00000000) and the kernel image file stored in Flash (e.g., reads zImage from address 0x80000000); then it calls the hash algorithm library in the security environment to calculate the current hash value of the two files respectively; finally, it compares the current hash value with the original hash value pre-stored in OTP byte by byte. If they are completely consistent, the verification passes; if there is any difference in any byte, the verification fails.

[0097] Verification Result Response: If the verification passes, the bootloader will normally enter the core hardware initialization stage (such as processor and memory controller configuration); if the verification fails, the security response mechanism will be triggered immediately: the boot process will be terminated through the hardware security controller (HSC) (stopping the execution of subsequent instructions), and the chip's key hardware interfaces (such as peripheral interfaces such as SPI, I2C, MIPI-CSI, etc., which are achieved by configuring the lock bit of the interface control register) will be locked. At the same time, the fault information (such as "firmware verification failed, error code 0x05") will be output through the security indicator light or encrypted log, and can only be unlocked by the manufacturer's dedicated debugging tools.

[0098] For example, when a smart security SoC chip leaves the factory, the original SHA-256 hash value (0x7a9f3d...8b2c) of the bootloader u-boot.bin and the original hash value (0x1e3b5c...9d7e) of the kernel image zImage are burned into the OTP area. During startup, the bootloader reads its own code and zImage, calculates the current hash values ​​as 0x7a9f3d...8b2c and 0x1e3b5c...9d7e respectively, which are completely consistent with the original OTP values. The verification passes, and core hardware initialization continues. However, if malicious tampering causes the current hash value of zImage to become 0x2f4c6e...0a1b (which does not match the original value), the verification fails, the system immediately terminates the boot process, and locks the SPI and MIPI-CSI interfaces to prevent the tampered firmware from running further or stealing image data.

[0099] This firmware security verification step, through integrity verification based on a hash algorithm, blocks the attack path of malicious firmware tampering (such as implanting backdoor programs or modifying startup logic) from the source of startup. It effectively prevents risks such as illegal control of devices due to firmware tampering (such as security cameras being hijacked to transmit fake videos) and leakage of sensitive data (such as the theft of vehicle vision data). At the same time, through the mechanism of storing the original hash value and locking the hardware interface via OTP, it ensures the immutability of the verification process and secure isolation after failure. This significantly improves the anti-attack capability of the intelligent vision SoC chip in security-sensitive scenarios such as security and autonomous driving, and provides underlying protection for the safe operation of the device throughout its entire life cycle.

[0100] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0101] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0102] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0103] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0104] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0105] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0106] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0107] The electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments in this application.

[0108] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and a bus connecting different system components (including memory and processor).

[0109] The memory stores program code that can be executed by the processor, causing the processor to perform the steps in the various embodiments described in this specification.

[0110] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0111] The memory may also include programs / utilities having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0112] A bus can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus that uses any of the various bus structures.

[0113] Electronic devices can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable user interaction with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be achieved through input / output (I / O) interfaces. Furthermore, electronic devices can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapters. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0114] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0115] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0116] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and are not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention.

Claims

1. A method for starting an intelligent vision SoC chip operating system, characterized in that, The method comprises the following steps: S1: after the intelligent vision SoC chip is powered on, a built-in boot program of the chip is started; the boot program first initializes core hardware of the chip, and the core hardware at least includes a processor, a memory controller and a bus; S2: hardware adaptation detection is performed through the boot program to identify a model, hardware architecture parameters and configuration information of a dedicated vision processing component of the current intelligent vision SoC chip; the dedicated vision processing component at least includes a vision processing unit, an image signal processor (ISP) and a video codec; S3: based on the detection result of step S2, a startup configuration file matched with the current chip is called from a preset hardware adaptation library; the startup configuration file contains module loading priority and vision hardware initialization parameters customized for the current chip; S4: according to the module loading priority in the startup configuration file, an operating system kernel and an intelligent vision dedicated driver module are preferentially loaded; the intelligent vision dedicated driver module at least includes a VPU driver, an ISP driver and an image sensor interface driver; S5: after the operating system kernel and the dedicated driver module are loaded, targeted initialization of the dedicated vision processing component is performed; including allocating computing resources for the VPU, configuring image preprocessing parameters of the ISP, and activating a working mode of the video codec; S6: after the initialization is completed, user layer core services and vision application basic modules of the operating system are loaded to complete startup of the operating system of the intelligent vision SoC chip. 2.The intelligent visual SoC chip operating system starting method according to claim 1, characterized in that, The hardware adaptation detection in step S2 specifically comprises: S21: reading a vendor identification register and a model register of the intelligent vision SoC chip to determine the chip vendor and the model; S22: acquiring the core number of the VPU, the image processing channel number of the ISP and the supported format of the video codec through bus scanning; S23: matching the detected hardware information with a pre-stored hardware feature library in the boot program to verify the integrity and validity of the hardware information.

3. The intelligent vision SoC chip operating system booting method according to claim 1, characterized in that, The setting rule of the module loading priority in step S4 is: First priority: operating system kernel core processes, including process scheduling and memory management; Second priority: intelligent vision dedicated driver modules, including a VPU driver, an ISP driver and an image sensor interface driver; Third priority: operating system basic services, including a file system and a network service; Fourth priority: vision application basic modules, including an image preprocessing library and a video stream management module.

4. The intelligent vision SoC chip operating system booting method of claim 1, wherein, The targeted initialization of the dedicated vision processing component in step S5 further comprises: S51: dynamically allocating computing resources of the VPU, and reserving at least one core resource for a real-time vision detection task according to the VPU core number of the current chip; S52: configuring white balance, exposure compensation and noise reduction parameters of the ISP, and the parameter values are called from a preset parameter library based on the image sensor model detected in step S2; S53: performing a function self-check on the video codec to verify whether H.264 / H.265 encoding and decoding functions of the video codec are normal through output test code streams.

5. The intelligent vision SoC chip operating system booting method according to claim 1, characterized in that, The method further comprises step S7: abnormality monitoring and processing during startup, specifically: If the hardware adaptation detection in step S2 fails, the boot program outputs an adaptation error code and triggers a backup adaptation scheme, calling general visual hardware configuration parameters for retry; If the visual processing component initialization in step S5 fails, the power supply path of the abnormal component is closed, abnormal logs are recorded, and a degraded working mode of the remaining normal components is started.

6. The intelligent vision SoC chip operating system booting method of claim 1, wherein, In step S6, when loading user layer core services and visual application basic modules, a demand-based lazy loading strategy is adopted: only the necessary visual application basic modules in the current scenario are loaded, and the modules not called are temporarily stored in the memory cache area, and the loading is completed when the subsequent application requests.

7. The intelligent vision SoC chip operating system booting method according to claim 1, characterized in that, After the boot program in step S1 is started, a firmware security verification step is further included: the integrity of the boot program itself and the operating system kernel image file is verified by a preset hash algorithm, and after the verification passes, the core hardware initialization is performed, and if the verification fails, the startup is terminated and the chip hardware interface is locked. 8.A non-transitory computer readable storage medium having stored therein at least one instruction or at least one piece of program, characterized in that, The at least one instruction or the at least one program is loaded and executed by the processor to implement the intelligent visual SoC chip operating system startup method of any one of claims 1-7.

9. An electronic device, comprising: The non-transitory computer readable storage medium of claim 8, wherein the processor is configured to load and execute the at least one instruction or the at least one program stored in the storage medium to implement the intelligent visual SoC chip operating system startup method of any one of claims 1-7.

Images