Storage medium-free implementation method for USB camera

Abstract

The invention provides a storage medium-free implementation method for a USB (Universal Serial Bus) camera, which comprises the following steps that: S1, equipment is powered on and firstly enters a recovery state, initial USB communication is established with HOST, and a communication mechanism is a self-defined private protocol transmission mode; s2, the HOST transmits the firmware to the Device end in a USB (Universal Serial Bus) mode; after the device receives the firmware, the firmware is temporarily stored in the OCM, then a code segment is carried to the IRAM, a data DATA segment is reserved in the OCM, and preparation is made for starting a normal system in the next step; s3, the HOST sends a VRPROGSTAGE2 command through the private protocol, and the control equipment jumps to the transmission firmware to run; s4, after the device receives the VRPROGSTAGE2 command, the device is restarted, at the moment, the HOST can carry out reenumeration on the device, and the device can be changed into a usb camera device from a recovery state; s5, the HOST sends a flow opening instruction through a standard UVC protocol, and the Device is controlled to flow out; and S6, after the Device receives the HOST stream opening instruction, preparing a video stream according to a standard UVC protocol, and opening video transmission. According to the method, an external flash memory medium is saved; and drive such as no or / nand flash does not need to be adapted.

Description

Technical Field

[0001] This invention belongs to the field of video image processing technology, and specifically relates to a method for implementing a USB camera without a storage medium. Background Technology

[0002] In existing technologies, the high-definition video image processing workflow can be divided into several stages: image acquisition, image processing at the transmitting end, signal transmission, image processing at the receiving end, and image display. Each stage requires the support of a specific video chip to function. Among these, USB cameras utilize a USB interface, offering plug-and-play functionality, user-friendly operation, and eliminating the need for a capture card, power supply, or disassembly of the chassis. They also support laptops. Compared to traditional surveillance cameras, they are lower in cost, more convenient, and easier to operate.

[0003] UVC stands for USB Video Class, a protocol standard defined for USB video capture devices. It was jointly developed by Microsoft and several other device manufacturers and has become part of the USB org standard.

[0004] Most mainstream operating systems today (such as Windows XP SP2 and later, Linux 2.4.6 and later, macOS 10.5 and later) provide UVC device drivers, so hardware devices that conform to the UVC specification can be used normally in the host without installing any drivers. Devices using UVC technology include webcams, digital cameras, analog-to-digital converters, TV sticks, and still image cameras. The latest UVC version is UVC 1.5, defined by the USB Implementers Forum, including the basic protocol and payload format.

[0005] Webcams were the first and most numerous UVC-enabled devices. Operating systems running Windows XP SP2 and later support UVC, with Vista being a given. Linux kernels since version 2.4 support a large number of device drivers and can support UVC devices.

[0006] Existing USB cameras almost always require external flash memory (NAND / NOR / SD card / eMMC or other storage media) during the boot process. During boot, the device first reads the firmware from the external storage medium into DDR, and then boots from DDR. This adds an extra step of external flash memory read / write to the entire process.

[0007] However, the shortcomings of the existing technology are:

[0008] (1) An additional external flash memory medium increases material costs;

[0009] (2) The operation process involves external flash memory read and write, so the startup time is relatively long;

[0010] (3) Waste pins.

[0011] In addition, commonly used terms in the prior art include:

[0012] (1) USB: Universal Serial Bus.

[0013] (2) OCM: On Chip Memory.

[0014] (3) UVC: USB Video Camera, a camera that transmits video using a USB interface.

[0015] (4) CPU: Central Processing Unit.

[0016] (5) Camera: In this invention, it refers to a USB Camera.

[0017] (6) Device: In this invention, it refers to a USB camera.

[0018] (7) HOST: host, in this invention, can be a laptop / tablet / mobile phone / TV or other terminal containing a USB host.

[0019] (8) NOR / NAND: NOR / NAND flash, an external storage medium that retains data even when power is off.

[0020] (9) Firmware: Firmware.

[0021] (10)DDR: Double Data Rate SDRAM, which refers to memory in this invention.

[0022] (11) IRAM: Instruction RAM, dedicated instruction RAM.

[0023] (12) Recovery: Recovery area, representing the initial state of the firmware.

[0024] (13) V0: Abbreviation for Ingenic's first-generation RISC-V CPU, also known as V0 processor.

[0025] (14) VR: Vendor Request (VR) is abbreviated as VR.

[0026] (15)EP: USB endpoint, short for USB endpoint, USB hardware transmission channel. For example, ep0 and ep1 are endpoint 0 and endpoint 1, respectively.

[0027] (16)SRAM: On-chip memory, in this invention it refers to OCM.

[0028] (17)RV32IMC: An abbreviation for an instruction set supported by RISC-V. Summary of the Invention

[0029] To address the aforementioned issues, the purpose of this application is to implement flash-free booting, meaning that the USB camera boots without accessing external flash memory (NAND / NOR flash). This is achieved by designing a proprietary protocol that utilizes the existing USB interface to complete the device boot process.

[0030] Specifically, the present invention provides a method for implementing a USB camera without a storage medium, the method comprising:

[0031] S1, when the device is powered on, it first enters the recovery state and establishes preliminary USB communication with the HOST. The communication mechanism is a custom private protocol transmission method.

[0032] S2, the HOST transmits the firmware to the device via USB. The firmware is integrated into the manufacturer's third-party software or released to Windows or Linux systems. The C200 chip is a custom camera chip for laptop platforms and can be released to Windows or Linux systems for updates, updating with the system. The firmware is the core of the entire system operation, including the core running program.

[0033] After receiving the firmware, the device temporarily stores the firmware in the OCM, then moves the code segment to the IRAM, and keeps the data segment in the OCM, in preparation for the next step of starting the normal system.

[0034] S3, the HOST sends the VR_PROG_STAGE2 command through the private protocol to control the device to jump to the above-mentioned transmission firmware for operation;

[0035] S4. After receiving the VR_PROG_STAGE2 command, the device restarts. At this time, the host will re-enumerate the device, and the device will change from recovery state to a USB camera device.

[0036] S5, the HOST sends an open stream command through the standard UVC protocol to control the device's outgoing stream;

[0037] S6. After receiving the HOST's command to start streaming, the Device prepares the video stream according to the standard UVC protocol and starts video transmission.

[0038] In the method,

[0039] USB: Used for data transfer;

[0040] CPU: Used to handle parsing and HOST communication;

[0041] OCM: Used to temporarily store system code and instructions;

[0042] IRAM: The space used to execute instructions;

[0043] HOST: The host system, used to communicate with the USB camera and boot from it.

[0044] During the boot process between the HOST and the USB Camera via USB: Steps S1, S2, and S3 utilize a newly added proprietary USB protocol. This protocol is used to transmit firmware and control the device to switch to normal USB camera operating mode. Table 1 below shows the custom proprietary protocol for the entire transmission process.

[0045]

[0046] Table 1

[0047] All data interactions described in the method follow the protocol commands in Table 1 above.

[0048] The details of the private protocol are as follows:

[0049] Stage 1: The HOST obtains Device descriptor information by sending the USB standard request GET_DESCRIPTOR; GET_DESCRIPTOR is a standard request of the USB protocol, and all USB device enumeration begins with it;

[0050] The intelligent video processor returns Device descriptor information to the host via a GET_DESCRIPTPR request;

[0051] Stage 2: Enumeration complete, obtain CPU INFO;

[0052] The host sends VR_GET_CPU_INFO custom requests (vendor requests);

[0053] The intelligent video processor sends VR_GET_CPU_INFO via ep0 and returns CPU information to the HOST;

[0054] Stage 3: Data transmission, including VR_SET_DATA_ADDR and VR_SET_DATA_LEN instructions;

[0055] Stage 4: Start the system and load the image.

[0056] In step Stage 3, the detailed process of data transmission is as follows:

[0057] 1) The HOST sends a VR_SET_DATA_ADDR request to the Device via usb ep0 to notify the SRAM location where the device firmware is stored;

[0058] 2) The HOST sends a VR_SET_DATA_LEN request to the Device via ep0 to notify the device of the size of the firmware to be sent;

[0059] 3) The HOST sends data to the Device through ep1 loop; after receiving the firmware, the Device stores the firmware in the specified SRAM address addr.

[0060] The aforementioned Stage 3 and Stage 4 can be further divided into three processes:

[0061] (1) The HOST sends firmware to the device in a loop through ep0 / ep1 and specifies the device's jump address entry;

[0062] (2) The HOST sends a VR_PROG_STAGE1 request to the Device via ep0, notifying the device to verify the firmware transfer and legality before operation;

[0063] (3) The HOST sends VR_CHECK_RESULT through ep0 to obtain firmware verification information. If the firmware is valid, the HOST sends VR_PROG_STAGE2 request to the Device through ep1. After receiving the instruction, the Device loads and runs the firmware from the entry address read in step (1), and the device switches to normal camera mode; otherwise, it returns to Stage2.

[0064] The HOST includes PCs / tablets, and can also be applied to platforms with USB hosts, such as mobile phones / TVs / industrial cameras.

[0065] The method uses a camera with a USB interface to transmit video via UVC, which is defined by the USB Implementers Forum and includes basic protocols and payload formats, and supports UVC.

[0066] The operating systems used by the host include Windows XP SP2 and later versions that support UVC, Vista that supports UVC, and Linux systems with kernel versions 2.4 and later that support multiple device drivers and simultaneously support UVC devices.

[0067] The device uses a 22-nanometer process, supports RGB-IR sensor or ordinary RGB sensor access, supports up to 1080p 60fps JPEG encoding, and is suitable for scenarios such as personal / home webcams, video conferencing, and laptop cameras.

[0068] The chip is an intelligent video processor, including the Ingenic C200 model. The C200 model chip is also used in professional security, medical, and 3D printing fields. In terms of performance, it has powerful hardware computing power and efficient video encoding capabilities. The C200 model chip is equipped with Ingenic's self-developed RISC-V single-core processor. The self-developed RISC-V includes 1) a RISC-V single core, supporting up to 400MHz; 2) a 32-bit, three-stage pipelined instruction structure; and 3) support for the RV32IMC instruction set architecture. This enables the C200 model chip to fully utilize hardware efficiency under limited hardware resources and greatly reduce the chip's power consumption.

[0069] Therefore, the advantage of this application is:

[0070] (1) Saves external flash memory media, eliminating the need to add a single component;

[0071] (2) Saves software development work, as there is no need to adapt to drivers such as NOR / NAND flash;

[0072] (3) Safe and reliable, requiring no external encryption chip or other encryption methods;

[0073] (4) Save chip pins. Attached Figure Description

[0074] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0075] Figure 1 This is a schematic diagram of the operation process of the entire device using the method of this application.

[0076] Figure 2 This is a schematic diagram of the USB custom proprietary protocol transmission and interaction in the scheme of this application. Detailed Implementation

[0077] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0078] This application proposes a storage-media-free implementation method for USB cameras. The following is an embodiment. This technology can currently be applied to the Ingenic C200 chip project. The C200 is a professional intelligent video application processor that Ingenic Technology is about to mass-produce. It adopts a 22-nanometer process, supports RGB-IR sensor access, and supports up to 1080p60fps JPEG encoding. This chip is suitable for personal / home webcams, video conferencing, laptop cameras, and other scenarios.

[0079] Furthermore, the C200 chip is also used in professional security, medical fields, 3D printers, and other areas. This is mainly due to its powerful hardware computing power and efficient video encoding capabilities. In terms of performance, the C200 is equipped with a RISC-V (V0) single-core processor developed by Ingenic, with a CPU of 400MHz and an OCM of 400MHz. This enables the C200 to fully utilize hardware efficiency under limited hardware resources, greatly reducing the chip's power consumption.

[0080] Webcams using USB interfaces to transmit video utilize UVC, with the latest version being UVC 1.5. Defined by the USB Implementers Forum, it includes the basic protocol and payload format. Webcams were the first and most numerous devices to support UVC. Operating systems running Windows XP SP2 and later support UVC, including Vista. Linux kernels since version 2.4 support a large number of device drivers and can support UVC devices.

[0081] The specific implementation of the method is as follows: Figure 1 As shown, the entire equipment processing flow is as follows:

[0082] S1, when the device is powered on, it first enters the recovery state and establishes preliminary USB communication with the HOST (PC). This communication mechanism is a custom proprietary protocol transmission method defined in this method.

[0083] S2, the HOST (PC) transmits the firmware to the Device. This firmware is integrated into the manufacturer's third-party software, or it may be published to Windows or Linux systems and updated with the system. This firmware is the core of the entire system's operation, including the core runtime program. In traditional solutions, this firmware is stored on external storage media (NOR flash / SD card, etc.). This invention transmits the firmware to the device via USB through the HOST (PC). The firmware can be integrated into the manufacturer's third-party software.

[0084] After receiving the firmware, the device temporarily stores the firmware in the OCM, then moves the code segment to the IRAM, and keeps the DATA segment in the OCM, in preparation for the next step of starting the normal system.

[0085] S3, the HOST (PC) sends the VR_PROG_STAGE2 command via a proprietary protocol to control the device to jump to the aforementioned transmission firmware for operation. For a detailed explanation of the proprietary protocol, please see the following text.

[0086] S4. After receiving the VR_PROG_STAGE2 command, the device restarts. At this time, the HOST (PC) will re-enumerate the device, and the device will change from recovery state to a USB camera device.

[0087] S5, HOST (PC) sends an open stream command via the standard UVC protocol to control the device's outgoing stream;

[0088] S6. After receiving the HOST (PC) start-up command, the Device prepares the video stream according to the standard UVC protocol and starts video transmission.

[0089] During the startup process between the HOST and the USB Camera via USB: a new proprietary USB protocol is used in steps S1, S2, and S3. This new proprietary protocol is used to transmit firmware and control the device to switch to normal USB camera operating mode.

[0090] like Figure 2 As shown, the details of the private protocol are as follows:

[0091] Stage 1: The HOST (PC) obtains Device descriptor information by sending a standard USB request (GET_DESCRIPTOR). GET_DESCRIPTOR is a standard request in the USB protocol, and all USB device enumeration begins with it.

[0092] The C200 processor returns device descriptor information to the host via a GET_DESCRIPTPR request.

[0093] Stage 2: Enumeration complete, obtain CPU INFO;

[0094] The HOST (PC) sends VR_GET_CPU_INFO custom requests, referred to as vendor requests; Vendor Request is the term for custom requests in this patent, abbreviated as VR;

[0095] The C200 processor sends VR_GET_CPU_INFO via ep0 and returns CPU information to the HOST (PC);

[0096] Stage 3: Data transmission, including commands such as VR_SET_DATA_ADDR and VR_SET_DATA_LEN. The detailed process of data transmission in Stage 3 is as follows:

[0097] 1) The HOST sends a VR_SET_DATA_ADDR request to the Device via usb ep0 to notify the SRAM location where the device firmware is stored;

[0098] 2) The HOST sends a VR_SET_DATA_LEN request to the Device via ep0 to notify the device of the size of the firmware to be sent;

[0099] 3) The HOST sends data to the Device through ep1 loop; after receiving the firmware, the Device stores the firmware in the specified SRAM address addr.

[0100] The aforementioned Stage 3 and Stage 4 can be further divided into three processes:

[0101] (1) The HOST sends firmware to the device in a loop through ep0 / ep1 and specifies the device's jump address entry;

[0102] (2) The HOST sends a VR_PROG_STAGE1 request to the Device via ep0, notifying the device to verify the firmware transfer and legality before operation;

[0103] (3) The HOST sends VR_CHECK_RESULT through ep0 to obtain firmware verification information. If the firmware is valid, the HOST sends VR_PROG_STAGE2 request to the Device through ep1. After receiving the instruction, the Device loads and runs the firmware from the entry address read in step (1), and the device switches to normal camera mode; otherwise, it returns to Stage2.

[0104] The HOST includes PCs / tablets, and can also be applied to platforms with USB hosts, such as mobile phones / TVs / industrial cameras.

[0105] The table below shows the private protocol customization for the entire transmission protocol. All data interactions described above are performed according to the following protocol commands.

[0106]

[0107] Table 1

[0108] The chip is an intelligent video processor, including the Ingenic C200 model. The C200 model chip is also used in professional security, medical, and 3D printing fields. In terms of performance, it has powerful hardware computing power and efficient video encoding capabilities. The C200 model chip is equipped with Ingenic's self-developed RISC-V single-core processor. The self-developed RISC-V includes 1) a RISC-V single core, supporting up to 400MHz; 2) a 32-bit, three-stage pipelined instruction structure; and 3) support for the RV32IMC instruction set architecture. This enables the C200 model chip to fully utilize hardware efficiency under limited hardware resources and greatly reduce the chip's power consumption.

[0109] In summary, by using the technical solution of this invention, the following can be achieved:

[0110] (1) Saves storage media

[0111] This invention: a 1920x1080 resolution camera that can work without an external NOR flash.

[0112] Current technology: 1920*1080 resolution cameras typically require 512k NOR flash memory. For 2k and other higher resolution cameras, the savings in material costs are even greater.

[0113] (2) Save ROM space and development workload

[0114] This invention eliminates the need to develop NOR / NAND flash drivers for the bootroom and system drivers, thereby saving bootroom space to some extent.

[0115] Existing technology: Existing solutions basically include flash memory, which increases the development workload at the bootroom and system driver levels.

[0116] (3) Safe and reliable

[0117] This invention: This method does not require external flash memory. It boots the device using a proprietary USB protocol, which greatly improves overall security.

[0118] Existing technology: Current solutions store firmware on flash memory, which can be read and cracked using third-party tools, leading to program copying. Other encryption methods are necessary to prevent cracking.

[0119] (4) Saves chip pins,

[0120] This invention eliminates the NOR / NAND flash interface, resulting in fewer I / O pins compared to the transmission chip, thus simplifying chip-side design.

[0121] Existing technology: Most existing solutions have NOR / NAND flash interfaces and I / O pins.

[0122] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for implementing a USB camera without a storage medium, characterized in that, The method includes: S1, when the device is powered on, it first enters the recovery state and establishes preliminary USB communication with the HOST. The communication mechanism is a custom private protocol transmission method. S2, the HOST transmits the firmware to the device via USB. The firmware is integrated into the manufacturer's third-party software or released to the Windows or Linux system and updates with the system. The firmware is the core of the entire system operation, including the core running program. After receiving the firmware, the device temporarily stores the firmware in the OCM, then moves the code segment to the IRAM, and keeps the data segment in the OCM, in preparation for the next step of starting the normal system. S3, the HOST sends the VR_PROG_STAGE2 command through the private protocol to control the device to jump to the above-mentioned transmission firmware for operation; S4. After receiving the VR_PROG_STAGE2 command, the device restarts. At this time, the host will re-enumerate the device, and the device will change from recovery state to a USB camera device. S5, the HOST sends an open stream command through the standard UVC protocol to control the device's outgoing stream; S6. After receiving the HOST's command to start streaming, the Device prepares the video stream according to the standard UVC protocol and starts video transmission. In the method, USB: Used for data transfer; CPU: Used to handle parsing and HOST communication; OCM: Used to temporarily store system code and instructions; IRAM: The space used to execute instructions; HOST: The host system, used to communicate with the USB camera and boot from it.

2. The method for implementing a USB camera without a storage medium according to claim 1, characterized in that, During the boot process between the HOST and the USB Camera via USB: Steps S1, S2, and S3 utilize a newly added proprietary USB protocol. This protocol is used to transmit firmware and control the device to switch to normal USB camera operating mode. Table 1 below shows the custom proprietary protocol for the entire transmission process. Table 1 All data interactions described in the method follow the protocol commands in Table 1 above.

3. The method for implementing a USB camera without a storage medium according to claim 2, characterized in that, The details of the private protocol are as follows: Stage 1: The HOST obtains Device descriptor information by sending the USB standard request GET_DESCRIPTOR; GET_DESCRIPTOR is a standard request of the USB protocol, and all USB device enumeration begins with it; The intelligent video processor returns Device descriptor information to the host via a GET_DESCRIPTPR request; Stage 2: Enumeration complete, obtain CPU INFO; The host sends VR_GET_CPU_INFO custom requests (vendor requests); The intelligent video processor sends VR_GET_CPU_INFO via ep0 and returns CPU information to the HOST; Stage 3: Data transmission, including VR_SET_DATA_ADDR and VR_SET_DATA_LEN instructions; Stage 4: Start the system and load the image.

4. The method for implementing a USB camera without a storage medium according to claim 3, characterized in that, In step Stage 3, the detailed process of data transmission is as follows: 1) The HOST sends a VR_SET_DATA_ADDR request to the Device via usb ep0 to notify the SRAM location where the device firmware is stored; 2) The HOST sends a VR_SET_DATA_LEN request to the Device via ep0 to notify the device of the size of the firmware to be sent; 3) The HOST sends data to the Device through ep1 loop; after receiving the firmware, the Device stores the firmware in the specified SRAM address addr.

5. A method for implementing a USB camera without a storage medium according to claim 3, characterized in that, The aforementioned Stage 3 and Stage 4 can be further divided into three processes: (1) The HOST sends firmware to the device in a loop through ep0 / ep1 and specifies the device's jump address entry; (2) The HOST sends a VR_PROG_STAGE1 request to the Device via ep0, notifying the device to verify the firmware transfer and legality before operation; (3) The HOST sends VR_CHECK_RESULT through ep0 to obtain firmware verification information. If the firmware is valid, the HOST sends VR_PROG_STAGE2 request to the Device through ep1. After receiving the instruction, the Device loads and runs the firmware from the entry address read in step (1), and the device switches to normal camera mode; otherwise, it returns to Stage2.

6. The method for implementing a USB camera without a storage medium according to claim 1, characterized in that, The HOST includes PCs / tablets, and can also be applied to platforms with USB hosts, such as mobile phones / TVs / industrial cameras.

7. The method for implementing a USB camera without a storage medium according to claim 1, characterized in that, The method uses a camera with a USB interface to transmit video via UVC, which is defined by the USB Implementers Forum and includes basic protocols and payload formats, and supports UVC.

8. A method for implementing a USB camera without a storage medium according to claim 7, characterized in that, The operating systems used by the host include Windows XP SP2 and later versions that support UVC, Vista that supports UVC, and Linux systems with kernel versions 2.4 and later that support multiple device drivers and simultaneously support UVC devices.

9. A method for implementing a USB camera without a storage medium according to claim 1, characterized in that, The device uses a 22-nanometer process, supports RGB-IR sensor or ordinary RGB sensor access, supports up to 1080p 60fps JPEG encoding, and is suitable for scenarios such as personal / home webcams, video conferencing, and laptop cameras.

10. A method for implementing a USB camera without a storage medium according to claim 9, characterized in that, The chip is an intelligent video processor, including the Ingenic C200 model chip. The C200 model chip is also used in professional security, medical fields, and 3D printing fields. In terms of performance, it has powerful hardware computing power and efficient video encoding capabilities. The C200 model chip is equipped with Ingenic's self-developed RISC-V single-core processor. The self-developed RISC-V includes 1) RISC-V single core, which supports a maximum of 400MHz; 2) 32-bit, three-stage pipelined instruction structure; 3) support for RV32IMC instruction set architecture.

Images