USB time camera

By using a combination of CAN interface and USB interface in USB time cameras, the problem of single-level trigger signals being easily disturbed and not suitable for long-distance transmission is solved, high-precision time synchronization and robot attitude control are achieved, and user experience is improved and the applicability of product structure.

CN118977278AActive Publication Date: 2024-11-19WUHAN YUGUANG VISION TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411103944.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-19
Estimated Expiration
2044-08-13

AI Technical Summary

Technical Problem

The existing single-level trigger signals are easily disturbed and are not suitable for long-distance transmission, resulting in image time label errors and product structure limitations, making it difficult to meet the precise needs of robot posture and position perception.

Method used

Using the combination of CAN interface and USB interface, the time information sent by the robot is received through the transmission interface module of the USB time camera, the system time is updated, and the timestamp is recorded and packaged when taking images. The MIPI protocol and UVC driver are used to achieve high-precision time synchronous.

Benefits of technology

It realizes high-precision and low-latency time synchronization in robot control, improves user experience, ensures precise control of robot posture and position, and has strong anti-interference and is suitable for long-distance transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118977278A_ABST
    Figure CN118977278A_ABST
Patent Text Reader

Abstract

The invention is suitable for the field of robots and industrial cameras, and provides a USB time camera which is used for sensing the posture and position of a robot. The USB time camera comprises a main control module, an image sensing module and a transmission interface module, wherein the image sensing module and the transmission interface module are connected with the main control module; the transmission interface module is integrated with a USB interface and a CAN interface, and the transmission interface module is used for receiving time information sent by a robot through the USB interface or the CAN interface so that the master control module can update system time; the main control module can record system time when the image sensing module shoots the image and package the system time and the shot image into image data with a timestamp so as to transmit the image data to the robot; according to the invention, a flexible mode of either the CAN interface or the USB interface is used, long-distance transmission can be realized, relatively good anti-interference performance can be realized, high-precision time synchronization is realized, and the time precision can reach a millisecond level.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of robots and industrial cameras, and in particular relates to a USB time camera. Background Art

[0002] In the field of robotics, robots use many sensors, and the data from each sensor needs to be time-tagged, so that the robot can accurately know its current posture and position and then proceed to the next action.

[0003] The USB time camera not only needs to transmit the captured pictures in real time, but also needs to record the time when the picture was taken and transmit it to the robot, so that the robot can make corresponding actions and judgments.

[0004] Figure 8 The method that is currently used more often is shown: the robot host gives a trigger signal, and the host records the time T1 when the trigger signal is generated. After receiving the trigger signal, the ordinary USB camera immediately exposes and shoots the picture and transmits the picture to the robot host via the USB cable. The robot host then prints the time T1 on the picture, and then triggers the next shooting action; the disadvantages of this method: the single-level trigger signal is easily interfered by the outside world, which can easily cause false triggering, resulting in the risk of incorrect image time labeling; the single-level trigger signal is not suitable for long-distance transmission, which will greatly limit the product structure and is inconvenient for product development, which is a problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a USB time camera, aiming to solve the problem that the existing single-level trigger signal is easily interfered and is not suitable for long-distance transmission; the present invention uses a flexible method of selecting between a CAN interface and a USB interface, which can not only achieve long-distance transmission, but also has better anti-interference performance, and the time accuracy can reach the millisecond level.

[0006] The present invention is implemented as follows: a USB time camera is used for the posture and position perception of a robot; the USB time camera comprises: a main control module, an image sensing module connected to the main control module and a transmission interface module;

[0007] The transmission interface module integrates a USB interface and a CAN interface. The transmission interface module is used to receive time information sent by the robot via the USB interface or the CAN interface so that the main control module can update the system time; the main control module can record the system time when the image sensor module takes the image, and package the system time and the taken image into image data with a timestamp for easy transmission to the robot.

[0008] Furthermore, the image sensing module includes a lens and an image sensor arranged on the image plane of the lens. The image sensor and the main control module are connected and configured based on the MIPI protocol, and the image data converted by the image sensor is transmitted to the main control module.

[0009] Furthermore, the transmission interface module includes a power supply, a USB bus and a CAN bus.

[0010] Furthermore, the image sensor adopts SC132GS.

[0011] Furthermore, the robot has a robot host, the USB bus is configured with a UVC driver as a transmitter, and the UVC driver is coupled with a synchronizer in the robot host;

[0012] The USB bus is configured with another UVC driver as a receiving end connected to the main control module; wherein the synchronizer sends a timestamp according to a specified period, the timestamp is transmitted to the main control module through the UVC driver, and the main control module updates the system time according to the timestamp.

[0013] Furthermore, the CAN bus is configured with a time synchronization protocol based on the UVACAN network.

[0014] Furthermore, when the main control module transmits the image data to the robot, it uses the display timestamp and system clock reference of the USB interface to transmit the image data.

[0015] Compared with the prior art, the USB time camera provided by the present invention can use a flexible method of selecting either a CAN interface or a USB interface to perform time synchronization, which can be transmitted over long distances and has relatively good anti-interference performance, and the time accuracy can reach the millisecond level; it has the characteristics of high precision, low latency, and multiple interfaces; when applied to robot control, the robot host actively sends time information to the USB time camera through the USB interface or the CAN interface, and then the USB time camera updates the system time after receiving the time, and keeps it consistent with the system clock of the robot; when the USB time camera takes a picture, it records the current system time, and then packages the picture and the recorded time and sends them to the robot host through the USB interface, so that the robot can make corresponding actions and judgments, realize precise control of the robot, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A structural block diagram of a USB time camera provided by an embodiment of the present invention;

[0017] Figure 2 This is a block diagram of the internal structure of a USB time camera in an embodiment of the present invention;

[0018] Figure 3 TYPE-C interface circuit diagram of the embodiment of the present invention;

[0019] Figure 4 is a circuit diagram of an image sensor in an embodiment of the present invention;

[0020] Figure 5 This is a connection diagram of the CAN bus in an embodiment of the present invention;

[0021] Figure 6 A framework diagram of USB bus time synchronization in an embodiment of the present invention;

[0022] Figure 7 A flowchart of USB bus time synchronization in an embodiment of the present invention;

[0023] Figure 8 This is the structural block diagram of a conventional USB camera. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] It can be understood that the terms "first", "second", etc. used in the present application can be used in this article to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish the first element from another element.

[0026] Combination Figure 1 to Figure 7 In one embodiment, a USB time camera is provided for the posture and position perception of a robot; in combination with the contents of the diagram, the USB time camera comprises: a main control module, an image sensing module connected to the main control module, and a transmission interface module;

[0027] The transmission interface module integrates a USB interface and a CAN interface. The transmission interface module is used to receive time information sent by the robot via the USB interface or the CAN interface so that the main control module can update the system time; the main control module can record the system time when the image sensor module takes the image, and package the system time and the taken image into image data with a timestamp for easy transmission to the robot.

[0028] The USB time camera provided in this embodiment can use a flexible method of selecting either a CAN interface or a USB interface, which can not only transmit over long distances, but also have relatively good anti-interference performance, and the time accuracy can reach the millisecond level; it has the characteristics of high precision, low latency, and multiple interfaces; when used in robot control, the robot host 200 actively sends time information to the USB time camera 100 through the USB interface or the CAN interface, and then the USB time camera 100 updates the system time after receiving the time to keep it consistent with the system clock of the robot; when the USB time camera 100 takes a picture (or image), it records the current system time, and then packages the picture (or image) and the recorded time and sends them to the robot host 200 through the USB interface, so that the robot can make corresponding actions and judgments, realize precise control of the robot, and improve user experience.

[0029] In an example of an embodiment, the image sensing module includes a lens and an image sensor arranged on the image plane of the lens. The image sensor and the main control module are connected and configured based on the MIPI protocol, and the image data converted by the image sensor is transmitted to the main control module; the main control module adopts an image processing unit, specifically a CPU (central processing unit).

[0030] In this example, the image sensor is used to collect image data. When collecting image data (equivalent to exposure), an exposure signal is sent out to notify the main control module, that is, the CPU, in the form of an interrupt; the CPU will record the time T1 of the exposure signal; thereafter, the collected image data is sent to the CPU via the MIPI protocol; the CPU receives the image data via the MIPI protocol. The image data is the original Raw Data data, which is processed by online programming to obtain the imaging effect. Then, the recorded T1 time and the processed image data are packaged together and transmitted to the robot host or host computer via the UVC protocol.

[0031] In an example of an embodiment, the main configuration of the USB time camera is given; see Table 1 for details.

[0032] Table 1 is a main configuration table of the USB time camera in this embodiment.

[0033]

[0034] In an example of an embodiment, the transmission interface module includes a power supply, a USB bus and a CAN bus; the power supply is a 5V power supply that meets the requirements of the USB interface and is provided as the main power supply for the USB time camera 100 to operate.

[0035] In an example scenario, such as Figure 2 As shown, the transmission interface module is a customized Type-C interface, which includes power supply, USB bus and CAN bus;

[0036] Among them, Figure 3 As shown in the figure, the USB bus uses the two data lines D+ and D- of USB 2.0; for the CAN bus, since the Type-C interface has reserved an interface position for the USB 3.0 protocol, and the customized Type-C interface does not use the USB3.0 protocol, the interface position of the USB 3.0 protocol is replaced with the CAN bus, which makes the installation of the robot simple and only requires a USB data cable to achieve connection.

[0037] In one example of an embodiment, Figure 4 As shown, it is an image sensor, wherein the LED Strobe pin (framed in the figure) is the signal function pin during exposure; if the image sensor is in the exposure period, the LED Strobe pin is always in a high level state until the exposure ends; in this way, the exact time when the image sensor is exposed can be accurately known through the signal function pin, and then this exact time can be recorded for subsequent packaging.

[0038] In this example, for this signal function pin, a command to enable the LED Strobe pin can be sent through an I2C command when initializing the image sensor;

[0039] After the CPU receives the exposure signal from the image sensor, it will immediately respond and record the system time of the interruption. Then the CPU waits for the completion of receiving the image via the MIPI protocol, and then stamps the image with the timestamp just recorded.

[0040] In an example of this embodiment, the image sensor may be SC132GS.

[0041] In an example of this embodiment, Figure 5 The figure shows a connection diagram of a CAN bus; the CAN bus has only two differential signal lines, CAN_H and CAN_L. The characteristics of the CAN bus are strong anti-interference, long transmission distance, fast transmission speed, and support for many nodes. All units in the CAN network are connected together through the two differential signal lines CAN_H and CAN_L. In this way, the USB time camera and the robot host will be mounted in this CAN network. The robot host will continuously broadcast its system time, and the node where the USB time camera is located will synchronize its clock after receiving the broadcast.

[0042] In an example of this embodiment, the CAN bus is configured with a time synchronization protocol based on the UVACAN network; the time synchronization protocol is an application layer protocol based on the CAN bus; it is very similar to the PTP (Precision Time Protocol) protocol, which is a time synchronization protocol mainly used for high-precision time synchronization between devices.

[0043] Therefore, the CAN bus can implement a high-precision time synchronization protocol because it uses the open source UVACAN network and a PTP-like high-precision clock synchronization protocol developed based on the CAN bus.

[0044] In an example of this embodiment, Figure 6 , Figure 7 As shown, the robot has a robot host 200, the USB bus is configured with a UVC driver as a transmitter, and the UVC driver is coupled with a synchronizer in the robot host 200;

[0045] The USB bus is configured with another UVC driver as a receiving end connected to the main control module; wherein the synchronizer sends a timestamp according to a specified period, the timestamp is transmitted to the main control module through the UVC driver, and the main control module updates the system time according to the timestamp.

[0046] In this example, the synchronizer in the robot host 200 may be an application thread, and the application thread sends a timestamp at intervals of about 1 second; the sent timestamp is the system time of the current robot;

[0047] The sent timestamp is sent to the lower computer, i.e., the USB time camera 100, via the UVC device and the USB bus;

[0048] After the UVC driver of the USB time camera 100 receives the timestamp, it will immediately update it into the system to achieve high-precision time synchronization.

[0049] In an example of this embodiment, the application thread can adjust the interval time according to the usage situation.

[0050] In an example of this embodiment, when the main control module transmits the image data to the robot, it uses the display timestamp and system clock reference of the USB interface to transmit the image data.

[0051] Among them, in the USB interface, the USB protocol defines the Payload Header and Payload Data of the video stream; because the default video stream header information contains the PTS (display timestamp) field, it is the location reserved for the timestamp; however, this example requires a higher time precision, and 64 bits are needed to store the high-precision timestamp. Therefore, this example occupies some data bits of the SCR as a carrier, which can parse the correct high-precision timestamp information and image data to the robot host.

[0052] As mentioned above, the PTS and SCR (i.e., display timestamp and system clock reference) of the Payload Header of the video stream in the USB protocol are used to transmit the high-precision timestamp of the captured image, which is the timestamp recorded by the interruption above; high-precision time synchronization can be achieved, and the time when the image is taken can be accurately recorded; while the ordinary USB camera 300 uses the time when the MIPI receives the image, which will result in inaccurate time; in comparison, this example uses the exposure interrupt method to improve the accuracy of recording the shooting time; at the same time, this example cancels the original USB camera 300 using the level trigger signal method, which is prone to interference; and the combination of USB bus and CAN bus only requires one Type-C connection line to simultaneously realize the transmission of CAN bus and USB bus (or USB2.0 data line and CAN data line in the figure), which can greatly increase the transmission distance and has strong anti-interference ability.

[0053] The USB time camera provided in the above embodiment realizes the combination of USB bus and CAN bus through a Type-C connection line, and has the characteristics of high precision, low latency and multiple interfaces. When used in robot control, the robot host 200 actively sends time information to the USB time camera 100 through the USB interface or the CAN interface, and then the USB time camera 100 updates the system time after receiving the time to keep it consistent with the system clock of the robot. When the USB time camera 100 takes a picture, it records the current system time, and then packages the picture and the recorded time and sends them to the robot host 200 through the USB interface, so that the robot can make corresponding actions and judgments.

[0054] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A USB time camera for robot posture and position perception; characterized in that: The USB time camera comprises: a main control module, an image sensor module connected to the main control module and a transmission interface module; The transmission interface module integrates a USB interface and a CAN interface. The transmission interface module is used to receive time information sent by the robot via the USB interface or the CAN interface so that the main control module can update the system time; the main control module can record the system time when the image sensor module takes the image, and package the system time and the taken image into image data with a timestamp for easy transmission to the robot.

2. The USB time camera according to claim 1, characterized in that: The image sensing module includes a lens and an image sensor arranged on the image plane of the lens. The image sensor and the main control module are connected and configured based on the MIPI protocol, and the image data converted by the image sensor is transmitted to the main control module.

3. The USB time camera according to claim 1 or 2, characterized in that: The transmission interface module includes a power supply, a USB bus and a CAN bus.

4. The USB time camera as claimed in claim 2, characterized in that: The image sensor adopts SC132GS.

5. The USB time camera as claimed in claim 3, characterized in that: The robot has a robot host, the USB bus is configured with a UVC driver as a transmitter, and the UVC driver is coupled with a synchronizer in the robot host; The USB bus is configured with another UVC driver as a receiving end connected to the main control module; wherein the synchronizer sends a timestamp according to a specified period, the timestamp is transmitted to the main control module through the UVC driver, and the main control module updates the system time according to the timestamp.

6. The USB time camera as claimed in claim 3, characterized in that: The CAN bus is configured with a time synchronization protocol based on the UVACAN network.

7. The USB time camera according to claim 1, characterized in that: When the main control module transmits the image data to the robot, it uses the display timestamp and system clock reference of the USB interface to transmit.

Citation Information

Patent Citations

  • Joint calibration method for monocular camera and millimeter wave radar

    CN113514803A

  • Data recording and playback method and device and computer storage medium

    CN114090827A

  • Multifunctional industrial camera system based on single-chip FPGA

    CN114584708A

  • Autonomous Vehicle Interface System

    US20150331422A1

  • Inspection planning using robotic systems

    US20210129326A1