Time synchronization method for video information and bus message information, and electronic devices.

Abstract

To provide an invention that relates to a technical field of a software development for a vehicle, and more particularly provide a method and system for time synchronization of video information and bus message information.SOLUTION: A time stamp T0 of a head frame of video information, a time stamp T1 of a tail frame of the video information, a time stamp T2 of the tail frame of a bus message, and a time stamp Ty of each frame of the bus message are recorded, and the time stamp of the head frame of the bus message is matched with the time stamp T0 of the head frame of the video information. A proportionality coefficient k=(T1-T0) / (T2-T0) is derived. All recorded bus messages are traversed and the time stamp Ty of each frame is modified to Tx=T0+(Ty-T0)*k and stored, so that the time stamp Tx of each frame of the bus message is recorded with reference to a clock of the video information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application claims the Chinese Patent Application No. 2023118632749 filed on December 29, 2023, and the US Patent Application No. 18 / 516,995 filed on November 22, 2023, based on priority, and all of its contents are incorporated herein by reference.

[0002] The present invention belongs to the technical field of vehicle software development, and specifically relates to a method and system for time synchronization of video information and bus message information.

Background Art

[0003] In the field of automotive ADAS (Advanced Driving Assistance System) development and verification, it often occurs that data from multiple devices that are not always related to each other needs to be collected. The timestamps between each device may be different. For example, the timestamp of the bus data collection device is from itself, the timestamp of the video collection device is from the computer, and the error between the timestamp of the bus data collection device and the timestamp of the computer is not fixed. The timestamp becomes larger or smaller, and when it accumulates to a certain extent, the error with the computer's timestamp becomes very large, so that the data collected between each device cannot be synchronized in time.

Summary of the Invention

[0004] The present invention relates to a method for time synchronization of video information and bus message information. This method includes the following. Define the timestamp of the video information as the base timestamp. When starting the recording, obtain the timestamp T0 of the first frame of the video information, align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment. When recording is stopped, the timestamp T1 of the video information tail frame and the timestamp T2 of the bus message tail frame are obtained. We derive the proportionality constant k = (T1 - T0) / (T2 - T0). All recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx = T0 + (Ty - T0) * k and saved, so that the timestamp Tx of each frame of the bus messages is recorded relative to the clock of the video information.

[0005] Other features and advantages of the present invention are described in the following specification and some will become apparent from the specification or will be understood by practicing the invention. The object and other advantages of the present invention are realized and obtained by the structures specifically pointed out in the specification and drawings.

[0006] To make the above-mentioned objectives, features, and advantages of the present invention easier to understand, preferred embodiments will be given below and described in detail in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0007] To more clearly describe specific embodiments of the present invention or technical solutions of the prior art, the drawings that are necessary for describing specific embodiments or prior art are briefly described below. The drawings described below are some embodiments of the present invention, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without any creative effort. [Figure 1] This is a flowchart illustrating a time synchronization method for video information and bus message information in several embodiments. [Figure 2] This is a principle block diagram of a bus adapter according to several embodiments. [Figure 3] This is a principle block diagram of a system for realizing a virtual channel for automobile buses according to several embodiments. [Figure 4] This is a principle block diagram of a system for realizing a virtual channel for automobile buses according to several embodiments. [Figure 5] This is a principle block diagram of an electronic device according to several embodiments. [Figure 6] This is a principle block diagram of an electronic device according to several embodiments. [Figure 7] This is a system principle block diagram relating to several embodiments. [Modes for carrying out the invention]

[0008] To further clarify the object, technical aspects and advantages of the embodiments of the present invention, the technical aspects of the present invention will be described below clearly and completely in reference to the accompanying drawings, although it is clear that the embodiments described are only some embodiments of the present invention and not all embodiments. All other embodiments obtained based on the embodiments of the present invention, without the creative work of a person skilled in the art, are within the scope of the protection of the invention.

[0009] In the field of automotive ADAS development and verification, it is always necessary to collect data from multiple unrelated devices. The timestamps between each device tend to differ. For example, the timestamp of a bus data acquisition device is from itself, and the timestamp of a video acquisition device is from the computer. The error between the bus data acquisition device's timestamp and the computer's timestamp is not fixed, and the timestamps become larger or smaller. Once accumulated to a certain extent, the error between the timestamp and the computer's timestamp becomes very large, and as a result, the data collected between each device cannot be synchronized in time.

[0010] In related technologies, a single independent device is customized to simultaneously possess both automotive video information acquisition and bus data acquisition functions. Alternatively, multiple independent devices are customized and connected by synchronization lines to achieve real-time synchronization during the data acquisition process. Achieving time synchronization between multiple unrelated devices using a simple and efficient method is difficult.

[0011] Therefore, at least one embodiment provides a method for time-synchronizing video information and bus message information. This method includes the following: Define the video information's timestamp as the base timestamp. When recording is initiated, the timestamp T0 of the first frame of the video information is obtained, the timestamp of the first frame of the bus message is set to match the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after the match is recorded. When recording is stopped, the timestamp T1 of the video information tail frame and the timestamp T2 of the bus message tail frame are obtained. We derive the proportionality constant k = (T1 - T0) / (T2 - T0). All recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx and saved, where Tx = T0 + (Ty - T0) * k, so that the timestamp Tx of each frame of the bus messages is recorded relative to the clock of the video information.

[0012] This method for synchronizing video and bus message information in the automotive ADAS development and verification field allows for simultaneous data collection and eventual data synchronization using unrelated video and bus data acquisition devices, without requiring device customization. This solves the timestamp problem in synchronized video and bus data acquisition from multiple devices and improves collection efficiency.

[0013] Various non-limiting embodiments of the present invention will be described in detail below with reference to the drawings. As shown in Figure 1, several embodiments provide a method for time-synchronizing video information and bus message information. This method includes the following: In S101, the timestamp of the video information is defined as the base timestamp. In S102, when recording is started, the timestamp T0 of the first frame of the video information is obtained, the timestamp of the first frame of the bus message is set to match the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after the match is recorded. In S103, when recording is stopped, the timestamp T1 of the video information tail frame and the timestamp T2 of the bus message tail frame are obtained. In S104, we derive the proportionality constant k = (T1-T0) / (T2-T0). In S105, all recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx and saved. Here, Tx = T0 + (Ty - T0) * k. As a result, the timestamp Tx of each frame of the bus messages is recorded relative to the clock of the video information.

[0014] The following is a detailed explanation of one example of how to synchronize video information and bus message information over time. In automotive development, video information is recorded using one camera V1, and power CAN message information is recorded using one bus adapter PCAN.

[0015] After simultaneously starting up camera V1 and bus adapter PCAN, the timestamp of the video frame sent from camera V1 in the next moment (for example, 2023-08-13 09:01:02.123, i.e., August 13, 2023, 9:01:02.123 milliseconds) is defined as the timestamp T0 of the first frame. The timestamps of bus message information after the timestamp T0 of the first frame are all set relative to the timestamp T0 of the first frame. That is, the timestamp Ty of the bus message information at the timestamp T0 of the first frame is the timestamp T0 of the first frame, and the timestamp Ty of the bus message information 1 second after the timestamp T0 of the first frame is 2023-08-13 09:01:03.123, i.e., August 13, 2023, 9:01:03.123 milliseconds.

[0016] Assuming that the recording stops 60 minutes after synchronous collection, when the recording stops, the time stamp T1 of the tail frame of the video information is 2023-08-13 10:01:02.123, that is, 10:01:02.123 milliseconds on August 13, 2023. The time stamp T2 of the tail frame of the bus adapter PCAN bus message information collected at this time is 3590 seconds. Therefore, the proportionality coefficient k = (T1 - T0) / (T2 - T0) = 3600 / 3590 = 1.002785515320334.

[0017] Traverse all the recorded bus messages and modify the time stamp Ty of each frame to Tx. Here, Tx = T0 + (Ty - T0)*k. Save the bus message data file, thereby realizing the time synchronization of the video information and the bus message information.

[0018] In some embodiments, aligning the time stamp of the head frame of the bus message with the time stamp T0 of the head frame of the video information, and recording the time stamp Ty of each frame of the bus message after alignment (step) includes the following. Define the time stamp of the head frame of the bus message as T3.

[0018] If the time stamp of any bus message in the recording process is defined as T, then the time stamp Ty of each frame of the bus message after alignment is Ty = T - T3 + T0.

[0019] The following details a method of aligning the time stamp of the head frame of the bus message with the time stamp T0 of the head frame of the video information and recording the time stamp Ty of each frame of the bus message hardware after alignment in one example. In the development of ADAS for automobiles, video information is recorded using one camera V1, and power CAN message information is recorded using one bus adapter PCAN. After simultaneously starting up camera V1 and bus adapter PCAN, the timestamp of the video frame sent from camera V1 in the next moment (for example, 2023-08-13 09:01:02.123, i.e., August 13, 2023, 9:01:02.123 milliseconds) is defined as the timestamp T0 of the first frame. The timestamp of the first frame of the bus message is defined as T3, following the timestamp T0 of the first frame. For example, T3 = 1,234,567 microseconds. The timestamp of any bus message after the timestamp T3 of the first frame is T, for example T=2,234,567 microseconds, and based on the method Ty=T-T3+T0(T-T3=1,000,000 microseconds, i.e., 1 second), Ty=2023-08-13 09:01:03.123, i.e., August 13, 2023, 9:01:03.123 milliseconds.

[0020] In some embodiments, video information is transmitted to a computer device in real time. Bus messages are also transmitted to the computer device via a bus adapter. Specifically, after synchronizing the collection times of the video information and bus message information as described above, the computer can collect the video information and bus messages in a synchronized manner.

[0021] In some embodiments, the transmission frame rate of bus messages is 1000 frames / second or higher. Specifically, the transmission frame rate is 1000 frames / second or higher, meaning the transmission interval of bus messages is less than 1 millisecond. This means that the time difference between the tail frame of the bus message and the tail frame of the video information is less than 1 millisecond, thereby guaranteeing the accuracy of synchronized collection of video information and bus message data.

[0022] As shown in Figure 2, some embodiments further provide a bus adapter. This bus adapter is configured to transmit bus messages in the time-synchronized system of video information and bus message information to a computer device and includes an MCU module and a communication module. The MCU module is configured to send and receive bus messages, and if the frame rate is lower than 1000 frames / second, it generates a frame rate compensation message and uses it as a bus message. The communication module is configured to connect to the MCU module and, when connected to the bus adapter, transmits bus messages to the computer device.

[0023] As shown in Figure 3, some embodiments further provide a time synchronization system for video information and bus message information. This system includes at least one bus adapter, at least one video information collector, and at least one computer device. At least one bus adapter is configured to send and receive bus messages. At least one video information collector is, for example, a camera, which is configured to collect video information. At least one computer device includes a processor, a display that communicates with the processor to display a graphics interface, a computer-readable storage medium, a communication bus, and a communication interface. The video information collector communicates with the processor via a communication interface. The above-mentioned processor, the above-mentioned computer-readable storage medium, and the above-mentioned communication interface communicate with the bus adapter via the above-mentioned communication bus. The above computer-readable storage medium stores command programs. The above processor is configured to execute the above command after acquiring bus messages and video information, and then perform the following operations: Define the video information timestamp as the base timestamp. When recording is started, acquire the timestamp T0 of the first frame of the video information, and synchronize the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after synchronization. When recording is stopped, acquire the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1-T0) / (T2-T0). Traversal all recorded bus messages and correct the timestamp Ty of each frame to Tx and save it. Here, Tx = T0 + (Ty-T0)*k. As a result, the timestamp Tx of each frame of the bus message is recorded relative to the clock of the video information. The above display shows time-synchronized bus messages and video information via a graphic interface.

[0024] As shown in Figure 4, some embodiments further provide a time synchronization system for video information and bus message information. This system includes a computer device and at least one of the bus adapters described above. The computer device is configured to include a definition module, an acquisition / setting module, and a traversal module. The definition module defines the timestamp of the video information as the base timestamp. The acquisition and configuration module is configured to acquire the timestamp T0 of the first frame of the video information when recording is started, match the timestamp T0 of the first frame of the bus message from at least one bus adapter with the timestamp T0 of the first frame of the video information, and record the timestamp Tyn of each frame of the bus message after the match. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2n of the tail frame of the bus message are acquired. The proportionality coefficient kn = (T1-T0) / (T2n-T0) is derived, where n is the bus adapter number. The traversal module traverses all recorded bus messages and saves the timestamp Tyn of each frame, correcting it to Txn. Here, Txn = T0 + (Tyn - T0) * kn. This ensures that the timestamp Txn of each frame of the bus messages is recorded relative to the clock of the video information. Here, the computer commands corresponding to the specific implementation functions of the definition module, acquisition / setting module, and traversal module are stored on a computer-readable storage medium and implemented in the computer device, and specifically refer to the content of the time synchronization method for the video information and bus message information mentioned above, so a detailed explanation is omitted here.

[0025] The following describes the electronic device relating to the embodiments of this disclosure from the perspective of hardware processing. The embodiments described herein do not limit the specific realization of electronic devices.

[0026] As shown in Figure 5, some embodiments further provide an apparatus. This apparatus includes at least one computer-readable storage medium and at least one processor. At least one computer-readable storage medium for storing commands. At least one processor executes the above command to perform the following operations: Define the video information timestamp as the base timestamp. When recording is started, obtain the timestamp T0 of the first frame of the video information, align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment. When recording is stopped, obtain the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traversal all recorded bus messages and save the timestamp Ty of each frame, correcting it to Tx. Here, Tx = T0 + (Ty - T0) * k. As a result, the timestamp Tx of each frame of the bus message is recorded relative to the clock of the video information. Specifically, the above device further includes a communication bus and a communication interface. The processor, the computer-readable storage medium, and the communication interface communicate with each other via the communication bus.

[0027] As shown in Figure 6, some embodiments further provide electronic equipment. This electronic equipment includes a processor, a display that communicates with the processor and displays a graphics interface, a computer-readable storage medium, a communication bus, and a communication interface. The readable storage medium stores command programs. The above processor is configured to perform the following operations by executing the above command: Define the video information timestamp as the base timestamp. When recording is started, obtain the timestamp T0 of the first frame of the video information, and align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment. When recording is stopped, obtain the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1-T0) / (T2-T0). Traversal all recorded bus messages and correct the timestamp Ty of each frame to Tx and save it. Here, Tx = T0 + (Ty-T0)*k. As a result, the timestamp Tx of each frame of the bus message is recorded relative to the clock of the video information. The above display shows time-synchronized bus messages and video information via a graphic interface.

[0028] In some embodiments, computer devices, industrial computers, and other equipment can also be considered types of electronic devices. Note that the configurations shown in Figures 5 and 6 are not limited to electronic devices and may include fewer or more components than those shown, and may be combinations of several components or arrangements of different components.

[0029] In some embodiments, the communication interface may be a communication interface that can be connected to an external bus adapter, such as RS232, RS485, USB port, and TYPE port. A wired or wireless network interface may also be included, and the network interface may optionally include wired and / or wireless interfaces (e.g., Wi-Fi interface, Bluetooth interface, etc.) that are typically used to establish a communication connection between the computer device and other electronic devices.

[0030] A readable storage medium or computer-readable storage medium includes at least one type of memory. Memory includes flash memory, hard disks, multimedia cards, card-type memory (e.g., SD memory), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, it may be an internal storage unit of a computer device, such as a hard disk. In other embodiments, memory may be an external storage device of a computer device, such as a plug-in hard disk, Smart Media Card (SMC) (registered trademark), Secure Digital (SD), or Flash Card. Furthermore, memory may include both internal storage units and external storage devices of a computer device. Memory is used to store various data, such as application software and computer program code installed on the computer device, as well as to temporarily store output data or data to be output.

[0031] In some embodiments, the processor may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip that executes program code stored in a computer-readable storage medium or processes data, for example, to run a computer program.

[0032] In some embodiments, the communication bus may be an input / output bus, such as a Peripheral Component Interconnect (PCI) bus or an Enhanced Industry Standard Architecture (EISA) bus. This bus can be divided into an address bus, a data bus, a control bus, etc.

[0033] Optionally, the computer device may further include a user interface. The user interface may include input units such as a display and a keyboard, and optionally, the user interface may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. In this case, the display is also called a display screen or display unit for displaying information processed within the computer device and for displaying a visualized user interface.

[0034] When the above processor executes the above program, it implements the steps in the embodiment of the time synchronization method for video information and bus message information shown in Figure 1. For example, steps S101 to S105 shown in Figure 1. Alternatively, when the processor executes the computer program, it implements the functions of each module or unit in the embodiment of each of the above devices.

[0035] Some embodiments further provide a computer-readable storage medium that stores any of the above possible methods for time-synchronizing video information and bus message information.

[0036] Some embodiments further provide a computer-readable storage medium. This storage medium stores computer-readable commands, which, when executed by at least one processor, cause the aforementioned time synchronization method for video information and bus message information to be performed. Specifically, the timestamp of the video information is defined as the base timestamp. When recording is started, the timestamp T0 of the first frame of the video information is obtained, the timestamp of the first frame of the bus message is synchronized with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after synchronization is recorded. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are obtained. The proportionality coefficient k = (T1-T0) / (T2-T0) is derived. All recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx and saved. Here, Tx = T0 + (Ty-T0)*k. As a result, the timestamp Tx of each frame of the bus message is recorded relative to the clock of the video information. A detailed explanation of the time synchronization method for video information and bus message information can be found elsewhere, so the explanation is omitted here.

[0037] Some embodiments further provide a computer program product. This computer program product includes a computer program or command. When the computer executes the above computer program or command, the computer is instructed to perform one of the above possible methods for time synchronization of video information and bus message information.

[0038] Some embodiments further provide a computer program product. This computer program product includes a computer-readable storage medium in which computer-readable program code is stored, the computer-readable program code includes commands, these commands cause at least one processor or at least one computer device to perform the following operations. Define the video information's timestamp as the base timestamp. When recording is initiated, the timestamp T0 of the first frame of the video information is obtained, the timestamp of the first frame of the bus message is set to match the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after the match is recorded. When recording is stopped, the timestamp T1 of the video information tail frame and the timestamp T2 of the bus message tail frame are obtained. We derive the proportionality constant k = (T1 - T0) / (T2 - T0). All recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx and saved. Here, Tx = T0 + (Ty - T0) * k. As a result, the timestamp Tx of each frame of the bus messages is recorded relative to the clock of the video information. Specifically, please refer to the detailed explanation of how video information and bus message information are synchronized in time; the explanation is omitted here.

[0039] As shown in Figure 7, some embodiments further provide a system. This system includes a computer device, a bus adapter, and a video information collector. The above-mentioned computer device includes a processor, a computer-readable storage medium, a communication bus, and a communication interface. The video information collector mentioned above communicates with the processor via a communication interface. The above-mentioned processor, the above-mentioned computer-readable storage medium, and the above-mentioned communication interface communicate with the bus adapter via the above-mentioned communication bus. The above-mentioned readable storage medium stores the command program. The above processor is configured to execute the above command after acquiring bus messages and video information, and then perform the following operations: Define the video information timestamp as the base timestamp. When recording is started, acquire the timestamp T0 of the first frame of the video information, and synchronize the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after synchronization. When recording is stopped, acquire the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1-T0) / (T2-T0). Traversal all recorded bus messages and correct the timestamp Ty of each frame to Tx and save it. Here, Tx = T0 + (Ty-T0)*k. As a result, the timestamp Tx of each frame of the bus message is recorded relative to the clock of the video information.

[0040] In some embodiments, the video information collector can communicate with a processor via a communication interface, such as a USB port, and acquire video information using a video acquisition device such as a video acquisition card.

[0041] Some embodiments further provide a method which includes sending and receiving bus messages by a bus adapter and collecting video information by a video information collector. After acquiring bus messages and video information, execute the above command to cause the processor to perform the following operations: Define the video information timestamp as the base timestamp. When recording is started, acquire the timestamp T0 of the first frame of the video information, and synchronize the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after synchronization. When recording is stopped, acquire the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1-T0) / (T2-T0). Traversal all recorded bus messages and correct the timestamp Ty of each frame to Tx and save it. Here, Tx = T0 + (Ty-T0)*k. As a result, the timestamp Tx of each frame of the bus message is recorded relative to the clock of the video information. The graphical interface displays time-synchronized bus messages and video information.

[0042] In some embodiments, the bus adapter may be a CAN (Controller Area Network) bus adapter, a CANFD (Controller Area Network Flexible Data Rate) bus adapter, a FastLIN (Fast Local Interconnect Network) bus adapter, a LIN (Local Interconnect Network) bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter, and may be one-to-many or many-to-many, and the specific implementation of the bus adapter is not limited in some other embodiments.

[0043] In some embodiments, the computer device can communicate via the UDS (Unified Diagnostic Services), XCP (Universal Calibration Protocol), or CCP (CAN Calibration Protocol) protocol to obtain bus messages.

[0044] In some embodiments, bus messages are transmitted by communicating with a debugging device, such as a vehicle ECU (Electronic Control Unit) and its related systems, via the UDS (Unified Diagnostic Services), XCP (Universal Calibration Protocol), or CCP (CAN Calibration Protocol) protocol.

[0045] In several embodiments, this method and system can cover time-synchronized scenes of video information and bus message information in the fields of automotive, aerospace, marine, machinery, and automation. Specifically in the automotive field, this may include vehicle ECUs (Electronic Control Units) and related systems. For example, but not limited to, devices such as electronic power steering systems (EPS), anti-lock braking systems (ABS), electronic stability control systems (ESC), automotive engine management systems, and battery management systems (BMS) can transmit bus messages by connecting to a computer device via a bus.

[0046] In some embodiments, the disclosed apparatus and methods can, of course, be implemented in other ways. The embodiments of the apparatus described above are merely illustrative; for example, the flowcharts and block diagrams in the drawings illustrate the implementable architectures, functions, and operations of apparatus, methods, and computer program products according to some embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code. The module, program segment, or part of code contains executable instructions for implementing one or more predetermined logical functions. In some alternative implementations, the functions represented in the blocks may occur in an order different from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes in reverse order, depending on the related functions. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may be implemented in a dedicated hardware-based system that performs a predetermined function or operation, or in a combination of dedicated hardware and computer instructions.

[0047] Furthermore, in each embodiment of the present invention, each functional module may be integrated to form a single independent part, each module may exist individually, or two or more modules may be integrated to form a single independent part.

[0048] The above functions can be implemented in the form of software function modules and, when sold or used as independent products, can be stored on a single computer-readable storage medium. Based on this understanding, the technical solutions of the present invention, in their essence or contribution to the prior art, or parts thereof, can be represented in the form of a software product. The computer software product is stored on a single storage medium and contains a plurality of instructions that cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0049] By illustrating the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications to the above-described content without departing from the technical spirit of the present invention. The technical scope of the present invention is not limited to the contents of the specification and must be determined based on the claims.

Claims

1. A method for synchronizing video information and bus message information over time, The timestamp T0 of the first frame of the video information, the timestamp T1 of the tail frame of the video information, the timestamp T2 of the tail frame of the bus message, and the timestamp Ty of each frame of the bus message are recorded, and the timestamp of the first frame of the bus message is set to match the timestamp T0 of the first frame of the video information. The steps include deriving the proportionality constant k = (T1-T0) / (T2-T0), A method for synchronizing video information and bus message information performed by a computer, comprising the steps of traversing all recorded bus messages and saving the timestamp Ty of each frame therein, with the format Tx = T0 + (Ty - T0) * k.

2. The timestamp of the first frame of a bus message is defined as T3, and the timestamp of any bus message in the recording process is defined as T. The time synchronization method according to claim 1, wherein the timestamp Ty of each frame of the bus message is recorded as Ty = T - T3 + T0.

3. The video information is transmitted to the computer device in real time. The time synchronization method according to claim 1, characterized in that a bus message is transmitted to the computer device via a bus adapter.

4. The time synchronization method according to claim 1, characterized in that the transmission frame rate of the bus message is 1000 frames / second or more.

5. A computer-readable storage medium, A computer-readable storage medium characterized by storing computer-readable commands and, when executed by at least one processor, causing it to execute a program for a time synchronization method according to any one of claims 1 to 4.

6. It is a device, It includes at least one computer-readable storage medium and at least one processor, The aforementioned computer-readable storage medium stores commands, The apparatus is characterized in that the processor executes the command in order to execute a program for the time synchronization method described in any one of claims 1 to 4.

7. It is an electronic device, Includes a processor, a display that communicates with the processor to display a graphics interface, a computer-readable storage medium, a communication bus and a communication interface, The processor, the computer-readable storage medium, and the communication interface communicate with each other via the communication bus. The aforementioned readable storage medium stores a command program. The processor is configured to execute a program for the time synchronization method described in any one of claims 1 to 4 by executing the command, The electronic device is characterized in that the display shows time-synchronized bus messages and video information via a graphic interface.

8. A time synchronization system for video information and bus message information, It includes at least one bus adapter, at least one video information collector, and at least one computer device, The bus adapter is configured to send and receive bus messages, The aforementioned video information collector is configured to collect video information, The computer device includes a processor, a display that communicates with the processor to display a graphics interface, a computer-readable storage medium, a communication bus, and a communication interface. The aforementioned video information collector communicates with the processor via a communication interface, The processor, the computer-readable storage medium, and the communication interface enable communication with the bus adapter via the communication bus. The aforementioned computer-readable storage medium stores a command program. The processor is configured to execute the command and then run the program for the time synchronization method described in any one of claims 1 to 4, after acquiring bus messages and video information. The aforementioned display is characterized by displaying time-synchronized bus messages and video information via a graphic interface, thereby providing a time-synchronization system for video information and bus message information.

9. The bus adapter is a CAN bus adapter, a CANFD bus adapter, a FastLIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter. The time synchronization system according to claim 8, characterized in that it communicates with a computer device via the UDS, XCP, or CCP protocol to send and receive bus messages to and from a processor in the computer device.

10. The bus adapter includes an MCU module and a communication module. The MCU module is configured to send and receive bus messages, and if the frame rate is lower than 1000 frames / second, it generates a frame rate compensation message and uses it as a bus message. The time synchronization system according to claim 8, characterized in that the communication module is configured to be connected to the MCU module and transmits bus messages to a computer device when connected to a bus adapter.

11. It is a bus adapter, A time-synchronization system for video information and bus message information according to any one of claims 1 to 4, configured to transmit bus messages to a computer device, comprising an MCU module and a communication module, The MCU module is configured to send and receive bus messages, and if the frame rate is lower than 1000 frames / second, it generates a frame rate compensation message and uses it as a bus message. A bus adapter characterized in that the communication module is configured to connect to an MCU module and transmits bus messages to a computer device when connected to the bus adapter.

12. The bus adapter is a CAN bus adapter, a CANFD bus adapter, a FastLIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter. The bus adapter according to claim 11, characterized in that it communicates with a computer device via the UDS, XCP, or CCP protocol to send and receive bus messages to and from a processor in the computer device.

13. It is a system, Includes computer equipment, bus adapter, and video information collector, The computer device includes a processor, a computer-readable storage medium, a communication bus, and a communication interface. The aforementioned video information collector communicates with the processor via a communication interface, The processor, the computer-readable storage medium, and the communication interface enable communication with the bus adapter via the communication bus. The aforementioned readable storage medium stores a command program. The system is characterized in that the processor is configured to execute the command and then run a program for the time synchronization method described in any one of claims 1 to 4, after acquiring bus messages and video information.

14. The bus adapter is a CAN bus adapter, a CANFD bus adapter, a FastLIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter. The system according to claim 13, characterized in that it communicates with a computer device via the UDS, XCP, or CCP protocol to send and receive bus messages to and from a processor in the computer device.

15. The bus adapter includes an MCU module and a communication module. The MCU module is configured to send and receive bus messages, and if the frame rate is lower than 1000 frames / second, it generates a frame rate compensation message and uses it as a bus message. The system according to claim 13, characterized in that the communication module is configured to be connected to the MCU module and transmits bus messages to a computer device when connected to a bus adapter.

Images