Time synchronization method of video information and bus message information and electronic apparatus

Abstract

To provide an invention that relates to a technical field of a software development for a vehicle, and more particularly provide a method and system for time synchronization of video information and bus message information.SOLUTION: A time stamp T0 of a head frame of video information, a time stamp T1 of a tail frame of the video information, a time stamp T2 of the tail frame of a bus message, and a time stamp Ty of each frame of the bus message are recorded, and the time stamp of the head frame of the bus message is matched with the time stamp T0 of the head frame of the video information. A proportionality coefficient k=(T1-T0) / (T2-T0) is derived. All recorded bus messages are traversed and the time stamp Ty of each frame is modified to Tx=T0+(Ty-T0)*k and stored, so that the time stamp Tx of each frame of the bus message is recorded with reference to a clock of the video information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 2023118632749, filed on December 29, 2023, and U.S. Patent Application No. 18 / 516,995, filed on November 22, 2023, the entire contents of which are hereby incorporated by reference.

[0002] The present invention relates to the field of vehicle software development, and more particularly to a method and system for time synchronization of video information and bus message information. [Background technology]

[0003] In the field of automotive ADAS (Advanced Driving Assistance System) development and verification, it is often necessary to collect data from multiple devices that are not always related to each other. The timestamps between each device may be different, for example, the timestamp of a bus data collection device is from itself, and the timestamp of a video collection device is from a computer, and the error between the timestamp of the bus data collection device and the timestamp of the computer is not fixed, and the timestamp becomes larger or smaller. After accumulating to a certain extent, the error with the timestamp of the computer becomes very large, so that the data collected by each device cannot be synchronized in time. Summary of the Invention

[0004] The present invention relates to a method for time synchronization of video information and bus message information, the method including: The timestamp of the video information is defined as the base timestamp. When recording is started, the timestamp T0 of the first frame of the video information is acquired, the timestamp of the first frame of the bus message is matched with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after the match is recorded. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are acquired. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). All recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx=T0+(Ty-T0)*k and saved, so that the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0005] Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The objectives and other advantages of the invention will be realized and obtained by the structure particularly pointed out in the description and drawings.

[0006] In order to make the above objects, features and advantages of the present invention more comprehensible, the following preferred embodiments will be described in detail in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0007] In order to more clearly describe the specific embodiments of the present invention or the technical solutions of the prior art, the following will briefly describe the drawings that need to be used to describe the specific embodiments or the prior art. The drawings described in the following description are some embodiments of the present invention, and it is obvious that those skilled in the art can obtain other drawings from these drawings without any creative efforts. [Figure 1] 1 is a flowchart of a method for time synchronizing video information and bus message information according to some embodiments. [Figure 2] FIG. 1 is a principle block diagram of a bus adapter according to some embodiments. [Figure 3] FIG. 1 is a principle block diagram of a system for implementing an automotive bus virtual channel according to some embodiments. [Figure 4] FIG. 1 is a principle block diagram of a system for implementing an automotive bus virtual channel according to some embodiments. [Figure 5] FIG. 1 is a principle block diagram of an electronic device according to some embodiments. [Figure 6] FIG. 1 is a principle block diagram of an electronic device according to some embodiments. [Figure 7] FIG. 1 is a principle block diagram of a system according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0008] In order to clarify the objectives, technical aspects and advantages of the embodiments of the present invention, the technical aspects of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. However, it is clear that the described embodiments are only some of the embodiments of the present invention and do not represent all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without performing creative work fall within the scope of protection of the invention.

[0009] In the field of automotive ADAS development and verification, it is often necessary to collect data from multiple devices that are not related to each other. The timestamps of each device tend to be different. For example, the timestamp of a bus data collection device is from itself, while the timestamp of a video collection device is from a computer. The error between the timestamp of the bus data collection device and the timestamp of the computer is not fixed; the timestamps become larger or smaller. After accumulating to a certain extent, the error with the computer timestamp becomes too large, causing the data collected by each device to be unable to be synchronized in time.

[0010] In related art, one independent device is customized to simultaneously perform both vehicle video information collection and bus data collection functions, or multiple independent devices are customized and connected by a synchronization line to achieve real-time synchronization in the data collection process. However, it is difficult to achieve time synchronization between multiple unrelated devices in a simple and efficient manner.

[0011] Accordingly, at least one embodiment provides a method for time synchronizing video information and bus message information, the method including: The timestamp of the video information is defined as the base timestamp. When recording is started, the timestamp T0 of the first frame of the video information is acquired, and the timestamp of the first frame of the bus message is matched with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after the match is recorded. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are acquired. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). All recorded bus messages are traversed, and the timestamp Ty of each frame is modified to Tx and saved, where Tx = T0 + (Ty - T0) * k, so that the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0012] This method for time synchronizing video information and bus message information is applicable to the field of automotive ADAS development and verification. When there is no need to customize devices, unrelated video and bus collection devices can be used to simultaneously collect data and ultimately synchronize the data, thereby solving the time stamp problem of synchronously collecting video and bus data from multiple devices and improving collection efficiency.

[0013] Various non-limiting embodiments of examples of the present invention will now be described in detail with reference to the drawings. As shown in Figure 1, some embodiments provide a method for time synchronization of video information and bus message information. The method includes: In S101, the timestamp of the video information is defined as a base timestamp. In S102, when recording is started, the timestamp T0 of the first frame of the video information is obtained, the timestamp of the first frame of the bus message is aligned with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after alignment is recorded. In S103, when recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are acquired. In S104, the proportionality coefficient k=(T1-T0) / (T2-T0) is derived. In S105, all recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx and saved, where Tx = T0 + (Ty - T0) * k. This allows the timestamp Tx of each frame of the bus message to be recorded based on the clock of the video information.

[0014] An example of a method for time synchronizing video information and bus message information will now be described in detail. In the development of an automobile, one camera V1 is used to record video information, and one bus adapter PCAN is used to record message information of the power CAN.

[0015] After the camera V1 and the bus adapter PCAN are started simultaneously, the timestamp of the video frame sent from the camera V1 the next time (for example, 2023-08-13 09:01:02.123, i.e., 9:01:02:123 ms on August 13, 2023) is defined as the timestamp T0 of the first frame. All timestamps of bus message information after the timestamp T0 of the first frame are set based on the timestamp T0 of the first frame. In other words, the timestamp Ty of the bus message information at the timestamp T0 of the first frame is the timestamp T0 of the first frame, and the timestamp Ty of the bus message information one second after the timestamp T0 of the first frame is 2023-08-13 09:01:03.123, i.e., 9:01:03:123 ms on August 13, 2023.

[0016] Assuming that recording is stopped after 60 minutes of synchronous collection, when recording is stopped, the timestamp T1 of the tail frame of the video information is 2023-08-13 10:01:02.123, or 10:01:02:123 milliseconds on August 13, 2023. The timestamp T2 of the tail frame of the bus adapter PCAN bus message information collected at this time is 3590 seconds. Therefore, the proportionality coefficient k = (T1 - T0) / (T2 - T0) = 3600 / 3590 = 1.002785515320334.

[0017] Traversing all recorded bus messages, modifying the timestamp Ty of each frame to Tx, where Tx = T0 + (Ty - T0) * k, and saving the bus message data file, thereby realizing time synchronization between the video information and the bus message information.

[0018] In some embodiments, aligning the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and recording the timestamp Ty of each frame of the bus message after alignment (step) includes: The timestamp of the first frame of the bus message is defined as T3. If the timestamp of any bus message in the recording process is defined as T, then the timestamp of each frame of the combined bus message is Ty=T-T3+T0.

[0019] The following is an example to explain in detail how to align the timestamp of the first frame of a bus message with the timestamp T0 of the first frame of video information, and record the hardware timestamp Ty of each frame of the bus message after alignment. In the development of automotive ADAS, one camera V1 is used to record video information, and one bus adapter PCAN is used to record message information of the power CAN. After simultaneously starting camera V1 and bus adapter PCAN, the timestamp of the video frame sent from camera V1 at the next instant (e.g., 2023-08-13 09:01:02.123, i.e., 9:01:02:123 ms on August 13, 2023) is defined as the timestamp T0 of the first frame. The timestamp of the first frame of the bus message after the timestamp T0 of the first frame is defined as T3, for example, T3 = 1234567 microseconds. The timestamp of any bus message after the timestamp T3 of the first frame is T, for example T = 2234567 microseconds, and based on the matching method Ty = T - T3 + T0 (T - T3 = 1000000 microseconds, or 1 second), Ty = 2023-08-13 09:01:03.123, or 9:01:03, 123 milliseconds on August 13, 2023.

[0020] In some embodiments, video information is transmitted in real time to a computer device, and bus messages are transmitted to the computer device via a bus adapter. Specifically, after synchronizing the collection times of the video information and bus message information, the computer can synchronously collect the video information and bus messages.

[0021] In some embodiments, the transmission frame rate of the bus messages is 1000 frames per second or greater. Specifically, the transmission frame rate is 1000 frames per second or more, meaning that the bus message transmission interval is less than 1 millisecond, meaning that the time error between the tail frame of the bus message and the tail frame of the video information is less than 1 millisecond, thereby ensuring the accuracy of the synchronous collection of video information and bus message data.

[0022] 2, some embodiments further provide a bus adapter, which is configured to transmit bus messages in the time synchronization system of the video information and bus message information to a computer device, and includes an MCU module and a communication module. The MCU module is configured to transmit and receive bus messages, and generates a frame rate compensation message as the bus message when the frame rate is lower than 1000 frames / second. The communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

[0023] Some embodiments further provide a system for time synchronization of video information and bus message information, as shown in Figure 3. The system includes at least one bus adapter, at least one video information collector, and at least one computer device. At least one bus adapter is configured to send and receive bus messages. The at least one video information collector, such as a camera, is configured to collect video information. At least one computing device includes a processor, a display in communication with the processor for displaying a graphical interface, a computer-readable storage medium, a communication bus, and a communication interface. The video information collector communicates with the processor via a communication interface. The processor, the computer-readable storage medium, and the communication interface realize communication with a bus adapter via the communication bus. The computer-readable storage medium stores a command program. After acquiring the bus message and video information, the processor is configured to execute the command and perform the following operations: Define the timestamp of the video information as a base timestamp; When recording is started, acquire the timestamp T0 of the first frame of the video information, align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after the alignment; When recording is stopped, acquire the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message; Derive a proportionality coefficient k = (T1 - T0) / (T2 - T0); Traversal all recorded bus messages, and correct the timestamp Ty of each frame to Tx, where Tx = T0 + (Ty - T0) * k; As a result, the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information. The display shows time-synchronous bus messages and video information via a graphic interface.

[0024] 4, some embodiments further provide a system for time synchronization of video information and bus message information, the system including a computer device and at least one of the bus adapters, the computer device configured to include a definition module, an acquisition and setting module, and a traversal module. The definition module defines the timestamp of the video information as the basic timestamp. The acquisition and setting module is configured to acquire a timestamp T0 of the first frame of video information when recording is started, align the timestamp of the first frame of a bus message from at least one bus adapter with the timestamp T0 of the first frame of the video information, and record the timestamp Tyn of each frame of the bus message after alignment. When recording is stopped, the acquisition and setting module acquires a timestamp T1 of the tail frame of the video information and a timestamp T2n of the tail frame of the bus message. The proportionality coefficient kn = (T1 - T0) / (T2n - T0) is derived, where n is the bus adapter number. The traversal module traverses all recorded bus messages and modifies the timestamp Tyn of each frame to Txn, where Txn = T0 + (Tyn - T0) * kn, so that the timestamp Txn of each frame of the bus message is recorded based on the clock of the video information. Here, the computer commands corresponding to the specific implementation functions of the definition module, acquisition setting module and traversal module are stored in a computer-readable storage medium and implemented in a computer device. Specifically, reference can be made to the content of the time synchronization method for the above-mentioned video information and bus message information, and therefore the description thereof will be omitted here.

[0025] Hereinafter, an electronic device according to an embodiment of the present disclosure will be described from the viewpoint of hardware processing. The embodiments of the present disclosure are not limited to specific implementations of electronic devices.

[0026] Some embodiments further provide an apparatus, as shown in Figure 5. The apparatus includes at least one computer-readable storage medium and at least one processor. At least one computer-readable storage medium storing commands. At least one processor executes the above commands to perform the following operations: Define the timestamp of the video information as a base timestamp; When recording is started, obtain the timestamp T0 of the first frame of the video information, and align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment; When recording is stopped, obtain the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message; Derive a proportionality coefficient k = (T1 - T0) / (T2 - T0); Traversal all recorded bus messages, and correct the timestamp Ty of each frame to Tx, and save it, where Tx = T0 + (Ty - T0) * k; As a result, the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information. Specifically, the device further includes a communication bus and a communication interface, and the processor, the computer-readable storage medium, and the communication interface communicate with each other via the communication bus.

[0027] 6, some embodiments further provide an electronic device, which includes a processor, a display communicating with the processor and configured to display a graphic interface, a computer-readable storage medium, a communication bus, and a communication interface, wherein the computer-readable storage medium stores a command program. The processor is configured to execute the following operations by executing the above commands: Define the timestamp of the video information as a base timestamp; When recording is started, obtain the timestamp T0 of the first frame of the video information, and align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment; When recording is stopped, obtain the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message; Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0); Traversal all recorded bus messages, and correct the timestamp Ty of each frame to Tx, where Tx = T0 + (Ty - T0) * k. As a result, the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information. The display shows time-synchronous bus messages and video information via a graphic interface.

[0028] In some embodiments, computer equipment, industrial computers, and devices may also be types of electronic equipment. It should be noted that the configurations shown in Figures 5 and 6 are not intended to limit the electronic device, which may include fewer or more components than those shown, may combine some components, or may have different component arrangements.

[0029] In some embodiments, the communication interface may be a communication interface connectable to an external bus adapter, such as RS232, RS485, a USB port, a TYPE port, etc. A wired or wireless network interface may also be included, and the network interface may optionally include a wired and / or wireless interface (e.g., a WI-FI interface, a Bluetooth interface, etc.) typically used to establish a communication connection between the computing device and other electronic devices.

[0030] A readable storage medium or computer-readable storage medium includes at least one type of memory. Memory may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, it may be an internal storage unit of a computer device, such as a hard disk of the computer device. In other embodiments, memory may be an external storage device of a computer device, such as a plug-in hard disk installed in the computer device, a SmartMedia Card (SMC)®, a Secure Digital Card (SD), a Flash Card, etc. Furthermore, memory may include both an internal storage unit of a computer device and an external storage device. Memory is used to store various data, such as application software and computer program code installed in the computer device, as well as to temporarily store output data or data to be output.

[0031] In some embodiments, the processor executes program code stored on a computer-readable storage medium or processes data, and may be, for example, a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip for executing a computer program.

[0032] In some embodiments, the communication bus may be an input / output bus, which may be a Peripheral Component Interconnect (PCI) bus or an Enhanced Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0033] Optionally, the computer device may further include a user interface. The user interface may include input units such as a display and a keyboard, and optionally, the user interface may also include a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. In this case, the display is also called a display screen or a display unit, since it displays information processed in the computer device and a visualized user interface.

[0034] When the processor executes the program, it implements the steps in the embodiment of the method for time synchronization of video information and bus message information shown in Fig. 1. For example, steps S101 to S105 shown in Fig. 1. Or, when the processor executes the computer program, it implements the functions of each module or unit in each of the embodiments of the device.

[0035] Some embodiments further provide a computer-readable storage medium, which stores any of the above possible methods for time synchronization of video information and bus message information.

[0036] Some embodiments further provide a computer-readable storage medium. The storage medium stores computer-readable commands that, when executed by at least one processor, cause the above-described method for time synchronization of video information and bus message information to be performed. Specifically, the timestamp of the video information is defined as a base timestamp. When recording is started, the timestamp T0 of the first frame of the video information is obtained, and the timestamp of the first frame of the bus message is aligned with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after alignment is recorded. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are obtained. A proportionality coefficient k = (T1 - T0) / (T2 - T0) is derived. All recorded bus messages are traversed, and the timestamp Ty of each frame is corrected to Tx and saved, where Tx = T0 + (Ty - T0) * k. As a result, the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information. A detailed description of the method for time synchronization of video information and bus message information is omitted here.

[0037] Some embodiments further provide a computer program product, which includes computer programs or commands that, when executed on a computer, cause the computer to perform any of the possible methods for time synchronization of video information and bus message information described above.

[0038] Some embodiments further provide a computer program product, which includes a computer-readable storage medium having computer-readable program code stored thereon, the computer-readable program code including instructions that cause at least one processor or at least one computing device to perform the following operations: The timestamp of the video information is defined as the base timestamp. When recording is started, the timestamp T0 of the first frame of the video information is acquired, and the timestamp of the first frame of the bus message is matched with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after the match is recorded. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are acquired. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). All recorded bus messages are traversed and the timestamp Ty of each frame is corrected to Tx and saved, where Tx = T0 + (Ty - T0) * k. This ensures that the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information. Specifically, please refer to the detailed description of the time synchronization method between the video information and the bus message information, and the description will be omitted here.

[0039] Some embodiments further provide a system, as shown in Figure 7. The system includes a computer device, a bus adapter, and a video information collector. The computing device includes a processor, a computer-readable storage medium, a communication bus, and a communication interface. The video information collector communicates with the processor via a communication interface. The processor, the computer-readable storage medium, and the communication interface realize communication with a bus adapter via the communication bus. The readable storage medium stores a command program. After acquiring the bus message and video information, the processor is configured to execute the command and perform the following operations: Define the timestamp of the video information as a base timestamp; When recording is started, acquire the timestamp T0 of the first frame of the video information, align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after the alignment; When recording is stopped, acquire the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message; Derive a proportionality coefficient k = (T1 - T0) / (T2 - T0); Traversal all recorded bus messages, and correct the timestamp Ty of each frame to Tx, where Tx = T0 + (Ty - T0) * k; As a result, the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0040] In some embodiments, the video information collector communicates with the processor via a communications interface, such as a USB port, and may obtain video information in the form of a video acquisition device, such as a video acquisition card.

[0041] Some embodiments further provide a method, the method including transmitting and receiving bus messages by a bus adapter and collecting video information by a video information collector. After acquiring the bus message and video information, the above command is executed to cause the processor to perform the following operations: Define the timestamp of the video information as the base timestamp; When recording is started, acquire the timestamp T0 of the first frame of the video information, and align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment; When recording is stopped, acquire the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message; Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0); Traversal is performed through all recorded bus messages, and the timestamp Ty of each frame is corrected to Tx and saved, where Tx = T0 + (Ty - T0) * k; As a result, the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information. A graphic interface displays time-synchronous bus messages and video information.

[0042] In some embodiments, the bus adapter may be a CAN (Controller Area Network) bus adapter, a CANFD (Controller Area Network Flexible Data Rate) bus adapter, a FastLIN (Fast Local Interconnect Network) bus adapter, a LIN (Local Interconnect Network) bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter, and may be one-to-many or many-to-many, and in some other embodiments, the specific implementation of the bus adapter is not limited.

[0043] In some embodiments, the computing device may communicate via Unified Diagnostic Services (UDS) or Universal Calibration Protocol (XCP) or CAN Calibration Protocol (CCP) protocols to obtain bus messages.

[0044] In some embodiments, the debugging device communicates with a vehicle's Electronic Control Unit (ECU) and its associated systems via the Unified Diagnostic Services (UDS) or Universal Calibration Protocol (XCP) or CAN Calibration Protocol (CCP) protocols to transmit bus messages.

[0045] In some embodiments, the method and system can cover time-synchronized scenes of video information and bus message information in the automotive, aerospace, marine, machinery, and automation fields. For the automotive field, specific examples include vehicle ECUs (Electronic Control Units) and related systems, such as, but not limited to, electronic power steering systems (EPS), anti-lock braking systems (ABS), electronic stability systems (ESC), automotive engine management systems, and battery management systems (BMS), which can be connected to a computer device via a bus to achieve bus message transmission.

[0046] It should be understood that in some embodiments, the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely exemplary. For example, the flowcharts and block diagrams in the figures illustrate possible architectures, functions, and operations of apparatuses, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, a program segment, or a portion of code. The module, program segment, or portion of code includes executable instructions for implementing one or more predetermined logical functions. Note that in some alternative implementations, the functions depicted in the blocks may occur in a different order than depicted in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, or may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated hardware-based system that performs the predetermined functions or operations, or in a combination of dedicated hardware and computer instructions.

[0047] Furthermore, each functional module in each embodiment of the present invention may be integrated together to form a single independent part, each module may exist independently, or two or more modules may be integrated to form a single independent part.

[0048] The above functions can be realized in the form of software functional modules and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention essentially or a part that contributes to the prior art or a part of the technical solution can be expressed in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention.

[0049] The above-described preferred embodiments of the present invention have been enlightened, and those skilled in the art can make various changes and modifications based on the above description without departing from the technical spirit of the present invention. The technical scope of the present invention is not limited to the content of the specification, but should be determined based on the claims.

Claims

1. 1. A method for time synchronizing video information and bus message information, comprising: a timestamp T0 of the first frame of the video information, a timestamp T1 of the tail frame of the video information, a timestamp T2 of the tail frame of the bus message, and a timestamp Ty of each frame of the bus message are recorded, and the timestamp of the first frame of the bus message is aligned with the timestamp T0 of the first frame of the video information; deriving a proportionality coefficient k=(T1-T0) / (T2-T0); traversing all recorded bus messages and modifying the timestamp Ty of each frame thereof to Tx=T0+(Ty-T0)*k and storing the modified timestamp Ty.

2. The timestamp of the first frame of the bus message is defined as T3, and the timestamp of any bus message in the recording process is defined as T, 2. The method of claim 1, wherein the timestamp Ty of each frame of the bus message is recorded as Ty=T-T3+T0.

3. Transmitting video information to a computer device in real time; 2. The time synchronization method according to claim 1, further comprising transmitting a bus message to said computer device via a bus adapter.

4. 2. The time synchronization method according to claim 1, wherein the transmission frame rate of the bus message is 1000 frames / second or more.

5. A computer-readable storage medium, comprising: A computer-readable storage medium storing computer-readable commands that, when executed by at least one processor, cause a program for the time synchronization method according to any one of claims 1 to 4 to be executed.

6. 1. An apparatus comprising: at least one computer-readable storage medium and at least one processor; The computer-readable storage medium stores commands; The apparatus, characterized in that the processor executes the command to execute a program of the time synchronization method according to any one of claims 1 to 4.

7. An electronic device, a processor, a display communicating with the processor to display a graphical interface, a computer-readable storage medium, a communication bus, and a communication interface; the processor, the computer-readable storage medium, and the communication interface communicate with each other via the communication bus; the readable storage medium stores a command program; the processor is configured to execute the program for the time synchronization method according to any one of claims 1 to 4 by executing the command; The display displays time-synchronous bus messages and video information via a graphic interface.

8. 1. A system for time synchronizing video information and bus message information, comprising: at least one bus adapter, at least one video information collector, and at least one computer device; the bus adapter is configured to send and receive bus messages; the video information collector is configured to collect video information; the computer device includes a processor, a display communicating with the processor to display a graphical interface, a computer-readable storage medium, a communication bus, and a communication interface; the video information collector communicating with a processor via a communication interface; the processor, the computer-readable storage medium, and the communication interface realize communication with a bus adapter via the communication bus; The computer-readable storage medium stores a command program; the processor is configured to execute the command to execute the program of the time synchronization method according to any one of claims 1 to 4 after acquiring the bus message and the video information; 10. A time-synchronized system for video and bus message information, wherein the display displays time-synchronized bus messages and video information via a graphic interface.

9. the bus adapter is a CAN bus adapter, a CAN FD bus adapter, a FastLIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter; 9. The time synchronization system of claim 8, wherein the system communicates with a computer device via UDS, XCP, or CCP protocols to send and receive bus messages to a processor in the computer device.

10. the bus adapter includes an MCU module and a communication module; The MCU module is configured to perform transmission and reception of bus messages, and when the frame rate is lower than 1000 frames / second, generates a frame rate compensation message as the bus message; 9. The time synchronization system of claim 8, wherein the communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

11. 1. A bus adapter, comprising: A system for transmitting bus messages in a time-synchronized system for video information and bus message information according to any one of claims 1 to 4 to a computer device, the system comprising: an MCU module; and a communication module; The MCU module is configured to perform transmission and reception of bus messages, and when the frame rate is lower than 1000 frames / second, generates a frame rate compensation message as the bus message; A bus adapter, wherein the communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

12. the bus adapter is a CAN bus adapter, a CAN FD bus adapter, a FastLIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter; 12. The bus adapter of claim 11, wherein the bus adapter communicates with a computer device via UDS, XCP, or CCP protocols to send and receive bus messages to a processor in the computer device.

13. 1. A system comprising: a computer device, a bus adapter, and a video information collector; the computing device includes a processor, a computer-readable storage medium, a communication bus, and a communication interface; the video information collector communicating with a processor via a communication interface; the processor, the computer-readable storage medium, and the communication interface realize communication with a bus adapter via the communication bus; the readable storage medium stores a command program; The system is characterized in that the processor is configured to execute the command and execute the program of the time synchronization method described in any one of claims 1 to 4 after obtaining bus messages and video information.

14. the bus adapter is a CAN bus adapter, a CAN FD bus adapter, a FastLIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter; 14. The system of claim 13, wherein the system communicates with a computer device via UDS or XCP or CCP protocols to send and receive bus messages to a processor in the computer device.

15. the bus adapter includes an MCU module and a communication module; The MCU module is configured to perform transmission and reception of bus messages, and when the frame rate is lower than 1000 frames / second, generates a frame rate compensation message as the bus message; 14. The system of claim 13, wherein the communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

Images