Method for time synchronization of video information and bus message information and electronic device

Abstract

To time-synchronize video information and bus message information.SOLUTION: A method according to the present invention includes: defining a time stamp of video information as a base time stamp; when recording is activated, acquiring a time stamp T0 of a head frame of the video information, matching a time stamp of a head frame of a bus message with the time stamp T0 of the head frame of the video information, and recording a time stamp Ty of each frame of the bus message with the matched time stamp; when the recording is stopped, acquiring a time stamp T1 of a tail frame of the video information and a time stamp T2 of a tale frame of the bus message; deriving a proportionality coefficient k=(T1-T0) / (T2-T0); traversing all the recorded bus messages; modifying the time stamp ty of each of the frame to Tx=T0+(Ty-T0)*k to store it; and recording a time stamp Tx of each of the frames of the bus messages using a clock of the video information as reference.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application claims priority to Chinese Patent Application No. 2023118632749 filed on December 29, 2023, and US Patent Application No. 18 / 516,995 filed on November 22, 2023, the entire contents of which are hereby incorporated herein by reference.

[0002] The present invention belongs to the technical field of vehicle software development, and specifically relates to a method and system for time synchronization of video information and bus message information.

Background Art

[0003] In the field of automotive ADAS (Advanced Driving Assistance System) development and verification, it often occurs that data from multiple devices that are not related to each other needs to be collected. The timestamps between each device may be different. For example, the timestamp of the bus data collection device is from itself, the timestamp of the video collection device is from the computer, and the error between the timestamp of the bus data collection device and the timestamp of the computer is not fixed. The timestamp becomes larger or smaller, and when it is accumulated to a certain extent, the error with the computer's timestamp becomes very large, so that the data collected between each device cannot be synchronized in time.

Summary of the Invention

[0004] The present invention relates to a method for time synchronization of video information and bus message information. This method includes the following. Define the timestamp of the video information as the base timestamp. When starting the recording, obtain the timestamp T0 of the first frame of the video information, align the timestamp of the first frame of the bus message with the timestamp T0 of the first frame of the video information, and record the timestamp Ty of each frame of the bus message after alignment. When the recording is stopped, obtain the time stamp T1 of the tail frame of the video information and the time stamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame as Tx = T0 + (Ty - T0) * k. In this way, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0005] Other features and advantages of the present invention are described in the following specification, and some are apparent from the specification or can be understood by implementing the present invention. The objectives and other advantages of the present invention are realized and obtained by the structures specifically pointed out in the specification and drawings.

[0006] To make the above objectives, features, and advantages of the present invention clearer, the following provides preferred embodiments and, in conjunction with the accompanying drawings, will be described in detail.

Brief Description of the Drawings

[0007] To more clearly explain the specific embodiments of the present invention or the technical solutions of the prior art, the following briefly explains the drawings that need to be used in the description of the specific embodiments or the prior art. The drawings described in the following explanations are some embodiments of the present invention, and it is obvious that those skilled in the art can obtain other drawings from these drawings without creative efforts.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0008] To make the objectives, technical aspects, and advantages of the embodiments of the present invention clearer, the technical aspects of the present invention will be clearly and completely described below in connection with the accompanying drawings. It is obvious that the described embodiments are only some embodiments of the present invention and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained on the premise that those skilled in the art do not perform creative labor belong to the scope of protection of the invention.

[0009] In the field of automotive ADAS development and verification, it is always necessary to collect data from a plurality of devices that are not related to each other. The timestamps between each device tend to be different. For example, the timestamp of the bus data collection device is from itself, the timestamp of the video collection device is from the computer, and the error between the timestamp of the bus data collection device and the timestamp of the computer is not fixed. The timestamp becomes larger and larger or smaller and smaller. When accumulated to a certain extent, the error with the timestamp of the computer becomes very large, so that the data collected between each device cannot be synchronized in time.

[0010] In related technologies, an independent device is customized to have both an automotive video information collection function and a bus data collection function at the same time. Or a plurality of independent devices are customized and connected by a synchronization line between them to realize real-time synchronization in the data collection process. It is difficult to realize time synchronization between a plurality of devices that are not related to each other using a simple and efficient method.

[0011] Therefore, at least one embodiment provides a method for time synchronization of video information and bus message information. This method includes the following. Define the time stamp of the video information as the base time stamp. When recording is started, obtain the time stamp T0 of the first frame of the video information, and align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When recording is stopped, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, modify and save the time stamp Ty of each frame to Tx, where Tx = T0 + (Ty - T0)*k. Thus, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0012] The method for time synchronization of the present video information and bus message information can, in the field of automotive ADAS development and verification, when there is no need to customize the device, simply use an unrelated video collection device and bus collection device to collect data simultaneously and finally perform data synchronization, solve the time stamp problem of video and bus data synchronization collection of multiple devices, and improve the collection efficiency.

[0013] The various non-limiting embodiments of the embodiments of the present invention will be described in detail below with reference to the drawings. As shown in FIG. 1, some embodiments provide a method for time synchronization of video information and bus message information. This method includes the following. In S101, define the time stamp of the video information as the base time stamp. In S102, when starting the recording, obtain the time stamp T0 of the first frame of the video information, align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. In S103, when stopping the recording, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. In S104, derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). In S105, traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame to Tx. Here, Tx = T0 + (Ty - T0) * k. In this way, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0014] Hereinafter, an example of the time synchronization method for video information and bus message information will be described in detail. In the development of an automobile, use one camera V1 to record video information and use one bus adapter PCAN to record the message information of the power CAN.

[0015] After starting the camera V1 and the bus adapter PCAN simultaneously, define the time stamp of the video frame sent from the camera V1 at the next moment (for example, 2023-08-13 09:01:02.123, that is, 9:01:02.123 milliseconds on August 13, 2023) as the time stamp T0 of the first frame. The time stamps of the bus message information after the time point of the time stamp T0 of the first frame are all set based on the time stamp T0 of the first frame. That is, the time stamp Ty of the bus message information at the time point of the time stamp T0 of the first frame is the time stamp T0 of the first frame, and the time stamp Ty of the bus message information 1 second after the time point of the time stamp T0 of the first frame is 2023-08-13 09:01:03.123, that is, 9:01:03.123 milliseconds on August 13, 2023.

[0016] Assuming that the recording stops 60 minutes after synchronous collection, when the recording stops, the time stamp T1 of the tail frame of the video information is 2023-08-13 10:01:02.123, that is, 10:01:02.123 milliseconds on August 13, 2023. The time stamp T2 of the tail frame of the bus adapter PCAN bus message information collected at this time is 3590 seconds. Therefore, the proportionality coefficient k = (T1 - T0) / (T2 - T0) = 3600 / 3590 = 1.002785515320334.

[0017] Traverse all the recorded bus messages and modify the time stamp Ty of each frame to Tx. Here, Tx = T0 + (Ty - T0) * k. Save the bus message data file, thereby realizing the time synchronization of the video information and the bus message information.

[0018] In some embodiments, aligning the time stamp of the head frame of the bus message with the time stamp T0 of the head frame of the video information, and recording the time stamp Ty of each frame of the bus message after alignment (step) includes the following. Define the time stamp of the head frame of the bus message as T3. If the time stamp of any bus message in the recording process is defined as T, then the time stamp Ty of each frame of the bus message after alignment is Ty = T - T3 + T0.

[0019] The following details a method of aligning the time stamp of the head frame of the bus message with the time stamp T0 of the head frame of the video information and recording the time stamp Ty of each frame of the bus message hardware after alignment in one example. In the development of automotive ADAS, video information is recorded using one camera V1, and power CAN message information is recorded using one bus adapter PCAN. After starting the camera V1 and the bus adapter PCAN simultaneously, define the timestamp of the video frame sent from the camera V1 at the next moment (for example, 2023-08-13 09:01:02.123, that is, 9:01:02.123 milliseconds on August 13, 2023) as the timestamp T0 of the first frame. After the timestamp T0 of the first frame, define the timestamp of the first frame of the bus message as T3. For example, T3 = 1234567 microseconds. The timestamp of any bus message after the timestamp T3 of the first frame is T. For example, T = 2234567 microseconds. Based on the method of alignment Ty = T - T3 + T0 (T - T3 = 1000000 microseconds, that is, 1 second), Ty = 2023-08-13 09:01:03.123, that is, 9:01:03.123 milliseconds on August 13, 2023.

[0020] In some embodiments, video information is transmitted to a computer device in real time. Also, bus messages are transmitted to the computer device via a bus adapter. Specifically, after synchronizing the collection times of the aforementioned video information and bus message information, the computer can collect the video information and bus messages synchronously.

[0021] In some embodiments, the transmission frame rate of the bus message is 1000 frames / second or more. Specifically, the transmission frame rate is 1000 frames / second or more, that is, the transmission interval of the bus message is less than 1 millisecond. This means that the time error between the tail frame of the bus message and the tail frame of the video information is less than 1 millisecond, thereby ensuring the synchronous collection accuracy of the video information and bus message data.

[0022] As shown in FIG. 2, some embodiments further provide a bus adapter. This bus adapter is configured to send the bus messages in the time synchronization system of the above video information and bus message information to a computer device, and includes an MCU module and a communication module. The MCU module is configured to execute the sending and receiving of bus messages, and when the frame rate is lower than 1000 frames / second, generate a frame rate compensation message as the bus message. The communication module is configured to be connected to the MCU module and send the bus message to the computer device when connected to the bus adapter.

[0023] As shown in FIG. 3, some embodiments further provide a time synchronization system for video information and bus message information. This system includes at least one bus adapter, at least one video information collector, and at least one computer device. At least one bus adapter is configured to send and receive bus messages. At least one video information collector is, for example, a camera and is configured to collect video information. At least one computer device includes a processor, a display that communicates with the processor to display a graphic interface, a computer-readable storage medium, a communication bus, and a communication interface. The video information collector communicates with the processor via the communication interface. The above processor, the above computer-readable storage medium, and the above communication interface realize communication with the bus adapter via the above communication bus. The above computer-readable storage medium stores a command program. After obtaining the bus message and video information, the above-mentioned processor is configured to execute the above command to perform the following operations. Define the time stamp of the video information as the base time stamp. When starting the recording, obtain the time stamp T0 of the first frame of the video information, and align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When stopping the recording, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame thereof to Tx. Here, Tx = T0 + (Ty - T0) * k. Thereby, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information. The above-mentioned display displays the time-synchronized bus message and video information through a graphic interface.

[0024] As shown in FIG. 4, some embodiments further provide a time synchronization system for video information and bus message information. This system includes a computer device and at least one of the above-mentioned bus adapters. The computer device is configured to include a definition module, an acquisition setting module, and a traversal module. The definition module defines the time stamp of the video information as the basic time stamp. When the recording is started, the acquisition setting module acquires the time stamp T0 of the first frame of the video information, aligns the time stamp of the first frame of the bus message from at least one bus adapter with the time stamp T0 of the first frame of the video information, and records the time stamp Tyn of each frame of the bus message after alignment. When the recording is stopped, the time stamp T1 of the last frame of the video information and the time stamp T2n of the last frame of the bus message are acquired. The proportionality coefficient kn = (T1 - T0) / (T2n - T0) is derived. Here, n is the number of the bus adapter. The traversal module traverses all the recorded bus messages, modifies the time stamp Tyn of each frame to Txn and saves it. Here, Txn = T0 + (Tyn - T0) * kn. In this way, the time stamp Txn of each frame of the bus message is configured to be recorded based on the clock of the video information. Here, the computer commands corresponding to the specific implementation functions of the definition module, the acquisition setting module, and the traversal module are stored in a computer-readable storage medium and realized in a computer device. Specifically, reference can be made to the content of the time synchronization method of the above video information and bus message information, so the description is omitted here.

[0025] Hereinafter, the electronic device according to the embodiment of the present disclosure will be described from the perspective of hardware processing. The embodiment of the present disclosure does not limit the specific implementation of the electronic device.

[0026] As shown in FIG. 5, some embodiments further provide a device. This device includes at least one computer-readable storage medium and at least one processor. At least one computer-readable storage medium stores commands. At least one processor, which executes the above command to perform the following operations. Define the time stamp of the video information as the base time stamp. When starting the recording, obtain the time stamp T0 of the first frame of the video information, and align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When stopping the recording, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame thereof to Tx. Here, Tx = T0 + (Ty - T0)*k. Thereby, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information. Specifically, the above device further includes a communication bus and a communication interface. The above processor, the above computer-readable storage medium, and the above communication interface realize communication with each other via the above communication bus.

[0027] As shown in FIG. 6, some embodiments further provide an electronic device. This electronic device includes a processor, a display that communicates with the processor to display a graphic interface, a computer-readable storage medium, a communication bus, and a communication interface. The above-readable Take storage medium stores a command program. The above-mentioned processor is configured to perform the following operations by executing the above-mentioned command. Define the time stamp of the video information as the base time stamp. When starting the recording, obtain the time stamp T0 of the first frame of the video information, and align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When stopping the recording, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame thereof to Tx. Here, Tx = T0 + (Ty - T0)*k. Thereby, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information. The above-mentioned display displays the time-synchronized bus message and video information via a graphic interface.

[0028] In some embodiments, the computer device, industrial computer, and device can also be regarded as a kind of electronic device. Note that the configurations shown in FIGS. 5 and 6 do not limit the electronic device, and it can include fewer or more components than shown in the figures, some components can be combined, or different components can be arranged.

[0029] In some embodiments, the communication interface may be a communication interface connectable to an external bus adapter, such as RS232, RS485, a USB port, and a TYPE port. A wired or wireless network interface may also be included, and the network interface may optionally include wired and / or wireless interfaces (e.g., a WI-FI interface, a Bluetooth interface, etc.) typically used to establish a communication connection between the computer device and other electronic devices.

[0030] The readable storage medium or computer-readable storage medium includes at least one kind of memory. The memory includes flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, it may be an internal storage unit of a computer device, such as a hard disk of the computer device. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard disk equipped in the computer device, a Smart Media Card (SMC) (registered trademark), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory may include both an internal storage unit and an external storage device of the computer device. The memory stores various data such as application software installed in the computer device and the code of computer programs, and is also used to temporarily store output data and data to be output.

[0031] In some embodiments, the processor may execute program code stored in a computer-readable storage medium or process data, for example, a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips for executing a computer program.

[0032] In some embodiments, the communication bus may be an input / output bus that can be, for example, a Peripheral Component Interconnect (PCI) bus or an Enhanced Industry Standard Architecture (EISA) bus. This bus can be divided into an address bus, a data bus, a control bus, and the like.

[0033] Optionally, the computer device may further include a user interface. The user interface can include input units such as a display and a keyboard, and optionally, the user interface can also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. In this case, the display is also called a display screen or a display unit for displaying the information processed within the computer device and for displaying the visualized user interface.

[0034] When the above processor executes the above program, it realizes the steps in the embodiment of the time synchronization method for video information and bus message information shown in FIG. 1 above. For example, steps S101 to S105 shown in FIG. 1. Alternatively, when the processor executes a computer program, it realizes the functions of each module or unit in the embodiments of the above respective devices.

[0035] Some embodiments further provide a computer-readable storage medium. This storage medium stores any of the possible time synchronization methods for video information and bus message information described above.

[0036] Some embodiments further provide a computer-readable storage medium. This storage medium stores computer-readable commands, which, when executed by at least one processor, cause the time synchronization method for video information and bus message information described above to be executed. Specifically, the time stamp of the video information is defined as the base time stamp. When recording is started, the time stamp T0 of the first frame of the video information is obtained, and the time stamp of the first frame of the bus message is adjusted to the time stamp T0 of the first frame of the video information, and the time stamp Ty of each frame of the adjusted bus message is recorded. When recording is stopped, the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message are obtained. The proportionality coefficient k = (T1 - T0) / (T2 - T0) is derived. All the recorded bus messages are traversed, and the time stamp Ty of each of its frames is corrected to Tx and saved. Here, Tx = T0 + (Ty - T0)*k. Thus, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information. Since a specific description of the time synchronization method for video information and bus message information can be referred to, the description is omitted here.

[0037] Some embodiments further provide a computer program product. This computer program product includes a computer program or commands. When the computer executes the above computer program or commands, the computer is caused to execute any of the possible time synchronization methods for video information and bus message information described above.

[0038] Some embodiments further provide a computer program product. This computer program product includes a computer-readable storage medium, on which computer-readable program code is stored. The computer-readable program code includes commands, and these commands cause at least one processor or at least one computer device to perform the following operations. Define the time stamp of the video information as the base time stamp. When starting the recording, obtain the time stamp T0 of the first frame of the video information, align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When stopping the recording, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, modify the time stamp Ty of each frame to Tx and save it. Here, Tx = T0 + (Ty - T0) * k. In this way, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information. Specifically, refer to the detailed description of the time synchronization method of the video information and the bus message information, and the description is omitted here.

[0039] As shown in FIG. 7, some embodiments further provide a system. This system includes a computer device, a bus adapter, and a video information collector. The above computer device includes a processor, a computer-readable storage medium, a communication bus, and a communication interface. The above video information collector communicates with the processor via the communication interface. The above processor, the above computer-readable storage medium, and the above communication interface realize communication with the bus adapter via the above communication bus. The above read Take The possible storage medium stores a command program. After obtaining the bus message and video information, the processor is configured to execute the above commands to perform the following operations. Define the time stamp of the video information as the base time stamp. When starting the recording, obtain the time stamp T0 of the first frame of the video information, align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When stopping the recording, obtain the time stamp T1 of the last frame of the video information and the time stamp T2 of the last frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame to Tx. Here, Tx = T0 + (Ty - T0)*k. Thus, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information.

[0040] In some embodiments, the video information collector may communicate with the processor via a communication interface, such as a USB port, and obtain video information in a manner of a video collection device such as a video collection card.

[0041] Some embodiments further provide a method. This method includes transmitting and receiving bus messages by a bus adapter and collecting video information by a video information collector. After obtaining the bus message and video information, execute the above command to cause the processor to perform the following operations. Define the time stamp of the video information as the base time stamp. When starting the recording, obtain the time stamp T0 of the first frame of the video information, and align the time stamp of the first frame of the bus message with the time stamp T0 of the first frame of the video information, and record the time stamp Ty of each frame of the bus message after alignment. When stopping the recording, obtain the time stamp T1 of the tail frame of the video information and the time stamp T2 of the tail frame of the bus message. Derive the proportionality coefficient k = (T1 - T0) / (T2 - T0). Traverse all the recorded bus messages, and modify and save the time stamp Ty of each frame to Tx. Here, Tx = T0 + (Ty - T0)*k. Thus, the time stamp Tx of each frame of the bus message is recorded based on the clock of the video information. The time-synchronized bus message and video information are displayed by the graphic interface.

[0042] In some embodiments, the bus adapter may be a CAN (Controller Area Network) bus adapter, a CANFD (Controller Area Network Flexible Data Rate) bus adapter, a FastLIN (Fast Local Interconnect Network) bus adapter, a LIN (Local Interconnect Network) bus adapter, an Ethernet bus adapter, a FlexRay bus adapter, and may be one-to-many or many-to-many. In some other embodiments, the specific implementation of the bus adapter is not limited.

[0043] In some embodiments, the computer device can communicate via the UDS (Unified Diagnostic Services) or XCP (Universal Calibration Protocol) or CCP (CAN Calibration Protocol) protocol to obtain bus messages.

[0044] In some embodiments, it communicates via the UDS (Unified Diagnostic Services) or XCP (Universal Calibration Protocol) or CCP (CAN Calibration Protocol) protocol with a debug device, such as a vehicle ECU (Electronic Control Unit) and its related systems, to transmit bus messages.

[0045] In some embodiments, the present method and system can cover the time synchronization scenario of video information and bus message information related to the automotive, aerospace, marine, machinery, and automation fields. Specifically for the automotive field, it may be a vehicle ECU (Electronic Control Unit) and its related systems. For example, but not limited to, devices such as an electronic power steering system EPS, an anti-lock braking system ABS, an electronic stability control system ESC, an automotive engine management system, and a battery management system BMS can achieve the transmission of bus messages if connected to the computer device by a bus method.

[0046] In some embodiments, it goes without saying that the disclosed apparatus and method can also be implemented in other ways. The embodiments of the apparatus described above are merely illustrative. For example, the flowcharts and block diagrams in the drawings show the architectures, functions, and operations that can be realized by the apparatus, method, and computer program product according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code. The above-mentioned module, program segment, or part of the code includes executable instructions for realizing one or more predetermined logical functions. It should be noted that in some alternative implementation manners, the functions represented by the blocks may occur in an order different from the order shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel and, depending on the related functions, can sometimes be executed in the reverse order. Also, each block of the block diagram and / or flowchart, as well as combinations of the blocks of the block diagram and / or flowchart, may be realized by a dedicated hardware-based system for executing a predetermined function or operation, or may be realized by a combination of dedicated hardware and computer instructions.

[0047] In addition, each functional module in each embodiment of the present invention may be integrated together to form an independent part, each module may exist alone, or two or more modules may be integrated to form an independent part.

[0048] When the above functions are realized in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art or a part of the technical solution, can be represented in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0049] Inspired by the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the technical idea of the present invention from the above description. The technical scope of the present invention is not limited to the content of the specification, and its technical scope must be determined based on the scope of the claims.

Claims

1. 1. A method for time synchronizing video information and bus message information, comprising the steps of: Define a timestamp of the video information as a base timestamp; When recording is started, a time stamp T0 of the first frame of the video information is obtained, and the time stamp of the first frame of the bus message is aligned with the time stamp T0 of the first frame of the video information, and the time stamp Ty of each frame of the bus message after alignment is recorded; When recording is stopped, a timestamp T1 of the tail frame of the video information and a timestamp T2 of the tail frame of the bus message are obtained; Derive the proportionality coefficient k=(T1-T0) / (T2-T0), A computer-implemented method for time synchronizing video information and bus message information, comprising: traversing all recorded bus messages; and modifying and storing the timestamp Ty of each frame thereof to Tx=T0+(Ty-T0)*k, whereby the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information.

2. A method for aligning a timestamp of a first frame of a bus message with a timestamp T0 of a first frame of video information, and recording a timestamp Ty of each frame of the bus message after the alignment, comprising: The timestamp of the first frame of the bus message is defined as T3. The method of claim 1, wherein if the timestamp of any bus message in the recording process is defined as T, the timestamp of each frame of the aligned bus message is Ty=T-T3+T0.

3. Transmitting the video information to a computer device in real time; 2. The method of claim 1, further comprising transmitting a bus message to said computer device via a bus adapter.

4. 2. The method of claim 1, wherein a transmission frame rate of the bus message is 1000 frames / second or more.

5. 1. A computer-readable storage medium, comprising: A computer-readable storage medium storing computer-readable instructions which, when executed by at least one processor, cause a program for carrying out the time synchronization method according to any one of claims 1, 2 and 4 to be executed.

6. An apparatus comprising: at least one computer readable storage medium and at least one processor; The computer-readable storage medium stores commands; 5. An apparatus, characterized in that the processor executes the commands to execute a program of the time synchronization method according to claim 1, 2 or 4.

7. An electronic device, a processor, a display communicating with the processor for displaying a graphic interface, a computer readable storage medium, a communication bus and a communication interface; the processor, the computer-readable storage medium, and the communication interface communicate with each other via the communication bus; The readable storage medium stores a command program; The processor is configured to execute the program for the time synchronization method according to any one of claims 1, 2 and 4 by executing the command, 1. An electronic device, characterized in that the display displays time-synchronous bus messages and video information via a graphic interface.

8. 1. A system for time synchronizing video information and bus message information, comprising: at least one bus adapter, at least one video information collector, and at least one computer device; the bus adapter is configured to send and receive bus messages; the video information collector is configured to collect video information; The computer device includes a processor, a display in communication with the processor for displaying a graphical interface, a computer-readable storage medium, a communication bus, and a communication interface; the video information collector in communication with a processor via a communication interface; the processor, the computer-readable storage medium, and the communication interface realize communication with a bus adapter via the communication bus; The computer-readable storage medium stores a command program; The processor is configured to execute the command to execute the program of the time synchronization method according to any one of claims 1, 2 and 4 after acquiring the bus message and the video information; 11. A system for time-synchronizing video and bus message information, comprising: a display for displaying time-synchronized bus messages and video information via a graphic interface.

9. the bus adapter is a CAN bus adapter, a CAN FD bus adapter, a Fast LIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter; 9. The time synchronization system of claim 8, which communicates with a computer device via UDS or XCP or CCP protocol to send and receive bus messages to a processor in the computer device.

10. The bus adapter includes an MCU module and a communication module; The MCU module is configured to perform transmission and reception of bus messages, and when the frame rate is lower than 1000 frames / second, generate a frame rate compensation message as the bus message; 9. The time synchronization system of claim 8, wherein the communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

11. 1. A bus adapter, comprising: A system for transmitting bus messages in a time-synchronized system of video information and bus message information according to any one of claims 1 to 4 to a computer device, the system comprising: an MCU module; and a communication module; The MCU module is configured to perform transmission and reception of bus messages, and when the frame rate is lower than 1000 frames / second, generate a frame rate compensation message as the bus message; A bus adapter, wherein the communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

12. the bus adapter is a CAN bus adapter, a CAN FD bus adapter, a Fast LIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter; 12. The bus adapter of claim 11, wherein the bus adapter communicates with a computer device via UDS or XCP or CCP protocols to send and receive bus messages to a processor in the computer device.

13. 1. A system comprising: A computer device, a bus adapter, and a video information collector, the computing device includes a processor, a computer-readable storage medium, a communication bus, and a communication interface; the video information collector in communication with a processor via a communication interface; the processor, the computer-readable storage medium, and the communication interface realize communication with a bus adapter via the communication bus; The readable storage medium stores a command program; The system, characterized in that the processor is configured to execute the command and execute the program of the time synchronization method described in any one of claims 1, 2 and 4 after acquiring bus messages and video information.

14. the bus adapter is a CAN bus adapter, a CAN FD bus adapter, a Fast LIN bus adapter, a LIN bus adapter, an Ethernet bus adapter, or a FlexRay bus adapter; 14. The system of claim 13, further comprising: a computer device that communicates with the computer device via UDS or XCP or CCP protocols to send and receive bus messages to a processor in the computer device.

15. The bus adapter includes an MCU module and a communication module; The MCU module is configured to perform transmission and reception of bus messages, and when the frame rate is lower than 1000 frames / second, generate a frame rate compensation message as the bus message; 14. The system of claim 13, wherein the communication module is configured to be connected to the MCU module and transmits bus messages to the computer device when connected to the bus adapter.

16. 1. A method comprising: transmitting and receiving bus messages by a bus adapter; and collecting video information by a video information collector; After obtaining the bus message and video information, the processor performs the following operations: The timestamp of the video information is defined as a base timestamp. When recording is started, the timestamp T0 of the first frame of the video information is obtained, and the timestamp of the first frame of the bus message is aligned with the timestamp T0 of the first frame of the video information, and the timestamp Ty of each frame of the bus message after alignment is recorded. When recording is stopped, the timestamp T1 of the tail frame of the video information and the timestamp T2 of the tail frame of the bus message are obtained, and a proportional coefficient k=(T1-T0) / (T2-T0) is derived. All recorded bus messages are traversed, and the timestamp Ty of each frame is modified to Tx=T0+(Ty-T0)*k and stored, so that the timestamp Tx of each frame of the bus message is recorded based on the clock of the video information; A computer implemented method for displaying time synchronous bus messages and video information through a graphical interface.

17. A method for aligning a timestamp of a first frame of a bus message with a timestamp T0 of a first frame of video information, and recording a timestamp Ty of each frame of the bus message after the alignment, comprising: The timestamp of the first frame of the bus message is defined as T3.

17. The method according to claim 16, characterized in that if the timestamp of any bus message in the recording process is defined as T, then the timestamp of each frame of the combined bus message is Ty=T-T3+T0.

18. 17. The method of claim 16, comprising transmitting video information in real time and the bus messages being transmitted by a bus adapter.

19. 17. The method according to claim 16, characterized in that the transmission frame rate of the bus messages is equal to or greater than 1000 frames / second.

Images