Data distribution device, control system with data distribution device and configuration method of data distribution device

Abstract

The invention provides a data distribution device, a control system with the data distribution device and a configuration method of the data distribution device. The data distribution device comprises a deserializer and a plurality of serializers connected with the deserializer, the plurality of serializers comprise a first serializer and a second serializer, each of the first serializer and the second serializer comprises a first interface used for being connected with a control device and a second interface used for being connected with the deserializer, and the first interface is used for being connected with the control device. The first serializer is configured to perform initialization according to an initialization command received from a first interface of the first serializer, and the second serializer is configured to perform initialization according to an initialization command received from a first interface of the second serializer, and in response to completion of initialization, the second serializer is configured to perform initialization according to an initialization command received from a second interface of the second serializer. And notifying the deserializer that the second serializer has completed initialization through a second interface of the second serializer.

Description

Technical Field

[0001] The present disclosure relates to autonomous driving technology, and in particular to a data diversion device, a control system having the device, and a configuration method thereof. Background Art

[0002] An autonomous vehicle may include various sensors (e.g., cameras) for sensing the environment around the vehicle and the vehicle's own conditions, as well as a control device. The control device may include various algorithm modules, which include various machine learning models (e.g., neural networks) to process data collected by sensors on the vehicle in real time or online for identifying, evaluating, and avoiding or overcoming potential obstacles in the environment where the vehicle is located. In order to avoid safety issues caused by failure of the control device, multiple redundant control devices may be installed on the autonomous vehicle. Some data collected by sensors (e.g., image data collected by a camera) may need to be transmitted to each control device. Summary of the invention

[0003] The present disclosure provides a data diversion device for transmitting (or diverting) data collected by a sensor (such as image data collected by a camera) to multiple control devices. The present disclosure also provides a control system having the device, and a configuration method for the device and system.

[0004] The present disclosure provides, in one aspect, a data distribution device, comprising:

[0005] Deserializer; and

[0006] a plurality of serializers connected to the deserializer,

[0007] The plurality of serializers include a first serializer and a second serializer,

[0008] The first serializer and the second serializer each include a first interface for connecting to a control device and a second interface for connecting to a deserializer,

[0009] Wherein, the first serializer is configured as:

[0010] performing initialization according to an initialization command received from the first interface of the first serializer,

[0011] Wherein, the second serializer is configured as:

[0012] performing initialization according to an initialization command received from the first interface of the second serializer,

[0013] In response to the completion of the initialization, the deserializer is notified through the second interface of the second serializer that the second serializer has completed the initialization.

[0014] In another aspect, the present disclosure provides a control system, comprising:

[0015] Image acquisition device;

[0016] a plurality of control devices, wherein the plurality of control devices comprises a first control device and a second control device; and

[0017] A data distribution device, comprising a deserializer and a plurality of serializers, and configured to transmit the image data acquired by the image acquisition device to the plurality of control devices, wherein:

[0018] The plurality of serializers include a first serializer and a second serializer,

[0019] The first control device is configured to initialize the first serializer,

[0020] The second control device is configured to initialize the second serializer,

[0021] The second serializer is configured to, in response to completion of initialization, notify the first control device through the deserializer and the first serializer that the second serializer has completed initialization.

[0022] In another aspect, the present disclosure provides a method for configuring a control system.

[0023] Wherein, the control system comprises:

[0024] Image acquisition device;

[0025] a plurality of control devices, wherein the plurality of control devices comprises a first control device and a second control device; and;

[0026] A data distribution device, the data distribution device includes a deserializer and a plurality of serializers, and is configured to transmit the image data collected by the image acquisition device to the plurality of control devices, wherein the plurality of serializers include a first serializer and a second serializer,

[0027] The method comprises:

[0028] The first control device initializes the first serializer;

[0029] The second control device initializes the second serializer;

[0030] In response to completing the initialization, the second serializer notifies the first control device through the deserializer and the first serializer that the second serializer has completed the initialization.

[0031] According to the technology disclosed in the present invention, a data splitting device with a simple structure can be realized, which is particularly beneficial for vehicle-mounted applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings, the same elements are represented by the same reference numerals.

[0033] Figure 1 is a schematic diagram of a vehicle located in an environment according to an example embodiment of the present disclosure;

[0034] Figure 2 is a schematic diagram of a control device according to an exemplary embodiment of the present disclosure;

[0035] Figure 3 is a schematic diagram of a control system of a vehicle according to an exemplary embodiment of the present disclosure;

[0036] Figure 4 According to an exemplary embodiment of the present disclosure Figure 3 A specific implementation of the control system shown;

[0037] Figure 5 is a flow chart of a method for configuring a control system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] In this disclosure, the term "plurality" means two or more, unless otherwise specified. In this disclosure, the term "and / or" describes the association relationship of associated objects, covering any one of the listed objects and all possible combinations. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0040] In the present disclosure, unless otherwise specified, the terms "first", "second", etc. are used to distinguish similar objects without intending to limit their positional relationship, temporal relationship, or importance relationship. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in ways other than those illustrated or described herein.

[0041] In addition, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, system, product, or apparatus.

[0042] Figure 1 1 is a schematic diagram of a vehicle 120 located in an environment 100 according to an example embodiment of the present disclosure. The vehicle 120 may be a car, a truck, a motorcycle, a bus, a recreational vehicle, an amusement park vehicle, a tram, a golf cart, a train, a trolley bus, or other vehicle. The vehicle 120 may have a control system, such as an autonomous driving system. Under the control of the control system, the vehicle 120 may operate in an autonomous driving mode, either completely or partially. The vehicle 120 may control itself in the autonomous driving mode, for example, the vehicle 120 may determine the current state of the vehicle 120 and the current state of the environment in which the vehicle 120 is located, determine the predicted behavior of at least one other vehicle in the environment, determine the trust level corresponding to the likelihood that the at least one other vehicle will perform the predicted behavior, and control the vehicle 120 itself based on the determined information. When in the autonomous driving mode, the vehicle 120 may operate without human interaction.

[0043] The control system of the vehicle 120 may include various sensors for sensing the environment 100 around the vehicle and the conditions of the vehicle itself, for example, an image acquisition device. Figure 1 In the example of , the image acquisition device includes cameras 121 and 122. In addition to the camera, the sensors on the vehicle 120 may also include one or more of an inertial measurement unit (IMU), a global navigation satellite system (GNSS) transceiver (e.g., a global positioning system (GPS) transceiver), a radio detection and ranging (RADAR) sensor, a laser detection and ranging (LiDAR) sensor, an acoustic sensor, and an ultrasonic sensor.

[0044] The control system of the vehicle 120 may also include one or more control devices. Each control device is capable of controlling some or all functions of the vehicle 120. Each control device may include, for example, an autonomous driving control unit, which may be composed of one or more algorithm modules, including various machine learning models (e.g., neural networks) that can perform real-time or onboard processing (e.g., object detection) on data collected by sensors on the vehicle 120 to identify, evaluate, and avoid or cross potential obstacles in the environment 100 where the vehicle is located. In some embodiments, the autonomous driving control unit is used to combine data from sensors to determine the driving path or trajectory of the vehicle 120.

[0045] For redundancy, the control system can have multiple control devices. Figure 1 In the example of , the vehicle 120 includes two control devices, namely, a control device 131 and a control device 132. It will be appreciated by those skilled in the art that the vehicle 120 may have more control devices. One of these control devices may serve as a main control device and the other as a backup control device. The autonomous driving control unit of the backup control device may have the same functions as the autonomous driving control unit of the main control device, or may have fewer functions than the autonomous driving control unit of the main control device. Under the condition that the main control device operates normally, the main control device controls the vehicle (e.g., controls some or all functions of the vehicle 120), and the backup control device does not perform any control on the vehicle. The algorithm module of the main control device may perform real-time or online processing (e.g., object detection) on sensor data (e.g., image data collected by cameras 121 and 122) to determine the driving path or trajectory of the vehicle 120, and / or to identify, evaluate, and avoid or cross potential obstacles in the environment in which the vehicle is located.

[0046] In the event of a failure of the main control device, the backup control device replaces the main control device to control the vehicle (eg, control part or all of the functions of the vehicle 120 ), and the main control device no longer performs any control over the vehicle.

[0047] In one example, the control device 131 serves as the main control device of the vehicle 120 , and the control device 132 serves as the backup control device of the vehicle 120 . In another example, the control device 132 serves as the main control device of the vehicle 120 , and the control device 131 serves as the backup control device of the vehicle 120 .

[0048] In addition to the sensors and the control device, the control system of the vehicle may also include elements for data transmission between the sensors and the control device (eg a data splitter).

[0049] Figure 2 1 is a schematic diagram of a control device according to an exemplary embodiment of the present application. Each control device (eg Figure 1The control device 131 and the control device 132 shown in the figure can both be implemented by the control device 200. The control device 200 can be a server, a personal computer (PC), a laptop computer, a tablet computer, a personal digital assistant (PDA), an electronic control unit (ECU), a cellular phone, a smart phone, a set-top box (STB), etc. An example of an electronic control unit is a domain controller. The example control device 200 may include a processor 202 (e.g., a system on a chip (SoC), a general processing core, a graphics core, and optional other processing logic) and a memory 204 that can communicate with each other via a bus 206 or other data transmission system. The control device 200 may also include various input / output (I / O) devices or interfaces 210 (such as a touch screen display, an audio jack, a voice interface) and a network interface 212. The network interface 212 can use 3G cellular communication (e.g., CDMA, EVDO, GSM / GPRS) or 4G cellular communication (e.g., WiMAX or LTE), and can also use 5G cellular communication. In an example embodiment, the network interface 212 can communicate with a wireless local area network (WLAN) (e.g., using ). In an example embodiment, the network interface 212 may include or support substantially any wired and / or wireless communication and data processing mechanism by which information / data may be communicated between the control device 200 and another control device or system via the network 214.

[0050] Memory 204 is an example of a computer-readable storage medium on which instruction sets, software, firmware, or other processing logic (e.g., logic 208) are stored. Logic 208 or a portion thereof may also reside completely or at least partially within processor 202 during execution by control device 200.

[0051] The term computer-readable storage medium may be understood to include a single non-transitory medium or multiple non-transitory media (e.g., centralized or distributed databases and / or associated caches and computing systems) that store one or more instruction sets. The term computer-readable storage medium may also be understood to include any non-transitory medium that can store, encode, or carry an instruction set for execution by a machine and cause the machine to perform any one or more of the methods of the various embodiments or that can store, encode, or carry a data structure utilized by or associated with such an instruction set. The term computer-readable storage medium may therefore be understood to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0052] The sensors on the vehicle need to be arranged rationally to save the number of sensors. This is because, on the one hand, the locations suitable for installing sensors on the vehicle are limited, and on the other hand, excessive sensors not only bring great challenges to vehicle integration, but also increase the cost of the vehicle. In addition, the power supply on the vehicle is limited.

[0053] Among the locations on the vehicle that are suitable for installing sensors, a few locations are more preferred. Sensors installed in these locations can be less blocked by the vehicle or have a better field of view than those installed in other locations. It is beneficial to allow various control devices of the vehicle (such as a main control device and a backup control device) to use the data of sensors on some sensors (especially preferred locations). The embodiments of the present disclosure provide a data diversion device and a control system (described in detail below), which can transmit sensor data (for example, image data) to multiple control devices at the same time, so that these control devices can all use the sensor data.

[0054] Figure 3 1 is a schematic diagram of a control system of a vehicle 120 according to an exemplary embodiment of the present disclosure. As described above, the control system of the vehicle 120 may include a plurality of image acquisition devices (eg, cameras) and a plurality of control devices.

[0055] like Figure 3 As shown, the control system 300 of the vehicle 120 may further include a data diversion device 310. The data diversion device is used to transmit or divert image data from at least one image acquisition device (such as cameras 121 and 122) to multiple control devices (such as control devices 131 and 132). The data diversion device 310 may include one or more data diversion units. In one example, the multiple data diversion units of the data diversion device 310 have substantially the same structure and function, for example, each data diversion unit includes a deserializer and the same number of multiple serializers. In another example, different data diversion units have different structures, for example, each data diversion unit includes a deserializer and different numbers of multiple serializers. Each data diversion unit is connected to at least one camera and at least two control devices to transmit the image data collected by the camera to each control device connected thereto, so that the image data collected by the camera can be shared among multiple control devices. For example, as Figure 3 As shown, the data splitting device 310 includes a data splitting unit 350 and a data splitting unit 360. The data splitting unit 350 is connected to the camera 121 and the control device 131 and the control device 132 to transmit the image data collected by the camera 121 to the control device 131 and the control device 132. The data splitting unit 360 is connected to the camera 122 and the control device 131 and the control device 132 to transmit the image data collected by the camera 122 to the control device 131 and the control device 132.

[0056] It should be noted that although Figure 3It is shown that the image data collected by the two cameras of the control system 300 are shared by each control device. Those skilled in the art will understand that the control system 300 can have more or fewer cameras to collect image data to share between the control devices. Accordingly, the data diversion device 310 can have more or fewer data diversion units. In some embodiments, the number of data diversion units is the same as the number of cameras that need to be shared, that is, each camera that needs to be shared is connected to a data diversion unit. It should also be noted that although Figure 3 One data splitting unit is connected to two control devices, but those skilled in the art will appreciate that each data splitting unit can be connected to more control devices, thereby transmitting image data to these control devices simultaneously.

[0057] Figure 4 yes Figure 3 A specific implementation of the control system shown in FIG. Figure 4 As shown, the control system 400 includes an image acquisition device (e.g., camera 121), multiple control devices (e.g., control devices 131 and 132), and a data splitting device 310. The data splitting device 310 is used to transmit the image data acquired by the image acquisition device to the control device. The data splitting device 310 may include a data splitting unit (e.g., data splitting unit 350). It should be noted that for the sake of simplicity, Figure 4 Not shown Figure 3 The data splitting unit 360 and the camera 122. Those skilled in the art can understand that the data splitting unit 360 can have substantially the same structure and function as the data splitting unit 350.

[0058] The data diversion unit 350 may include a deserializer and multiple serializers. For example, the data diversion unit 350 includes a deserializer 453 for connecting to the camera 121 and serializers 451 and 452 connected to the deserializer. Serializers 451 and 452 can be connected to different control devices respectively, for example. Serializer 451 can be connected to the control device 131, and serializer 452 can be connected to the control device 132. Since the same deserializer of the data diversion unit is connected to different serializers, the deserializer can send each image received from the camera to different serializers at the same time, and then the serializer sends it to the corresponding control device, so that different control devices can receive the same image data almost at the same time. In this way, the image data collected by the camera can be fully utilized.

[0059] It should be noted that although Figure 4 In the example, two serializers are connected to the deserializer 453, but those skilled in the art will appreciate that more serializers may be connected to the deserializer 453, and these serializers may be connected to different control devices, respectively. Figure 4The deserializer 453 is connected to one camera. Those skilled in the art will appreciate that the deserializer 453 may be connected to multiple cameras to transmit or divert image data collected by multiple cameras to different control devices through different serializers.

[0060] like Figure 4 As shown, the deserializer 453 may include multiple interfaces (e.g., pins) to facilitate connection with the camera and the serializer. Specifically, the deserializer 453 can be connected to the camera 121 through the interface C, so as to establish a communication link between the deserializer 453 and the camera 121. Through the communication link, the deserializer 453 can receive image data from the camera 121, send a command (e.g., a command from the control device 131 to initialize or configure the camera 121) to the camera 121, and power the camera 121. In one example, the interface C is a GMSL interface (e.g., a GMSL2 interface) to establish a GMSL link between the deserializer 453 and the camera 121. GMSL is the abbreviation of Gigabit Multimedia Serial Link, including GMSL1 and GMSL2.

[0061] The deserializer 453 can be connected to the corresponding interface A of the serializer 451 through the interface A, and can be connected to the corresponding interface B of the serializer 452 through the interface B, so as to establish a data link between the deserializer 453 and the serializers 451 and 452, respectively. Through this link, the deserializer 453 can send the image data received from the camera 121 (after the deserialization operation) to the serializers 451 and 452, so that the serializers 451 and 452 can receive the same image data from the deserializer 453 almost at the same time. In an example, the deserializer 453 can send each frame of the image data received from the camera 121 to the serializers 451 and 452. In an example, both interfaces A and B are mipi interfaces to establish mipi links between the deserializer 453 and the serializers 451 and 452, respectively. mipi is the abbreviation of Mobile Industry Processor Interface.

[0062] The deserializer 453 can be connected to the corresponding interface M of the serializer 451 through the interface M, so as to establish a command link between the deserializer 453 and the serializer 451. Through this link, the deserializer 453 can receive commands from the control device 131 forwarded by the serializer 451 (for example, commands for initializing or configuring the deserializer 453 and the camera 121), and through this link, the deserializer 453 can send the status information of the deserializer 453 (for example, the status of the interface of the deserializer 453) to the control device 131 through the serializer 451. In one example, the interface M is an I2C interface to establish an I2C (Inter-Intergrated Circuit) link between the deserializer 453 and the serializer 451.

[0063] In some embodiments, the deserializer 453 is also connected to the corresponding interfaces of the serializer 451 (e.g., interfaces P0, P1, P4, and P6 of the serializer 451) through multiple interfaces, such as interfaces P0, P1, P4, and P6. These interfaces can be general purpose input and output (GPIO) interfaces, and their functions can be configured or initialized by the control device 131. For example, the control device 131 can configure or initialize the interface P0 of the deserializer 453 and the serializer 451 to transmit the frame synchronization signal FSYNC used by the camera 121 to capture images. With the help of interface P0, the control device 131 can transmit the frame synchronization signal FSYNC to the camera 121 through the deserializer 453 and the serializer 451 to control the frame rate of the camera to capture and output images. The control device 131 can configure or initialize the interface P1 of the deserializer 453 and the serializer 451 to transmit the Lock signal, indicating the status of the link between the camera 121 and the deserializer 453. For example, the interface P1 of the deserializer 453 and the serializer 451 indicates that the link between the camera 121 and the deserializer 453 has been established at one level (e.g., a high level), and the interface P1 of the deserializer 453 and the serializer 451 indicates that the link between the camera 121 and the deserializer 453 has not been established or is in a disconnected state at another level (e.g., a low level). In this way, with the help of the interface P1, the control device 131 can learn the state of the link between the camera 121 and the deserializer 453 through the serializer 451. The control device 131 can configure the interface P4 of the deserializer 453 and the serializer 451 to transmit the error signal ERRB indicating the failure of the deserializer 453. In this way, with the help of the interface P4, the control device 131 can receive the error signal of the deserializer 453 through the serializer 451. The control device 131 can configure the interface P6 of the deserializer 453 and the serializer 451 to transmit the error signal ERROR of the camera. In this way, by means of the interface P6 , the control device 131 can receive the error signal ERROR of the camera through the deserializer 453 and the serializer 451 .

[0064] The serializer 451 can be connected to the control device 131 through the interface D, so as to establish a communication link between the serializer 451 and the control device 131. Through the communication link, the control device 131 can send an initialization or configuration command to the serializer 451 to initialize or configure the serializer 451. Through the communication link, the control device 131 can receive image data from the serializer 451 and power the data splitting unit 350. At the same time, through the communication link, the serializer 451 can receive a command from the control device 131 to initialize or configure the camera 121 and the deserializer 453, and then the serializer 451 forwards the command to the deserializer 453 or further forwards it to the camera 121 through the deserializer 453. Through the communication link, the serializer 451 can also forward the status information of the deserializer 453 to the control device 131. In an example, the interface D of the serializer 451 is a GMSL interface (e.g., a GMSL2 interface) to establish a GMSL link between the serializer 451 and the control device 131.

[0065] The serializer 452 can be connected to the control device 132 through its interface D, so as to establish a communication link between the serializer 452 and the control device 132. Through the communication link, the control device 132 can send an initialization or configuration command to the serializer 452 to initialize or configure the serializer 452, and receive image data from the serializer 452. In one example, the interface D of the serializer 452 is a GMSL interface (e.g., a GMSL2 interface) to establish a GMSL link between the serializer 452 and the control device 132.

[0066] The control device 131 can not only initialize or configure the serializer 451, but also initialize or configure the deserializer 453. In some embodiments, the initialization (or configuration) of the deserializer 453 and the serializer 451 by the control device 131 includes setting the address (e.g., I2C address) of the deserializer 453 and the serializer 451, setting the working mode, rate, number of channels, etc. of the GMSL interface of the deserializer 453 and the serializer 451, setting the data format of the mipi interface of the deserializer 453 and the serializer 451, and / or setting the functions of some of the interfaces of the deserializer 453 and the serializer 451 (e.g., interfaces P0, P1, P4, and P6 of the deserializer 453 and the serializer 451). The initialization (or configuration) of the deserializer 453 by the control device 131 can also include setting the number of links for the deserializer 453 to output image data, for example, setting the deserializer 453 to output two paths of image data (to be output to the serializers 451 and 452, respectively).

[0067] After the control device 132 completes the initialization or configuration of the serializer 452, the serializer 452 can notify the control device 131 that it has completed the initialization through the deserializer 453 and the serializer 451. In some embodiments, the initialization (or configuration) of the serializer 452 by the control device 132 includes setting the address (e.g., I2C address) of the serializer 452, setting the working mode, rate, number of channels, etc. of the GMSL interface of the serializer 452, and / or setting the data format of the mipi interface of the serializer 452, etc.

[0068] In some embodiments, the deserializer 453 can be connected to the corresponding interface P7 of the serializer 452 through the interface P7, so as to establish a notification link between the deserializer 453 and the serializer 452. After the serializer 452 is initialized by the control device 132, the serializer 452 can send a notification to the deserializer 453 through the notification link to notify the deserializer 453 that it has completed initialization. Then the deserializer 453 notifies the control device 131 through the serializer 451 that the serializer 452 has completed initialization. The interface P7 of the deserializer 453 and the serializer 452 can be a GPIO interface, and its function can be pre-set, for example, when the data diversion device is manufactured.

[0069] In one example, the P7 interface of the deserializer 453 can be pre-set as a state interface, and its state (e.g., the size of the level) can be changed by the serializer 452. The serializer 452 can change the state of the P7 interface of the deserializer 453 (e.g., change the size of the level of the interface) to notify the deserializer that it has completed initialization. For example, after the control device 132 initializes the serializer 452, the serializer 452 changes the state of the interface P7 of the deserializer 453 through its interface P7, and changes the state of the interface P7 of the deserializer 453 from a low level to a high level, thereby notifying the deserializer 453 that the serializer 452 has completed initialization. In other examples, after the control device 132 initializes the serializer 452, the serializer 452 changes the state of the interface P7 of the deserializer 453 through its interface P7, and changes the state of the interface P7 of the deserializer 453 from a high level to a low level, thereby notifying the deserializer 453 that the serializer 452 has completed initialization.

[0070] In some embodiments, the control device 131 may monitor the state of the interface P7 of the deserializer 453 through the deserializer 453 and the serializer 451 (and through the I2C link established between the deserializer 453 and the interface M of the serializer 451). When the control device 131 obtains a notification of a change in the state of the interface P7 of the deserializer 453 through the deserializer 453 and the serializer 451 (for example, the control device 131 detects a change in the voltage of the interface P7 of the deserializer 453 through the deserializer 453 and the serializer 451), the control device 131 determines that the state of the interface P7 has changed, and the control device 131 further determines that the serializer 452 has completed initialization.

[0071] In some embodiments, in addition to serializers 451 and 452, the data diversion unit 350 may also include more serializers, each of which may have an interface with the same function as interface B and P7 of serializer 452, and be connected to the corresponding interface of deserializer 453 through these interfaces (these interfaces of deserializer 453 may have the same function as interface B and P7 of deserializer 453). At the same time, each of these serializers may have an interface with the same function as interface D of serializer 452, and be respectively connected to different control devices (control devices of the control system other than control devices 131 and 132) through the interface.

[0072] The control device 131 determines that the serializer 452 has completed initialization (in the case where the data distribution unit 350 includes more serializers in addition to the serializers 451 and 452, the control device 131 also determines that these serializers have also completed initialization), and after the control device 131 completes the initialization of the deserializer 453 and the serializer 451, the control device 131 initializes the camera 121 through the deserializer 453 and the serializer 451 (and through the I2C link established between the deserializer 453 and the interface M of the serializer 451), so that the camera 121 collects and outputs image data. In some embodiments, the initialization (or configuration) of the camera 121 by the control device 131 includes setting the address of the camera 121 (for example, the I2C address), setting the working mode, rate, number of channels of the GMSL interface of the camera 121, and / or setting the fault error reporting mode of the camera 121.

[0073] After the camera 121 completes initialization, the camera 121 will capture images of the surrounding environment and send the captured image data to the deserializer 453 at a set frequency (i.e., frame rate). In one example, the frame rate of the camera 121 is determined by the frame synchronization signal FSYNC. Before the camera 121 sends the image data to the deserializer 453, the serializer in the camera 121 will serialize the image data. The deserializer 453 deserializes the received image data, and the deserializer 453 sends the deserialized image data to the serializer 451, and also sends the deserialized image data to the serializer 452. In this way, the serializer 451 and the serializer 452 will receive the same image data at the same time. The serializer 451 will serialize the received image data and send it to the control device 131, and the serializer 452 will serialize the received image data and send it to the control device 132. In the case where the data splitting unit 350 includes more serializers in addition to the serializers 451 and 452 , these serializers, like the serializer 452 , receive the image data from the deserializer 453 , and serialize the received image data and send it to the respective connected control devices.

[0074] The control system 400 also includes other data distribution units (such as Figure 3 The data splitter unit 360 shown) and other cameras (e.g. Figure 3 In the case of the camera 122 shown in the figure, the control devices 131 and 132 can also initialize the deserializer and serializer of the data splitting unit and the camera connected to the data splitting unit.

[0075] In some embodiments, the deserializer 453 includes an interface PWDN, which is a state interface and can have different states (e.g., different levels), and the state of the interface PWDN (e.g., the magnitude of the level) can be changed. When the interface PWDN is in one state (e.g., a high level), the deserializer 453 can be disabled or turned off, and when the interface PWDN is in another state (e.g., a low level), the deserializer 453 can be enabled or turned on.

[0076] In some embodiments, the serializer 452 includes an interface PWDN, which is a state interface and can have different states (e.g., different levels), and the state of the interface PWDN (e.g., the magnitude of the level) can be changed. When the interface PWDN is in one state (e.g., a high level), the serializer 452 can be disabled or turned off, and when the interface PWDN is in another state (e.g., a low level), the serializer 452 can be enabled or turned on.

[0077] In some embodiments, the serializer 451 may be connected to the interface PWDN of the deserializer 453 through its interface P8 to control the state of the interface. The serializer 451 may be connected to the interface PWDN of the serializer 452 through its interface P5 to control the state of the interface. The interfaces P5 and P8 of the serializer 451 may be GPIO interfaces, and their functions may be pre-set, for example, when the data splitting device is manufactured.

[0078] When the control device 131 finds that the data diversion unit 350 has a fault, it can control the deserializer 453 to be turned off and on (i.e., restart) through the interface P8 of the serializer 451 and the interface PWDN of the deserializer 453. The control device 131 can control the serializer 452 to be turned off and on (i.e., restart) through the interface P5 of the serializer 451 and the interface PWDN of the serializer 452.

[0079] In some embodiments, each GMSL interface of the data splitter unit 350 has a transient voltage suppression (TVS) protection device to prevent damage caused by static electricity.

[0080] In one example, the serializers 451 and 452 are both MAX96717 serializers manufactured by Maxim Corporation, and the deserializer 453 is MAX9296A deserializer manufactured by Maxim Corporation.

[0081] In some embodiments, the control device 131 is connected to the serializer 451 of the data splitting unit 350 through a coaxial cable having an inner conductor and an outer conductor. The control device 132 is connected to the serializer 452 of the data splitting unit 350 through a coaxial cable having an inner conductor and an outer conductor. The camera 121 is connected to the deserializer 453 of the data splitting unit 350 through a coaxial cable having an inner conductor and an outer conductor. In some embodiments, the data splitting unit 350 includes connectors 471, 472, and 473, and the control devices 131, 132, and the camera 121 are connected to the data splitting unit 350 through the connectors. For example, the control device 131 is connected to the serializer 451 through the connector 471. The control device 132 is connected to the serializer 452 through the connector 472. The camera 121 is connected to the deserializer 453 through the connector 473. Each of the connectors 471, 472, and 473 may be a coaxial connector, for example, an HFM (High-Speed ​​FAKAR-Mini) connector.

[0082] In some embodiments, one or more of the connectors 471, 473, the control devices 131, 132, and the camera 121 include a coaxial power (PoC) element. Data (e.g., video or image data), commands, and power can be transmitted over a coaxial cable through the PoC element. For example, the control device 131 and the connector 471 include a coaxial power (PoC) element to enable transmission of signals (including data (e.g., image data) and various control commands) and power between the control device 131 and the data splitting unit 350 over a single coaxial cable. The camera 121 and the connector 473 also include a coaxial power (PoC) element to enable transmission of signals and power between the camera 121 and the connector 473 over a single coaxial cable.

[0083] The control device 132 may also include a coaxial power supply (PoC) element, so that when the control device 132 is directly connected to a camera (not shown in the figure), a single coaxial cable can be used to transmit signals and power between the control device 132 and the camera. In the case where the data splitting unit 350 is powered by the control device 131, although the control device 132 has a POC, the data splitting unit 350 will not be powered by the POC.

[0084] The data splitting unit and data splitting device according to the embodiments of the present application have simple structures and are particularly suitable for vehicle-mounted applications. For vehicle-mounted applications, it is desirable that the device structure is simple. Generally, the fewer the number of components in the device, the more beneficial it is to the safety of the vehicle. In addition, for a control system with redundancy, the fewer the number of components shared by different data paths, the better. For example, Figure 4 The data splitting device shown has a simple structure and uses a deserializer and a serializer, and can achieve the purpose of data splitting without other complex components. Figure 4 In the control system shown, there are two data paths, one path includes the camera 121, the deserializer 453, the serializer 451 and the control device 131. When the data diversion device includes a connector, the path includes the camera 121, the connector 473, the deserializer 453, the serializer 451, the connector 471 and the control device 131. The other path includes the camera 121, the deserializer 453, the serializer 452 and the control device 132. When the data diversion device includes a connector, the path includes the camera 121, the connector 473, the deserializer 453, the serializer 452, the connector 472 and the control device 132. It can be seen that the common elements of the two paths are the camera 121 and the deserializer 453 (or the camera 121, the connector 473 and the deserializer 453). It can be seen that according to the data diversion unit and the data diversion device of the embodiment of the present application, the two data paths of the control system can share a small number of elements.

[0085] In order to compare multiple elements that are related to each other (such as Figure 4 The camera 121, deserializer 453, serializer 451, and serializer 452 shown in the figure are configured or initialized. If a processor or controller is considered to be added to the data diversion device of the control system, the initialization of these components is uniformly managed. This will increase the complexity of the data diversion device and is not conducive to the safety of the vehicle. According to an embodiment of the present application, after initialization, a serializer notifies the control device through the deserializer and other serializers that it has completed initialization. In this way, no additional processor or controller is required, which simplifies the structure of the data diversion unit and the data diversion device.

[0086] Figure 5 It is a flowchart of a configuration method of a control system according to an example embodiment of the present disclosure. The control system may include one or more image acquisition devices, multiple control devices (such as a first control device and a second control device), and a data diversion device for transmitting image data acquired by the image acquisition device to each control device. The data diversion device may include a deserializer and multiple serializers (such as a first serializer and a second serializer). In one example, the data diversion device may include one or more data diversion units, each data diversion unit including a deserializer and multiple serializers, wherein one of the data diversion units includes a deserializer and first and second serializers. The control system may be, for example Figure 3 The control system 300 shown or Figure 4 A control system 400 is shown.

[0087] In step 510, a control device of the control system, such as a first control device, powers a data diversion device (such as a data diversion unit of the data diversion device), and the data diversion unit then powers an image acquisition device (such as a camera) connected to the data diversion unit.

[0088] For example, refer to Figure 4 As shown, after the control device 131 is started, its POC is enabled, so that the control device 131 can power the data diversion unit 350 (including serializers 451, 452 and deserializer 453) of the data diversion device 310, and after the data diversion unit 350 is powered, it will power the camera 121.

[0089] After the data distribution unit 350 is powered, a communication link (e.g., a GMSL2 link) is established between the control device 131 and the serializer 451. After the data distribution unit 350 is powered, if the control device 132 is started at this time, a communication link (e.g., a GMSL2 link) is also established between the control device 132 and the serializer 452.

[0090] In step 520, after the first control device supplies power to the data diversion unit, it configures (ie, initializes) a serializer (eg, a first serializer) of the data diversion unit, and the first control device may also configure (ie, initialize) a deserializer of the data diversion unit.

[0091] For example, refer to Figure 4 As shown, after the data diversion unit 350 is powered, the control device 131 configures (i.e., initializes) the serializer 451 and the deserializer 453. For example, the control device 131 initializes the serializer 451 and the deserializer 453 through the communication link established between the control device 131 and the serializer 451. After the serializer 451 and the deserializer 453 are initialized, the control device 131 determines whether the serializer 452 has completed initialization. For example, the control device 131 determines whether the serializer 452 has completed initialization by monitoring the state of the interface P7 of the deserializer 453. The control device 131 can monitor the state of the interface P7 of the deserializer 453 through the command link (e.g., I2C link) between the serializer 451 and the deserializer 453. In some embodiments, before the serializer 452 completes initialization, the interface P7 of the deserializer 453 is at a low level, and after the serializer 452 completes initialization, the interface P7 of the deserializer 453 is at a high level.

[0092] In step 530, after the first control device supplies power to the data splitting unit, other serializers of the data splitting unit are configured (i.e., initialized) by their corresponding control devices (e.g., control devices respectively connected to these serializers). For example, the second control device may configure (i.e., initialize) the second serializer of the data splitting unit.

[0093] For example, refer to Figure 4As shown, once the control device 132 finds or detects that a communication link (e.g., a GMSL2 link) is established between the control device 132 and the serializer 452, the control device 132 configures (i.e., initializes) the serializer 452. The control device 132 can initialize the serializer 452 through the communication link (e.g., a GMSL2 link) between the control device 132 and the serializer 452. After the control device 132 completes the initialization of the serializer 452, it will notify the control device 131 through the serializer 451 and the deserializer 453 that the serializer 452 has completed initialization. In one example, the control device 132 changes the state of the interface P7 of the deserializer 453 through the interface P7 of the serializer 452 to notify the control device 131 through the deserializer 453 that the serializer 452 has completed initialization. For example, the control device 132 controls the interface P7 of the serializer 452 to output a high level as a signal that the serializer 452 has completed initialization. When the interface P7 of the serializer 452 outputs a high level, the interface P7 of the deserializer 453 changes from a low level to a high level. The control device 131 determines that the interface P7 of the deserializer 453 changes from a low level to a high level through a communication link (e.g., a GMSL2 link) between the control device 131 and the serializer 451 and a command link (e.g., an I2C link) between the serializer 451 and the deserializer 453, thereby the control device 131 determines that the serializer 452 has completed initialization.

[0094] In step 540, after the data splitting unit completes initialization (ie, both the serializer and the deserializer of the data splitting unit complete initialization), the first control device configures (ie, initializes) the image acquisition device connected to the data splitting unit.

[0095] For example, refer to Figure 4 As shown, after the serializer 451, the serializer 452 and the deserializer 453 have completed initialization, the control device 131 configures (i.e., initializes) the camera 121 through the communication link (e.g., GMSL2 link) between the control device 131 and the serializer 451, the command link (e.g., I2C link) between the serializer 451 and the deserializer 453, and the communication link (e.g., GMSL2 link) between the deserializer 453 and the camera 121.

[0096] In step 550, after completing initialization, the image acquisition device acquires and outputs image data to the data distribution unit connected thereto. The image acquisition device can continuously acquire multiple frames of images and output these images in sequence according to the time sequence in which the images were acquired. Such image data is also called video.

[0097] For example, refer to Figure 4As shown, the image data output by the camera 121 will be transmitted to the deserializer 453 through the communication link (e.g., GMSL2 link) between the deserializer 453 and the camera 121. The deserializer 453 deserializes the received image data, and the deserializer 453 sends the deserialized image data to the serializer 451 through the data link (e.g., mipi link) between the serializer 451 and the deserializer 453, and also sends the deserialized image data to the serializer 452 through the data link (e.g., mipi link) between the serializer 452 and the deserializer 453. In this way, the serializer 451 and the serializer 452 will receive the same image data at the same time. The serializer 451 will serialize the received image data and send it to the control device 131 through the communication link (e.g., GMSL2 link) between the control device 131 and the serializer 451, and the serializer 452 will serialize the received image data and send it to the control device 132 through the communication link (e.g., GMSL2 link) between the control device 132 and the serializer 452.

[0098] It should be noted that, in the case where the data splitting device includes multiple data splitting units, the above steps 510-550 can be performed similarly for each data splitting unit.

[0099] Although this paper includes many details, these details should not be interpreted as limitations to the scope of the claimed invention, but are interpreted as descriptions of features peculiar to specific embodiments. Some features described in the context of different embodiments herein may also be combined in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be separated or implemented in multiple embodiments with any suitable sub-combination. In addition, although features may be described as acting with some combinations and even initially claimed above, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may be directed to a variation of a sub-combination or a sub-combination. Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring to perform such operations in the particular order shown or in a sequential order, or to perform all described operations to achieve desired results.

[0100] Note that the above are only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the attached claims.

Claims

1. A data distribution device, comprising: Deserializer; and a plurality of serializers connected to the deserializer, The plurality of serializers include a first serializer and a second serializer, The first serializer and the second serializer each include a first interface for connecting to a control device and a second interface for connecting to a deserializer, Wherein, the first serializer is configured as: performing initialization according to an initialization command received from the first interface of the first serializer, Wherein, the second serializer is configured as: performing initialization according to an initialization command received from the first interface of the second serializer; and In response to the completion of the initialization, the deserializer is notified through the second interface of the second serializer that the second serializer has completed the initialization.

2. The device according to claim 1, wherein: The second serializer is configured to, in response to completion of initialization, notify the deserializer through a second interface of the second serializer that the second serializer has completed initialization, including the second serializer being configured to: In response to the second serializer completing initialization, a state of a predetermined interface of the deserializer is modified through a second interface of the second serializer.

3. The device according to claim 2, wherein: The state of the predetermined interface includes the level of the predetermined interface.

4. The device according to claim 2, wherein: The deserializer comprises: a first interface for connecting to a second interface of the first serializer; and a second interface for connecting to a second interface of a second serializer, The predetermined interface of the deserializer is the second interface of the deserializer.

5. The device according to claim 4, wherein: The first interface of the deserializer and the second interface of the first serializer are both I2C interfaces, The first interface of the first serializer and the first interface of the second serializer are both GMSL interfaces.

6. The device according to claim 1, wherein: The deserializer is configured as: receiving an initialization command from the second interface of the first serializer; and Perform initialization according to the received initialization command.

7. The device according to claim 1, wherein: The deserializer includes an interface for connecting to an image acquisition device, The deserializer is configured as: receiving an initialization command from the second interface of the first serializer; and An initialization command is output from an interface connected to the image acquisition device.

8. The device according to claim 7, wherein: The interface connected to the image acquisition device is the GMSL interface.

9. The device according to claim 1, wherein: The deserializer includes an interface for connecting to an image acquisition device, and the deserializer is configured as follows: Each frame of the image data received from the interface connected to the image acquisition device is sent to each of the first serializer and the second serializer.

10. The device according to claim 1, wherein: Each of the first serializer and the second serializer includes a third interface for connecting to a deserializer, and the third interfaces of the first serializer and the second serializer are both mipi interfaces.

11. A control system comprising: Image acquisition device; a plurality of control devices, wherein the plurality of control devices comprises a first control device and a second control device; and A data distribution device, comprising a deserializer and a plurality of serializers, and configured to transmit the image data acquired by the image acquisition device to the plurality of control devices, wherein: The plurality of serializers include a first serializer and a second serializer, The first control device is configured to initialize the first serializer, The second control device is configured to initialize the second serializer, The second serializer is configured to, in response to completion of initialization, notify the first control device through the deserializer and the first serializer that the second serializer has completed initialization.

12. The system according to claim 11, wherein: The image acquisition device is a first image acquisition device, the deserializer is a first deserializer, and the plurality of serializers are a first plurality of serializers, The system further comprises: a second image acquisition device; a second deserializer connected to the second image acquisition device; a second plurality of serializers coupled to the second deserializer; The first control device and the second control device are further configured to respectively initialize different serializers in the second group of multiple serializers.

13. The system according to claim 11, wherein: The second serializer is further configured to, in response to completion of initialization, notify the first control device through the deserializer and the first serializer that the second serializer has completed initialization, including the second serializer being configured as: In response to the second serializer completing initialization, a state of the predetermined interface of the deserializer is modified through the predetermined interface of the second serializer.

14. The system according to claim 11, wherein: The first control device is configured to initialize the image acquisition device in response to receiving the notification from the second serializer.

15. The system of claim 14, wherein: The first control device is further configured to initialize the deserializer through the first serializer.

16. The system of claim 15, wherein: The first control device is configured to initialize the image acquisition device in response to receiving a notification from the second serializer, including the first control device being configured to: After the first serializer and the deserializer are initialized, the image acquisition device is initialized in response to receiving a notification from the second serializer.

17. A method for configuring a control system, in, The control system comprises: Image acquisition device; a plurality of control devices, wherein the plurality of control devices comprises a first control device and a second control device; and; a data distribution device, the data distribution device comprising a deserializer and a plurality of serializers, and configured to transmit the image data acquired by the image acquisition device to each of the plurality of control devices, wherein the plurality of serializers comprises a first serializer and a second serializer, The method comprises: The first control device initializes the first serializer; The second control device initializes the second serializer; and In response to completing the initialization, the second serializer notifies the first control device through the deserializer and the first serializer that the second serializer has completed the initialization.

18. The method according to claim 17, wherein: In response to the completion of initialization, the second serializer notifies the first control device through the deserializer and the first serializer that the second serializer has completed initialization, including: In response to completing the initialization, the second serializer modifies the state of the predetermined interface of the deserializer through the predetermined interface of the second serializer.

19. The method according to claim 18, wherein: The state of the predetermined interface of the deserializer includes the level of the predetermined interface of the deserializer.

20. The method of claim 17, further comprising: In response to receiving the notification from the second serializer, the first control device initializes the image acquisition device.

21. The method according to claim 20, further comprising: The first control device initializes the deserializer through the first serializer.

22. The method according to claim 21, wherein: The first control device initializes the image acquisition device in response to receiving the notification from the second serializer, including: After initializing the first serializer and the deserializer, the first control device initializes the image acquisition device in response to receiving a notification from the second serializer.

Images