Communication device and communication system

Abstract

One aspect of the present disclosure relates to a communication device, including: a determination unit configured to determine the feasibility of a retransmission request during retransmission control based on priorities assigned to each region within a frame; and a transmission unit configured to generate retransmission request data and transmit the retransmission request data to another communication device if the determination unit permits the retransmission request.

Description

Technical Field

[0001] The present disclosure relates to a communication device and a communication system. Background Art

[0002] In recent years, in order to implement automatic vehicle driving technology, various sensors are installed in a vehicle. For example, an image sensor including a complementary metal oxide semiconductor (CMOS) image sensor (CIS) or a distance sensor using time of flight (TOF) is included. In addition, it has been considered to use the camera serial interface (CIS)-2 standard for communication between these sensors and an application processor (AP: application processor).

[0003] In addition, Patent Document 1 proposes a system that can reduce the number of data buses when coupling a processing device and a plurality of image sensors by using the CIS-2 standard.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2017-211864 Summary of the Invention

[0007] Incidentally, in a case where retransmission of sensor data frequently occurs due to deterioration of transmission path characteristics in communication between various sensors and an AP, there is a possibility that retransmission exceeding the transmission start timing of the next frame may occur. In such a case, there is a concern that the delay may extend to the next frame and thereafter, and thus it may not be possible to maintain a desired frame rate. Therefore, it is desirable to provide a communication device and a communication system capable of preserving a desired frame rate.

[0008] A communication device according to a first aspect of the present disclosure includes: a determination unit that determines the admissibility of a retransmission request based on priorities specified for each region within a frame in retransmission control; and a transmission unit that generates retransmission request data and transmits it to another communication device when the retransmission request is permitted in the determination unit.

[0009] In the communication device according to the first aspect of the present disclosure, in retransmission control, the admissibility of a retransmission request is determined based on priorities specified for each region within a frame, and when the retransmission request is permitted, retransmission request data is generated and the retransmission request data is transmitted to another communication device. This makes it possible to prevent the delay from extending to the next frame and subsequent frames.

[0010] A communication device according to a second aspect of the present disclosure includes: a receiving unit that receives retransmission request data in retransmission control; a transmitting unit that transmits data to another communication device; a storage unit that stores the data transmitted from the transmitting unit as transmitted data; and a control unit that performs retransmission control of the transmitted data based on the retransmission request data.

[0011] In the communication device according to the second aspect of the present disclosure, when retransmission request data is received in retransmission control, retransmission control of the transmitted data stored in the storage unit is performed based on the retransmission request data. This makes it possible to prevent the delay from extending to the next frame and subsequent frames.

[0012] A communication device according to a third aspect of the present disclosure includes: a determination unit that determines the retransmissibility of a transmitted packet based on priorities specified for each region within a frame in retransmission control; and a transmitting unit that transmits the transmitted packet to another communication device when the determination unit permits retransmission of the transmitted packet.

[0013] In the communication device according to the third aspect of the present disclosure, in retransmission control, the retransmissibility of a transmitted packet is determined based on priorities specified for each region within a frame, and when retransmission of the transmitted packet is permitted, the transmitted packet is transmitted to another communication device. This makes it possible to prevent the delay from extending to the next frame and subsequent frames.

[0014] A communication system according to a fourth aspect of the present disclosure includes a transmitting device and a receiving device, and the transmitting device and the receiving device communicate with each other. In this communication system, the receiving device includes: a determination unit that determines the permissibility of a retransmission request based on priorities specified for each region within a frame in retransmission control; and a first transmitting unit that generates retransmission request data and transmits it to the transmitting device when the determination unit permits the retransmission request. In this communication system, the transmitting device includes: a receiving unit that receives the retransmission request data; a second transmitting unit that transmits data to the receiving device; a storage unit that stores the data transmitted from the second transmitting unit as transmitted data; and a control unit that performs retransmission control of the transmitted data based on the retransmission request data.

[0015] In the communication system according to the fourth aspect of the present disclosure, in retransmission control, the permissibility of a retransmission request is determined based on priorities specified for each region within a frame, and when the retransmission request is permitted, retransmission request data is generated and transmitted from the transmitting device to the receiving device. This makes it possible to prevent the delay from extending to the next frame and subsequent frames.

[0016] A communication system according to a fifth aspect of the present disclosure includes a transmitting device and a receiving device, and the transmitting device and the receiving device communicate with each other. In this communication system, the receiving device includes a first transmitting unit that generates retransmission request data and transmits it to the transmitting device in retransmission control. In this communication system, the transmitting device includes: a receiving unit that receives the retransmission request data; a second transmitting unit that transmits data to the receiving device; a storage unit that stores the data transmitted from the second transmitting unit as transmitted data; and a determination unit that determines the retransmissibility of the transmitted data based on the retransmission request data and the priority assigned to each region within a frame. The second transmitting unit transmits the transmission packet to the receiving device when the determination unit permits the retransmission of the packet.

[0017] In the communication system according to the fifth aspect of the present disclosure, in retransmission control, the retransmissibility of a transmitted packet is determined based on the priority assigned to each region within a frame, and the transmitted packet is transmitted from the transmitting device to the receiving device in the case of permitting or not permitting the retransmission of the transmitted packet. This makes it possible to prevent the delay from extending to the next frame and subsequent frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic configuration examples of communication systems according to embodiments of the present disclosure are shown.

[0019] Figure 2 Shows in Figure 1 Examples of frame formats and packet formats used in communication according to the A-PHY standard in the communication system of.

[0020] Figure 3 Examples of functional blocks of communication devices in a sensor unit and a control unit are shown.

[0021] Figure 4 Shows in Figure 1 Examples of retransmission processes when an error occurs in the communication system of.

[0022] Figure 5 Shows in Figure 1 Examples of retransmission processes when an error occurs in the communication system of.

[0023] Figure 6 Shows in Figure 1 Examples of retransmission processes when an error occurs in the communication system of.

[0024] Figure 7 Shows Figure 3 Examples of functional blocks of a retransmission controller in.

[0025] Figure 8 Shows Figure 3 Examples of functional blocks of a transmission buffer in.

[0026] Figure 9 Show Figure 3 An example of a functional block of a transmission scheduler in

[0027] Figure 10 Show an example of the priority of retransmission within a frame.

[0028] Figure 11 Show an example of the setting of a register.

[0029] Figure 12 Show an example of the priority of retransmission within a frame.

[0030] Figure 13 Show an example of the setting of a register.

[0031] Figure 14 Show an example of the priority of retransmission within a frame.

[0032] Figure 15 Show an example of the setting of a register.

[0033] Figure 16 Show Figure 1 An example of retransmission control in a communication system in

[0034] Figure 17 Show Figure 1 An example of retransmission control in a communication system in

[0035] Figure 18 Show Figure 3 A modified example of the functional block of a retransmission controller in

[0036] Figure 19 Show Figure 3 A modified example of the functional block of a transmission buffer in

[0037] Figure 20 Show an example of the schematic configuration of an imaging system.

[0038] Figure 21 Is a block diagram showing an example of the schematic configuration of a vehicle control system.

[0039] Figure 22 Is a diagram helping to illustrate an example of the installation positions of an outside-vehicle information detection unit and an imaging unit. Detailed Description of the Invention

[0040] Hereinafter, modes for implementing the present disclosure will be described in detail with reference to the drawings. The following description gives specific examples of the present disclosure, and the present disclosure is not limited to the following embodiments.

[0041] <Example of the Configuration of a Communication System>

[0042] Figure 1 FIG. 1 shows a schematic configuration example of a communication system 1 according to an embodiment of the present disclosure. For example, the communication system 1 is installed in a vehicle such as an automobile, and includes a sensor unit 10 and a control unit 20. The sensor unit 10 and the control unit 20 are coupled to each other via a bus 30. A-PHY (as a standard for the physical layer developed by the Mobile Industry Processor Interface (MIPI) Alliance) is used for signal transmission between the sensor unit 10 and the control unit 20 via the bus 30. It is assumed that A-PHY corresponds to a transmission distance of, for example, up to 15 m.

[0043] For example, the sensor unit 10 includes an image sensor 11 (CIS) and a communication device 12. The image sensor 11 is coupled to the communication device 12 via a bus according to the I3C standard. For example, the control unit 20 includes a communication device 21 and an application processor 22 (AP). The application processor 22 is coupled to the communication device 21 via a bus according to the I3C standard. The communication device 12 and the communication device 21 communicate with each other via the bus 30 according to the A-PHY standard.

[0044] Figure 2 FIG. 2 shows examples of a frame format and a packet format to be used in communication according to the A-PHY standard in the communication system 1.

[0045] For example, in the frame format of a frame for transmitting an image, for each row of the image, a packet storing the data of that row is generated; or, for each data obtained by dividing that row multiple times, a packet storing the data is generated between a frame start (FS: Frame Start) indicating the start of the frame and a frame end (FE: Frame End) indicating the end of the frame. Further, for example, in the frame format of a frame for transmitting an image, for each plurality of pixels included in the image, a packet storing the data of the plurality of pixels is generated between a frame start (FS) indicating the start of the frame and a frame end (FE) indicating the end of the frame.

[0046] In the packet format of A-PHY, a packet header is set in the header and a packet tail is set in the tail. Then, a payload storing data is arranged between the packet header and the packet tail. The packet tail includes, for example, PHY CRC-32. For example, the packet header includes an adaptation descriptor, a service descriptor, a placement descriptor, PHY2, a destination address, PHY3, a payload length, and a PHY header CRC. The adaptation descriptor includes, for example, an adaptation type value (AdaptationTypeValue). Examples of the adaptation type value include I 2 , ,

[0047] C, I3C, GPIO, etc.

[0047] Furthermore, the packet header includes, for example, a message count (MC: Message Count) for differentiating packets. As the message count, the transmission count that increments each time a packet is sent is used. For example, in addition to the message count, the packet header includes various types of information required to apply the communication system 1 to a vehicle such as an automobile. The communication system 1 uses the message count included in the packet header to individually differentiate packets, thereby enabling retransmission for each packet.

[0048] <Functional blocks of communication devices 12 and 21>

[0049] Figure 3 An example of the functional blocks of communication devices 12 and 21 is shown.

[0050] In the communication system 1, for example, the communication device 12 and the communication device 21 transmit and receive control data to and from each other via a low-speed and two-way control channel. Furthermore, in the communication system 1, the communication device 12 transmits sensor data such as images to the communication device 21 via a high-speed and one-way data channel.

[0051] The communication device 12 includes, for example, a transmitter 51, a receiver 52, an error detection code calculator 53, an error detector 54, a control data discriminator 55, a transmit buffer 56, a transmit scheduler 57, an error detection code calculator 58, and a transmitter 59. Each block arranged above the virtual line shown in the communication device 12 is for the processing of transmitting and receiving control data, and each block arranged below the virtual line is for the processing of transmitting sensor data.

[0052] The communication device 21 includes, for example, a transmitter 61, a receiver 62, an error detection code calculator 63, an error detector 64, a transmit / receive scheduler 65, a receiver 66, an error detector 67, a retransmission detector 68, and a retransmission controller 69. Each block arranged above the virtual line shown in the communication device 21 is for the processing of transmitting and receiving control data, and each block arranged below the virtual line is for the processing of receiving sensor data.

[0053] The transmitter 51 transmits the control data provided from the error detection code calculator 53 to the communication device 21 via the control channel. The receiver 52 receives the control data transmitted from the communication device 21 via the control channel, and provides the received control data to the error detector 54. The error detection code calculator 53 calculates a parity bit or a cyclic redundancy check (CRC), for example, as an error detection code for the control data provided via the control data discriminator 55. Then, the error detection code calculator 53 adds the calculated error detection code to the control data, and provides it to the transmitter 51.

[0054] The error detector 54 detects whether an error has occurred in the control data by using an error detection code added to the control data provided from the receiver 52. Then, when no error has occurred in the control data, the error detector 54 provides the control data as it is to the control data discriminator 55 according to the result of error detection in the control data. At the same time, when an error has occurred in the control data, the error detector 54 notifies that an error has occurred in the control data according to the result of error detection in the control data, and supplies the control data to the control data discriminator 55.

[0055] For example, the control data discriminator 55 receives the input of the control data and outputs it to the control circuit (not shown) of the control image sensor 11, and discriminates the control data transmitted from the communication device 21. That is, the control data discriminator 55 provides the control data input from the control circuit to the error detection code calculator 53, and outputs the control data provided from the error detector 54 to the control circuit; at this time, the control data discriminator 55 discriminates the control data. For example, in the case of transmitting the retransmission request data described later, as the control data provided from the error detector 54, from the communication device 21, the control data discriminator 55 detects the retransmission request data, and provides the detected retransmission request data to the transmission buffer 56.

[0056] The transmission packet to be transmitted from the communication device 12 to the communication device 21 is supplied from the transmission scheduler 57 to the transmission buffer 56. Then, the transmission buffer 56 temporarily stores the transmission packet (transmission data) provided from the transmission scheduler 57 as a transmission packet (transmitted transmission data). In addition, when the retransmission request data is provided from the control data discriminator 55, the transmission buffer 56 appropriately provides the transmitted packet (transmitted transmission data) among the stored transmitted packets (transmitted transmission data) that the retransmission request data requests to retransmit as a retransmission packet (retransmission data) to the transmission scheduler 57. It should be noted that the sensor data stored in the payload of the transmission packet is also appropriately referred to as transmission data hereinafter. The sensor data stored in the payload of the retransmission packet is also appropriately referred to as retransmission data hereinafter.

[0057] Sensor data is input from an imaging element (not shown) of the image sensor 11 to the transmission scheduler 57. Here, a packet stores, in the payload, sensor data newly input from the image sensor 11 to the transmission scheduler 57. That is, the number of sensors that store, in the payload of the packet, sensor data that is not retransmission data is appropriately referred to as a new packet. Further, as described later, the transmission scheduler 57 exchanges a retransmission data transmission request and a retransmission data transmission permission with the transmission buffer 56, and provides a retransmission packet from the transmission buffer 56. Then, the transmission scheduler 57 adjusts the corresponding transmission timing of the new packet or the retransmission packet, and sequentially provides the packet as a transmission packet to be transmitted from the communication device 12 to the communication device 21 to the transmission buffer 56 and the error detection code calculator 58.

[0058] The error detection code calculator 58 calculates a parity bit or a CRC, for example, as a code for error detection in the transmission packet supplied from the transmission scheduler 57, adds the code for error detection to the packet, and supplies it to the transmitter 59. The transmitter 59 transmits the transmission packet provided from the error detection code calculator 58 to the communication device 21 via a data channel.

[0059] The transmitter 61, the receiver 62, the error detection code calculator 63, and the error detector 64 are respectively configured similarly to the transmitter 51, the receiver 52, the error detection code calculator 53, and the error detector 54.

[0060] For example, the transmission / reception scheduler 65 receives an input of control data and outputs it from a control circuit (not shown) of the control application processor 22 to the control circuit, and manages the scheduling of control data to be transmitted to and received from the communication device 12. That is, the transmission / reception scheduler 65 outputs the control data provided from the error detector 64 to the control circuit, and provides the control data input from the control circuit to the error detection code calculator 63 at a predetermined timing for transmission to the communication device 12. Further, when retransmission request data is provided from the retransmission controller 69, the transmission / reception scheduler 65 provides the retransmission request data as control data to the error detection code calculator 63 for transmission to the communication device 12.

[0061] The receiver 66 receives the transmission packet transmitted from the communication device 12 via a data channel, and provides the received transmission packet to the error detector 67.

[0062] The error detector 67 uses the error detection code added to the transmission packet provided from the receiver 66 to detect whether an error has occurred in the transmission packet. Then, the error detector 67 uses the error detection code added to the transmission packet to detect whether an error has occurred in the transmission packet, and outputs the sensor data stored in the payload of the transmission packet to the processing circuit at a subsequent stage.

[0063] Further, the error detector 67 provides the retransmission detector 68 with an error detection result indicating an error that has occurred in the transmitted packet. For example, when it has been detected that a header error or a payload error has occurred in the transmitted packet received by the communication device 21, the error detector 67 provides the retransmission detector 68 with an error detection result indicating the detected header error or payload error. In addition, the error detector 67 provides the retransmission detector 68 with the message count included in the additional packet header of the transmitted packet received by the communication device 21.

[0064] The retransmission detector 68 performs a retransmission detection process to detect the message count of the transmitted packet that has not been received by the communication device 21 and needs to be retransmitted based on the error detection result and the message count provided from the error detector 67. Then, the retransmission detector 68 detects the message count detected by the retransmission detection process as a retransmission candidate message count, which is a candidate for requesting retransmission, and provides the detected message count to the retransmission controller 69. It should be noted that the retransmission detection process to be performed by the retransmission detector 68 will be described later.

[0065] Based on the retransmission candidate message count provided from the retransmission detector 68 and various types of information transmitted from the communication device 12, the retransmission controller 69 performs a retransmission request process to generate retransmission request data for requesting the transmission of a retransmitted packet and provides the generated retransmission request data to the transmission / reception scheduler 65. In addition, the retransmission request process of the retransmission detection unit 68 will be described later.

[0066] Here, the communication system 1 embeds various types of information required for the retransmission controller 69 to perform the retransmission request process into the header of the transmitted packet that stores sensor data in the payload, and transmits the embedded various types of information from the communication device 12 to the communication device 21. For example, the communication device 12 uses the user-defined area of the packet header specified by CSI-2 to transmit various types of information required for the retransmission controller 69 to perform the retransmission request process, and provides the information to the retransmission controller 69 via the error detector 67.

[0067] According to the above configurations of the communication device 12 and the communication device 21, it is possible to implement retransmission for each packet by using the message count included in the packet header specified in A-PHY. That is, the communication device 12 and the communication device 21 perform retransmission control of MIPI A-PHY.

[0068] ]Next, with reference to Figure 4 the flowchart shown in Figure 4 a schematic description of the packet retransmission process to be performed in the communication system 1 will be given.

[0069] In step S101, the retransmission detector 68 of the communication device 21 performs a retransmission detection process. For example, based on the error detection result and the message count provided by the error detector 67, the retransmission detector 68 detects the message count of the transmitted packet that the communication device 21 fails to receive and needs to retransmit.

[0070] In step S102, the retransmission controller 69 of the communication device 21 performs a retransmission request process. For example, the retransmission controller 69 creates a retransmission list that registers the message count of the packet for which retransmission is requested, while performing filtering of the oldest message count stored in the transmission buffer 56 among the retransmission candidate message counts provided by the retransmission detector 68. This allows the retransmission list to register a message count that is newer than the earliest message count stored in the transmission buffer 56 among the retransmission candidate message counts. Then, the retransmission control unit 69 determines the admissibility of the retransmission request based on the priority specified for each region within the frame, and generates retransmission request data when the retransmission request is allowed. Specifically, the retransmission controller 69 determines the admissibility of the retransmission request based on the priority specified for each region within the frame, the retransmittable frequency specified for each region, and the number of retransmissions counted for each region, and generates retransmission request data when the retransmission request is allowed.

[0071] In step S103, the control data discriminator 55 and the transmission buffer 56 of the communication device 12 perform a retransmission acceptance process. For example, the control data discriminator 55 discriminates the retransmission request data generated by the retransmission controller 69 from the control data provided by the error detector 54, and provides the packet for which retransmission is requested to the transmission buffer 56. Then, when the transmitted packet saved is searched using the message count specified by the retransmission request data and the transmitted packet for which retransmission is requested is obtained as the search result, the transmission buffer 56 saves the transmitted packet as a retransmission packet. At the same time, in the case where the transmitted packet for which retransmission is requested cannot be obtained as the search result, the transmission buffer 56 discards the retransmission request data used for the search.

[0072] In step S104, the transmission scheduler 57 of the communication device 12 performs a retransmission data transmission process. For example, the transmission scheduler 57 adjusts the timing of transmission of the retransmission packet ensured by the transmission buffer 56 and the new packet that stores newly input sensor data in the payload, and appropriately transmits the new packet or the retransmission packet as a transmission packet.

[0073] As described above, the communication system 1 can request retransmission for each transmitted packet sent from the communication device 12 to the communication device 21, and perform the transmission of the retransmission packet for which retransmission has been requested.

[0074] <Example of the retransmission detection process>

[0075] Refer toFigure 5 and Figure 6 give a description of the retransmission detection process in communication system 1. Figure 5 and Figure 6 both show examples of the retransmission detection process when a payload error occurs in communication according to the A-PHY standard in communication system 1. Figure 5 and Figure 6 both show the retransmission detection process triggered by receiving the message count included in the header, and the numbers (1 to 6) enclosed by the rectangular box shown in Figure 6 represent the message count. Figure 5 and Figure 6 both show the state where payload errors have occurred in the transmitted packets with message counts 2, 3, and 5.

[0076] When receiving the transmitted packet with message count 2, communication device 21 checks the header of the transmitted packet to obtain the message count 2 included in the header. At this time, in the case where a payload error has occurred in the transmitted packet, communication device 21 performs a predetermined error detection process to identify that a payload error has occurred in the transmitted packet with message count 2.

[0077] Communication device 21 determines the admissibility of a retransmission request based on the priority specified for each area within the frame, and generates retransmission request data when the retransmission request is allowed. Specifically, communication device 21 determines the admissibility of the retransmission request based on the priority specified for each area within the frame, the number of retransmissions specified for each area, and the number of retransmissions counted for each area, and generates retransmission request data when the retransmission request is allowed. Communication device 21 sends the generated retransmission request data to communication device 12. When receiving the retransmission request data from communication device 21, communication device 12 ensures the transmitted packet corresponding to the message count 2 specified by the received retransmission request data as a retransmission packet. Communication device 12 sends the ensured retransmission packet as a transmitted packet to communication device 21 at a predetermined timing. In addition, when a payload error occurs in the transmitted packet with message count 3 or message count 5, a similar process to the above is performed. In addition, a similar process to the above is also performed when a header error has occurred in the transmitted packet.

[0078] <Configuration example of retransmission controller 69 and processing example of retransmission request processing>

[0079] Next, a configuration example of retransmission controller 69 and a processing example of retransmission request processing are described.

[0080] Figure 7 shows an example of the functional blocks of retransmission controller 69. As Figure 7As shown, for example, the retransmission controller 69 includes a retransmission list storage unit 71, a retransmission frequency storage unit 72, a timing control unit 73, and a retransmission request data generation unit 74.

[0081] Here, various types of information are provided to the retransmission controller 69. For example, when the information is embedded in the header of a transmitted packet, it is sent. Examples of this information include information indicating the oldest message count of the transmission packet buffer 84 of the transmission buffer 56 ( Figure 10 ).

[0082] The retransmission list storage unit 71 stores a retransmission list of the message counts of the packets for which retransmission is requested, and manages this retransmission list. For example, the retransmission list storage unit 71 manages the retransmission list so as to register only the message counts that are newer than the earliest message count stored in the transmission buffer 56 among the retransmission candidate message counts detected by the retransmission detector 68.

[0083] For example, when message counts 98 to 102 are registered in the retransmission list according to the retransmission candidate message counts, it is assumed that message count 100 is newly supplied as the oldest message count of the transmission packet buffer 84 of the transmission buffer 56. In this case, the retransmission list storage unit 71 removes message counts 98 and 99 that are older than message count 100 from the retransmission list, so that only message counts 100 to 102 that are newer than message count 100 are registered in the retransmission list.

[0084] The retransmission frequency storage unit 72 includes a register ( Figure 9 ) in which data corresponding to the concept shown in Figure 8 is stored. Figure 8 An example is given of a state in which priorities are set for each predetermined area within a frame (one image data). In the frame (one image data) of Figure 8 , priorities are set for each predetermined number of rows from top to bottom. For example, a priority of "low" is set for each line of a row N1 starting from the topmost layer (first area); a priority of "medium" is set for each line of a row N2 starting from the line where the priority "low" is set (second area); a priority of "high" is set for each line of N3 rows starting from the line where the priority "medium" is set (third area), and a priority of "medium" is set for each line of N4 rows starting from the line where the priority "high" is set (fourth area). As Figure 9As shown, for example, in a register, for the first region, the retransmission frequency RN corresponding to the priority is set to N5; for the second region, the retransmission frequency RN corresponding to the priority is set to N6; for the third region, the retransmission frequency RN corresponding to the priority is set to N7; for the fourth region, the retransmission frequency RN corresponding to the priority is set to N8. N5 + N6 + N7 + N8 is equivalent to the retransmission frequency Nmax (N maximum) per frame (one image data).

[0085] From the perspective of the A-PHY standard, it is necessary to send packets to allow the logical layer not to exceed 97.5% of the effective transmission rate of the physical layer. Therefore, from the perspective of the A-PHY standard, the retransmission frequency Nmax is preferably set to a frequency such that the packets can be sent within a range that allows the logical layer not to exceed 97.5% of the effective transmission rate of the physical layer.

[0086] In the register, various setting values are specified for each region, such as the line number LN, the retransmission frequency RN, the own-region bearability flag CA, and the bearability flag M. In the register, the setting values of each region are recorded in order. For example, in the register, the setting values of the first region, the second region, the third region, and the fourth region are recorded in this order.

[0087] Here, the number of lines LN is the number of pixel lines included in the relevant region. The retransmission frequency RN is the upper limit value of the frequency of possible retransmission requests in the relevant region. The own-region bearability flag CA is a flag as follows: when the frequency (RNa) of the retransmission requests actually made in the relevant region is less than the retransmission frequency RN set in the register for the relevant region, it is set whether to add the value (RN - RNa) obtained by subtracting RNa from RN to the retransmission frequency RN in the next region. The frequency (RNa) of the retransmission requests actually made in each region is recorded in the register.

[0088] When the own-region bearability flag CA is "disabled", RN - RNa is not added to the retransmission frequency RN in the next region. When the own-region bearability flag CA is "enabled", RN - RNa is added to the retransmission frequency RN in the next region. When RN - RNa is added to the retransmission frequency RN, the retransmission frequency RN in this region is a value larger than the retransmission frequency RN set in the register by RN - Rna.

[0089] The bearability flag M is a flag that sets whether the remaining unused retransmission frequency (remaining frequency RNb) is added to the retransmission frequency RN in the next region, regardless of whether RN - Rna in the region before its own region is added to the retransmission frequency RN in its own region. When the bearability flag M is "disabled", the remaining frequency RNb is not added to the retransmission frequency RN in the next region. When the bearability flag M is "enabled", the remaining frequency RNb is added to the retransmission frequency RN in the next region.

[0090] Based on the priorities set for each region within a frame (image data), the timing control unit 73 performs control to retransmit the transmitted packet (hereinafter referred to as "transmitted packet X") corresponding to the message count in the retransmission list registered in the retransmission list storage unit 71 as a retransmission packet. Specifically, the timing control unit 73 determines whether to request the transmission / reception scheduler 65 to send a retransmission request for retransmitting the transmitted packet X as a retransmission packet based on various settings set in the register for the region (hereinafter referred to as "region α") corresponding to the message count in the retransmission list registered in the retransmission list storage unit 71 and the frequency of the retransmission request (RNa) actually made in region α.

[0091] For example, when the frequency of the retransmission request (RNa) actually made in region α does not exceed the retransmission frequency RN set in the register for region α, the timing control unit 73 determines to permit the transmission request for the retransmission request and requests the transmission / reception scheduler 65 to send a retransmission request for retransmitting the transmitted packet X as a retransmission packet. However, when the bearability flag M for region α is set to "enabled" in the register and when the frequency of the retransmission request (RNa) actually made in region α does not exceed the frequency obtained by adding the remaining frequency RNb to the retransmission frequency RN set in the register for region α, for example, the timing control unit 73 determines to permit the transmission request for the retransmission request and requests the transmission / reception scheduler 65 to send a retransmission request for retransmitting the transmitted packet X as a retransmission packet. Additionally, the timing control unit 73 determines to permit the transmission request for the retransmission request and requests the transmission / reception scheduler 65 to send a retransmission request for retransmitting the transmitted packet X as a retransmission packet when, for example, the frequency of the retransmission request actually made within the frame (image data) does not exceed the retransmittable frequency Nmax specified for the frame (image data).

[0092] When obtaining transmission permission for the transmission request issued by the timing control unit 73 from the transmission / reception scheduler 65, the retransmission request data generation unit 74 generates retransmission request data for requesting retransmission of the transmitted packet corresponding to the message count read from the retransmission list of the retransmission list storage unit 71 and provides the generated retransmission request data to the transmission / reception scheduler 65.

[0093] <Configuration example of transmission buffer 56 and processing example of retransmission reception processing>

[0094] Next, a configuration example of the transmission buffer 56 and a processing example of retransmission reception processing will be described.

[0095] Figure 10 An example of a functional block of the transmission buffer 56 is shown. For example, as Figure 10 shown, the transmission buffer 56 includes a write control unit 81, a transfer control unit 82, a read control unit 83, a transmission packet buffer 84, and a retransmission packet buffer 85.

[0096] The write control unit 81 executes write control to write the transmitted packet supplied from the transmission scheduler 57 into the transmission packet buffer 84.

[0097] The transfer control unit 82 searches for the transmitted packet with the message count specified by the retransmission request data provided from the control data discriminator 55 among the transmitted packets stored in the transmission packet buffer 84. Then, when the transmitted packet with the message count specified by the retransmission request data can be obtained as a search result, the transfer control unit 82 executes transfer control to transfer the transmitted packet to the retransmission packet buffer 85. It should be noted that when the transmitted packet with the message count specified by the retransmission request data cannot be obtained as a search result, the transfer control unit 82 discards the retransmission request data.

[0098] The read control unit 83 manages the address of the retransmission packet stored in the retransmission packet buffer 85, and issues a retransmission data transmission request for requesting the transmission scheduler 57 to transmit the retransmission packet stored in the retransmission packet buffer 85. Then, when a retransmission data transmission permission is obtained from the transmission scheduler 57, the read control unit 83 executes read control to read the retransmission packet from the retransmission packet buffer 85, and provides the read retransmission packet to the transmission scheduler 57. The transmission packet buffer 84 stores transmitted packets, and the retransmission packet buffer 85 stores retransmission packets.

[0099] <Configuration example of transmission scheduler 57>

[0100] Next, a configuration example of the transmission scheduler 57 will be described.

[0101] Figure 11 An example of a functional block of the transmission scheduler 57 is shown. The transmission scheduler 57 includes, for example, a sensor data buffer 91, a switching unit 92, and a control unit 93, as Figure 11 shown.

[0102] For example, the sensor data buffer 91 temporarily stores a new packet, which stores sensor data newly input from an imaging element (not shown) of the image sensor 11 in the payload. For example, the sensor data buffer 91 is necessary because it is assumed that a new packet storing sensor data in the payload is input during the output of a retransmission packet provided from the transmission buffer 56.

[0103] Under the control of the control unit 93, the switching unit 92 switches between a new packet stored in the sensor data buffer 91 and a retransmission packet provided from the transmission buffer 56, and outputs a packet from the transmission scheduler 57. The control unit 93 controls the switching of the output of the switching unit 92 when acquiring a retransmission data transmission request issued from the transmission buffer 56. For example, when acquiring a retransmission data transmission request issued from the transmission buffer 56, the control unit 93 switches the output of the switching unit 92 to the retransmission data side.

[0104] <Effect>

[0105] Next, the effect of the communication system 1 according to the present embodiment will be described.

[0106] In the present embodiment, based on the priority specified for each region within a frame in the retransmission control, the admissibility of a retransmission request is determined, and when the retransmission request is allowed, retransmission request data is generated and sent to other communication devices. This makes it possible to prevent the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0107] In the present embodiment, in the register provided in the retransmission frequency storage unit 72, a retransmittable frequency corresponding to the priority is specified for each region within a frame. Thus, based on the retransmittable frequency specified for each region within a frame, the admissibility of a retransmission request is determined, and when the retransmission request is allowed, retransmission request data is generated and sent to other communication devices. This makes it possible to prevent the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0108] In the present embodiment, when the frequency of actually requesting retransmission in each region does not exceed the retransmittable frequency, it is determined that the retransmission request is allowed. This makes it possible to prevent the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0109] In this embodiment, regions are set for each predetermined number of rows in a frame. Here, for example, it is assumed that the image sensor 11 is provided in a moving body such as an automobile, and image data is obtained by imaging the front part of the moving body with the image sensor 11. At this time, the lower region in the image data may reflect the hood of the automobile, the middle region in the image data may reflect the road in front of the automobile; and the upper region in the image data may reflect the sky in front of the automobile. In this case, the importance of the middle region in the image data is higher than that of other regions, and thus regions within the frame are set for each predetermined number of rows in the frame so that retransmission of regions of high importance in the image data can be ensured while maintaining the desired frame rate.

[0110] <Modification Example>

[0111] [Modification Example A]

[0112] In the above embodiment, priorities are set for each predetermined number of rows from the top to the bottom within a frame (one image data). However, in the above embodiment, for example, as Figure 12 shown, priorities can be set for each predetermined number of columns from the left to the right within a frame (one image data). In this case, in the frame format for transmitting one frame of an image, for each column of the image, packets storing the data of that column are generated; or for each data obtained by dividing a column multiple times, packets storing that data are generated between the frame start (FS) indicating the start of the frame and the frame end (FE) indicating the end of the frame.

[0113] In Figure 12 the frame (one image data), priorities are set for each predetermined number of columns starting from the left. For example, a priority of "low" is set for each column of N1 columns starting from the left (first region); a priority of "high" is set for each column of N2 columns starting from the column where the priority "low" is set (second region); a priority of "medium" is set for each column of N3 columns starting from the column where the priority "high" is set (third region), and a priority of "low" is set for each column of N4 columns starting from the column where the priority "medium" is set (fourth region). For example, for the first region, the retransmission frequency RN is set to N5; for the second region, the retransmission frequency RN is set to N6; for the third region, the retransmission frequency RN is set to N7; for the fourth region, the retransmission frequency RN is set to N8. N5 + N6 + N7 + N8 corresponds to the retransmission frequency Nmax per frame (one image data).

[0114] In the register, for example, various setting values are specified for each region, such as the column number CN, the retransmission frequency RN, the own-region bearability flag CA, and the bearability flag M, as Figure 13As shown. In the register, the set values of each region are recorded in order. For example, in the register, the set value of the first region, the set value of the second region, the set value of the third region, and the set value of the fourth region are recorded in this order. The column number CN is the number of pixel columns included in the relevant region.

[0115] In this modification, regions are set for each predetermined column number in a frame. Here, for example, it is assumed that the image sensor 11 is provided in a moving body such as an automobile, and image data is obtained by imaging the left front of the moving body with the image sensor 11. At this time, the right end region in the image data can reflect the vehicle body and pillars of the automobile, the middle region in the image data can reflect the road in front of the left automobile; the left end region in the image data can reflect the road shoulder or guardrail. In this case, the importance of the middle region in the image data is higher than that of other regions, and thus regions within the frame are set for each predetermined column number in the frame so that retransmission of the high-importance region in the image data can be ensured while maintaining the desired frame rate.

[0116] [Modification B]

[0117] In the foregoing embodiment, priorities are set for each predetermined row number from the top to the bottom within a frame (one image data). However, in the above embodiment, for example, as Figure 14 shown, priorities can be set for each predetermined plurality of pixels in a frame (one image data). In this case, in the frame format for transmitting one frame of an image, for each plurality of pixels included in the image, between the frame start (FS) indicating the start of the frame and the frame end (FE) indicating the end of the frame, a packet storing data of the plurality of pixels is generated.

[0118] In Figure 14In a frame (one image data), priorities are set for each of a predetermined plurality of pixels. For example, a priority of "high" is set for an inverted V-shaped region (first region); a priority of "medium" is set for a region (second region) adjacent to the lower part of the region (first region) where the priority of "high" is set; a priority of "low" is set for each region (third region) adjacent to the upper right part and the upper left part of the region (first region) where the priority of "high" is set; and a priority of "ultra-low" is set for a region (fourth region) adjacent to the upper part of each region (third region) where the priority of "low" is set and for a region (fourth region) adjacent to the lower part of the region (second region) where the priority of "medium" is set. For example, for the first region, the retransmission frequency RN is set to N8; for the second region, the retransmission frequency RN is set to N7; for the third region, the retransmission frequency RN is set to N6; and for the fourth region, the retransmission frequency RN is set to N5. N5 + N6 + N7 + N8 corresponds to the retransmission frequency Nmax per frame (one image data).

[0119] In a register, for example, various setting values such as the retransmission frequency RN, the own-region bearability flag CA, and the bearability flag M are specified for each region, as Figure 15 shown. In the register, the setting values of each region are recorded in order. For example, in the register, the setting values of the first region, the setting values of the second region, the setting values of the third region, and the setting values of the fourth region are recorded in this order. In the register, for example, the coordinates for dividing each region are further specified, as Figure 15 shown.

[0120] In this modification example, regions are set for each of a predetermined plurality of pixels in a frame. Here, for example, it is assumed that the image sensor 11 is provided in a moving body such as an automobile, and image data is obtained by imaging the front part of the moving body with the image sensor 11. At this time, there is a possibility that the inverted V-shaped region in the image data may reflect a road extending straight away from the front of the automobile; the region adjacent to the lower part of the inverted V-shaped region may reflect the hood of the automobile; and the regions adjacent to the upper left part and the upper right part of the inverted V-shaped region in the image data may reflect the road shoulder or the guardrail. In this case, the importance of the inverted V-shaped region in the image data is higher than that of other regions, and thus regions within the frame are set for each of a predetermined plurality of pixels in the frame so that retransmission of high-importance regions in the image data can be ensured while maintaining the desired frame rate.

[0121] [Modification Example C]

[0122] In the above-described embodiment and its modification examples, for example, as Figure 16As shown, not only can the retransmission frequency Nmax per frame (one image data) be specified, but also the retransmission frequency per multiple frames can be specified. In this case, the concept of a block is introduced for each multiple frames, and the upper limit value of the retransmission frequency is specified for each block size BS. This makes it possible to prevent the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0123] [Modification Example D]

[0124] In the above-described embodiments and their modification examples, for example, when the frequency of the retransmission request (RNa) actually made in the relevant area exceeds the retransmission frequency RN set in the register in the relevant area, the retransmission controller 69 may send an error flag indicating that RNa exceeds the retransmission frequency RN to the transmission / reception scheduler 65, as Figure 16 and Figure 17 shown. In this case, when the error flag is provided from the retransmission controller 69, the transmission / reception scheduler 65 provides the error flag as control data to the error detection code calculator 63 for transmission to the communication device 12.

[0125] When the communication device 12 receives the error flag as control data from the communication device 21, the control data discriminator 55 provides the error flag to the transmission buffer 56. When the error flag is acquired, the transmission buffer 56 stops providing the retransmission packet to the transmission scheduler 57. When the error flag is received, the communication device 12 further adds an NG flag indicating that retransmission is not allowed to the end of the frame.

[0126] In this way, in this modification example, when RNa exceeds the retransmission frequency RN, an error flag indicating that RNa exceeds the retransmission frequency RN is sent to the communication device 12. This allows the transmission of the retransmission packet to be stopped, thereby making it possible to prevent the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0127] [Modification Example E]

[0128] In the foregoing embodiments and their modification examples, for example, as Figure 18 and Figure 19 shown, the retransmission frequency storage unit 72 may be provided in the transmission buffer 56 instead of being provided in the retransmission controller 69.

[0129] In this case, the timing control unit 73 performs control to retransmit the transmitted packet (hereinafter referred to as "transmitted packet X") corresponding to the message count in the retransmission list registered in the retransmission list storage unit 71 as a retransmission packet. Specifically, the timing control unit 73 requests the transmission / reception scheduler 65 to send a retransmission request to retransmit the transmitted packet X as a retransmission packet.

[0130] Based on the priorities set for each region within a frame (image data), the transmission control unit 82 performs control to retransmit the transmitted packet corresponding to the message count specified by the retransmission request data provided by the control data discriminator 55 (hereinafter referred to as "transmitted packet X") as a retransmitted packet. Specifically, based on various settings set in the register for the region corresponding to the message count specified by the retransmission request data provided by the control data discriminator 55 (hereinafter referred to as "region α") and the frequency of the actual retransmission requests (RNa) made in region α, the transmission control unit 82 determines whether to retransmit transmitted packet X.

[0131] For example, when the frequency of the actual retransmission requests (RNa) made in region α does not exceed the retransmittable frequency RN set in the register for region α, the transmission control unit 82 determines to allow the retransmission of transmitted packet X and performs transmission control to transfer transmitted packet X to the retransmission packet buffer 85. However, for example, when the bearability flag M for region α is set to "enabled" in the register and the frequency of the actual retransmission requests (RNa) made in region α does not exceed the frequency obtained by adding the remaining frequency RNb to the retransmittable frequency RN set in the register for region α, the transmission control unit 82 determines to allow the retransmission of transmitted packet X and performs transmission control to transfer transmitted packet X to the retransmission packet buffer 85. Further, for example, when the frequency of the retransmission requests actually made within the frame (image data) does not exceed the retransmittable frequency Nmax set for the frame (image data), the transmission control unit 82 determines to allow the retransmission of transmitted packet X and performs transmission control to transfer transmitted packet X to the retransmission packet buffer 85.

[0132] When the retransmission of transmitted packet X is allowed, the transmission control unit 82 searches for transmitted packet X from the transmitted packets stored in the transmitted packet buffer 84. Then, when transmitted packet X can be obtained as the search result, the transmission control unit 82 performs transmission control to transfer transmitted packet X to the retransmission packet buffer 85. It should be noted that when the transmitted packet with the message count specified by the retransmission request data cannot be obtained as the search result, the transmission control unit 82 discards the retransmission request data.

[0133] In this modified example, retransmission control using the retransmission frequency storage unit 72 is performed in the communication device 12 (transmission buffer 56). Also in this case, it is possible to prevent the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0134] <3. Application Example>

[0135] Figure 20An example of a schematic configuration of an imaging system 2 showing a communication system 1 according to any one of the above-described embodiments and modifications thereof is shown. The imaging system 2 includes, for example, an optical system 210, a shutter device 220, a communication system 1, a signal processing circuit 230, and a display unit 240.

[0136] The optical system 210 forms an image of image light (incident light) from an object on the imaging surface of the communication system 1 (image sensor 11). The shutter device 220 is provided between the optical system 210 and the imaging system 2, and controls the period of light irradiation and light blocking with respect to the communication system 1 (image sensor 11). The communication system 1 receives incident image light (incident light) from the outside through the image sensor 11, and outputs a pixel signal corresponding to the received image light (incident light) to the signal processing circuit 230. The signal processing circuit 230 processes the pixel signal input from the communication system 1 and generates image data. The signal processing circuit 230 further generates an image signal corresponding to the generated image data and outputs the generated image signal to the display unit 240. The display unit 240 displays an image based on the image signal input from the signal processing circuit 230.

[0137] In this application example, the communication system 1 according to any one of the foregoing embodiments and modifications thereof is applied to the imaging system 2. This enables appropriate communication according to the capacity and speed of the data to be transmitted, for example, so that a high-quality captured image can be provided for the imaging system 2.

[0138] <Example of practical application>

[0139] The technology (this technology) according to the embodiments of the present disclosure is applicable to various products. For example, the technology according to the embodiments of the present disclosure can be implemented in the form of a device to be installed in any kind of moving body. Non-limiting examples of the moving body may include an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, any personal mobility device, an airplane, an unmanned aerial vehicle (UAV), a ship, and a robot.

[0140] Figure 21 is a block diagram showing an example of a schematic configuration of a vehicle control system that is an example of a moving body control system to which the technology according to the embodiments of the present disclosure can be applied.

[0141] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In Figure 21In the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an inside vehicle information detection unit 12040, and an integrated control unit 12050. In addition, as a functional structure of the integrated control unit 12050, a microcomputer 12051, a sound / image output unit 12052, and a vehicle-mounted network interface (I / F) 12053 are illustrated.

[0142] The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 serves as a control device for a driving force generation device (such as an internal combustion engine, a drive motor, etc.) that generates the driving force of the vehicle, a driving force transmission mechanism that transmits the driving force to the wheels, a steering mechanism that adjusts the steering angle of the vehicle, a braking device that generates the braking force of the vehicle, etc.

[0143] The body system control unit 12020 controls the operation of various devices provided to the body according to various programs. For example, the body system control unit 12020 serves as a control device for a keyless entry system, a smart key system, an electric window device, or various lights such as headlights, rear lights, brake lights, turn signals, fog lights, etc. In this case, radio waves or signals from various switches sent from a mobile device as an alternative to a button can be input to the body system control unit 12020. The body system control unit 12020 receives these input radio waves or signals and controls the door lock device, electric window device, lights, etc. of the vehicle.

[0144] The outside vehicle information detection unit 12030 detects information outside the vehicle including the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside vehicle information detection unit 12030. The outside vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. In addition, the outside vehicle information detection unit 12030 can also perform processing for detecting objects such as people, vehicles, obstacles, signs, characters on the road surface, etc., or processing for detecting their distances based on the received image.

[0145] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of received light of the light. The imaging unit 12031 can output the electrical signal as an image, or can output the electrical signal as information about the measured distance. In addition, the light received by the imaging unit 12031 can be visible light, or can be invisible light such as infrared light.

[0146] The in-vehicle information detection unit 12040 detects information regarding the interior of the vehicle. The in-vehicle information detection unit 12040 is connected, for example, to a driver state detection unit 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that photographs the driver. Based on the detection information input from the driver state detection unit 12041, the in-vehicle information detection unit 12040 can calculate the driver's fatigue or concentration, or can determine whether the driver is dozing off.

[0147] The microcomputer 12051 can calculate control target values for a driving force generation device, a steering mechanism, or a braking device based on information regarding the interior or exterior of the vehicle obtained by the out-vehicle information detection unit 12030 or the in-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control for realizing functions of an advanced driver assistance system (ADAS), which includes anti-collision or shock absorption for the vehicle, following driving based on a following distance, maintaining the vehicle speed of driving, warning of vehicle collision, warning of deviation of the vehicle from a lane, and the like.

[0148] In addition, the microcomputer 12051 can perform cooperative control for autonomous driving by controlling a driving force generation device, a steering mechanism, a braking device, etc. based on information regarding the exterior or interior of the vehicle obtained by the out-vehicle information detection unit 12030 or the in-vehicle information detection unit 12040, which enables the vehicle to automatically drive without relying on the driver's operations, etc.

[0149] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on information regarding the exterior of the vehicle obtained by the out-vehicle information detection unit 12030. For example, the microcomputer 12051 can perform cooperative control for preventing glare by controlling the headlamp to change from high beam to low beam according to the positions of a preceding vehicle or an oncoming vehicle detected by the external vehicle information detection unit 12030.

[0150] The sound / image output unit 12052 sends an output signal of at least one of sound and image to an output device that can visually or auditorily notify information to the occupants of the vehicle or to the outside of the vehicle. In Figure 21 the example, an audio speaker 12061, a display unit 12062, and a dashboard 12063 are shown as output devices. For example, the display unit 12062 may include at least one of an on-board display and a head-up display.

[0151] Figure 22 is a diagram illustrating an example of the installation position of the imaging unit 12031.

[0152] InFigure 22 In this case, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0153] The imaging units 12101, 12102, 12103, 12104, and 12105 are disposed, for example, at positions on the front nose, side mirrors, rear bumper, and rear door of the vehicle 12100 and at a position on the upper part of the windshield inside the vehicle. The imaging unit 12101 disposed at the front nose inside the vehicle and the imaging unit 12105 disposed at the upper part of the windshield mainly obtain images in front of the vehicle 12100. The imaging units 12102 and 12103 disposed on the side mirrors mainly obtain images on the sides of the vehicle 12100. The imaging unit 12104 disposed on the rear bumper or rear door mainly obtains images at the rear of the vehicle 12100. The imaging unit 12105 disposed at the upper part of the windshield inside the vehicle is mainly used to detect a vehicle ahead, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0154] Incidentally, Figure 22 Examples of the imaging ranges of the imaging units 12101 to 12104 are described. The imaging range 12111 represents the imaging range of the imaging unit 12101 disposed on the front nose. The imaging ranges 12112 and 12113 represent the imaging ranges of the imaging units 12102 and 12103 disposed on the side mirrors, respectively. The imaging range 12114 represents the imaging range of the imaging unit 12104 disposed on the rear bumper or rear door. For example, a bird's-eye view image of the vehicle 12100 viewed from above is obtained by superimposing the image data imaged by the imaging units 12101 to 12104.

[0155] At least one of the imaging units 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0156] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the temporal change in the distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Thus, the nearest three-dimensional object traveling on the travel path of the vehicle 12100 in substantially the same direction as the vehicle 12100 at a prescribed speed (e.g., 0 km / h or more) is extracted as the preceding vehicle. In addition, the microcomputer 12051 can preset a following distance to be maintained in front of the preceding vehicle and perform automatic braking control (including following stop control), automatic acceleration control (including following start control), and the like. Therefore, cooperative control for autonomous driving can be executed so that the vehicle automatically travels regardless of the driver's operation or the like.

[0157] For example, the microcomputer 12051 can classify the three-dimensional object data regarding the three-dimensional objects into the three-dimensional object data of two-wheeled vehicles, standard-sized vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatically avoiding obstacles. For example, the microcomputer 12051 identifies the obstacles around the vehicle 12100 as the obstacles that can be visually recognized by the driver of the vehicle 12100 and the obstacles that are difficult for the driver of the vehicle 12100 to visually recognize. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle. In the case where the collision risk is equal to or higher than the set value and thus there is a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display unit 12062 and performs forced deceleration or avoidance steering via the drive system control unit 12010. The microcomputer 12051 can thereby assist driving to avoid a collision.

[0158] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. The microcomputer 12051 can identify a pedestrian, for example, by determining whether a pedestrian exists in the captured images of the imaging units 12101 to 12104. Such identification of a pedestrian is performed, for example, by a process of extracting feature points in the captured images of the imaging units 12101 to 12104 that are infrared cameras and by a process of performing pattern matching processing on a series of feature points representing the contour of an object to determine whether it is a pedestrian. When the microcomputer 12051 determines that a pedestrian exists in the captured images of the imaging units 12101 to 12104 and thus identifies the pedestrian, the sound / image output unit 12052 controls the display unit 12062 so that a square contour line for emphasis is displayed superimposed on the identified pedestrian. The sound / image output unit 12052 may also control the display unit 12062 so that an icon representing the pedestrian is displayed at a desired position.

[0159] A description of an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied has been given above. The technology according to an embodiment of the present disclosure can be applied to the imaging unit 12031 among the components of the above-described configuration. Specifically, the imaging system 2 can be applied to the imaging unit 12031. Applying the technology according to an embodiment of the present disclosure to the imaging unit 12031 allows for high-quality captured images, so that highly accurate control can be performed in the mobile body control system using the captured images.

[0160] Although the present disclosure has been described above with reference to embodiments, modification examples, and application examples, the present disclosure is not limited to the above-described embodiments and the like, and can be modified in various ways. It should be noted that the effects described herein are merely illustrative. The effects of the present disclosure are not limited to those described herein. The present disclosure may have other effects in addition to the effects described herein.

[0161] In addition, the present disclosure may also have the following configuration. (1)

[0163] A communication device, comprising:

[0164] A determination unit that determines the admissibility of a retransmission request based on priorities specified for each region within a frame in retransmission control; and

[0165] A transmission unit that generates retransmission request data and transmits it to another communication device when the retransmission request is allowed in the determination unit. (2)

[0167] The communication device according to (1) further includes a register in which a retransmittable frequency corresponding to the priority is set for each region. (3)

[0169] The communication device according to (2), wherein, when the frequency of the retransmission requests actually performed in each area does not exceed the retransmissible frequency, the determination unit determines to allow the retransmission requests. (4)

[0171] The communication device according to (2), wherein, when the frequency of the retransmission requests actually performed within a frame does not exceed the retransmissible frequency specified for the frame, the determination unit determines to allow the retransmission requests. (5)

[0173] The communication device according to any one of (1) to (4), wherein the area is set for each predetermined number of rows in the frame. (6)

[0175] The communication device according to any one of (1) to (4), wherein the area is set for each predetermined number of columns in the frame. (7)

[0177] The communication device according to any one of (1) to (4), wherein the area is set for each predetermined number of pixels in the frame. (8)

[0179] The communication device according to any one of (1) to (7), wherein the communication device performs retransmission control of MIPI A-PHY. (9)

[0181] A communication device, comprising:

[0182] A receiving unit that receives retransmission request data in retransmission control;

[0183] A transmitting unit that transmits data to other communication devices;

[0184] A holding and saving unit that saves the data transmitted from the transmitting unit as transmitted data; and

[0185] A control unit that performs retransmission control of the transmitted data based on the retransmission request data. (10)

[0187] A communication device, comprising:

[0188] A determination unit that determines the retransmissibility of a transmitted packet based on the priority specified for each area within a frame in retransmission control; and

[0189] A transmitting unit that transmits the transmitted packet to other communication devices when the determination unit allows the retransmission of the transmitted packet. (11)

[0191] The communication device according to (10), wherein the determination section determines that retransmission of the transmitted packet is permitted when a frequency of retransmission requests actually made for each area does not exceed a retransmission possible frequency specified for each area. (12)

[0193] The communication device according to (10) or (11), wherein the determination unit determines that retransmission of the transmitted packet is permitted when the frequency of retransmission requests actually made within the frame does not exceed a retransmission possible frequency specified for the frame. (13)

[0195] A communication system comprising:

[0196] a transmitting device; and

[0197] The receiving device, the transmitting device and the receiving device communicate with each other;

[0198] The receiving device includes:

[0199] a determination unit that, in retransmission control, determines feasibility of a retransmission request based on a priority assigned to each region within a frame; and

[0200] a first transmitting unit that generates retransmission request data and transmits the retransmission request data to the transmitting device when the retransmission request is permitted by the determining unit; and

[0201] The sending device includes:

[0202] a receiving unit, receiving resend request data;

[0203] a second sending unit, sending data to a receiving device;

[0204] a holding unit that holds the data sent from the second sending unit as sent data; and

[0205] The control unit performs retransmission control on the transmitted data based on the retransmission request data. (14)

[0207] A communication system comprising:

[0208] a transmitting device; and

[0209] The receiving device, the transmitting device and the receiving device communicate with each other;

[0210] The receiving device includes:

[0211] A first transmitting unit generates retransmission request data and transmits the retransmission request data to a transmitting device during retransmission control; and

[0212] The sending device includes:

[0213] A receiving unit that receives retransmission request data;

[0214] A second transmitting unit that transmits data to a receiving device;

[0215] A holding and saving unit that saves the data transmitted from the second transmitting unit as transmitted data; and

[0216] A determining unit that determines the retransmissibility of the transmitted data based on the retransmission request data and the priorities specified for each region within a frame; wherein

[0217] When the retransmission of the transmitted packet is allowed in the determining unit, the second transmitting unit transmits the transmitted packet to the receiving device.

[0218] The communication device according to the first aspect of the present disclosure determines the retransmissibility of a retransmission request based on the priorities specified for each region within a frame in retransmission control, and when the retransmission request is allowed, generates retransmission request data and transmits the retransmission request data to another communication device, thereby preventing the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0219] The communication device according to the second aspect of the present disclosure, when receiving retransmission request data in retransmission control, performs retransmission control of the transmitted data saved by the saving unit based on the retransmission request data, thereby preventing the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0220] The communication device according to the third aspect of the present disclosure determines the retransmissibility of a transmitted packet based on the priorities specified for each region within a frame in retransmission control, and when the retransmission of the transmitted packet is allowed, transmits the transmitted packet to another communication device, thereby preventing the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0221] The communication system according to the fourth aspect of the present disclosure determines the suitability of a retransmission request based on the priorities specified for each region within a frame in retransmission control, and when the retransmission request is allowed, generates retransmission request data and transmits it from a transmitting device to a receiving device, thereby preventing the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0222] The communication system according to the fifth aspect of the present invention determines the retransmissibility of a transmitted packet based on the priorities specified for each region within a frame in retransmission control, and transmits the transmitted packet from a transmitting device to a receiving device whether the retransmission of the transmitted packet is allowed or not, thereby preventing the delay from extending to the next frame and subsequent frames. As a result, the desired frame rate can be maintained.

[0223] This application claims the benefit of Japanese Priority Patent Application JP2020-056057, filed with the Japanese Patent Office on March 26, 2020, the entire content of which is incorporated herein by reference.

[0224] Those skilled in the art should understand that various modifications, combinations, sub-combinations, and changes can be made according to design requirements and other factors, as long as they are within the scope of the appended claims or their equivalents.

Claims

1. A communication device, comprising: A determination unit that, in the retransmission control of MIPI A-PHY, determines the feasibility of a retransmission request based on the priority assigned to each region within a frame; And A transmission unit that, when the retransmission request is permitted by the determination unit, generates retransmission request data and transmits it to another communication device; A register in which a retransmission frequency corresponding to the priority is set for each of the regions; Wherein, when the frequency of the retransmission request actually performed in each region does not exceed the retransmission frequency, the determination unit determines to permit the retransmission request, and when the frequency of the retransmission request actually performed within the frame does not exceed the retransmission frequency specified for the frame, the determination unit determines to permit the retransmission request.

2. The communication device according to claim 1, wherein, The region is set for each predetermined number of rows in the frame.

3. The communication device according to claim 1, wherein, The region is set for each predetermined number of columns in the frame.

4. The communication device according to claim 1, wherein, The region is set for each predetermined number of pixels in the frame.

5. A communication device, comprising: A receiving unit that receives retransmission request data determined based on the priority assigned to each region within a frame in the retransmission control of MIPI A-PHY; A transmission unit that transmits data to another communication device; A storage unit that stores the data transmitted from the transmission unit as transmitted data; And A control unit that performs retransmission control of the transmitted data based on the retransmission request data, Wherein a retransmission frequency corresponding to the priority is set for each of the regions; Wherein, when the frequency of the retransmission request actually performed in each region does not exceed the retransmission frequency, it is determined to permit the retransmission request, and when the frequency of the retransmission request actually performed within the frame does not exceed the retransmission frequency specified for the frame, it is determined to permit the retransmission request.

6. A communication device, comprising: A determination unit that, in the retransmission control of MIPI A-PHY, determines the feasibility of retransmitting a transmitted packet based on the priority assigned to each region within a frame; And A transmission unit that, when the retransmission of the transmitted packet is permitted by the determination unit, transmits the transmitted packet to another communication device; A register in which a retransmission frequency corresponding to the priority is set for each of the regions; Wherein, when the frequency of the retransmission request actually performed in each region does not exceed the retransmission frequency, the determination unit determines to permit retransmission of the transmitted packet, and when the frequency of the retransmission request actually performed within the frame does not exceed the retransmission frequency specified for the frame, the determination unit determines to permit retransmission of the transmitted packet.

7. A communication system, comprising: A transmitting device; And A receiving device that communicates with the transmitting device; The receiving device includes: A determination unit that, in the retransmission control of MIPI A-PHY, determines the feasibility of a retransmission request based on the priority assigned to each region within a frame; and A first transmitting unit that, when the retransmission request is permitted in the determination unit, generates retransmission request data and transmits the data to the transmitting device; A register in which a retransmission frequency corresponding to the priority is set for each of the regions; wherein, when the frequency of the retransmission requests actually made in each of the regions does not exceed the retransmission frequency, the determination unit determines to permit the retransmission request, and when the frequency of the retransmission requests actually made within the frame does not exceed the retransmission frequency specified for the frame, the determination unit determines to permit the retransmission request; and The transmitting device includes: A receiving unit that receives the retransmission request data; A second transmitting unit that transmits data to the receiving device; A storage unit that stores the data transmitted from the second transmitting unit as transmitted data; and A control unit that performs retransmission control of the transmitted data based on the retransmission request data.

8. A communication system, comprising: A transmitting device; And A receiving device, where the transmitting device and the receiving device communicate with each other; The receiving device includes: A first transmitting unit that, in retransmission control, generates retransmission request data and transmits the data to the transmitting device; A register in which a retransmission frequency corresponding to the priority is set for each region; wherein, when the frequency of the retransmission requests actually made in each of the regions does not exceed the retransmission frequency, the determination unit determines to permit retransmission of the transmitted packet, and when the frequency of the retransmission requests actually made within the frame does not exceed the retransmission frequency specified for the frame, the determination unit determines to permit retransmission of the transmitted packet; and The transmitting device includes: A receiving unit that receives the retransmission request data; A second transmitting unit that transmits data to the receiving device; A storage unit that stores the data transmitted from the second transmitting unit as transmitted data; and A determination unit that determines the feasibility of retransmission of the transmitted data based on the retransmission request data and the priority specified for each region within the frame; wherein when the retransmission of the transmitted data is permitted in the determination unit, the second transmitting unit transmits the transmitted data to the receiving device.

Images