Single-channel optical communication for vehicle registration

Abstract

Techniques and examples related to single-channel line-of-sight (LOS) communications are described. A transmitting end of the LOS communication involves direct modulation of light emitted from an optical transmitter. A receiving end of the LOS communication involves receiving a video stream of the optical transmitter that emits the light, wherein the video stream includes a plurality of video frames that are temporally consecutive. The receiving end of the LOS communication also involves converting the video stream into a binary string including a plurality of binary bits. Each of the binary bits corresponds to a respective one of the plurality of video frames, and the binary string contains a repeating binary pattern representing a message being communicated. The receiving end of the LOS communication also involves extracting the binary pattern from the binary string and then decoding the binary pattern to obtain the message.

Description

Technical Field

[0001] The present disclosure relates generally to wireless data communications, and more particularly to single-channel optical modulation for non-networked line-of-sight (LOS) communications for manned and unmanned transportation vehicles. Background Art

[0002] The regulation of unmanned aerial vehicles (UAVs) depends on the registration of UAVs so that the operator of the UAV can be identified at any time. For example, without some form of on-vehicle registration, a UAV operator could violate Federal Aviation Administration (FAA) regulations and avoid penalties by simply abandoning the UAV. Without a system that links or associates a vehicle with its owner / operator, UAV laws cannot be enforced.

[0003] Current methods for identifying UAV registration have various limitations. For example, a registration tag affixed to a UAV (similar to a license plate or a vehicle identification number affixed to or engraved on a car) can help identify the UAV when the UAV is recovered or possessed, but it may be difficult to read from a distance when the UAV is in the air. Alternatively, a cellular or radio networking solution can be adopted. For example, an aerial UAV can send an identification signal to a nearby base station or a radio transponder of a cellular or mobile phone / data network for identifying the UAV itself, but this will not be feasible when the UAV flies over an area of ​​radio or cellular darkness (such as, a remote area where mobile phone service coverage is not reached). In addition, radio-based UAV registration may be subject to electronic spoofing, used as a carrier of malware, or cause misidentification when multiple UAVs are flying in the same area.

[0004] An important consideration when registering a UAV is the ability for a person, vehicle, or device to quickly identify the UAV when it is within line of sight of the person, vehicle, or device, even in areas lacking cellular network service. A specific example is identifying unauthorized UAVs flying over U.S. national parks, which typically lack network service in large portions of the park. Summary of the invention

[0005] The present disclosure is intended to provide methods, systems and devices related to single-channel optical communication for vehicle registration. According to one embodiment, a method according to the present disclosure may involve: receiving a video stream including a plurality of temporally consecutive video frames. The method may also involve: converting the video stream into a binary string including a plurality of binary bits, each binary bit corresponding to a corresponding one of the plurality of video frames, the binary string containing a repeating binary pattern representing a message. The method may further involve: extracting the binary pattern from the binary string. The method may also involve: decoding the binary pattern to obtain the message. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Non-limiting and non-exhaustive embodiments of the present disclosure are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various figures unless otherwise specified.

[0007] Figure 1 is a diagram depicting an exemplary scenario in which embodiments according to the present disclosure may be utilized.

[0008] Figure 2 is a diagram showing an encoding / decoding table that may be utilized according to an embodiment of the present disclosure.

[0009] Figure 3 is a flow chart depicting an exemplary process of a transmitting end of a communication system according to an embodiment of the present disclosure.

[0010] Figure 4 is a flow chart depicting an exemplary process of a receiving end of a communication system according to an embodiment of the present disclosure.

[0011] Figure 5 The present invention is described in accordance with an embodiment of the present invention. Figure 4 A flowchart of an exemplary process of a portion of an exemplary process.

[0012] Figure 6 The present invention is a diagram showing an embodiment of the present invention. Figure 5 A diagram of details of a portion of an exemplary process.

[0013] Figure 7 The present invention is a diagram showing an embodiment of the present invention. Figure 5 A flow chart of details of another portion of an exemplary process.

[0014] Figure 8 The present invention is described in accordance with an embodiment of the present invention. Figure 4 A flowchart of an exemplary process for another portion of an exemplary process.

[0015] Fig. 9 is a block diagram depicting an exemplary transport vehicle according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] In the following description, reference is made to the accompanying drawings which form a part thereof and in which are shown by way of illustration specific exemplary embodiments in which the present disclosure may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the concepts disclosed herein, and it is understood that the various disclosed embodiments may be modified and other embodiments may be utilized without departing from the scope of the present disclosure. Therefore, the following detailed description should not be considered in a limiting sense.

[0017] Although examples of the present disclosure are provided primarily in the context of UAVs, the scope of the present disclosure is not limited thereto. That is, the various implementations, concepts, schemes, systems, methods, and devices described herein may be applicable to other types of transportation vehicles, surveillance vehicles, or service vehicles, including manned or unmanned and manual or automatic automobiles, drones, airborne vessels, and / or watercraft. Furthermore, the scope of the present disclosure extends beyond communications between vehicles and may include communications between vehicles and non-vehicle objects such as people, machines, infrastructure, or buildings.

[0018] The present disclosure relates to a communication method that enables a UAV to convey the registration information of the UAV to a recipient in a non-networked environment. The present disclosure proposes an optical communication system for use with a UAV to generate and send messages that may include registration information. The optical communication system of the present disclosure can constitute a single communication channel having a transmitting end and a receiving end. On the transmitting end, the optical communication system can use a light-emitting device such as a light-emitting diode (LED), which is turned on and off by a microcontroller to encode the message through direct modulation. The message may include an identity associated with the UAV (e.g., an FAA registration number). On the receiving end, a video recording device (e.g., a camera of a smart phone) can record or otherwise receive the modulated light signal. An algorithm implemented by software, hardware, or a combination of both (e.g., an application running on a smart phone) can analyze the received video and decode the message from the video. Based on the decoded message, the identity of the UAV (e.g., the registered owner of the UAV) can be known. It can be used in Figure 1 Various embodiments of optical communication systems are implemented in the depicted exemplary scenario, as described below.

[0019] Figure 1 An exemplary scenario 100 is depicted in which embodiments of the proposed optical communication system may be implemented. The scenario 100 may include a non-networked environment 099 in which a UAV 010 may traverse. That is, the environment 099 may not have a cellular or mobile phone / data network that the UAV 010 may communicate with as the UAV 010 traverses the environment 099. The UAV 010 may be equipped with a light transmitter 017 that continuously emits a light signal that conveys a message as the UAV 010 traverses the environment 099. In some embodiments, the light transmitter 017 may also be used as a safety light (e.g., an anti-collision light) that continuously operates while the UAV 010 is in the air or in motion. The light transmitter 017 may be controlled by a microcontroller (not shown) embedded in the UAV 010 to emit the light signal.

[0020] Devices with video recording capabilities, such as smartphones 020, may also be present in environment 099. Figure 1As shown, a fictitious LOS 123 may exist between smartphone 020 and UAV 010. LOS 123 may be tracking UAV 010 and therefore move accordingly within environment 099 as UAV 010 traverses or otherwise moves through environment 099. Assuming that the user holding smartphone 020 may also be moving within environment 099 as UAV 010 traverses environment 099, LOS 123 will move within environment 099 based on the movement of UAV 010 and the user.

[0021] Specifically, the smart phone 020 may be equipped with a video camera ( Figure 1 010 traversing environment 099. In addition to UAV 010, video stream 021 may also capture other objects or backgrounds present in environment 099, such as house 091, tower 092, tree 093, hill 094, road 095, and / or airplane 096. In addition, smartphone 020 may implement an algorithm using software or hardware embedded in smartphone 020, or a combination of both (e.g., an application running on a smartphone) to analyze video stream 021 and decode messages conveyed in light signals emitted from light transmitter 017 of UAV 010 recorded in video stream 021.

[0022] The message conveyed in the optical signal emitted by the optical transmitter 017 can represent a plurality of alphanumeric digits (i.e., letters or numbers). In some embodiments, the plurality of alphanumeric digits can be the registration number of the UAV 010 as assigned by the FAA. The registration number can have a fixed length, such as a length of ten digits. In some embodiments, the registration number can begin with the letter 'F' or the letter 'FA'. That is, for all UAVs registered by the FAA, the FAA assigns a unique registration number of ten digits long to each UAV as the identification of the UAV, beginning with the letter 'F' or 'FA'. The letter 'F' or the letter 'FA' is used as a message identifier, and is predetermined based on the communication protocol adopted by the optical communication system. The FAA can also maintain a database indexed by the registration number, each entry of which records the relevant information of the corresponding UAV (e.g., year, brand, model, owner's name, owner's address). In the case of knowing the registration number of a specific UAV, the relevant information of the UAV can also be understood through the database.

[0023] When the microcontroller embedded in the UAV 010 modulates the light signal according to the message, the light signal emitted by the optical transmitter 017 can convey the message (e.g., the registration number of the UAV 010). That is, the microcontroller can encode the message into the light signal by modulating the light signal at a predetermined frequency, which is also predetermined according to the communication protocol adopted by the optical communication system. In some embodiments, the microcontroller can modulate the light signal according to the American Standard Code for Information Interchange (ASCII) encoding method. Figure 2 As shown in the ASCII table 200 of FIG. 1 , each alphanumeric digit of the message can be converted into an eight-bit binary code (i.e., a binary representation consisting of eight bits of 0, 1, or a combination of both). For example, Figure 1 The FAA registration number of UAV 010 may be a ten-digit number “FA12345678.” Each of the ten digits of the registration number may be converted into an eight-bit binary representation (ie, an ASCII code) according to the ASCII table 200 . That is, the digit 'F' can be converted into the ASCII code '01000110', the digit 'A' can be converted into the ASCII code '01000001', the digit '1' can be converted into the ASCII code '00110001', the digit '2' can be converted into the ASCII code '00110010', the digit '3' can be converted into the ASCII code '00110011', the digit '4' can be converted into the ASCII code '00110100', the digit '5' can be converted into the ASCII code '00110101', the digit '6' can be converted into the ASCII code '00110110', the digit '7' can be converted into the ASCII code '00110111', and the digit '8' can be converted into the ASCII code '00111000'.

[0024] Figure 3 FIG. 3 is a flowchart depicting an exemplary process 300 employed on a transmitting end of an optical communication system. In particular, process 300 illustrates a processor (eg, embedded in a processor 300 ) on a transmitting end of an optical communication system. Figure 1 How a processor in UAV 010 can modulate a signal transmitter (e.g., Figure 1 The optical transmitter 017) can transmit light to the receiving party (for example, Figure 1 Smart phone 020) to convey the message (e.g., Figure 1 The process 300 may include the following steps shown as blocks (such as, Figure 3310, 320, 330, 340 and 350). Although shown as discrete blocks, the various blocks of process 300 may be divided into additional blocks, combined into fewer blocks, or eliminated depending on the desired implementation. Process 300 may start at block 310.

[0025] At 310 , process 300 may involve a processor embedded in UAV 010 extracting the first digit of registration number “FA12345678,” namely digit “F.” Process 300 may proceed from 310 to 320 .

[0026] At 320, process 300 may involve the processor converting the first digit "F" into a binary representation. For example, the processor may convert the first digit "F" into a binary representation according to Figure 2 The ASCII table 200 converts the digit "F" into the eight-bit ASCII code '01000110'. Process 300 may proceed from 320 to 330. Block 330 may include sub-blocks 331, 332, 333, 334, 335, and 336. Specifically, process 300 may proceed from block 320 to sub-block 331 of block 330.

[0027] At 330, the process 300 may involve the processor modulating the signal transmitter according to the eight-bit ASCII code '01000110' representing the first digit "F" of the registration number of UAV 010, one bit at a time. Specifically, at sub-block 331, the process 300 may involve the processor extracting the first bit of the ASCII code '01000110', i.e., the most significant bit (MSB) '0'. The process 300 may proceed from 331 to 332.

[0028] At 332, process 300 may involve the processor determining whether the bit previously extracted at sub-block 331 is '0' or '1'. If the processor determines that the extracted bit is '0', process 300 may proceed from 332 to 333, where process 300 may involve the processor causing the signal transmitter to remain off for 1 duration. If the processor determines that the extracted bit is '1', process 300 may proceed from 332 to 334, where process 300 may involve the processor causing the signal transmitter to remain on for 1 duration. The duration that the signal transmitter is turned off at sub-block 333 and turned on at sub-block 334 is determined by a predetermined modulation frequency of the communication protocol employed. Specifically, the time length of the duration is equal to the mathematical inverse of the modulation frequency. Process 300 may proceed from 333 or 334 to 335.

[0029] At 335, process 300 may involve the processor determining whether the bit previously extracted at sub-block 331 is the last bit of the ASCII code, i.e., its least significant bit (LSB). If the bit previously extracted at 331 is not yet the LSB of the eight-bit ASCII code representing the previously extracted digit, the process may proceed from 335 to 336 to extract the next bit of the ASCII code, and then proceed from 336 to 332. If the bit previously extracted at 331 is already the LSB of the eight-bit ASCII code representing the previously extracted digit, process 300 may proceed from 335 to 340.

[0030] At 340, process 300 may involve the processor determining whether the digit previously extracted at subblock 336 is the last digit of the ten-digit registration number "FA12345678". If the previously extracted digit is not yet the last digit of the registration number, process 300 may proceed from 340 to 350 and extract the next digit of the registration number. If the previously extracted digit is already the last digit of the registration number, process 300 may proceed from 340 to 310 and repeat the modulation of the signal transmitter again.

[0031] In summary, process 300 may extract one digit of the registration number and convert the digit into a binary representation, and modulate the signal transmitter according to the binary representation before moving on to the next digit of the registration number, one digit at a time. That is, process 300 may begin by processing digit "F" of the registration number, then digit "A", then digit "1", then digit "2", then digit "3", ..., and finally digit "8". Once each digit of the message (i.e., the registration number) has been processed (i.e., extracted, converted, and the light transmitter modulated accordingly), process 300 may be repeated as long as UAV 010 is airborne or in operation.

[0032] Figure 4 FIG. 4 is a flowchart depicting an exemplary process 400 employed at a receiving end of an optical communication system. In particular, process 400 illustrates a processor (eg, embedded in a processor) at a receiving end of an optical communication system. Figure 1 How a processor in a smartphone 020 can decode a message conveyed in a modulated optical signal sent from a transmitting end of an optical communication system. Process 400 may include blocks (such as, Figure 4 410, 420, 430, 440 and 450). Although shown as discrete blocks, the various blocks of process 400 may be divided into additional blocks, combined into fewer blocks, or eliminated depending on the desired implementation. Process 400 may start at block 410.

[0033] At 410, process 400 may involve a processor embedded in smartphone 020 receiving a video stream, such as Figure 1 As mentioned above, the video stream 021 may be generated by a video camera of the smartphone 020. The video camera may be positioned as the UAV 010 passes through the Figure 1 099 hours of tracking and recording of the environment Figure 1 UAV 010. Figure 5 As shown, the video stream 021 may be composed of a plurality of video frames F(1), F(2), . . . , F(N) recorded or otherwise captured by a video camera over a period of time. Process 400 may proceed from 410 to 420.

[0034] At 420, process 400 may involve the processor converting video stream 021 into a binary string 041, such as Figure 5 As shown. Figure 5 As shown, the binary string 041 may include multiple binary bits, Bit(1), Bit(2), ..., Bit(N). The value of each of the binary bits Bit(1) to Bit(N) is "0" or "1". In addition, each of the binary bits Bit(1) to Bit(N) has a one-to-one correspondence with one of the video frames F(1) to F(N). For example, Bit(1) is derived from F(1) via boxes 510, 520, and 530, so Bit(2) is derived from F(2), ..., and Bit(N) is derived from F(N). In addition, the binary string 041 may include a binary pattern that is repeated within the binary string 041. The binary pattern may be represented by Figure 1 The message conveyed in the light signal emitted by the light transmitter 017. In some embodiments, the message conveyed may be Figure 1 The binary pattern may be repeated within the binary string 041 because it can be repeated according to Figure 3 The process 300 transmits the FAA registration number over and over again via the optical transmitter 017 of the UAV 010. More details about block 420 will be described later in this disclosure. Figure 5 The process 500 and Figure 7 The process 400 may proceed from 420 to 430.

[0035] At 430, process 400 may involve the processor extracting a binary pattern from binary string 041. The binary pattern may have a length predetermined by the communication protocol of the optical communication system and may consist of a string of 0 bits and 1 bits. The binary pattern may substantially represent the message being communicated. In some embodiments, the binary pattern may be a message encoded in ASCII code.

[0036] In order for the process 400 to successfully decode the binary pattern and ultimately obtain the message conveyed in the optical signal, it may be necessary for the binary pattern to contain a sufficient number of bits. Figure 5 The video stream 021 may need to have a sufficient number of video frames F(1) to F(N). Assuming that the message is a ten-digit FAA registration number encoded according to the ASCII table 200, the total number of bits in the binary pattern will need to be at least (10 digits per registration number) x (8 bits per digit) = 80 bits. In addition, there may be an oversampling relationship between the modulation frequency of the light emitter 017 and the frame rate of the camera recording UVA 010. For example, the light emitter 017 may be operating at a modulation frequency of 15 Hz, while the camera records at a frame rate of 30-fps (i.e., the camera generates 30 frames per second). This may translate into a 2x oversampling rate, meaning that the video stream 021 may need to have at least 160 video frames in order for the process 400 to generate an 80-bit long binary pattern. Typical practice may require an additional 2x redundancy and noise immunity, so in practice Figure 5 The number N may be as large as 320 in order for the process 400 to decode the FAA registration number of the UAV 010 .

[0037] The following description will be given later in this disclosure. Figure 8 More details about block 430 are given in the section of process 800 of FIG. From 430 , process 400 may proceed to 440 .

[0038] At 440, process 400 may involve the processor decoding the binary pattern to obtain the message. In some embodiments, the processor may use Figure 2 ASCII table 200 is used to decode the binary pattern into a message (e.g., the FAA registration number of UAV 010).

[0039] refer to Figures 5 to 7 , the following is about Figure 4 4. More details of block 420 of process 400 are provided below.

[0040] Figure 5 Shows that depiction may be Figure 4 420 of process 400. Process 500 may include blocks (such as, Figure 5 One or more operations, actions or functions of process 500 (510, 520 and 530). Although shown as discrete blocks, the various blocks of process 500 may be divided into additional blocks, combined into fewer blocks or eliminated depending on the desired implementation. Process 500 may start at block 510.

[0041] At 510, process 500 may involve a processor (e.g., embedded in a processor) at a receiving end of an optical communication system. Figure 1 A processor in a smartphone 020 of the optical communication system (or the like) extracts a corresponding region of interest (ROI) for each of the video frames F(1) to F(N) of the video stream 021. As mentioned above, the video stream 021 generated or recorded by a camera on the receiving end of the optical communication system can record or otherwise capture the UAV 010 traversing the environment 099 over a period of time. That is, the video frames F(1) to F(N) can be continuous in time, with each frame captured at a certain time interval after the previous frame. The time interval can be a feature of the camera. For example, if the camera records the video stream 021 at a frame rate of 30fps, the time interval is 1 / 30 second. In addition, each of the video frames F(1) to F(N) can include the UAV 010 in addition to some other objects or backgrounds present in the environment 099 (such as a house 091, a tower 092, a tree 093, a hill 094, a road 095, and / or an airplane 096).

[0042] Figure 6 The relationship between the video frame and the ROI of the video frame is shown. The video frame 621 can represent Figure 5 Any one of the video frames F(1) to F(N). Figure 6 As shown, video frame 621 captures UAV 010 and its light emitter 017. In addition, other objects within environment 099 (such as house 091, tower 092, tree 093, and road 095) are also captured in video frame 621. However, these other objects do not provide useful information about the message conveyed in the light signal emitted from light emitter 017 of UAV 010 because the message is inherent in the light signal emitted from light emitter 017, but not in the other objects captured in video frame 621. Therefore, at block 510 of process 500, the processor embedded in smartphone 020 may extract the ROI (e.g., Figure 6ROI 651 or ROI 657). ROI 651 may include primarily UAV 010, while ROI 657 may simply include only light emitter 017 of UAV 010. ROI 651 and ROI 657 are "regions of interest" because the message is conveyed via a light signal emitted from light source 017 of UAV 010. Blocks 520 and 530 of process 500 following block 510 may ignore portions of video frame 621 outside of ROI 651 (or even ROI 657) because there is little information therein related to the message being conveyed. In short, the ROI allows real-time tracking of light emitter 017 and masking of unimportant portions of video stream 021. Additionally, using ROI 651 or ROI 657 rather than the entire video frame 621 also provides the benefit of reducing image noise caused by stray light captured in video frame 621. For example, tower 092 may emit strong anti-collision lights in all directions, and some of the anti-collision lights may be captured by smartphone 020 as stray light on video frame 621. Since the subsequent steps of process 500, namely block 520, may be applied to ROI 651 or ROI 657 instead of the entire video frame 621, the noise caused by stray light will be minimized.

[0043] Also at 510, the processor on the receiving end may use the object tracker 682 to extract the ROI 651 or ROI 657, such as Figure 6 That is, the object tracker 682 can Figure 5 The object tracker 682 may be implemented using software, hardware, or a combination of software and hardware. For example, the object tracker 682 may be implemented by a cascade neural network (CNN) circuit or by one or more software classifiers. When more than one software classifier is used, the classifiers are cascaded and the classification results of the classifiers are aggregated to produce a stronger cascade classifier. To be used as the object tracker 682, the CNN circuit or cascade classifier may need to be pre-trained with various training video streams. The training process may involve generating low-level visual textures from similar local pixel clusters in the training video stream. After training, the CNN circuit or cascade classifier may be able to identify the light emitter 017 in the video frames F(1) to F(N) of the video stream 021, and create or otherwise extract an ROI containing the light emitter 017 of the UAV 010 for each of the video frames F(1) to F(N). Process 500 may proceed from 510 to 520.

[0044] At 520, process 500 may involve a processor at the receiving end of the optical communication system converting the ROI of each of the video frames F(1) to F(N) extracted in block 510 into a corresponding binary image, such as Figure 7 The binary image 630 or 631. For example, Figure 7 As shown, the processor may use one or more color filters 084 to convert the ROI 657 (i.e., Figure 6 The output of the color filter 084 may be the binary image 630 or the binary image 631, depending on whether the light emitter 017 is turned on or off as captured in the ROI 657. Figure 7 As shown, the light emitter 017 may include 6 LEDs arranged in an array geometric arrangement. If the light emitter 017 is captured as off in the ROI 657, the color filter 084 may have a binary image 630 as output, which is almost a pure black or dark image. If the light emitter 017 is captured as on in the ROI 657, the color filter 084 may have a binary image 631 as output, in which some lighter points are shown in a dark background. Figure 7 As shown, the location of the brighter dots may correspond approximately to the location of light emitter 017 relative to ROI 657. In some embodiments, a hue-saturation value spatial filter may be used as filter 084. Process 500 may proceed from 520 to 530.

[0045] At 530, process 500 may involve a processor at the receiving end of the optical communication system determining the value of the binary bit based on the binary image generated in block 520. Figure 7 As shown, process 500 can determine the value of binary bit 641 by using binary image classifier 086. If color filter 084 outputs binary image 630, binary image classifier 086 can determine that binary bit 641 has a value of '0'. If color filter 084 outputs binary image 631, binary image classifier 086 can determine that binary bit 641 has a value of '1'. Similar to Figure 6 Object Tracker 082, Figure 7 The binary image classifier 086 can also be implemented by a hardware CNN or a software cascade classifier or a combination of the two. Similarly, the binary image classifier 086 may also need to be pre-trained using training data. Alternatively or in addition, the binary image classifier 086 can determine the value of the binary bit 641 by counting the edge number of the binary image 630 or 631 using a computer vision algorithm.

[0046] use Figure 5The process 500 can be based on a corresponding one of the video frames F(1) to F(N) of the video stream 021 to determine Figure 5 That is, using process 500, a processor at a receiving end of an optical communication system can determine the value of binary bit Bit(1) based on video frame F(1), determine the value of binary bit Bit(2) based on video frame F(2), ..., and determine the value of binary bit Bit(N) based on video frame F(N).

[0047] refer to Figure 8 , the following is about Figure 4 4. More details of block 430 of process 400 are provided below.

[0048] Figure 8 Shows that depiction may be Figure 4 430 of process 400. Process 800 may include blocks (such as, Figure 8 810, 820, 830, 832, 834, 840, 850, 860, and 870). Although shown as discrete blocks, the various blocks of process 800 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. Process 800 may begin at block 810.

[0049] At 810, process 800 may involve a processor (e.g., embedded in a processor) at a receiving end of an optical communication system. Figure 1 The processor in the smart phone 020) generates the binary string (e.g., Figure 5 The binary string 041 is converted from the time domain representation of the binary string to the frequency domain representation of the binary string. An exemplary binary string 041 may be:

[0050] "0011110000001100110000000000110000001111000000110011000000000011000000111100000011001100000000001100"----------[t1]

[0051] The representation [t1] can be a time domain representation of a binary string, which can be equivalently understood as "2-bit 0, followed by 4-bit 1, followed by 6-bit 0, followed by 2-bit 1, followed by 2-bit 0, followed by 2-bit 1, followed by 10-bit 0, followed by 2-bit 1, followed by 6-bit 0, followed by 4-bit 1, followed by 6-bit 0, followed by 2-bit 1, followed by 2-bit 0, followed by 2-bit 1, followed by 10-bit 0, followed by 2-bit 1, followed by 6-bit 0, followed by 4-bit 1, followed by 6-bit 0, followed by 2-bit 1, followed by 2-bit 0, followed by 2-bit 1, followed by 10-bit 0, followed by 2-bit 1, followed by 2-bit 0". With this understanding, the representation [t1] can be converted into the frequency domain and becomes:

[0052] "2,4,6,2,2,2,10,2,6,4,6,2,2,2,10,2,6,4,6,2,2,2,10,2,2" ----------[f1]

[0053] That is, the representation [f1] may be a frequency domain representation of the binary string. Process 800 may proceed from 810 to 820 .

[0054] At 820, process 800 may involve the processor determining a peak position in the frequency domain representation of the binary string. In some embodiments, a position in the frequency domain representation of the binary string may be determined as a peak position when the value at the position is at least three standard deviations away from the mean of the frequency domain representation of the binary string. For example, for representation [f1], the processor may determine the peak position to be the position of 10 in representation [f1]. Process 800 may proceed from 820 to 830 to determine a frequency domain representation of the binary pattern. Block 830 may include subblocks 832 and 834. Specifically, process 800 may proceed from 820 to 832.

[0055] At sub-block 832, process 800 may involve the processor dividing the frequency domain representation of the binary string into a plurality of segments based on the peak position. For example, the processor may divide the representation [f1] into the following four segments:

[0056] "2,4,6,2,2,2"----------[fs1]

[0057] "10,2,6,4,6,2,2,2"----------[fs2]

[0058] "10,2,6,4,6,2,2,2"----------[fs3]

[0059] "10,2,2"----------[fs4]

[0060] Process 800 may proceed from 832 to 834 .

[0061] At sub-block 834, process 800 may involve the processor determining one of the multiple segments as a frequency domain representation of a binary pattern. The processor may achieve this by using a voting algorithm. For example, the processor may apply a voting algorithm to segments [fs1], [fs2], [fs3], and [fs4] and determine that segment [fs2] (same as [fs3]) may be a frequency domain representation of a binary pattern. That is, the voting algorithm may select the most common representation of segments [fs1] to [fs4] as the frequency domain representation of the binary pattern. In the actual case where the binary string 041 may be susceptible to various noises present in the environment 099, the multiple segments may include the following segments, such as:

[0062] "10,2,6,4,6,2,2,2"----------[fs1']

[0063] "11,1,6,4,6,2,2,2"----------[fs2']

[0064] "10,2,6,4,6,2,2,2"----------[fs3']

[0065] "10,2,6,4,6,3,1,2"----------[fs4']

[0066] "10,2,6,3,6,2,2,2"----------[fs5']

[0067] "9,2,6,4,6,2,2,2"----------[fs6']

[0068] However, the voting algorithm may still determine that “10, 2, 6, 4, 6, 2, 2, 2” is the frequency domain representation of the binary pattern in segments [fs1′] to [fs6′]. Process 800 may proceed from 834 to 840 .

[0069] At box 840, process 800 may involve the processor generating a self-joined frequency domain representation of the binary pattern by joining the frequency domain representation of the binary pattern with a copy of the frequency domain representation of the binary pattern. Without generating the self-joined frequency domain representation, the processor may be unable to locate the message identifier in box 870. To illustrate this, consider the frequency domain representation of the binary pattern, "10,2,6,4,6,2,2,2", as determined at subbox 834. Recall that the binary pattern begins at the peak position located at box 820. The peak position is located purely statistically based on the frequency domain representation of the binary string, and may or may not coincide with the message identifier of the message (e.g., the letter "F" or the letters "FA" for an FAA registration number). The self-joined frequency domain representation will remove the chance of an accident where the peak position happens to coincide with a portion of the message identifier, and thus split the message identifier into two parts. When the frequency domain representation of the binary pattern is “10, 2, 6, 4, 6, 2, 2, 2”, the processor may generate the self-concatenated frequency domain representation of the binary pattern as:

[0070] "10,2,6,4,6,2,2,2,10,2,6,4,6,2,2,2"----------[f2]

[0071] Process 800 may proceed from 840 to 850 .

[0072] At block 850, process 800 may involve the processor converting the self-concatenated frequency domain representation of the binary pattern into a time domain representation of the binary pattern. For example, the processor may convert the self-concatenated frequency domain representation of the binary pattern [f2] into the time domain as follows:

[0073] "0000000000110000001111000000110011000000000011000000111100000011001100" ----------[t2]

[0074] Representation [t2] may be a time domain representation of a binary pattern. Process 800 may proceed from 850 to 860.

[0075] At block 860, process 800 may involve the processor removing redundancy caused by oversampling of the camera generating the video stream in the time domain representation of the binary pattern. That is, the processor may perform an anti-aliasing operation on the representation [t2]. As mentioned above, Figure 1The light transmitter 017 of the UAV 010 may have a modulation frequency of 15 Hz, while the camera of the smartphone 020 may have a video frame rate of 30 fps. This may translate into an oversampling of 2. In the case of 2x oversampling, the processor can remove the redundancy caused by the 2x oversampling by simply removing every other bit in the representation [t2], resulting in:

[0076] "00000100011000101000001000110001010"----------[t2']

[0077] Representation [t2′] may be a time domain representation of the binary pattern after redundancy due to 2× oversampling has been removed. Process 800 may proceed from 860 to 870 .

[0078] At block 870, process 800 may involve the processor determining a binary pattern by locating instances of a predetermined message identifier in a time domain representation of the binary pattern. For example, the processor may locate an instance of a message identifier (the letter "F") used in the communication protocol of the FAA registration number of the UAV. Knowing that the ASCII code for the letter "F" is "01000110", the processor may locate two instances of the message identifier, as described below (the instances of the message identifier are shown in bold):

[0079] "00000100011000101000001000110001010"----------[t2']

[0080] Having located the instance of the message identifier, the processor can therefore determine the binary pattern as:

[0081] "01000110001010000" ----------[tp3]

[0082] Although the various embodiments of single-channel optical communications described above and to be disclosed below are primarily illustrated using UAVs as examples, the application of the optical communication system can be extended beyond UAV registration and include other manned or unmanned transport vehicles, such as land-based vehicles, watercraft, or aircraft. In addition, both the transmitting and receiving ends of the optical communication system can be implemented on the same vehicle. For example, in addition to broadcasting its own FAA registration number, the UAV is also able to decode optical signals emitted from nearby UAVs to identify them. For another example, a land-based vehicle (such as a car) can be equipped with the transmitting and receiving components of the optical communication system so that V2V (i.e., vehicle-to-vehicle) or V2X (i.e., vehicle-to-environment) communication can be achieved when the land-based vehicle travels through a non-networked area.

[0083] As described above, the wavelength of the optical signal used in the optical communication system may exceed the range of visible light. For example, the optical transmitter 017 may include an infrared light source, and the smart phone 020 or other device on the receiving end may be equipped with an infrared camera and thus be able to receive the modulated infrared signal emitted from the infrared light source.

[0084] Fig. 9 An exemplary block diagram of a transport vehicle 900 is shown. The transport vehicle 900 may include a computer or host processor 910, such as a processor embedded in a Figure 1 Smartphone 020 can be executed Figure 4 The transport vehicle 900 may also include a processor capable of recording moving or stationary objects (e.g., Figure 1 The object may emit a modulated light signal from a light transmitter of the object (e.g., light transmitter 017 of UAV 010). The light signal may be transmitted along a LOS (e.g., Figure 1 The LOS 123 between the UAV 010 and the smart phone 020) arrives at the transport vehicle 900. Specifically, the video camera 950 may be capable of generating a video stream (e.g., Figure 5 The video stream 021). The video stream may consist of a plurality of video frames recorded continuously in time (e.g., Figure 5 The video frames F(1) to F(N) are composed of, and each of the video frames may have an image of a light emitter of the object (e.g., Figure 5 Video frame 621 having an image of light emitter 017).

[0085] In addition, the transport vehicle 900 may include a computer vision engine 980 for analyzing and / or processing the video stream recorded by the video camera 950. Specifically, the main processor 910 may utilize the computer vision engine 980 to execute Figure 5 500, and converting the video stream into a binary string (e.g., Figure 5 A binary string can consist of multiple binary bits (for example, Figure 5 The binary string may include a repeated binary pattern that represents a message conveyed in the modulated light signal sent by the object.

[0086] In some embodiments, computer vision engine 980 may include an object tracker 982 (such as Figure 6 The object tracker 682 of FIG. 1 may be used by the host processor 910 to extract a ROI (such as, for example, Figure 6In some embodiments, the computer vision engine 980 may also include one or more color filters 984 (such as, Figure 7 The color filter 084) can be used by the host processor 910 to convert the ROI of each video frame into a binary image (such as, Figure 7 In some embodiments, the computer vision engine 980 may also include a binary image classifier 986 (such as, Figure 7 A binary image classifier 086) can be used by the main processor 910 to determine the value ('0' or '1') of a corresponding one of the binary bits based on the binary image.

[0087] In addition, the main processor 910 may execute Figure 8 Process 800 is used to extract a binary pattern from a binary string (e.g., the binary representation [tp3] shown above).

[0088] In some embodiments, the main processor 910 may include an ASCII codec 912 to decode the binary pattern and obtain the message. In some embodiments, the transport vehicle 900 may also include a memory 920 that is capable of storing various binary representations (e.g., [t1], [f1], [fs1] to [fs4], [fs1'] to [fs6'], [t2], [t2'], and [tp3] as shown above) when the main processor 910 executes the process 800, and / or storing the message decoded by the ASCII codec 912 from the binary pattern. In some embodiments, the transport vehicle 900 may also include a communication module 930 that is capable of transmitting the decoded message to a third party. For example, the ASCII codec 912 may decode the binary pattern and obtain the FAA registration number of the UAV performing the unauthorized flight, and the main processor 910 may be able to notify the law enforcement department of the FAA registration number via the communication module 930. In some embodiments, the transport vehicle 900 may also include a display 940 that is capable of displaying the decoded message.

[0089] In some embodiments, the transport vehicle 900 may include a light emitter microcontroller 960 and a light emitter 970 (such as, Figure 1 The main processor 910 can use the light emitter microcontroller 960 and the light emitter 970 to execute Figure 3That is, the main processor 910 may modulate the light emitter 970 via the light emitter microcontroller 960 so that the light emitter 970 may emit or otherwise broadcast a light signal that conveys the vehicle registration number of the transport vehicle 900 itself. The ASCII codec 912 may encode the vehicle registration number and modulate the light emitter 970 accordingly. In some embodiments, the processor 910 may modulate the light emitter 970 via the light emitter microcontroller 960 to emit a light signal that conveys a message other than the vehicle registration number of the transport vehicle 900.

[0090] In some embodiments, light emitter 970 may have multiple LEDs controlled by host processor 910 via light emitter microcontroller 960, such as LEDs 970(1), 970(2), ..., 970(M). Fig. 9 As shown. LEDs 970(1) to 970(M) can form a geometric arrangement, such as an array. In some embodiments, LEDs 970(1) to 970(M) can be controlled to turn on or off simultaneously. In some embodiments, LEDs 970(1) to 970(M) can be controlled so that only a portion of LEDs 970(1) to 970(M) can be turned on simultaneously instead of all. That is, LEDs 970(1) to 970(M) can be controlled to emit light signals together in various geometric patterns. Various geometric patterns can help increase the bandwidth or throughput of optical communications by adding additional degrees of freedom when emitting light signals. In some embodiments, some or all of LEDs 970(1) to 970(M) may be able to emit light having different wavelengths, thereby displaying different colors. Different colors can also help increase the bandwidth or throughput of optical communications by adding another additional degree of freedom when emitting light signals.

[0091] In the above disclosure, reference is made to the accompanying drawings that form a part thereof, and specific implementations in which the present disclosure may be practiced are shown by way of illustration in the accompanying drawings. It should be understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References in the specification to "one embodiment", "an embodiment", "an example embodiment", etc. indicate that the embodiment described may include specific features, structures or characteristics, but each embodiment may not necessarily include the specific features, structures or characteristics. In addition, such phrases do not necessarily refer to the same embodiment. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, whether or not it is explicitly described, it is within the knowledge of a technician in the field to implement such a feature, structure or characteristic in conjunction with other embodiments.

[0092] The implementation of the systems, devices, apparatuses and methods disclosed herein may include or utilize a special-purpose or general-purpose computer, which includes computer hardware, such as one or more processors and system memory, as discussed herein. Implementations within the scope of the present disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media that can be accessed by a general-purpose or special-purpose computer system. A computer-readable medium storing computer-executable instructions is a computer storage medium (device). A computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example and not limitation, implementations of the present disclosure may include at least two distinct computer-readable media: a computer storage medium (device) and a transmission medium.

[0093] Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid-state drives ("SSD") (e.g., RAM-based), flash memory, phase-change memory ("PCM"), other types of memory, other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general or special purpose computer.

[0094] The implementation of the device, system and method disclosed herein can communicate through a computer network. "Network" is defined as one or more data links that can transmit electronic data between a computer system and / or module and / or other electronic device. When transmitting or providing information to a computer through a network or another communication connection (hardwired, wireless or any combination of hardwired or wireless), the computer appropriately regards the connection as a transmission medium. Transmission media may include a network and / or a data link, which may be used to carry desired program code means in the form of computer executable instructions or data structures and may be accessed by a general or special purpose computer. The above combination should also be included in the scope of computer-readable media.

[0095] Computer executable instructions include, for example, instructions and data that, when executed at a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions. Computer executable instructions can be, for example, binary files, intermediate format instructions (such as assembly language), or even source code. Although the subject matter has been described in a language specific to structural features and / or method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the above-described features or actions. Rather, the features and actions are disclosed as example forms of implementing the claims.

[0096] Those skilled in the art will appreciate that the present disclosure can be practiced in a network computing environment with many types of computer system configurations, including built-in vehicle computers, personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablet computers, pagers, routers, switches, various storage devices, etc. The present disclosure can also be practiced in a distributed system environment, where both local and remote computer systems linked by a network (by hardwired data links, wireless data links, or by any combination of hardwired data links and wireless data links) perform tasks. In a distributed system environment, program modules can be located in both local and remote memory storage devices.

[0097] In addition, where appropriate, the functions described herein may be performed in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and processes described herein. Certain terms are used throughout the description and claims to refer to specific system components. It will be understood by those skilled in the art that components may be referred to by different names. This document is not intended to distinguish between components that have different names but the same function.

[0098] It should be noted that the sensor embodiments discussed above may include computer hardware, software, firmware, or any combination thereof to perform at least a portion of their functions. For example, the sensor may include computer code configured to be executed in one or more processors, and may include hardware logic / circuitry controlled by the computer code. These exemplary devices are provided herein for illustrative purposes and are not intended to be limiting. As known to those skilled in the relevant art, the embodiments of the present disclosure may be implemented in other types of devices.

[0099] At least some embodiments of the present disclosure relate to a computer program product that includes such logic stored (e.g., in software form) on any computer-usable medium. Such software, when executed in one or more data processing devices, causes the devices to operate as described herein.

[0100] Although various embodiments of the present disclosure have been described above, it should be understood that these embodiments are presented by way of example only and not by way of limitation. It will be appreciated by those skilled in the relevant art that various changes in form and detail may be made without departing from the spirit and scope of the present disclosure. Therefore, the breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should only be defined in accordance with the following claims and their equivalents. The foregoing description has been presented for illustration and description purposes. The foregoing description is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In view of the above teachings, many modifications and variations are possible. In addition, it should be noted that any or all of the aforementioned optional implementations may be used in any desired combination to form additional hybrid implementations of the present disclosure.

Claims

1. A communication method, comprising: receiving a video stream comprising a plurality of temporally consecutive video frames; converting the video stream into a binary string comprising a plurality of binary bits, each binary bit corresponding to a respective one of the plurality of video frames, the binary string comprising a repeating binary pattern representing a message; extracting the binary pattern from the binary string; as well as decoding the binary pattern to obtain the message, wherein extracting the binary pattern comprises: converting the binary string from a time domain representation of the binary string to a frequency domain representation of the binary string; locating a peak position in the frequency domain representation of the binary string; determining a frequency domain representation of the binary pattern based on the peak locations; generating a self-concatenated frequency domain representation of the binary pattern by concatenating the frequency domain representation of the binary pattern with a copy of the frequency domain representation of the binary pattern; converting the self-concatenated frequency domain representation of the binary pattern into a time domain representation of the binary pattern; and The binary pattern is determined by locating instances of a predetermined message identifier in the time domain representation of the binary pattern.

2. The method of claim 1, wherein the video stream records a vehicle traversing an environment, the vehicle being equipped with a light transmitter that emits a signal related to the message, and wherein converting the video stream into the binary string comprises: extracting a region of interest (ROI) of a corresponding video frame from each of the plurality of video frames, the ROI including the light emitter; for each of the plurality of video frames, converting a corresponding ROI into a binary image using one or more color filters; as well as For each of the plurality of video frames, a value of a corresponding one of the plurality of bins is determined based on a respective binary image.

3. The method of claim 2, wherein said extracting the ROI of the corresponding video frame comprises passing the corresponding video frame through a pre-trained cascade neural network (CNN), and wherein the CNN is configured to be pre-trained with training data.

4. The method of claim 1, wherein said determining the frequency domain representation of the binary pattern comprises: dividing the frequency domain representation of the binary string into a plurality of segments based on the peak positions; as well as A voting algorithm is used to determine a segment of the plurality of segments as the frequency domain representation of the binary pattern.

5. The method of claim 1, wherein after said converting said self-concatenated frequency domain representation of said binary pattern into said time domain representation of said binary pattern and before determining said binary pattern, said method further comprises: Redundancy caused by oversampling of a camera generating the video stream is removed in the time domain representation of the binary pattern.

6. The method of claim 1, wherein said decoding the binary pattern to obtain the message comprises: The binary pattern is decoded according to the American Standard Code for Information Interchange (ASCII) encoding method.

7. The method of claim 1, wherein the video stream records a vehicle traversing an environment, and wherein the vehicle is equipped with one or more light emitters that emit a signal related to the message.

8. The method of claim 7, wherein the one or more light emitters include a plurality of light emitting diodes (LEDs), and wherein the signal includes a modulated light signal emitted from one or more of the plurality of LEDs, the modulated light signal being presented in one or more colors, one or more geometric patterns, or both of the one or more of the plurality of LEDs.

9. The method of claim 7, wherein the vehicle comprises an unmanned aerial vehicle (UAV), a watercraft, or a land-based vehicle, and wherein the message includes a registration number of the UAV, the watercraft, or the land-based vehicle.

10. A communication receiver, comprising: a camera capable of generating a video stream comprising a plurality of temporally consecutive video frames; a computer vision engine capable of converting the video stream into a binary string, wherein the binary string comprises a plurality of binary bits, each binary bit corresponding to a respective one of the plurality of video frames, and wherein the binary string contains a repeating binary pattern representing a message; and a processor capable of extracting the binary pattern from the binary string and decoding the binary pattern to obtain the message, wherein extracting the binary pattern comprises: converting the binary string from a time domain representation of the binary string to a frequency domain representation of the binary string; locating a peak position in the frequency domain representation of the binary string; determining a frequency domain representation of the binary pattern based on the peak locations; generating a self-concatenated frequency domain representation of the binary pattern by concatenating the frequency domain representation of the binary pattern with a copy of the frequency domain representation of the binary pattern; converting the self-concatenated frequency domain representation of the binary pattern into a time domain representation of the binary pattern; and The binary pattern is determined by locating instances of a predetermined message identifier in the time domain representation of the binary pattern.

11. The communication receiver of claim 10, wherein the video stream records passing vehicles having light transmitters that emit signals related to the message, and wherein the computer vision engine comprises: an object tracker capable of extracting a region of interest (ROI) of each of the plurality of video frames, the ROI including the light emitter; a color filter capable of converting a corresponding ROI into a binary image for each of the plurality of video frames; as well as A binary image classifier is capable of determining, for each of the plurality of video frames, a value of a corresponding one of the plurality of bins based on a corresponding binary image.

12. The communication receiver of claim 11, wherein the color filter comprises a hue-saturation value spatial color filter.

13. The communications receiver of claim 11, wherein the signal comprises modulated infrared light, and wherein the camera comprises an infrared camera.

14. The communication receiver of claim 11, wherein the object tracker comprises a pre-trained cascaded neural network (CNN).

Images