Method and system for data transmission
By using the characteristics of electronic shutters and CMOS sensors in an automated system, reading image information row by row or column by row and calculating signal frequency, the problems of high hardware costs and difficulty in access protection in equipment diagnosis and maintenance are solved, and efficient and safe data transmission and robustness.
Patent Information
- Application Number
- CN202380034290.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-02-16
- Filing Date
- 2023-02-15
- Publication Date
- 2024-11-22
AI Technical Summary
In automated systems, remote diagnosis and maintenance of simple devices such as power supplies have high hardware costs, access protection and fault sensitivity challenges, and existing interfaces such as radio systems require dedicated hardware, making it difficult to achieve simple and secure data transmission .
By using electronic shutter characteristics and CMOS sensors in the camera, image information is read row by row or column by row to form a data stream, and using fast Fourier transform or Gzier algorithm to calculate the signal frequency, the optical signal transmission with high data rate is achieved. Ensure access protection and anti-interference capabilities.
It realizes efficient and secure data transmission on mobile devices, reduces hardware costs, improves the robustness and anti-interference ability of the system, and avoids the risks of unauthorized access and failure.
Smart Images

Figure CN119013919A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for data transmission in a system, comprising a transmitter with a modulation device and an optical transmitter, and a receiver with a camera, in which images are detected in a chronological order in the form of image rows or image columns, and a computing device with a memory, wherein the following steps are performed:
[0002] a) Provide at least one piece of data information;
[0003] b) modulating a carrier signal having a carrier frequency by means of the modulation device with the aid of the at least one piece of data information to form a modulated signal;
[0004] c) transmitting the modulated signal as an optical signal through an optical transmitter of the transmitter;
[0005] d) The light signal is received by a camera of the receiver, which detects an image in the form of image rows or image columns in time sequence, the image rows or image columns being detected at a speed at least twice the period duration of the carrier signal.
[0006] The invention also relates to a system for data transmission, comprising: a transmitter having a modulation device and an optical transmitter; and a receiver having a camera and a computing device, in which images are detected in time sequence in the form of image rows or image columns, and the computing device has a memory. Background Art
[0007] Automation components in electrical systems are becoming increasingly powerful and complex. For this reason, the need for inspection and maintenance is also increasing. While many devices already support remote maintenance and diagnostics, simpler components such as power supplies are often not integrated into the automation network, yet they play a key role in the functionality of the entire system.
[0008] In order to allow digital maintenance and diagnosis even for such devices, corresponding interfaces and diagnostic devices are required. The hardware and software outlay in the device itself and for the diagnostic device is enormous, especially in the case of a large number of corresponding devices. Ideally, no additional hardware should be required for such an interface. No special hardware (special diagnostic or reading devices) should be required for the diagnostic device either.
[0009] For components connected via the system network, remote diagnosis is in principle possible. However, diagnosis and maintenance are usually performed on site. If one now wants to perform diagnosis of a device, the device must first be found in the automation system. In addition, there are challenges with regard to on-site connectivity, device identification or device access (security), as well as device access restrictions and isolation of other participants in the automation system or communication network (i.e. unauthorized access by service technicians to other critical system parts). Therefore, even if the device is in principle integrated into the automation system, a local interface would also be advantageous here.
[0010] Until now, diagnostics in automation components have been performed either via the system network and the automation system or, in the case of simpler unconnected devices, via local interfaces.
[0011] For this purpose, simple wired interfaces such as RS232 or RS485 are usually created and communicate via simple proprietary protocols. In addition to the additional cost of the hardware required for the interface (including connectors, protection class), there are further challenges in terms of non-reaction, EMC (EMC) and compliance. In addition, for this type of interface, special hardware is required for diagnosis (e.g. a handheld computer or PC with a corresponding interface card).
[0012] Alternatively, wireless interfaces such as Bluetooth, WiFi or similar proprietary radio systems can be used. However, this requires a significant expansion of the hardware, which in turn brings corresponding additional costs (hardware, EMC, compliance, ...). When using proprietary systems, special hardware is also required for diagnostics (e.g. own diagnostic equipment or PC interface). In addition, for radio systems, there are further challenges in terms of access protection, security and fault sensitivity. In certain industrial sectors, the emission of radio waves is also a key topic. Summary of the invention
[0013] The object of the present invention is to create a digital interface for a technical device which transmits data from the device to a recipient, such as a mobile device, in a simple manner, in particular with regard to access protection, security and susceptibility to errors.
[0014] The object according to the invention is achieved by a method of the type mentioned at the outset, wherein the following further steps are performed:
[0015] e) determining, by means of a computing device, first image information and at least one second image information from the image by reading out row by row or column by column, similar to their detection in step d), which image information are based on at least a part of the corresponding image row or image column, respectively;
[0016] f) forming a data stream by linking the first and at least one second image information by a computing device;
[0017] g) calculating, by a calculating device, a signal frequency according to the data stream, and identifying the at least one piece of data information according to the signal frequency.
[0018] This makes it possible in a simple manner to transmit data at a high data rate from the device to a recipient such as a mobile device.
[0019] Improved access protection is also ensured, since the data can only be received in the vicinity via optical transmission and cannot be decrypted by unauthorized persons by penetrating the connected IT network and thus cannot have a negative impact on data security.
[0020] Furthermore, the solution is less prone to failures, since the system is based on a very simple and robust principle.
[0021] The technical method of the invention is based on the use of the properties of an electronic shutter together with a CMOS sensor, which is used in conventional cameras with an electronic image sensor, such as in mobile phones, in particular for detecting communications in the visible light range.
[0022] These properties also manifest themselves in a “rolling shutter” effect, which is generally undesirable for cameras of the type mentioned in the introduction.
[0023] The rolling shutter effect (e.g.: "Rolling-Shutter") is a positional error that can occur in photographs or video recordings of moving objects.
[0024] First of all, it seems obvious: the exposure of all points of the camera's light-sensitive sensor elements begins at exactly the same point in time, just as in the case of analog photography using a mechanical shutter.
[0025] However, some cameras are not suitable for the entire surface due to their structural type.However, due to the time-sequential row-by-row or column-by-column reading, the received light signal may change during the reading process of the row or column.
[0026] When all lines are combined to form a common image, the image changes relative to the light signal that changes during this time, so that the light signal cannot be imaged as a whole at once, but can only be imaged line by line, for example, at different points in time.
[0027] The present invention utilizes this effect in order to improve and increase the frame rate by detecting a light signal for each row-by-row or column-by-column readout process and thereby obtaining a sensor data sequence that is sampled much more frequently in time than is the case for the entire image.
[0028] The rolling shutter effect can occur, for example, in video cameras with digital cameras that use image converters using CMOS sensor technology, since these sensors read out the image row by row or column by column.
[0029] In one embodiment of the invention, it is provided that for the determination the first and the at least one second piece of image information are obtained from the image by grouping together parts of image lines or image columns.
[0030] Thereby, the anti-interference capability can be further improved, and the optical sensitivity of the camera can be improved.
[0031] In one embodiment of the invention, it is provided that the signal frequency is calculated from the data stream by means of a fast Fourier transformation.
[0032] Thereby, the signal frequency can be reliably determined and the interference sensitivity can be increased.
[0033] Furthermore, if the evaluation interval is chosen to be correspondingly long, ie to extend over a plurality of bits in the at least one piece of data information, a plurality of modulation states in chronological order can be detected simultaneously.
[0034] In one embodiment of the invention, it is provided that the signal frequency is calculated from the data stream by means of a Goertzel algorithm.
[0035] The Gezil algorithm is a digital signal processing method and a special form of discrete Fourier transform (DFT). Unlike various fast calculation methods in discrete Fourier transform (FFT) that always calculate all discrete spectral components in one block, the Gezil algorithm can only calculate individual discrete spectral components.
[0036] The algorithm is based on a structure consisting of a digital filter, which is extended with a state control. These states divide the calculation into a backward branch, in which the input values sampled in the time domain are loaded, and a forward branch, which provides the output signal. The backward loop is traversed for each digital sample value (English "sample"), and the backward loop is constructed as a recursive digital filter with two state memories and an accumulator. The forward branch is traversed only once after N sample values, and the calculated complex output value is provided from the state memory - that is, the spectral components according to amplitude and phase.
[0037] By choosing the filter coefficients used here, the frequency selectivity can be adjusted. By choosing the number of sample values, the quality factor can be influenced. N can take any natural number.
[0038] However, for each spectral component, a separate Goertzel structure is required. The algorithm can therefore be applied advantageously and with low computational effort, in particular when the entire spectrum is not to be calculated, but only individual spectral components thereof.
[0039] The Gezil algorithm is particularly advantageous and easy to implement. In addition, compared to other methods known in the prior art, the calculation time is shorter, especially when the target frequency to be identified is known, as in the present method.
[0040] In one embodiment of the invention, it is provided that the modulation of the modulation signal is an amplitude, frequency and / or phase modulation and that the modulation frequency of the modulation signal is at least 50 Hz, preferably at least 500 Hz, particularly preferably at least 1000 Hz.
[0041] As a result, the transmission rate of the data information can be significantly increased compared to the evaluation of the entire camera image.
[0042] In one embodiment of the invention, it is provided that the modulation of the modulation signal is a frequency modulation and that the modulation frequencies of the individual modulation states of the modulation do not generate harmonics with respect to one another.
[0043] The calculation of the signal frequency is thereby facilitated since ambiguities are avoided which could have a negative impact on the evaluation.
[0044] In one embodiment of the invention, it is provided that the modulation of the modulation signal has a pulse duty factor of at least 50%, preferably at least 75%.
[0045] As a result, a high light intensity of the optical transmitter can be achieved, which increases the range and the fault sensitivity between sender and receiver, without at the same time impairing the signal functions of the transmitter, such as, for example, operating status displays.
[0046] In one embodiment of the invention, it is provided that the modulation of the modulation signal has a constant pulse duty factor in the individual modulation states.
[0047] This makes it possible for the light transmitter to achieve that the emitted light does not flicker for an observer, which improves the information about the reliability of the technical device without undermining the observer's confidence in the device.
[0048] In particular, when the optical transmitter is simultaneously used for an operating display of the operating state of the technical device, flickering is absolutely undesirable, since an observer could erroneously interpret it as a malfunction of the technical device.
[0049] In one embodiment of the invention, it is provided that the carrier frequency is constant.
[0050] This makes it possible to calculate the signal frequency in a simple manner from the data stream and to increase the robustness of the system.
[0051] In one embodiment of the invention, it is provided that the at least one piece of data information comprises device-specific information, authorization information or authentication information.
[0052] This makes it possible, for example, to send device-specific information, access information, authorization information or authentication information from the device via the optical interface to a receiver, for example a receiver in a smartphone.
[0053] Since only optical signal reception is possible locally, a high level of anti-eavesdropping capabilities is achieved.
[0054] The object according to the invention is also achieved by a system of the type mentioned at the outset, in which the method steps according to the invention are carried out.
[0055] In one embodiment of the invention, it is provided that the transmitter is comprised by a technical device having an optical operating state indicator, and the optical transmitter is formed by the operating state indicator, and that the receiver is comprised by a mobile device, in particular a smartphone or a tablet computer.
[0056] As a result, the system according to the invention can be realized simply and cost-effectively.
[0057] The operating status indicator may be, for example, a light emitting diode or an element of a digital display.
[0058] In one embodiment of the invention, it is provided that the at least one piece of data information comprises device-specific information, authorization information or authentication information of the technical device, which is provided for data transmission in the system.
[0059] The device-specific information, authorization information or authentication information may be provided in the form of data information, sent by a sender and received by a receiver. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] The present invention will be described in more detail below with reference to the embodiments shown in the accompanying drawings.
[0061] Figure 1 A block diagram showing an embodiment of the present invention is shown;
[0062] Figure 2 shows a flow chart of the method according to the present invention;
[0063] Figure 3An example of a modulated optical signal is shown;
[0064] Figure 4 Shown according to Figure 3 An example of utilizing a light signal received by a camera sensor;
[0065] Figure 5 Shown according to Figure 4 An example of a signal read out column by column of an image portion;
[0066] Figure 6 Shows Figure 5 An example of an aggregate signal of;
[0067] Figure 7 Shown by Figure 6 Example of a data stream formed by aggregated signals. DETAILED DESCRIPTION
[0068] Figure 1 A block diagram of an embodiment of the present invention is shown.
[0069] A system S for data transmission comprises a technical device DEV having a transmitter TX having a modulation device MOD and an optical transmitter.
[0070] The optical transmitter may be an operational status indicator of the device DEV and may be, for example, a light emitting diode or an element of a digital display of the device DEV.
[0071] The system S also comprises a mobile radio device MD, a smartphone, having a receiver RX with a camera and a computing device CPU with a memory.
[0072] The computing device may be, for example, a microprocessor or a digital signal processor, programmable logic or the like.
[0073] In the camera, the images are recorded in a chronological order in the form of image rows Z1-Z8 or alternatively in the form of image columns S1-S8, such as Figure 4 As explained in more detail in .
[0074] The light signal LS is in the visible light range and is transmitted from the transmitter to the camera sensor.
[0075] The invention is based on a fundamentally rather disadvantageous property of a camera and utilizes the properties of an “electronic shutter” which leads to a “rolling shutter” effect.
[0076] With an electronic shutter, the full image is not exposed at exactly the same time during image detection.
[0077] The image information is exposed and read out row by row or column by column at a relatively high speed.
[0078] Thus, the signal can be transmitted by a sufficiently large light transmitter, ie the image segment extends over a plurality of rows or columns with the light transmitter. The row frequency and the exposure time of the rows or columns are important in this case.
[0079] Figure 2 A flow chart of the method according to the invention is shown, wherein the following steps are performed:
[0080] a) Provide at least one piece of data information;
[0081] b) modulating, by means of the modulation device MOD, a carrier signal having a carrier frequency with the aid of the at least one piece of data information to form a modulated signal;
[0082] c) transmitting the modulated signal as an optical signal LS via an optical transmitter of the transmitter TX;
[0083] d) receiving the light signal LS by a camera of the receiver RX, which detects the image B comprising the image lines Z1-Z8 and the image columns S1-S8 in the form of image lines Z1-Z8 or image columns S1-S8 in time sequence, wherein the detection speed of the image lines Z1-Z8 or image columns S1-S8 is at least twice the period duration TP of the carrier signal;
[0084] e) determining, by the computing device CPU, from the image B by reading out row by row or column by column, first image information BI1 and at least one second image information BI2-BI8, similar to their detection in step d), which are based on at least a part TM of the corresponding image row Z1-Z8 or image column S1-S8, respectively;
[0085] f) forming a data stream by linking the first and at least one second image information BI1, BI2-BI8 by the computing device CPU;
[0086] g) The computing device CPU calculates the signal frequency according to the data stream, and identifies the at least one piece of data information DI according to the signal frequency.
[0087] The modulation of the modulation signal may be amplitude modulation, frequency modulation and / or phase modulation.
[0088] For high data transmission rates, the modulation frequency of the modulation signal is at least 50 Hz or higher, preferably at least 500 Hz, particularly preferably at least 1000 Hz.
[0089] If the modulation of the modulation signal is frequency modulation, the modulation frequencies of the respective modulation states of the modulation should not generate harmonics with each other.
[0090] For example, the frequency in the modulated signal for transmitting a logic "0" state may be 500 Hz, and the frequency in the modulated signal for transmitting a logic "1" state may be 700 Hz.
[0091] Multi-value modulation is also possible, so that for example to transmit three-value information, three different frequencies can be selected, such as 1000 Hz, 1100 Hz and 1200 Hz.
[0092] The modulation of the modulation signal has a duty cycle of at least 50% in order to obtain a high brightness of the light emitting diode.
[0093] In case of higher duty cycles, the brightness of the LED is only slightly affected and is not perceptible or barely perceptible to the user of the device DEV.
[0094] Furthermore, the modulation of the modulation signal has a constant duty cycle over the various modulation states in order to prevent the user of the device DEV from perceiving undesirable flickering.
[0095] For simple evaluation of the data stream, the carrier frequency is constant.
[0096] Figure 3 An example of a modulated optical signal with a period duration TP is shown.
[0097] Figure 4 An example of a light signal LS received by a camera sensor according to the previous figure is shown.
[0098] The camera of the receiver RX detects images B in chronological order in the form of image sequences S1 - S8 .
[0099] The image has image rows Z1-Z8 and image columns S1-S8.
[0100] In this case, the detection speed of the image sequences S1 - S8 is at least twice the period duration TP of the carrier signal.
[0101] For each column S1-S8, a subset TM is formed over a plurality of rows Z4, Z5, respectively.
[0102] Figure 5 Shown according to Figure 4 Example of a column-by-column readout of a portion of an image.
[0103] The first image information BI1 and the further image information BI2 - BI8 are determined from the image B by reading out the camera column by column.
[0104] Figure 6 Shows Figure 5 Example of an aggregate signal.
[0105] Optionally for this determination, the first and second image information BI1 , BI2 - BI8 can be obtained from the image B by TM-aggregating parts of the image lines Z1 - Z8 AG in order to improve the signal quality.
[0106] Alternatively, only one row in image B may also be further processed.
[0107] Then, by the computing device CPU, a data stream is formed by linking the first and second image information BI1, BI2-BI8.
[0108] Figure 7 Shown by Figure 6 Example of a data stream formed by aggregated signals.
[0109] The signal frequency can be calculated from the data stream by means of a Fast Fourier Transform.
[0110] This makes it possible, for example, to improve the signal quality of the light signal LS.
[0111] For example, ambient light due to LED lighting can be filtered out in a simple manner.
[0112] Lighting technology often uses very similar frequencies in the 600-1000Hz range.
[0113] Therefore, if an FFT or similar method is applied to a segment of a frame to extract and identify the different frequencies, the fundamental frequencies and thus the corresponding signals can be clearly identified in the frequency domain, respectively.
[0114] During the FFT evaluation, depending on the size of the LED, the distance between the smartphone and the device and thus the relative size of the LED in the image, and the line frequency, clearly different values also result for the different frequencies used for transmission.
[0115] Different parameters that influence the “measured” frequency, such as smartphone technology, the size of the LED, etc., are not important for identification, as only the relative frequency jumps between the bits in the data stream are relevant for decoding.
[0116] Despite the diversity of smartphone technology, the frame rate and therefore also the line frequency are comparable in normal video mode and are sufficient for this recognition.
[0117] The SlowMotion mode implemented in various ways in many smartphones is not necessary. In principle, the FFT can be calculated column by column on the image, so that each of these contains a temporally and spatially offset series of image points.
[0118] The values of individual or selected FFTs may be added, which produces a single discrete spectrum and facilitates decoding.
[0119] Alternatively, for the frequency measurement, the points in time TD1 , TD2 , TD3 of a signal change in the data stream can also be determined.
[0120] For this purpose, the start time TS can first be determined by providing a start sequence SI to the transmitter in addition to the data information DI, transmitting the start sequence and receiving the start sequence with the receiver. If the start sequence SI is recognized, it is thereby signaled that a data sequence follows.
[0121] From the time points TD1, TD2, TD3, the signal frequency can be easily calculated.
[0122] The bit intervals in the data stream span multiple frames of the video recording because decoding occurs asynchronously with encoding.
[0123] Advantageously, when using a simple UART transmission with a start bit, 8 data bits, a parity bit and a frame rate of, for example, 60 fps (16.6 ms per frame), the bit interval is >50 ms long, so that it is ensured that at least two frames decode the bit completely and correctly, and that the decoding can be performed using a majority vote. In this case, however, a constant bit rate between sender and receiver should be agreed upon in advance.
[0124] List of reference numerals:
[0125] 1-8 Data series, aggregated image row portion across columns
[0126] 1A-8A, 1B-8B Image row portion spanning columns
[0127] AE Start Sequence
[0128] AG Aggregation of image row parts across columns
[0129] B-image
[0130] BI1-BI8 Image Information
[0131] CPU computing device
[0132] DEV Devices
[0133] DI data information
[0134] LS optical signal, visible light
[0135] MD Mobile Devices
[0136] MOD Modulation Device
[0137] RX Receiver with camera
[0138] S system
[0139] S1-S8 Image Series
[0140] SI Start Information
[0141] t Timeline
[0142] T1-T8 Sampling time points, reading time points from the camera
[0143] TD1-TD3 The time point when the data information changes
[0144] TM Subset
[0145] TP The cycle time of the optical signal
[0146] TS start time
[0147] TX Transmitter with optical transmitter
Claims
1. A method for data transmission in a system (S), the system comprising: a transmitter (TX) having a modulation device (MOD) and an optical transmitter; and a receiver (RX) having a camera, in which the images are recorded in the form of image rows or image columns in a chronological order, and a computing device (CPU) having a memory, wherein the following steps are performed: a) Provide at least one piece of data information; b) modulating, by means of said modulation means (MOD), a carrier signal having a carrier frequency with the aid of said at least one piece of data information to form a modulated signal; c) transmitting the modulated signal as an optical signal (LS) via an optical transmitter of the transmitter (TX); d) receiving the light signal (LS) by means of a camera of the receiver (RX), the camera detecting an image (B) comprising image rows (Z1-Z8) and image columns (S1-S8) in the form of image rows (Z1-Z8) or image columns (S1-S8) in time sequence, wherein the detection speed of the image rows (Z1-Z8) or image columns (S1-S8) is at least twice the period time (TP) of the carrier signal, It is characterized by the following other steps: e) determining, by the computing device (CPU), from the image (B) by reading out row by row or column by column, first image information (BI1) and at least one second image information (BI2-BI8), similar to their detection in step d), said image information being based on at least a part (TM) of the corresponding image row (Z1-Z8) or image column (S1-S8), respectively; f) forming a data stream by linking the first and at least one second image information (BI1, BI2-BI8) by the computing device (CPU); g) calculating, by the computing device (CPU), the signal frequency according to the data stream, and identifying the at least one piece of data information (DI) according to the signal frequency.
2. The method according to the preceding claim, wherein: For the determination, first and at least one second piece of image information (BI1, BI2-BI8) are obtained from the image (B) by aggregating (AG) parts (TM) of image lines (Z1-Z8) or image columns (S1-S8).
3. A method according to any one of the preceding claims, wherein: The signal frequency is calculated from the data stream by means of a Fast Fourier Transform.
4. The method according to any one of claims 1 to 3, wherein: The signal frequency is calculated from the data stream using the Geisel algorithm.
5. A method according to any one of the preceding claims, wherein: The modulation of the modulation signal is amplitude, frequency and / or phase modulation, and the modulation frequency of the modulation signal is at least 50 Hz, preferably at least 500 Hz, particularly preferably at least 1000 Hz.
6. A method according to any one of the preceding claims, wherein: The modulation of the modulation signal is frequency modulation, and the modulation frequencies of the respective modulation states of the modulation do not generate harmonics with each other.
7. A method according to any one of the preceding claims, wherein: The modulation of the modulation signal has a duty cycle of at least 50%, preferably at least 75%.
8. A method according to any one of the preceding claims, wherein: The modulation of the modulation signal has a constant duty cycle in each modulation state.
9. A method according to any one of the preceding claims, wherein: The carrier frequency is constant.
10. A method according to any one of the preceding claims, wherein: The at least one piece of data information (DI) comprises device specific information, authorization information or authentication information.
11. A system (S) for data transmission, the system comprising: a transmitter (TX) having a modulation device (MOD) and an optical transmitter; and a receiver (RX) having a camera in which images are captured in a chronological order in the form of image rows or image columns, and a computing device (CPU) having a memory, wherein the method steps according to any of the preceding claims are performed.
12. System (S) according to the preceding claim, wherein: The transmitter is comprised by a technical device having an optical operating status indicator, and the optical transmitter is formed by the operating status indicator, and the receiver (RX) is comprised by a mobile device (MP), in particular a smartphone or a tablet computer.
13. System (S) according to the preceding claim, wherein: The at least one piece of data information (DI) comprises device-specific information, authorization information or authentication information of the technical device, which is provided for data transmission in the system (S).
