Surveillance system

Abstract

A lighting device 110, provided in an observation location 202, irradiates reference beam S1 with the spatial intensity distribution thereof modulated on the basis of random pattern sets I1-Ir. A detection device 120 is provided in the observation location, includes a photodetector 122 which measures reflected light S2 reflected from an object OBJ, and transmits a data string b1-IM, which indicates output of the photodetector 122 for each of the random patterns I1-bM, to a sensor 130 located away from the observation location 102. A calculation processing device 140, provided located away from the observation location 102, acquires the data string b1-bM from the server 130 and reconstructs a restored image G(x, y) of the object OBJ on the basis of the data string b1-bM and the pattern set I1-IM.

Description

[Technical field]

[0001] The present disclosure relates to surveillance systems. [Background technology]

[0002] IoT (Internet of Things) technology is expected to be used in social infrastructure, especially for maintenance and inspection, crime prevention, and safety confirmation.

[0003] Fig. 1 shows a conventional surveillance system 1. A camera 12 and a transmitter 14 are placed in a remote location 10 to be monitored. The transmitter 14 transmits image data IMG captured by the camera 12 via a network 30 to a destination 20 where the image information will be used. The destination 20 is provided with a receiver 22 that acquires the image data IMG and an image reproduction device 24 that reproduces the image data IMG. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-092172 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the conventional surveillance system 1, the image data IMG itself is transmitted via a network, so there is a risk that the image data IMG may be stolen through hacking. Although it is possible to increase robustness against hacking by using information processing techniques such as encryption, if the encrypted image data is stolen, there is a risk that the original image may be restored.

[0006] The present disclosure has been made in this context, and one exemplary purpose of an aspect thereof is to provide a surveillance system with improved security. [Means for solving the problem]

[0007] An aspect of the present disclosure relates to a surveillance system, comprising: an illumination device provided at an observation location for irradiating a reference light whose spatial intensity distribution is modulated based on a set of random patterns; a detection device provided at the observation location, including a photodetector for measuring reflected light from an object, and for transmitting a data sequence indicating an output of the photodetector for each random pattern to a server located away from the observation location; and a calculation processing device provided at a location away from the observation location for acquiring the data sequence from the server and reconstructing a restored image of the object based on the data sequence and the set of random patterns.

[0008] Another aspect of the present disclosure relates to a surveillance system, comprising: a camera provided at an observation location, a first processor for converting an output image of the camera into a data sequence using a set of random images having a spatially random intensity distribution, a transmitter for transmitting the data sequence to a server located away from the observation location, and a second processor provided away from the observation location for acquiring the data sequence from the server and reconstructing the output image of the camera based on the data sequence and pattern information defining the set of random images. Effect of the Invention

[0009] According to certain aspects of the present disclosure, the security of a surveillance system can be improved. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates a conventional surveillance system. [Diagram 2] FIG. 1 is a diagram showing a monitoring system according to a first embodiment. [Diagram 3] FIG. 11 is a diagram showing a monitoring system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Overview of the embodiment) A summary of some exemplary embodiments of the present disclosure will be described. This summary is intended to provide a simplified summary of some concepts of one or more embodiments in order to provide a basic understanding of the embodiments as a prelude to the detailed description that follows, and is not intended to limit the scope of the invention or disclosure. Furthermore, this summary is not intended to be a comprehensive overview of all possible embodiments, nor is it intended to limit essential components of the embodiments. For convenience, the term "one embodiment" may be used to refer to one embodiment (example or variant) or multiple embodiments (examples or variants) disclosed in this specification.

[0012] In this specification, "intensity distribution is random" does not mean that it is completely random, but it is sufficient that it is random to the extent that an image can be reconstructed in ghost imaging. Therefore, in this specification, "random" can include a certain degree of regularity. In addition, "random" does not require that it is unpredictable, but may be predictable and reproducible.

[0013] An embodiment disclosed herein relates to a surveillance system, comprising: an illumination device provided at an observation location and configured to irradiate a reference light whose spatial intensity distribution is modulated based on a set of random patterns; a detection device provided at the observation location and configured to include a photodetector for measuring reflected light from an object and transmit a data sequence indicating an output of the photodetector for each random pattern to a server located away from the observation location; and a calculation processing device provided at a location away from the observation location and configured to acquire the data sequence from the server and reconstruct a restored image of the object based on the data sequence and the set of random patterns.

[0014] According to this configuration, even if the data string, which is the energy information, is stolen, the original image cannot be restored unless the corresponding random pattern set is also stolen, thereby improving security.

[0015] In one embodiment, the resolution of the random pattern may be controllable from the processor, thereby allowing control of the resolution of the image.

[0016] In one embodiment, the number of patterns in the set of random patterns may be controllable from the processor, thereby allowing control of the image resolution.

[0017] In one embodiment, the set of random patterns may be changeable. If the same set of random patterns is used for a long period of time, there is a possibility that the pattern information may be leaked. Therefore, by changing the set of random patterns, the leakage of the pattern information can be prevented.

[0018] In one embodiment, the set of random patterns may be generated by a pseudo-random number generator, the seed of which may be shared between the lighting device and the processing unit.

[0019] In one embodiment, the transmitter may transmit the data stream by scrambling the order of the data contained therein, thereby providing an additional level of security.

[0020] An embodiment disclosed herein relates to a surveillance system, comprising: a camera provided at an observation location, a first processor for converting an output image of the camera into a data sequence using a set of random images having a spatially random intensity distribution, a transmitter for transmitting the data sequence to a server remote from the observation location, and a second processor provided at a distance from the observation location for acquiring the data sequence from the server and reconstructing the output image of the camera based on the data sequence and pattern information defining the set of random images.

[0021] Even if the data string is stolen, the original image cannot be restored unless the set of random images (pattern information) used to generate it is stolen, which increases security.

[0022] In one embodiment, the set of random images may be changeable.

[0023] In one embodiment, the set of random images may be generated by a pseudo-random number generator, the seed of which may be shared between the first and second processing units.

[0024] (Embodiment) Hereinafter, some embodiments will be described with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and duplicated descriptions will be omitted as appropriate. In addition, the embodiments are not intended to limit the disclosure or invention, but are merely examples, and all features and combinations thereof described in the embodiments are not necessarily essential to the disclosure or invention.

[0025] (Embodiment 1) 2 is a diagram showing a monitoring system 100 according to embodiment 1. The monitoring system 100 mainly includes a lighting device 110, a detection device 120, a server 130, and a processing device 140, and monitors the state of an observation location 102 from a remote location 104.

[0026] The observation point 102 is provided with an illumination device 110 and a detection device 120. The illumination device 110 is a pseudo-thermal light source, and outputs a plurality of M pattern signals PTN1 to PTN M The spatial intensity distribution I(x, y) of the reference beam S1 is modulated based on (a set of random patterns), and the modulated reference beam S1 is irradiated onto the object OBJ. The random intensity distribution of the reference beam S1 is also referred to as a random pattern.

[0027] The illumination device 110 includes a light source 112, a patterning device 114, and a pattern generator 116. The light source 112 generates light S0 having a uniform intensity distribution. The light source 112 may be a laser, a light-emitting diode, or the like. The wavelength and spectrum of the reference light S1 are not particularly limited, and may be white light having multiple or continuous spectra, or monochromatic light including a predetermined wavelength. The wavelength of the reference light S1 may be infrared or ultraviolet.

[0028] Patterning device 114 has a plurality of pixels arranged in a matrix, and is configured to be able to spatially modulate light intensity distribution I based on a combination of on and off states of the plurality of pixels. In this specification, a pixel in an on state is called an on pixel, and a pixel in an off state is called an off pixel. In the following description, for ease of understanding, each pixel will take only two values ​​(1, 0), on and off, but this is not limited thereto, and pixels may take intermediate gradations.

[0029] A reflective DMD (Digital Micromirror Device) or a transmissive liquid crystal device can be used as the patterning device 114. A pattern signal PTN (image data) generated by a pattern generator 116 is provided to the patterning device 114.

[0030] The pattern generator 116 generates a random pattern I of the reference beam S1. r The pattern signal PTN that specifies r and generates a pattern signal PTN for each irradiation period. r Switch sequentially (r=1,2,…M).

[0031] The detection device 120 includes a photodetector 122 that measures reflected light S2 from an object OBJ, and detects random patterns I1 to I M Data strings b1 to b2 each showing the output of the photodetector 122 M is transmitted via a network 132 to a server 130 located away from the observation location 102.

[0032] The output of the photodetector 122 corresponds to a random pattern I r is a spatial integral value of the light energy (or intensity) incident on the photodetector 122 when the object OBJ is irradiated with a reference light having a value of . Therefore, the photodetector 122 can be a single-pixel photodetector.

[0033] The output signal of the photodetector 122 is converted into a digital signal by an A / D converter 124 .

[0034] The data processing unit 126 receives the output of the A / D converter 124 and converts the random patterns I1 to I M Data strings (also called energy information) b1 to b2 indicating the output of the photodetector 122 corresponding to M The data processing unit 126 is supplied with data (value r) from the pattern generator 116 indicating which random pattern is currently being used.

[0035] A random pattern I r Assume that the reference light S1 of a random pattern is irradiated for a certain irradiation period. If the sampling period of the A / D converter 124 is sufficiently shorter than the irradiation period of the reference light S1 of one random pattern, the A / D converter 124 generates multiple sampled values ​​for each irradiation period (each random pattern). The data processing unit 126 calculates the integral value, average value, or maximum value of the multiple sampled values ​​for a certain random pattern I. r The detection intensity b corresponding to r Alternatively, some of the sampled values ​​may be selected, and the integral, average, or maximum value of the selected sampled values ​​may be output as the detection intensity b r The selection of the multiple sampled values ​​may be, for example, by extracting the xth to yth sampled values ​​counting from the maximum value, by excluding sampled values ​​lower than an arbitrary threshold value, or by extracting sampled values ​​in a range where the magnitude of signal fluctuation is small.

[0036] When a device capable of setting the exposure time and exposure timing, such as a camera, is used as the photodetector 122, the output of the photodetector 122 is used as it is as the detection intensity b r It can be said that:

[0037] The transmitter 128 receives the data sequence b1 to b2 generated by the data processing unit 126. M to a server 130 located away from the observation location. The transmitter 128 may encrypt the data strings b1 to bM before transmitting them.

[0038] The output of the A / D converter 124 may be sent directly to the network 132. In this case, the processing of the data processing unit 126 described above may be executed by the arithmetic processing device 140.

[0039] The arithmetic processing device 140 is provided in the remote location 104. The arithmetic processing device 140 receives data sequences b1 to b2 from the network 132. M Then, the arithmetic processing device 140 obtains the data sequence b1 to b M and random pattern I1~I M Based on the set of x, y, we reconstruct a restored image G(x, y) of the object OBJ.

[0040] The arithmetic processing device 140 includes a pattern generator 142 and a reconstruction processing unit 144. The arithmetic processing device 140 is, for example, a computer, and can be implemented as a combination of a processor (hardware) such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcomputer, and a software program executed by the processor (hardware). The arithmetic processing device 140 may be a combination of multiple processors. Alternatively, the arithmetic processing device 140 may be implemented as dedicated hardware.

[0041] The pattern generator 142 generates a plurality of random patterns I1 to I2 used in the lighting device 110. M Play a set of.

[0042] The reconstruction processing unit 144 generates a plurality of random patterns I1 to I M and multiple detection intensities b1 to b M Based on this, a restored image G(x, y) of the object OBJ is reconstructed by computation.

[0043] The restored image G(x,y) may be reconstructed using the correlation function of Equation (1). r is the intensity distribution of the r-th reference beam S1, and b r is the detected intensity b corresponding to the rth intensity distribution Ir r The value is

number

[0044] The above is the configuration of the monitoring system 100. In this monitoring system 100, the server 130 stores data strings b1 to b M is stored, but the corresponding random patterns I1 to I M The set of data b1 to b2 does not exist in the server 130. Therefore, a malicious third party may obtain the data sequence b1 to b2 by unauthorized access. M Even if you can steal random patterns I1~I M Since the original image G(x,y) cannot be restored because the original image cannot be stolen, security can be improved compared to conventional surveillance systems that transmit image data itself.

[0045] Next, the pattern generator 116 of the lighting device 110 and the pattern generator 142 of the arithmetic processing device 140 generate random patterns I1 to I r The method of sharing will be described with reference to several examples.

[0046] Example 1 In the simplest case, the pattern generators 116 and 142 are each provided with fixed random patterns I1 to I M A set of random patterns I1 to I MIf one were to obtain this set, it would be necessary to identify the location of the observation point 102, go to the actual observation point 102, and measure the output S1 of the lighting device 110, which is practically impossible. Therefore, even with this simple configuration, a sufficiently high level of security can be ensured.

[0047] Example 2 In the second embodiment, the pattern generators 116 and 142 are provided with fixed random patterns I1 to I M A plurality of sets of random patterns are stored, and the set of random patterns to be used is changed.

[0048] One of the arithmetic processing device 140 and the lighting device 110 operates as a master, and the other as a slave. The master determines the set of random patterns to be used and transmits it to the slave. In this case, the master may transmit to the slave an identification number of the set to be used each time the set to be used is changed. Alternatively, the master may transmit to the slave data indicating the order in which the multiple sets are to be used.

[0049] This makes it possible to further improve security compared to the first embodiment.

[0050] Example 3 In the third embodiment, the pattern generators 116 and 142 generate N random patterns I1 to I N In the third embodiment, either the arithmetic processing device 140 or the lighting device 110 operates as a master, and the other operates as a slave. The master generates N random patterns I1 to I N The slave transmits data indicating the M random patterns to be used and their order from the above.

[0051] Example 4 The pattern generator 116 and the pattern generator 142 may be configured with the same pseudorandom number generator, and the master may transmit to the slave the seed of the pseudorandom number generator to be used.

[0052] Next, a modification of the first embodiment will be described.

[0053] (Variation 1) Random patterns I1 to I modulate the reference beam S1 M The resolution of the random pattern G(x,y) may be controllable from the processor 140. For example, the resolution provided to the image may vary from situation to situation due to security or privacy issues. Since the resolution of the restored image G(x,y) depends on the resolution of the random pattern, the resolution of the restored image can be controlled by changing the resolution of the random pattern. When a DMD is used as the patterning device 114, the resolution of the random pattern can be controlled by changing the number of mirrors used or the number of bins (the number of mirrors that represent one pixel).

[0054] (Variation 2) The number of irradiations for one measurement, i.e., random patterns I1 to I M The number M of patterns included in the set may be controllable by the arithmetic processing device 140. Since the resolution of the restored image G(x, y) depends on the number of irradiations of the random pattern, the resolution of the restored image G(x, y) can be controlled by controlling the number of irradiations.

[0055] (Variation 3) The transmitter 128 receives the data sequence b1 to b M may be transmitted with the order of the data contained therein scrambled, which can further increase security.

[0056] (Embodiment 2) 3 is a diagram showing a monitoring system 200 according to embodiment 2. The monitoring system 200 includes a camera 210, a first arithmetic processing device 220, a server 230, and a second arithmetic processing device 240.

[0057] The camera 210 and the first arithmetic processing device 220 are provided at the observation location 202. An image F(x, y) captured by the camera 210 is input to the first arithmetic processing device 220. The first arithmetic processing device 220 converts the output image F(x, y) of the camera 210 into a set I1(x, y) to I2(x, y) of random images I(x, y) having a spatially random intensity distribution. M Using (x, y), data sequence b1 to b M A random image I(x,y) is generated by the pattern generator 222.

[0058] The data string generation unit 224 converts the camera image F(x, y) into a random image I r By calculating the data b r For example, the formula (2) can be used for the conversion.

number

[0059] Equation (2) expresses the camera image F(x,y) and the random image I r This represents the process of multiplying the corresponding pixel values ​​of (x, y) and accumulating them for all pixels. r The object is then applied with a random pattern I r The first calculation processing unit 220 measures all the random images I1 to I M The same calculation is performed for M pieces of data b1 to b M Generate.

[0060] The transmitter 226 receives the data sequence b1 to b M is transmitted to a server 230 located away from the observation point 202.

[0061] The second arithmetic processing device 240 is provided in the remote location 204. The configuration and functions of the second arithmetic processing device 240 are similar to those of the arithmetic processing device 140 in the first embodiment.

[0062] According to the second embodiment, security can be improved as in the first embodiment. In addition, since there is no need to provide a complex lighting device 110 on the observation point 202 side, costs can be reduced compared to the first embodiment.

[0063] The relationship between the pattern generator 222 and the pattern generator 242 corresponds to the relationship between the pattern generator 116 and the pattern generator 142 in Fig. 2. Therefore, Examples 1 to 4 described in the first embodiment are also applicable to the second embodiment.

[0064] The present invention has been described above based on an embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present invention. Such modifications will be described below.

[0065] In the embodiment, the illumination device 110 is configured by a combination of the light source 112 and the patterning device 114, but this is not limited thereto. For example, the illumination device 110 may be configured by an array of a plurality of semiconductor light sources (LEDs (light emitting diodes) and LDs (laser diodes)) arranged in a matrix, and the on / off (or brightness) of each semiconductor light source may be controlled.

[0066] In the embodiment, a method using correlation calculation has been described as a ghost imaging (or single pixel imaging) method, but the image reconstruction method is not limited to this. In some embodiments, instead of correlation calculation, an image may be reconstructed by an analytical method using Fourier transform or inverse Hadamard transform, a method for solving an optimization problem such as sparse modeling, or an algorithm using AI or machine learning.

[0067] The present invention has been described in terms of specific embodiments, but the embodiments merely show one aspect of the principles and applications of the present invention, and many modifications and changes in arrangement are possible within the scope of the invention as defined in the claims. [Industrial Applicability]

[0068] The present invention relates to a surveillance system. [Explanation of symbols]

[0069] 100 Surveillance System 102 Observation points 104 Remote Area 110 Lighting equipment 112 Light source 114 Patterning Device 116 Pattern Generator 120 Detection Device 122 Photodetector 124 A / D converter 126 Data Processing Unit 128 Transmitter 130 Server 132 Network S1 reference light S2 reflected light 140 Processing Unit 142 Pattern Generator 144 Reconstruction processing unit 200 Surveillance System 202 Observation points 204 Remote Area 210 Camera 220 First Processing Unit 222 Pattern Generator 224 Data String Generation Unit 226 Transmitter 230 Server 232 Network 240 Second Processing Unit 242 Pattern Generator 244 Reconstruction processing section,

Claims

1. an illumination device that is provided at an observation point and that irradiates a reference light whose spatial intensity distribution is modulated based on a set of random patterns; A detection device that is provided at the observation location, includes a photodetector that measures reflected light from an object, and transmits a data string indicating an output of the photodetector for each of the random patterns to a server away from the observation location; a processor that is provided away from the observation location, acquires the data sequence from the server, and reconstructs a restored image of the object based on the data sequence and the set of random patterns; A monitoring system comprising:

2. 2. The surveillance system according to claim 1, wherein the resolution of the random pattern is controllable from the processor.

3. 3. The monitoring system according to claim 1, wherein the number of patterns included in the set of random patterns is controllable by the arithmetic processing device.

4. 4. The surveillance system according to claim 1, wherein the set of random patterns is changeable.

5. 5. The monitoring system according to claim 1, wherein the detection device transmits the data string after scrambling the order of the data contained therein.

Images