Surveillance system and method

Abstract

To provide a monitoring system capable of transmitting monitor information while improving confidentiality and usefulness, and a monitoring method.SOLUTION: A monitoring system comprises: a detection unit for detecting a monitoring target from a video image; an encoding unit for acquiring and encoding information relating to the detected target; a superposing unit for superposing the encoded information on a video image as an invisible pattern; and a transmission unit for transmitting the video image on which the information is superposed by the superposing unit. The monitoring system further comprises: a reception unit for receiving the video image transmitted by the transmission unit; a code acquisition unit for acquiring a code superposed on the video image received by the reception unit; and an information acquisition unit for acquiring the information relating to the detected target from the code acquired by the code acquisition unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a monitoring system and a monitoring method, and more particularly to a monitoring system and a monitoring method including a configuration for transmitting monitoring information. [Background technology]

[0002] In addition to video, surveillance systems transmit surveillance information acquired by sensors and video analysis. Conventional technologies have either established a separate communication from the video to transmit surveillance information, or used a method of visually superimposing detection information on the video. One method of visually superimposing detection information is to surround a detected person with a square, for example.

[0003] On the other hand, Patent Document 1 discloses that in order to prevent the leakage of personal information in a face matching system, a lossy feature database is provided that stores lossy features that represent the characteristics of the facial images of registered persons and that make it impossible to restore the facial image, and a matching device calculates the lossy features based on the facial image of a person included in an image captured by an imaging device, and compares them with the lossy features registered in the lossy feature database to determine whether or not a person included in the captured image is registered in the database. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 003400 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when surveillance information is visually superimposed on the video from the network camera and transmitted, the receiving side can only obtain the superimposed video. When checking the saved video one frame at a time, the superimposed information is often a hindrance to the video with the surveillance information superimposed. Despite this, it was impossible to erase the superimposed information later. In addition, if the video is leaked to a third party, it is easy to determine what is being monitored, which leads to low security.

[0006] On the other hand, when transmitting surveillance information via a separate communication channel from the video, it is necessary to accurately synchronize the video and detection information. In particular, in systems that include video recording, the video and surveillance information must be managed as separate files, which creates the issue of cumbersome data management. In addition, when playing back video together with surveillance information, a system and processing power are required for this purpose.

[0007] In addition, in Patent Document 1, the result of face matching processing is output to a display device, but the above-mentioned problem still exists.

[0008] In view of the above-mentioned problems, an object of the present invention is to provide a monitoring system and a monitoring method capable of transmitting monitoring information with improved confidentiality and usefulness. [Means for solving the problem]

[0009] In order to achieve the above-mentioned objective, one representative surveillance system of the present invention is characterized by comprising a detection unit that detects a surveillance target from an image, a coding unit that obtains and codes information about the detected target, a superimposition unit that superimposes the coded information on the image as an invisible pattern, and a transmission unit that transmits the image superimposed by the superimposition unit.

[0010] Furthermore, one of the surveillance methods of the present invention is characterized by having a detection step of detecting a surveillance target from an image, a coding step of obtaining and coding information relating to the detected target, a superimposition step of superimposing the coded information on the image as an invisible pattern, and a transmission step of transmitting the image superimposed in the superimposition step. Effect of the Invention

[0011] According to the present invention, in a monitoring system and a monitoring method, monitoring information can be transmitted with improved confidentiality and usefulness. Problems, configurations and effects other than those described above will become apparent from the following embodiments. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a computer system for implementing aspects according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram showing an embodiment of the monitoring system of the present invention. [Diagram 3] FIG. 3 is a flow chart showing an example of processing on the transmission side of the monitoring system of the present invention. [Figure 4] FIG. 4 is a flow chart showing an example of the process on the receiving side of the monitoring system of the present invention. [Diagram 5] FIG. 5 is a diagram for explaining a specific example of processing on the transmission side of the monitoring system of the present invention. [Figure 6] FIG. 6 is a diagram for explaining a specific example of the processing on the receiving side of the monitoring system of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] An embodiment for carrying out the present invention will be described.

[0014] <Computer system for implementing aspects according to the embodiment> 1 is a block diagram of a computer system 300 for implementing aspects according to an embodiment of the present disclosure. The mechanisms and devices of various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 300 include one or more processors 302, memory 304, terminal interface 312, storage interface 314, I / O (input / output) device interface 316, and network interface 318. These components may be interconnected via a memory bus 306, an I / O bus 308, a bus interface unit 309, and an I / O bus interface unit 310.

[0015] Computer system 300 may include one or more processing devices 302A and 302B, collectively referred to as processor 302. Each processor 302 executes instructions stored in memory 304 and may include an on-board cache. In some embodiments, computer system 300 may include multiple processors, while in other embodiments computer system 300 may be a single processing device system. The processing devices may be a central processing unit (CPU), a field-programmable gate array (FPGA), a graphics processing unit (GPU), a digital signal processor (DSP), or the like.

[0016] In some embodiments, memory 304 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. In some embodiments, memory 304 represents the entire virtual memory of computer system 300, and may include virtual memory of other computer systems connected to computer system 300 via a network. While memory 304 may be conceptually considered as a single entity, in other embodiments, memory 304 may be a more complex arrangement, such as a hierarchy of caches and other memory devices. For example, memory may exist as multiple levels of caches, and these caches may be divided by function. As a result, one cache may hold instructions, while other caches hold non-instruction data used by the processor. Memory may be distributed and associated with various different processing units, such as in a so-called Non-Uniform Memory Access (NUMA) computer architecture.

[0017] The memory 304 may store all or a portion of the programs, modules, and data structures implementing the functions described herein. For example, the memory 304 may store the latent factor identification application 350. In an embodiment, the latent factor identification application 350 may include instructions or descriptions to execute the functions described below on the processor 302, or may include instructions or descriptions interpreted by another instruction or description. In an embodiment, the latent factor identification application 350 may be implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices instead of or in addition to a processor-based system. In an embodiment, the latent factor identification application 350 may include data other than instructions or descriptions. In an embodiment, a camera, sensor, or other data input device (not shown) may be provided to communicate directly with the bus interface unit 309, the processor 302, or other hardware of the computer system 300. In such a configuration, the need for the processor 302 to access the memory 304 and the latent factor identification application may be reduced.

[0018] Computer system 300 may include a bus interface unit 309 for communication between processor 302, memory 304, display system 324, and I / O bus interface unit 310. I / O bus interface unit 310 may couple to I / O bus 308 for transferring data to and from various I / O units. I / O bus interface unit 310 may communicate with a number of I / O interface units 312, 314, 316, and 318, also known as I / O processors (IOPs) or I / O adapters (IOAs), via I / O bus 308. Display system 324 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both data to display device 326. Computer system 300 may also include one or more sensors or other devices configured to collect data and provide the data to processor 302. For example, computer system 300 may include environmental sensors to collect humidity data, temperature data, pressure data, etc., and motion sensors to collect acceleration data, movement data, etc. Other types of sensors may also be used. Display memory may be dedicated memory for buffering video data. Display system 324 may be connected to a display device 326, such as a separate display screen, a television, a tablet, or a handheld device. In some embodiments, display device 326 may include a speaker for rendering audio. Alternatively, the speaker for rendering audio may be connected to an I / O interface unit. In other embodiments, the functionality provided by display system 324 may be implemented by an integrated circuit including processor 302. Similarly, the functionality provided by bus interface unit 309 may be implemented by an integrated circuit including processor 302.

[0019] The I / O interface unit provides the ability to communicate with various storage or I / O devices. For example, the terminal interface unit 312 may be attached to user I / O devices 320, such as user output devices such as a video display, a television with speakers, and user input devices such as a keyboard, a mouse, a keypad, a touchpad, a trackball, buttons, a light pen, or other pointing device. A user may use a user interface to enter input data or instructions to the user I / O devices 320 and the computer system 300, and receive output data from the computer system 300, by manipulating the user input devices. The user interface may be displayed on a display, played through speakers, or printed via a printer, for example, via the user I / O devices 320.

[0020] The storage interface 314 may be attached to one or more disk drives or direct access storage devices 322 (typically magnetic disk drive storage devices, but may also be an array of disk drives or other storage devices configured to appear as a single disk drive). In some embodiments, the storage devices 322 may be implemented as any secondary storage device. Contents of the memory 304 may be stored in the storage devices 322 and retrieved from the storage devices 322 as needed. The network interface 318 may provide a communications path to enable the computer system 300 and other devices to communicate with each other. This communications path may be, for example, a network 330.

[0021] 1 includes a bus structure providing direct communication paths between processor 302, memory 304, bus interface 309, display system 324, and I / O bus interface unit 310, although in other embodiments computer system 300 may include point-to-point links, multiple hierarchical buses, parallel or redundant communication paths in a hierarchical, star, or web configuration. Additionally, while I / O bus interface unit 310 and I / O bus 308 are shown as a single unit, in reality computer system 300 may include multiple I / O bus interface units 310 or multiple I / O buses 308. Additionally, although multiple I / O interface units are shown to separate I / O bus 308 from various communication paths leading to various I / O devices, in other embodiments some or all of the I / O devices may be directly connected to a single system I / O bus.

[0022] In some embodiments, computer system 300 may be a device that receives requests from other computer systems (clients) without a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer, etc. In other embodiments, computer system 300 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

[0023] <Block diagram> Fig. 2 is a block diagram showing an embodiment of a surveillance system of the present invention. The surveillance system shown in Fig. 2 includes an imaging device 10, a transmitting-side processing device 20, a storage device 30, a receiving-side processing device 40, a storage device 50, a display device 60, and an alarm device 70. The transmitting-side processing device 20 and the receiving-side processing device 40 are capable of transmitting information via a network 80.

[0024] The imaging device 10, the transmission side processing device 20, and the storage device 30 may be configured as separate devices so as to be able to exchange information as the transmission side, or may be configured as an integrated device. The reception side processing device 40, the storage device 50, the display device 60, and the alarm device 70 may be configured as separate devices so as to be able to exchange information as the reception side, or may be configured as an integrated device. These can use the configuration of the computer system 300 shown in Fig. 1. In this case, the computer system 300 can be applied to both the transmission side and the reception side. A plurality of transmission sides and reception sides may be provided.

[0025] The imaging device 10 has a camera configuration and can be placed in various locations. For example, it can be placed at a monitoring location as a surveillance camera. The imaging device 10 can be configured as a camera that obtains information by forming an image of incident light on an imaging element via a lens and an aperture. Examples of the imaging element include a CCD (Charge-Coupled Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The imaging device 10 captures images at, for example, 3 frames per second (3 fps) or more, and the information is sent to the transmission-side processing device 20. A plurality of imaging devices 10 can be installed depending on the situation.

[0026] The transmission-side processing device 20 receives and processes the video from the imaging device 10. The transmission-side processing device 20 may use a processing device of an appropriate type as required, such as a CPU, FPGA, GPU, or DSP. The transmission-side processing device 20 has the functions of an object detection unit 21, an encoding unit 22, a superimposition unit 23, and a transmission unit 24. Note that an object refers to a detection target, and is a concept that includes people.

[0027] The object detection unit 21 detects a specific object based on the image from the imaging device 10. The detection can use a learning model prepared in advance to detect the monitoring target. The presence or absence of the detection target and the coordinates of the detection target are obtained by performing image analysis on the image. A specific example of the processing of the object detection unit 21 is shown in S102 of FIG. 3 described later.

[0028] The encoding unit 22 acquires information about the detected object, and encodes it as a specific two-dimensional pattern such as a barcode or QR code. Here, the information about the detected object includes the name, importance, display method, etc., and is acquired from a table 31 in the storage device 30. A specific example of the processing of the encoding unit 22 will be shown in S105 of Fig. 3 described later.

[0029] The superimposing unit 23 superimposes the pattern information coded by the coding unit 22 onto the image of the video from the imaging device 10. This pattern is superimposed onto the image of the video from the imaging device 10 like a digital watermark. When superimposing, the coded pattern is superimposed on the video in an invisible state. For example, it is superimposed on the image with a difference in pixel values ​​that is invisible to the human eye. A specific example of the process of the superimposing unit 23 is shown in S106 of FIG. 3, which will be described later.

[0030] The transmitting unit 24 transmits the video on which the code has been superimposed by the superimposing unit 23 to the reception-side processing device 40 via the network 80. The process here corresponds to S107 in Fig. 3, which will be described later.

[0031] The storage device 30 is a device that records information required for processing by the transmission-side processing device 20. Here, information in table 31 is stored. Also, images captured by the imaging device 10 may be recorded. The storage device 30 can apply an appropriate method as required, such as a hard disk drive (HDD), a solid state drive (SSD), or a digital data storage (DDS). The table 31 is a table that lists in advance a learning model for detecting a monitoring target, and the name, importance, display method, etc. of the monitoring target.

[0032] The receiving-side processing device 40 processes the information sent from the transmitting-side processing device 20. The receiving-side processing device 40 may use a processing device of an appropriate type as necessary, such as a CPU, an FPGA, a GPU, or a DSP. The receiving-side processing device 40 includes the functions of a receiving unit 41, a code acquiring unit 42, an information acquiring unit 43, a visualization unit 44, and an alarm issuing unit 45.

[0033] The receiving unit 41 receives the video transmitted from the transmitting unit 24 of the transmission-side processing device 20. The process here corresponds to S151 in Fig. 4, which will be described later.

[0034] The code acquisition unit 42 acquires the code from the video on which the code is superimposed and received by the receiving unit 41. A specific example of the process of the code acquisition unit 42 will be shown in S152 of Fig. 4, which will be described later.

[0035] The information acquisition unit 43 acquires information about the detected object from the acquired code. If necessary, encrypted information may be decrypted. A specific example of the process of the information acquisition unit 43 will be shown in S153 of FIG. 4, which will be described later.

[0036] When information on a monitored object is acquired, the visualization unit 44 generates additional information such as a frame and superimposes it on the image so that the detected object can be easily identified. At this time, it is possible to select whether or not to superimpose the additional information based on the setting contents of the setting information 51. The additional information may be a figure or a character, and for example, the detected object is visualized on the image by a rectangular frame or the like that surrounds the image. It is also possible to determine whether or not to superimpose the additional information on the image based on the setting information 51. Specific examples of the processing by the visualization unit 44 are shown in S155 to S157 of FIG. 4, which will be described later.

[0037] The alarm issuing unit 45 transmits a signal for issuing an alarm in the alarm device 70. The signal can be transmitted based on information related to the detected object acquired by the information acquiring unit 43.

[0038] The storage device 50 is a device that records information required for processing by the receiving-side processing device 40. Here, setting information 51 is stored. Information output by the receiving-side processing device 40 may also be recorded. The storage device 50 may use a suitable method as required, such as an HDD, SSD, or DDS. The setting information 51 stores information on whether or not additional information such as a frame is to be superimposed on the video in the visualization unit 44. This setting information 51 can be set in advance by an operation by a user or the like.

[0039] The display device 60 is a device capable of displaying the contents of the video and the like output by the receiving-side processing device 40. For example, the contents are displayed using a liquid crystal display (LCD), an organic electroluminescence (OEL) display, a touch panel, or the like. The display device 60 may also be provided with an operation button, and may be capable of inputting settings for the receiving-side processing device 40 and the setting information 51.

[0040] The alarm device 70 is a device that issues information other than video, separately from the display device 60, based on the results of processing by the receiving-side processing device 40. The alarm device 70 can be an alarm device that issues sound using a buzzer, speaker, etc., or an alarm device that issues light using an LED, etc. The method of issuing an alarm may be changed according to the level of importance. For example, as the level of importance increases, the sound may be increased, the type of warning sound may be changed, the intensity of the light may be increased, the light may be made to blink, the rate at which the light blinks may be shortened, the color of the light may be changed, etc. Here, the level of importance can be a concept that includes the degree of danger of the monitored object.

[0041] The network 80 is a line capable of data communication connecting each device, and can be applied regardless of the type of line, such as a dedicated line, an intranet, or an IP network such as the Internet.

[0042] <Flowchart> FIG. 3 is a flow chart showing an example of processing on the transmission side of the monitoring system of the present invention. Here, the processing in the transmitting side processor 20 will be described with reference to the flowchart in FIG.

[0043] First, in S101, an image of a subject is captured by performing a start operation such as turning on the power in the camera that is the imaging device 10. The captured image is sent from the imaging device 10 to the transmission-side processing device 20 and acquired by the transmission-side processing device 20.

[0044] Next, in S102, object detection processing is performed by image processing. Here, object detection refers to detecting some object or person in the image of the captured video. Coordinates of the object detected during object detection are also obtained.

[0045] Object detection can be performed by identifying a target that has been registered in advance. For example, it can identify that the target is a person and then identify the characteristics of that person. It can also identify the person's behavior or the person carrying a specific object (such as a knife). If the target is an object, it can also identify the characteristics of the object. Object detection can also be performed by referring to multiple frames, not just one frame.

[0046] Various methods can be applied as object detection methods. For example, AI (artificial intelligence) can be used to detect objects. One example of AI is a process that uses neural networks or deep learning to extract features. People and objects can be identified by using trained models. Methods that do not use AI can also be applied, such as detection according to predefined patterns.

[0047] Next, in S103, whether or not an object is detected is determined by object detection processing. If an object is detected (Yes), the process proceeds to S104. On the other hand, if an object is not detected (No), the process proceeds to S107, where the video is transmitted without any special information being superimposed.

[0048] In S104, it is determined whether the detected object is defined. Here, it is determined whether it is a monitored object such as a specific object or person registered in advance. This makes it possible to determine whether the detected object is a monitored object. If the detected object is defined (Yes), the process proceeds to S105. If the detected object is not defined (No), the process proceeds to S107, where the video is distributed as is without any special information being superimposed.

[0049] Whether or not a detected object is defined is determined using table 110, which defines the importance of the object to be detected, etc. Table 110 here corresponds to table 31 in FIG. 2. In table 110, information on the characteristics of the object to be detected is recorded, and it is possible to identify whether or not the characteristics match those of the object detected in S102. For example, if the degree of match is equal to or higher than a predetermined level, it can be determined that the detected object is defined. Furthermore, table 110 also stores information on the object, such as its name, importance, and display method. As a result, when it is determined that the detected object is defined, the name, importance, and display method of the object can be obtained as information at the same time.

[0050] In S105, information about the detected object is coded two-dimensionally. Coding in two dimensions means converting into a pattern that can be expressed two-dimensionally according to a specific rule. For example, a code such as a barcode (one-dimensional code or two-dimensional code) or a QR code (two-dimensional code) can be used here. These codes can store information about the detected object as information within the code by, for example, converting the information into a numerical value. The coded pattern can be applied as a pattern that combines two types of information, for example, black and white.

[0051] The information on the detected object includes the name, importance, display method, etc. of the object. This information is stored in the above-mentioned table 110. If the object is a person, the name includes the attributes and name of the person, and if the object is an object, the type and name of the object. The importance is the importance of the object, and may be defined in several stages (for example, two or more stages, three or more stages, four or more stages, etc.). The display method is a display method when the receiving side processing device 40 performs display processing on the object. For example, when the specified object is displayed as a square, the information includes information such as covering the object with a square frame based on the coordinates of four points near the object. The coordinates in this case are acquired in S102. In addition, the display method may be changed according to the importance, such as changing the thickness of the frame, changing the color of the frame, changing whether or not the frame blinks, and changing the blinking interval.

[0052] Furthermore, when the information about the detected object is coded in two dimensions, encryption may be performed to enhance confidentiality. In this case, the information (numerical information, etc.) before coding is encrypted. For example, the transmitting side processing device 20 and the receiving side processing device 40 have the same key, and the data is encrypted into a numerical value that looks just like a random number. When this encryption is performed, a password or the like is set to prevent a third party from easily decrypting the data.

[0053] Next, in S106, the two-dimensional coded pattern is superimposed on the image. The two-dimensional coded pattern was created in S105, and the image is the image received in S101. The superimposition here is a display method that is not usually visible to the naked eye, but allows the coded pattern to be obtained by a specific process of the receiving side processing device 40. That is, the superimposition is performed by applying an invisible gradient to the pixel values ​​of the pixels of the image in the image. For example, the color of the coded pattern is gradually lightened, and the information is placed on the image by making the pixel value difference invisible to the human eye. Here, the coded pattern can be a part of the image, but it is better to superimpose it on the entire image. This increases the number of images, and therefore allows more information to be placed on the image.

[0054] Next, in S107, the image on which the coded pattern has been superimposed is transmitted from the transmitting-side processing device 20 to the receiving-side processing device 40. If it is determined in S103 that no object has been detected, or if it is determined in S104 that the detected object has not been defined, the image captured by the imaging device 10 in S101 will be transmitted as is. Since the image on which the coded pattern has been superimposed contains information in each image within the image, there is almost no change in the amount of information compared to when there is no superimposed information.

[0055] 4 is a flow chart showing an example of the processing on the receiving side of the monitoring system of the present invention. Here, the processing on the receiving side processing device 40 will be described with reference to the flow chart of FIG.

[0056] First, in S151, after a start operation such as turning on the power of the reception-side processing device 40 is performed, the image transmitted from the transmission-side processing device 20 is received by the reception-side processing device 40. The image here is the image output in S107 of FIG.

[0057] Next, in S152, the two-dimensional coded pattern is identified from the video received in S151, and the code is extracted. When the coded pattern is superimposed, it has a characteristic color gradient, and can be identified by the receiving-side processing device 40. For example, if the receiving-side processing device 40 also holds information on the type of pattern coded by the transmitting-side processing device 20, it will be possible to extract the coded pattern more accurately.

[0058] Next, in S153, the extracted code is decrypted to obtain information about the detected object to be monitored. The code here is a barcode, QR code, or other code, and numerical and other information can be obtained from the pattern information. Furthermore, if the obtained information has been encrypted in S105 of FIG. 3, it can be decrypted by, for example, entering a password stored in the receiving processing device 40. The decrypted information becomes information about the detected object. It is stated that by predetermining the rules for this numerical and other information, it is possible to obtain it as information about the detected object. The information here is the name, importance, display method, etc. of the object.

[0059] When information about the detected object is acquired in S153, the process proceeds to S154, where an alert can be issued to a device that issues an alert by a method other than a video display, such as a siren or a warning light. The issuance of the alert here can be performed by the alarm device 70 shown in FIG. 2 based on information from the receiving-side processing device 40.

[0060] After S153, the process proceeds to S155. In S155, the display setting information 160 is acquired, and when the image is displayed based on the display setting information, it is determined whether or not a superimposed display of a frame or the like is permitted on the image. The display setting information 160 is recorded on the reception side processing device 40 side, and is stored in the recording device of FIG. The display setting information 160 corresponds to setting information 51 of S150. The display setting information 160 holds information on whether or not it is permitted to superimpose additional information such as a frame on the image. If superimposition is permitted on the image (Yes), the process proceeds to S156. If superimposition is not permitted on the image (No), the process proceeds to S158, and the image received in S151 is output as is.

[0061] In S156, additional information such as a frame surrounding the detected object is generated based on information such as coordinates and importance. The information here is based on the information about the detected object acquired in S153. At this time, the display of the frame may be changed according to the importance, for example, by changing the line thickness or color, or whether or not it blinks. For example, when the importance is high, the line thickness may be made thicker, or the color may be changed from another color to red, or the frame may be made to blink, or the blinking interval may be shortened, etc.

[0062] Next, in S157, the additional information such as the frame generated in S156 is superimposed on each corresponding image in the video. The video here is the video received in S156. The frame covers the detected object to make it easier for the user to visually identify the detected object. Note that the frame is not superimposed on an image in which the target detected object does not exist.

[0063] Next, in S158, the image is output. If additional information such as a frame has been superimposed on the image in S157, the image with the superimposed information is output. If superimposition on the image is not permitted in S155, the image received in S156 is output. The output destination is display device 60 shown in Fig. 2 or the like, and the content of the image can be confirmed on this display device.

[0064] <Example> FIG. 5 is a diagram for explaining a specific example of processing on the transmission side of the monitoring system of the present invention.

[0065] Fig. 5(a) shows an example of S101 in Fig. 3, which shows an image 201, which is an example of one image in the acquired video captured by the imaging device 10. The object detection process of S102 in Fig. 3 is performed on this image. The object detection here shows an example of detection to identify people. Two people, person 210 and person 211, are detected by this object detection.

[0066] FIG. 5B shows an example of S104 in FIG. 3, in which it is determined whether an object is defined. Here, by referring to the table 110 in FIG. 3, it is determined whether the two people 210 and 211 are defined, for example, by using a feature amount. FIG. 5B shows an example in which the person 210 is defined and the person 211 is not defined. From the information in the table 110, the name of the person 210 is "xx" and the importance is "B". The coordinates are specified from the position in the image. FIG. 5B shows an example of "(x1, y1), (x1, y2), (x2, y1), (x2, y2)". The coordinates are those surrounding the face of the person 210, and this also indicates a method of displaying a frame. These pieces of information become information 230 about the detected object.

[0067] Fig. 5(c) shows an example of S105 in Fig. 3, in which information 230 relating to the detected object is coded two-dimensionally. The code shown here is an example of a barcode 240. Information 230 relating to the detected object is converted into barcode 240 by, for example, converting it into a numerical value. At this time, it is also possible to convert into barcode 240 using a numerical value that has been encrypted once.

[0068] FIG. 5(d) shows an example of S106 in FIG. 3, in which an image 202 is superimposed with a coded pattern. Specifically, a barcode 240 is superimposed on the image 201. The superimposition can be performed by placing the barcode 240 over the entire image 201. When the barcode 240 is superimposed, a pixel value gradient that is not visible to the human eye is applied. For example, in the case of 256 gradations, a pixel value of 5 or less, or even 3 or less, or even 1 is added to the part of the barcode 240 that corresponds to black. The image 202 superimposed in this way cannot be distinguished by the human eye due to the slight difference in pixel values.

[0069] FIG. 6 is a diagram for explaining a specific example of the processing on the receiving side of the monitoring system of the present invention.

[0070] Fig. 6(a) shows an example of S151 in Fig. 4, which is an image 202 that is an example of a video image transmitted from the transmitting-side processing device 20 and received by the receiving-side processing device 40. When compared with image 201 in Fig. 5(a), it appears the same, but barcode 240 information, which is coded pattern information, is superimposed.

[0071] FIG. 6B shows an example of S152 in FIG. 4, in which the pattern of the barcode 240 is identified from the image 202 and the barcode 240 is extracted.

[0072] Figure 6 (c) shows an example of S153 in Figure 4, in which barcode 240 is decoded to obtain information about the detected object. When barcode 240 is identified, numerical and other information can be obtained, which is then converted into information about the detected object. It is advisable to determine rules in advance for converting numerical and other information into information about the detected object. Furthermore, if the numerical and other information read from barcode 240 is encrypted, it can be decoded by entering a password, for example.

[0073] Fig. 6(d) shows an example of S156 in Fig. 4 in which a frame 250 is superimposed around the face of person 210 as monitoring information. This frame is formed based on the information of coordinates (x1, y1), (x1, y2), (x2, y1), (x2, y2). In addition, since the importance is B, the frame is displayed (with a color, thickness, etc.) according to this importance.

[0074] As described above, since surveillance information can be transmitted using only video information, there is no need to add a system for transmitting new information, and existing systems can be easily reused. For example, existing recording devices can be used as is. In this case, even if the video sent to the receiving processing device 40 is recorded as is, the surveillance information remains stored on the video, and no device for adding surveillance information is required.

[0075] Furthermore, it is possible to transmit video from the transmitting-side processing device 20 with high confidentiality of the surveillance information. At this time, since the surveillance information is invisibly superimposed on the video, even if a third party obtains the video, the presence or absence of information cannot be easily determined. Also, since the amount of data of the video does not increase even if the surveillance information is superimposed, it is not possible to determine from the amount of data that the video has surveillance information superimposed. Therefore, it is unlikely to be targeted by packet analysis in unauthorized access, and information is unlikely to leak even in unauthorized access.

[0076] Furthermore, the usefulness of the monitoring system is improved since it is possible to select whether to display or not display the monitoring information on the receiving side processing device 40. For example, if an overlapping frame or the like is an obstacle to checking the monitoring, the information can be made not to be displayed.

[0077] As described above, the embodiment of the present invention has been described, but the present invention is not limited to the above-mentioned examples, and various modified examples are included. For example, the above-mentioned examples have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one example with the configuration of another example, and it is also possible to add the configuration of another example to the configuration of one example. In addition, it is possible to add, delete, or replace a part of the configuration of each example with another configuration.

[0078] For example, although the imaging device 10 is configured using a camera, the imaging device 10 may also use information obtained from a sensor other than a camera.

[0079] In addition, in (d) of FIG. 6, an example of superimposing a square frame as additional information is shown, but other than this, various shapes such as circles, ellipses, polygons, straight lines, curves, etc. may be used as long as the information is recognizable as monitoring information. Furthermore, when superimposing with a frame of some shape, the inside of the frame may be transparent (colorless), semi-transparent, or colored. Furthermore, for example, if it is not desired to display the detected object, the entire inside of the frame may be covered. In addition, text information may be added. [Explanation of symbols]

[0080] 10...imaging device, 20...transmitting side processing device, 21...object detection unit, 22...encoding unit, 23...superimposing unit, 24...transmitting unit, 30...storage device, 31...table, 40...receiving side processing device, 41...receiving unit, 42...code acquisition unit, 43...information acquisition unit, 44...visualization unit, 45...alarm unit, 50...storage device, 51...setting information, 60...display device, 70...alarm device, 80...network, 110...table, 160...display setting information, 201, 202...image, 210, 211...person, 230...information regarding detected object, 240...barcode, 250...frame, 300...code Computer system, 302... processor, 302A... processing unit, 304... memory, 306... memory bus, 308... I / O bus, 309... bus interface unit, 310... I / O bus interface unit, 312... terminal interface unit, 314... storage interface, 316... I / O device interface, 318... network interface, 320... user I / O device, 324... display system, 326... display device, 330... network, 350... latent factor identification application

Claims

1. A detection unit that detects a surveillance target from video; An encoding unit that acquires and encodes information about the detected object; a superimposing unit for superimposing the coded information on an image as an invisible pattern; a transmitting unit that transmits the video superimposed by the superimposing unit.

2. 2. The monitoring system according to claim 1, a receiving unit that receives the video transmitted by the transmitting unit; a code acquisition unit that acquires a code superimposed on the video image received by the receiving unit; and an information acquisition unit that acquires information about the detected target from the code acquired by the code acquisition unit.

3. 3. The monitoring system according to claim 2, A surveillance system comprising a visualization unit that superimposes additional information, including figures or text, on the video image received by the receiving unit based on the information acquired by the information acquisition unit.

4. 4. The monitoring system according to claim 3, The monitoring system is characterized in that the visualization unit acquires display setting information and determines whether or not to superimpose the additional information based on the display setting information.

5. The monitoring system according to any one of claims 2 to 4, A monitoring system comprising an alarm issuing unit that transmits information to an alarm device that issues an alarm based on the information acquired by the information acquiring unit.

6. The monitoring system according to any one of claims 1 to 5, A monitoring system, characterized in that the information regarding the detected object includes information regarding the importance of the monitored object.

7. A detection step of detecting a surveillance target from the video; An encoding step of acquiring and encoding information about the detected object; a superimposing step of superimposing the coded information on an image as an invisible pattern; a transmitting step of transmitting the video superimposed in the superimposing step.

8. The monitoring method according to claim 7, a receiving step of receiving the video transmitted in the transmitting step; a code acquisition step of acquiring a code superimposed on the video image received in the receiving step; and an information acquiring step of acquiring information about the detected target from the code acquired in the code acquiring step.

Images