Monitoring support system, method and program

Abstract

To assist a monitor in efficiently detecting a suspicious person or the like by appropriately displaying a video of interest from a monitoring video.SOLUTION: A camera unit (100) includes: a detection unit (151) that detects, for a plurality of layer images corresponding to a plurality of layers captured with a camera at a predetermined time point and at distances different from the camera, a specific action of a person in the layer images by a detection method corresponding to each layer; an identifying unit (152) that, when the specific action is detected, identifies a group of images captured by the camera from the predetermined time point to prior to a first time; and a setting unit (153) that sets priority to the identified image group so as to be transmitted preferentially over images of normal priority.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a monitoring support system, method, and program. [Background technology]

[0002] Generally, a surveillance camera samples some frame images from the captured video data and transmits them to a monitor device. The monitor device displays the received frame images on a screen at intervals longer than the actual frame rate at which the video was captured. Therefore, to the observer monitoring on the monitor screen, the video captured by the surveillance camera appears to be played back frame by frame. This can cause delays in the observer's detection of suspicious individuals, etc.

[0003] Patent Document 1 discloses a technique for changing the shooting area of ​​a nearby camera when the number of people in a shot video is equal to or greater than a predetermined value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-117542 Summary of the Invention [Problem to be solved by the invention]

[0005] As a result of the above, there is a problem that it is difficult for a monitor to detect a suspicious person or the like from the video images captured by a surveillance camera installed in a store or on the street and displayed on a monitor device. Note that the technology disclosed in the above-mentioned Patent Document 1 does not detect suspicious movements.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a surveillance support system, method, and program for assisting a surveillance person in efficiently detecting suspicious persons, etc., by appropriately displaying noteworthy images from surveillance footage. [Means for solving the problem]

[0007] A first aspect of the present disclosure provides a surveillance support system including: a detection unit that detects specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time point and corresponding to a plurality of layers at different distances from the camera using a detection method corresponding to each layer; an identification unit that, when the specific behavior is detected, identifies a group of images captured by the camera between the predetermined time point and a first time point; and a setting unit that sets a priority for the identified group of images so that they are transmitted preferentially over images with normal priority.

[0008] A second aspect of the present disclosure provides a surveillance support method in which a computer performs the steps of: detecting specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time point and corresponding to a plurality of layers at different distances from the camera using a detection method corresponding to each layer; if the specific behavior is detected, identifying a group of images captured by the camera between the predetermined time point and a first time before; and setting a priority for the identified group of images so that they are transmitted preferentially over images with normal priority.

[0009] A third aspect of the present disclosure provides a surveillance support program that causes a computer to execute the following steps: detecting specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time and corresponding to a plurality of layers at different distances from the camera using a detection method corresponding to each layer; if the specific behavior is detected, identifying a group of images captured by the camera between the predetermined time and a first time before; and setting a priority for the identified group of images so that they are transmitted preferentially over images with normal priority. [Effects of the Invention]

[0010] The present disclosure makes it possible to provide a surveillance support system, method, and program for assisting a surveillance person in efficiently detecting suspicious individuals, etc., by appropriately displaying noteworthy images from surveillance footage. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an overall configuration including a monitoring support system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing the configuration of a camera unit according to the first embodiment. [Figure 3] FIG. 2 is a block diagram showing the configuration of a switch unit according to the first embodiment. [Figure 4] FIG. 2 is a block diagram showing the configuration of a monitor unit according to the first embodiment. [Figure 5] 5 is a flowchart showing the flow of a monitoring support process in the camera unit according to the first embodiment. [Figure 6] 5 is a flowchart showing the flow of a monitoring support process in the camera unit according to the first embodiment. [Figure 7] 10A and 10B are diagrams for explaining the concept of pattern matching and layer images according to the distance from the camera according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing an example of a frame image transmitted with normal priority and displayed on a monitor unit according to the first embodiment. [Figure 9] FIG. 10 is a diagram showing an example of a frame image transmitted when the upper region matches a pattern (higher priority than normal) and displayed on a monitor unit according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a frame image transmitted when the lower area matches a pattern (higher priority than normal) and displayed on a monitor unit according to the first embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a frame image transmitted (with a higher priority than normal) and displayed on a monitor unit when a person moves from an upper region to a lower region according to the first embodiment. [Figure 12] FIG. 10 is a block diagram showing a hardware configuration of a monitoring support device according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, specific embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same elements are denoted by the same reference numerals, and for clarity of explanation, duplicated explanations will be omitted as necessary.

[0013] <Embodiment 1> FIG. 1 is a block diagram showing an overall configuration including a monitoring support system 1000 according to the first embodiment. The monitoring support system 1000 includes camera units 100-1 to 100-n (n is a natural number equal to or greater than 1) and a switch unit 200. Monitor units 300-1 to 300-m (m is a natural number equal to or greater than 1) appropriately display video data captured by the monitoring support system 1000 on a screen. An observer monitors the screen of the monitor unit 300-1 or the like to detect a suspicious person or the like on the screen. Therefore, the monitoring support system 1000 is an information system for assisting the observer in efficiently detecting a suspicious person or the like by appropriately displaying noteworthy images from the monitoring video. Here, each of the camera units 100-1 to 100-n is connected to the switch unit 200. Each of the monitor units 300-1 to 300-m is connected to the switch unit 200.

[0014] Camera unit 100-1 and the like are examples of surveillance support devices that include surveillance cameras installed in stores or on the street. Each of camera units 100-1 to 100-n has the same configuration, and unless there is a need to distinguish between them, they will be simply referred to as camera unit 100 in the following description. Furthermore, each of monitor units 300-1 to 300-m has the same configuration, and unless there is a need to distinguish between them, they will be simply referred to as monitor unit 300 in the following description.

[0015] The camera unit 100 takes images at predetermined intervals, sets priorities for the captured images, and transmits them to the switch unit 200. In particular, the camera unit 100 acquires the captured images as multiple layer images according to the distance from the camera, and when a specific behavior of a person is detected from each layer image, sets the priority of the frame image corresponding to the layer image higher than normal.

[0016] FIG. 2 is a block diagram showing the configuration of the camera unit 100 according to the first embodiment. The camera unit 100 can be considered an example of a monitoring support system or a monitoring support device. The camera unit 100 includes a camera 110, a depth sensor 120, a timer 130, an image buffer 140, a detection unit 151, an identification unit 152, a setting unit 153, and a transmission unit 154. The camera 110 captures images at predetermined intervals, for example, at a frame rate, and acquires the captured frame images. The depth sensor 120 is a sensor that measures the distance between the camera 110 and a subject as depth. Therefore, the camera 110 and the depth sensor 120 may be collectively referred to as a TOF (Time of Flight) camera. At least, the camera 110 acquires a plurality of layer images corresponding to the distance between the camera 110 and a subject based on the depth measured by the depth sensor 120, along with a frame image captured at a predetermined time point. Here, the layer images are images of a predetermined area within a frame image captured at the same time point. However, multiple layer images captured at the same time point may have overlapping areas within a frame image. Furthermore, camera 110 stores the acquired frame images and the corresponding multiple layer images in image buffer 140.

[0017] Image buffer 140 is a storage area that holds frame images 141, 142, etc. that were captured at multiple different points in time. For example, frame image 141 includes layer images 1411, 1412, etc. Alternatively, frame image 141 is stored in association with layer images 1411, 1412, etc. Images after frame image 142 have a similar configuration. Note that image buffer 140 is, for example, a semiconductor memory or the like.

[0018] The timer 130 periodically outputs a sampling pulse to the detection unit 151. Here, the interval between the sampling pulses is set to be sufficiently longer than the interval between images captured by the camera 110. For example, if the camera 110 is capturing images at 60 fps (frames per second), the sampling pulse is set to be output at intervals of three frames or more. The timer 130 is, for example, an analog circuit or a digital circuit.

[0019] The detection unit 151 performs processing such as image recognition on the layer image for each layer at the timing when it receives a sampling pulse from the timer 130. Specifically, the detection unit 151 detects specific behavior of a person in the layer image of the corresponding layer using a detection method according to the layer. For example, the detection unit 151 determines whether or not a person in the layer image is likely to be a suspicious person, and if it determines that there is a possibility, detects the specific behavior. Alternatively, the detection unit 151 determines whether or not a person in the layer image is engaging in suspicious behavior or behaviour, and if it determines that the person is engaging in suspicious behavior, etc., detects the specific behavior.

[0020] Furthermore, the detection unit 151 may detect a specific behavior of a person by pattern matching between images as a detection method. For example, the detection unit 151 compares a layer image captured at a first time with a layer image of the same layer captured at a second time that is later than the first time, and calculates a difference in data size or a change in pixels. If the difference or the like is equal to or greater than a predetermined value, the detection unit 151 may determine that the difference or the like matches a pattern of a specific behavior and detect that the person is performing the specific behavior. Examples of the specific behavior include the person moving or a specific part of the person moving. For example, if the positions of the person in the layer images captured at the first and second times are separated by a predetermined distance or more, the detection unit 151 may determine that the person moved between the first time and the second time and detect the specific behavior. Furthermore, when a specific part of a person in an image changes at a predetermined distance or angle between the layer images at the first and second times, the detection unit 151 may detect that the specific part of the person has moved from the first time to the second time as a specific behavior. Here, the specific part may be, for example, an arm, a fingertip, a face direction, etc., but is not limited to these.

[0021] Furthermore, when the layer indicates a distance less than a predetermined distance from the camera 110, the detection unit 151 may detect a specific behavior by performing pattern matching of a specific part of a person in the layer image as a detection method. Furthermore, when the layer indicates a distance equal to or greater than a predetermined distance from the camera 110, the detection unit 151 may detect a specific behavior by performing pattern matching of the entire position of the person in the layer image as a detection method.

[0022] Furthermore, the detection unit 151 calculates the average value of pixel change amounts in a predetermined number of layer images corresponding to the layer and captured at different time points within a second time period for each layer according to the distance from the camera 110. The detection unit 151 may then detect a specific behavior using a detection method corresponding to the layer for layer images captured at a predetermined time point corresponding to the layer whose average value is equal to or greater than a predetermined value. Note that each of the first time period and the second time period described above is equal to or greater than the image capturing interval by the camera 110 and equal to or less than the interval between sampling pulses, and is a preset time period. The first time period and the second time period may be different times. For example, the second time period is the interval between the image capturing time of a frame image including one layer image and the image capturing time of a frame image including the other layer image. In other words, the second time period may be different for each layer.

[0023] When a specific behavior is detected, the identification unit 152 identifies a group of images captured by the camera 110 between a predetermined time point and a first time point. For example, if the capture interval, in which the capture times are predetermined intervals, is times t1, t2, t3, t4, and so on, the timing of the sampling pulse is assumed to be times t1, t4, and so on. In this case, the period between the predetermined time point and the first time point refers to the time from the time when the detection unit 151 last received a sampling pulse from the timer 130, for example, from time t1 onward, to the time when the detection unit 151 most recently received a sampling pulse, for example, time t4. In other words, the period between the predetermined time point and the first time point refers to the time from either time t1, t2, or t3 to time t4. For example, if the capture interval of the camera 110 is 60 fps, the period between the predetermined time point and the first time point refers to, but is not limited to, 1 / 20 seconds.

[0024] The setting unit 153 sets a priority for the identified image group so that it is transmitted with priority over normal priority images. Note that normal priority images refer to images with a normal level of priority for transmission from the camera unit 100 to the monitor unit 300 via the switch unit 200. For example, normal priority images are frame images obtained by sampling some frame images of captured video data at a sampling interval. In other words, the "priority for transmitting with priority over normal priority images" is set for a frame image group, i.e., video data, with a higher transmission priority than frame images in which no specific behavior was detected. Here, the priority may be expressed as a numerical value, a level value, or the like. For example, the priority may be the degree of suitability of pattern matching, the importance of suspicious behavior, or the like.

[0025] The transmitting unit 154 transmits the group of images for which priority has been set to the display device. Specifically, the transmitting unit 154 transmits packets of each image to the switch unit 200, with the monitor unit 300 as the destination, regardless of the priority. Note that the transmitting unit 154 may preferentially transmit packets with high priority. In this case, packets with low priority may not be transmitted.

[0026] The detecting unit 151, the identifying unit 152, the setting unit 153, and the transmitting unit 154 are functional blocks, and are realized by hardware, software, or a combination of hardware and software.

[0027] 3 is a block diagram showing the configuration of the switch unit 200 according to the first embodiment. The switch unit 200 includes a buffer 210, a receiving unit 220, a scheduler 230, and a transmitting unit 240. The buffer 210 is a storage area that holds a plurality of packets 211, 212, ... received by the switch unit 200. The buffer 210 is, for example, a semiconductor memory. The packet 211 includes a priority 2111 in addition to a source address, a destination address, and a packet size in its header. The packet 211 also includes compressed data corresponding to a frame image in its payload. Similarly, the packet 212 includes a priority 2121 in its header, and includes compressed data corresponding to a frame image in its payload.

[0028] Receiving unit 220 receives packets from each of multiple camera units 100-1 to 100-n via a network interface corresponding to each camera unit, and stores the received packets in buffer 210. Receiving unit 220 is, for example, a circuit including a communication interface module for communication with each camera unit.

[0029] The scheduler 230 selects packets to be transmitted from the buffer 210 according to their priority. Specifically, the scheduler 230 preferentially selects packets with higher priority as packets to be transmitted. The scheduler 230 is, for example, a hardware circuit or a processor that executes a scheduler program.

[0030] The transmitter 240 transmits the packet selected by the scheduler 230 to the monitor unit 300. The transmitter 240 is, for example, a circuit including a communication interface module for communication with the monitor unit 300.

[0031] 4 is a block diagram showing the configuration of the monitor unit 300 according to the first embodiment. The monitor unit 300 includes a receiving unit 310, a decoder 320, and a monitor 330. The receiving unit 310 receives packets from the switch unit 200 via a network interface and outputs the packets to the decoder 320. The receiving unit 310 is, for example, a circuit including a communication interface module with the switch unit 200.

[0032] The decoder 320 decodes the compressed data included in the payload of the packet received from the receiving unit 310 and restores the image data. The decoder 320 is, for example, an analog circuit or a digital circuit that performs decoding. The monitor 330 then displays the restored image data on a screen. The monitor 330 is, for example, a liquid crystal display device.

[0033] 5 and 6 are flowcharts showing the flow of the monitoring support process in the camera unit according to the present embodiment 1. First, the camera 110 acquires a plurality of layer images captured at a predetermined time point based on the depth measured by the depth sensor 120. Then, the camera 110 stores the acquired plurality of layer images in the image buffer 140 (S101).

[0034] 7 is a diagram for explaining the concept of pattern matching and layer images according to the distance from the camera according to embodiment 1. In this example, layer image 41 shows the lower region of a frame image, and layer image 42 shows the upper region of the same frame image.

[0035] Layer image 41 is image data in layer 1, which is relatively close to camera 110. Layer 1 indicates a location less than a predetermined distance from camera 110. Layer image 41 includes pixel information indicating the presence of a person at position P2 in layer 1. Furthermore, even if a person at position P2 moves their arm from position P2-1 to P2-4, this person is considered to remain in the same layer 1. Therefore, layer image 41 includes pixel information indicating the presence of an arm at positions P2-1 and P2-4 in layer 1.

[0036] Layer image 42 is image data in layer 2, which is relatively far from camera 110. Layer 2 indicates a location that is at least a predetermined distance from camera 110. Layer image 42 includes pixel information indicating that a person is present at position P1-1 or P1-4 in layer 2. Furthermore, even if a person present at position P1-1 moves to position P1-4, this person is considered to remain in the same layer 2. Therefore, layer image 42 includes pixel information indicating that a person is present at position P1-1 or P1-4 in layer 2.

[0037] Returning to FIG. 5, the explanation will be continued. After step S101, for example, it is assumed that the timer 130 outputs a sampling pulse to the detection unit 151. At this time, the detection unit 151 selects an unselected layer from among the multiple layers (S102). Then, the detection unit 151 reads out from the image buffer 140 the most recent layer image corresponding to the selected layer up to a predetermined time ago (S103). It is assumed that the detection unit 151 reads out at least the most recent and the layer image just before that, that is, a total of two or more layer images that have not yet been transmitted. On the other hand, the detection unit 151 does not target layer images that have already been transmitted. Alternatively, the camera unit 100 may delete image data captured before the most recently transmitted frame image.

[0038] Then, the detection unit 151 calculates the amount of pixel change between consecutive layer images (S104). Here, consecutive layer images are layer images captured at adjacent times. For example, the detection unit 151 encodes the layer images and calculates the bit rate for each picture type. The picture type is an encoding type such as an I picture, a B picture, or a P picture. The detection unit 151 then calculates the bit rate difference for each picture type for consecutive layer images. Alternatively, the detection unit 151 calculates the pixel amount of each layer image and calculates the difference in pixel amount between consecutive layer images. Alternatively, the detection unit 151 calculates the data size of the difference frame for each layer image and calculates the difference in data size after encoding. The pixel change amount is the number of pixels whose luminance, etc. has changed by a predetermined value or more between layer images.

[0039] Then, the detection unit 151 calculates the average value of the pixel change amounts (S105). For example, if three or more layer images are read out in step S103, there will be two or more pairs of adjacent layer images. Then, the detection unit 151 calculates two or more pixel change amounts or difference values ​​in step S104. Therefore, the detection unit 151 calculates the average value of the pixel change amounts, etc. calculated in step S104. Note that if two layer images are read out in step S103, there will be one pair of adjacent layer images, so step S105 may be omitted and the value calculated in step S104 may be interpreted as the average value.

[0040] Thereafter, the detection unit 151 determines whether the average value is greater than a threshold value corresponding to the layer (S106). Here, the threshold value corresponding to the layer is, for example, in the case of layer 1, a pixel change amount value that may indicate a specific part's suspicious behavior, and in the case of layer 2, a pixel change amount value that may indicate a person's movement amount is a suspicious behavior. Note that the threshold value corresponding to the layer can be arbitrarily set and changed for each layer.

[0041] If it is determined in step S106 that the average value is greater than the threshold value corresponding to the layer, the detection unit 151 selects a detection method corresponding to the selected layer (S107). For example, the detection method may be a pattern matching method for detecting the movement of a person's arm in the case of layer 1, and a pattern matching method for detecting the movement of a person in the case of layer 2.

[0042] Then, the detection unit 151 performs pattern matching between the most recently captured layer image and the layer image captured immediately before using the selected detection method (S108). For example, in the case of layer 1, the detection unit 151 detects the area of ​​a person's arm from each layer image using image recognition, and calculates the difference in the position, length, angle, etc. of the arm area between the two layer images. In addition, in the case of layer 2, the detection unit 151 detects the area of ​​the person's overall position from each layer image using image recognition, and calculates the difference in the detected area between the two layer images.

[0043] Then, the detection unit 151 determines whether or not the pattern matches (S109). Specifically, the detection unit 151 determines whether or not the difference calculated in step S108 is equal to or greater than a predetermined value. If the difference is equal to or greater than the predetermined value, the detection unit 151 determines that the pattern matches. For example, in the case of layer 1, if the difference in the position, length, angle, etc. of the arm area is equal to or greater than a predetermined value, the arm movement is deemed suspicious and matches the pattern. Also, in the case of layer 2, if the difference in the overall position of the person, that is, the movement distance, is equal to or greater than a predetermined value, the movement is deemed suspicious because it is a sudden movement, and matches the pattern. Note that examples of matching a pattern are not limited to these.

[0044] If it is determined in step S109 that the pattern matches, the detection unit 151 determines a higher priority than normal for the layer image corresponding to the layer selected at the time of shooting (S110). Note that the frame image 51 may be recorded in a memory or the like in association with the layer image and the priority.

[0045] After step S110, or if it is determined in step S109 that the pattern does not match, the detection unit 151 determines whether all layers have been selected (S111). Note that step S111 is also executed if it is determined in step S106 that the average value is equal to or less than the threshold value corresponding to the layer.

[0046] If not all layers have been selected in step S111, that is, if there are unselected layers, the process returns to step S102 and repeats the subsequent processes. If all layers have been selected in step S111, the identification unit 152 determines whether a higher priority than normal has been determined for the processed layer image (S112). Specifically, the identification unit 152 refers to a memory or the like and determines whether the priority associated with the layer image is higher than normal.

[0047] If it is determined in step S112 that a higher priority than normal has been determined, the identification unit 152 identifies a group of frame images from the predetermined time point to a first time before (S113). The predetermined time point is the time point when the detection unit 151 receives a sampling pulse after step S101, that is, the time point when the frame image or layer image to be processed was captured. The identification unit 152 then reads out, from the image buffer 140, the frame image at the predetermined time point and one or more frame images captured from the predetermined time point back to the first time point, as a group of frame images. As a result, a group of frame images with an interval shorter than the sampling interval is identified.

[0048] Then, setting unit 153 sets the priority determined in step S110 to each packet of the identified group of frame images (S114). That is, setting unit 153 sets compressed data obtained by compressing each image of the identified group of frame images in the payload of a packet for transmission, and sets a higher priority than normal in the header. Note that setting unit 153 may also divide one image into multiple packets.

[0049] If it is determined in step S112 that a priority higher than normal has not been determined, the identification unit 152 identifies a frame image at a predetermined time point (S115). The identification unit 152 reads the frame image at the predetermined time point from the image buffer 140. The setting unit 153 then sets normal priority to each packet of the identified frame image (S116). That is, the setting unit 153 sets compressed data obtained by compressing the image of the identified frame image in the payload of a packet to be transmitted, and sets normal priority in the header. Note that the setting unit 153 may divide one image into multiple packets.

[0050] After step S114 or S116, the transmitting unit 154 transmits each packet to the switch unit 200 (S117). At this time, the transmitting unit 154 may transmit each packet in the order of when it was captured.

[0051] Note that the processes of steps S104 to S106 may be omitted. In that case, steps S107 and after are executed after step S103. Alternatively, the processes of steps S107 to S109 may be omitted. In that case, if it is determined in step S106 that the average value is greater than the threshold value corresponding to the layer, steps S110 and after are executed.

[0052] Thereafter, switch unit 200 stores the packets received from each of multiple camera units 100-1 to 100-n in buffer 210, and transmits the packets to monitor units 300-1 to 300-m in the order according to the priority of each packet.

[0053] The monitor unit 300 displays image data corresponding to each received packet, so the video is displayed according to the reception interval. Therefore, if the switch unit 200 transmits packets to the monitor unit 300 according to normal priority, the monitor unit 300 will play frame images at sampling intervals. Therefore, if the reception interval of packets from the switch unit 200 is long, the monitor unit 300 will see the video displayed frame by frame to the observer.

[0054] Meanwhile, the camera unit 100 sets high priority for a frame image in which a specific behavior is detected by the detection unit 151 and a predetermined number of frame images before that, and transmits them to the switch unit 200. Therefore, the switch unit 200 transmits to the monitor unit 300 a group of packets set with a higher priority than packets with normal priority. Therefore, the monitor unit 300 receives the group of high-priority packets at shorter reception intervals than normal, and displays the decoded frame images on the screen. In other words, the monitor unit 300 can display video in which suspicious behavior is detected at a higher frame rate than normal. Therefore, a monitor can visually recognize the movements of a person who may be a suspicious person in smoother and more detailed video, enabling efficient detection of suspicious people.

[0055] FIG. 8 is a diagram showing an example of frame images transmitted with normal priority according to the first embodiment, displayed on the monitor unit 300. Here, each of the frame images 51, 53, and 54 includes an upper region 511 and a lower region 512, an upper region 531 and a lower region 532, and an upper region 541 and a lower region 542. The upper region 511 and the lower region 512 may have some overlapping areas. The same applies to the other regions hereinafter. The camera 110 captures images at intervals of times t1, t2, t3, t4, and so on. Meanwhile, the sampling interval of the camera unit 100 is time t4 after time t1. The same applies to the other explanations hereinafter.

[0056] At time t1, monitor unit 300 displays frame image 51. In upper region 511 of frame image 51, a person is present at position P1-1, which is far from camera 110, and in lower region 512, a person is present at position P2, which is close to camera 110, with the person's arm at position P2-1.

[0057] Subsequently, the camera unit 100 performs the processes shown in FIGS. 5 and 6 for the frame image 54 captured at time t4, a sampling interval after time t1. At this time, the camera unit 100 compares the layer images of the frame image 54 and the frame image 53 captured immediately before at time t3 to determine whether the patterns match. In this example, the camera unit 100 determines that the movement of the person in the upper region, which is layer 2, is small because the pixel change amount between position P1-b in the upper region 541 and position P1-a in the upper region 531 is below a threshold. Furthermore, the camera unit 100 determines that the movement of the person's arm in the lower region, which is layer 1, is small because the pixel change amount between positions P2 and P2-1 in the lower region 542 and positions P2 and P2-1 in the lower region 532 is below a threshold. Therefore, the camera unit 100 sets normal priority to the frame image 54 and transmits it to the switch unit 200. Conversely, camera unit 100 does not transmit frame image 53 captured at time t3. Then, switch unit 200 transmits a packet of frame image 54 to monitor unit 300. Therefore, monitor unit 300 displays frame image 54 at time t4. In other words, after displaying frame image 51 at time t1, frame image 51 is not updated at times t2 and t3, and frame image 54 is displayed at time t4. Therefore, to a monitor, it appears as if the frame images are being updated frame by frame from 51 to 54. However, the person in upper area 541 has simply moved from position P1-1 to position P1-b, and there is little need to alert the monitor of this suspicious behavior, so there is little impact on monitoring.

[0058] 9 is a diagram showing an example of a frame image transmitted when the upper region matches a pattern (higher priority than normal) according to the first embodiment, and displayed on the monitor unit 300. The frame image 61 displayed at time t1 is the same as the frame image 51 described above.

[0059] Subsequently, at time t4, when a sampling interval has elapsed since time t1, camera unit 100 determines that the amount of movement of the person in the upper region is large because the pixel change amount between position P1-4 in upper region 641 of frame image 64 and position P1-3 in upper region 631 is greater than the threshold. Therefore, the pattern matches in layer 2, and camera unit 100 identifies frame images 62 to 64 from time t4 to time t2, which is the first time before, sets a higher priority than normal for each packet, and transmits each packet to switch unit 200. In other words, camera unit 100 also transmits frame images 62 and 63, which would not be transmitted with normal priority. Then, switch unit 200 transmits packets of frame images 62, 63, and 64 to monitor unit 300 with priority. Therefore, monitor unit 300 displays frame image 51 at time t1, then displays frame image 62 at time t2, frame image 63 at time t3, and frame image 54 at time t4. Therefore, the observer can visually see the person in the upper region moving smoothly from position P1-1 to P1-2, P1-3, and P1-4 in frame image 61 to 64. This allows for efficient determination of whether the person in the upper region is suspicious.

[0060] 10 is a diagram showing an example of a frame image transmitted when the lower area matches a pattern (higher priority than normal) according to the first embodiment, and displayed on the monitor unit 300. The frame image 71 displayed at time t1 is the same as the frame image 51 described above.

[0061] Subsequently, at time t4, a sampling interval after time t1, the camera unit 100 determines that the amount of movement of the person's arm in the lower region is large because the pixel change amount between position P2-4 in the lower region 742 of frame image 74 and position P2-3 in the lower region 732 is greater than the threshold. Therefore, the pattern matches in layer 1, and the camera unit 100 identifies frame images 72 to 74 from time t4 to time t2, which is the first time before, sets a higher priority than normal for each packet, and transmits each packet to the switch unit 200. In other words, the camera unit 100 also transmits frame images 72 and 73, which would not be transmitted with normal priority. The switch unit 200 then transmits packets of frame images 72, 73, and 74 to the monitor unit 300 with priority. Therefore, after displaying frame image 71 at time t1, the monitor unit 300 displays frame image 72 at time t2, frame image 73 at time t3, and frame image 74 at time t4. Therefore, the observer can visually see the smooth movement of the arm of the person in the lower area moving from position P2-1 to position P2-2, P2-3, and P2-4 in frame images 71 to 74. This allows for efficient determination of whether the person in the lower area is suspicious. In this example, the observer can also visually see the person in the upper area moving from position P1-1 to P1-3 between times t2 and t3.

[0062] FIG. 11 is a diagram showing an example of a frame image transmitted when a person moves from the upper region to the lower region (higher priority than normal) and displayed on the monitor unit 300 according to the first embodiment.

[0063] At time t1, the monitor unit 300 displays a frame image 81. In an upper region 811 of the frame image 81, a person is present at a position P3-1 far from the camera 110, and in a lower region 812, no person is present.

[0064] Subsequently, at time t4, when a sampling interval has elapsed since time t1, camera unit 100 determines that the amount of movement of the person in the lower region is large because the pixel change amount between position P3-4 in lower region 842 of frame image 84 and position P3-3 in lower region 832 is greater than the threshold. Therefore, the pattern matches in layer 1, and camera unit 100 identifies frame images 82 to 84 from time t4 to time t2, which is the first time before, sets a higher priority than normal for each packet, and transmits each packet to switch unit 200. In other words, camera unit 100 also transmits frame images 82 and 83, which would not be transmitted with normal priority. Then, switch unit 200 transmits packets of frame images 82, 83, and 84 to monitor unit 300 with priority. Therefore, monitor unit 300 displays frame image 81 at time t1, then displays frame image 82 at time t2, frame image 83 at time t3, and frame image 84 at time t4. Therefore, the observer can visually see the smooth movement of the person in the upper region of frame image 81 to 84 moving from position P3-1 to P3-2 to P3-3 to P3-4, thereby enabling an efficient determination of whether the person on the screen is suspicious.

[0065] To detect suspicious individuals and narrow down the images to be monitored, it is possible to train an image recognition algorithm to learn the behavior of suspicious individuals for each application. However, this requires cost and time. In contrast, in this embodiment, there is no need to train an advanced image recognition algorithm; notable images are identified from the monitoring images and transmitted to the display device with increased communication priority. Therefore, when the possibility of suspicious behavior is low, communication load and costs can be reduced, while when the possibility of suspicious behavior is high, the images can be displayed at an appropriate frame rate.

[0066] <Other embodiments> The camera unit 100 described above may be realized by an information processing device that is a monitoring support device. Fig. 12 is a block diagram showing the hardware configuration of a monitoring support device 100a according to another embodiment. The monitoring support device 100a includes a camera 110, a depth sensor 120, a timer 130, an image buffer 140, a control unit 150, a storage unit 160, a memory 170, and a communication unit 180. The camera 110, the depth sensor 120, the timer 130, and the image buffer 140 are the same as those shown in Fig. 2 above, and therefore description thereof will be omitted.

[0067] The storage unit 160 is an example of a storage device such as a hard disk, a flash memory, etc. The storage unit 160 stores a monitoring assistance program 161. The monitoring assistance program 161 is a computer program that implements the monitoring assistance processing and the like according to the above-described embodiment.

[0068] The memory 170 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily storing information during operation of the control unit 150. The communication unit 180 is a communication interface circuit between the internal configuration of the monitoring support device 100a and the switch unit 200.

[0069] The control unit 150 is a processor, i.e., a control device, that controls each component of the monitoring support device 100a. The control unit 150 loads the monitoring support program 161 from the storage unit 160 into the memory 170 and executes the monitoring support program 161. In this way, the control unit 150 realizes the functions of the above-mentioned detection unit 151, identification unit 152, setting unit 153, transmission unit 154, etc.

[0070] Alternatively, each component of the monitoring support device 100a may be realized by dedicated hardware. Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and programs. Furthermore, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), quantum processor (quantum computer control chip), etc., may be used as the processor.

[0071] The present invention has been described above in accordance with the above-mentioned embodiment, but the present invention is not limited to the configuration of the above-mentioned embodiment, and naturally includes various modifications, alterations, and combinations that a person skilled in the art can make within the scope of the invention as defined in the claims of this application.

[0072] Although the above-described embodiment has been described as a hardware configuration, the present disclosure is not limited to this. Any processing in the present disclosure can also be realized by causing a CPU to execute a computer program.

[0073] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals. [Explanation of symbols]

[0074] 1000 Monitoring Support System 100 camera unit 110 Camera 120 Depth Sensor 130 Timer 140 Image Buffer 141 frame images 1411 Layer Images 1412 Layer Images 142 frame images 151 Detection unit 152 Specific part 153 Setting section 154 Transmitter 200 Switch Unit 210 buffers 211 packets 2111 Priority 212 packets 2121 Priority 220 Receiving unit 230 Scheduler 240 Transmitter 300 monitor unit 310 Receiving unit 320 decoder 330 monitor 100a Monitoring support device 150 control section 160 Storage section 161 Surveillance Support Program 170 memory 180 Communications Department 41, 42 Layer images 100-1~100-n Camera Unit 300-1~300-m monitor unit

Claims

1. a detection unit that detects specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time point and corresponding to a plurality of layers at different distances from the camera, using a detection method corresponding to each layer; an identification unit that, when the specific behavior is detected, identifies a group of layer images captured from the layer image captured at the predetermined time point to a first time period; a setting unit that sets a priority for the specified image group so that a transmission unit that transmits images to be displayed on a display device transmits the images with priority over images with normal priority, The detection unit If the layer shows less than a predetermined distance from the camera, detecting the specific behavior by performing pattern matching of a specific part of a person in the layer image as the detection method; If the layer indicates a distance greater than a predetermined distance from the camera, the specific behavior is detected by performing pattern matching of the overall position of the person in the layer image as the detection method. Surveillance support system.

2. a detection unit that detects specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time point and corresponding to a plurality of layers at different distances from the camera, using a detection method corresponding to each layer; an identification unit that, when the specific behavior is detected, identifies a group of layer images captured from the layer image captured at the predetermined time point to a first time period; a setting unit that sets a priority for the specified image group so that a transmission unit that transmits images to be displayed on a display device transmits the images with priority over images with normal priority, The detection unit calculating an average value of pixel change amounts of the object corresponding to the layer in a predetermined number of layer images that correspond to the layer and that are captured at different times within a second time period that corresponds to the distance from the camera; The specific behavior is detected by the detection method for the layer image captured at the predetermined time point corresponding to the layer whose average value is equal to or greater than a predetermined value. Surveillance support system.

3. The computer a step of detecting a specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time and corresponding to a plurality of layers at different distances from the camera, using a detection method corresponding to each layer; When the specific behavior is detected, a step of identifying a group of layer images taken from the layer image taken at the predetermined time point to a first time period before the layer image taken at the predetermined time point; a step of setting a priority for the specified group of images so that a transmitting unit that transmits images to be displayed on a display device transmits the images with priority over images with normal priority; The detecting step includes: If the layer shows less than a predetermined distance from the camera, detecting the specific behavior by performing pattern matching of a specific part of a person in the layer image as the detection method; If the layer indicates a distance greater than a predetermined distance from the camera, the specific behavior is detected by performing pattern matching of the overall position of the person in the layer image as the detection method. Monitoring support method.

4. The computer a step of detecting a specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time and corresponding to a plurality of layers at different distances from the camera, using a detection method corresponding to each layer; When the specific behavior is detected, a step of identifying a group of layer images taken from the layer image taken at the predetermined time point to a first time period before the layer image taken at the predetermined time point; a step of setting a priority for the specified group of images so that a transmitting unit that transmits images to be displayed on a display device transmits the images with priority over images with normal priority; The detecting step includes: calculating an average value of pixel change amounts of the object corresponding to the layer in a predetermined number of layer images that correspond to the layer and that are captured at different times within a second time period that corresponds to the distance from the camera; The specific behavior is detected by the detection method for the layer image captured at the predetermined time point corresponding to the layer whose average value is equal to or greater than a predetermined value. Monitoring support method.

5. a step of detecting a specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time and corresponding to a plurality of layers at different distances from the camera, using a detection method corresponding to each layer; When the specific behavior is detected, a step of identifying a group of layer images taken from the layer image taken at the predetermined time point to a first time period before the layer image taken at the predetermined time point; a step of setting a priority for the specified image group so that a transmission unit that transmits images to be displayed on a display device transmits the images with priority over images with normal priority; The detecting step includes: If the layer shows less than a predetermined distance from the camera, detecting the specific behavior by performing pattern matching of a specific part of a person in the layer image as the detection method; If the layer indicates a distance greater than a predetermined distance from the camera, the specific behavior is detected by performing pattern matching of the overall position of the person in the layer image as the detection method. Monitoring support program.

6. a step of detecting a specific behavior of a person in a plurality of layer images captured by a camera at a predetermined time and corresponding to a plurality of layers at different distances from the camera, using a detection method corresponding to each layer; When the specific behavior is detected, a step of identifying a group of layer images taken from the layer image taken at the predetermined time point to a first time period before the layer image taken at the predetermined time point; a step of setting a priority for the specified image group so that a transmission unit that transmits images to be displayed on a display device transmits the images with priority over images with normal priority; The detecting step includes: calculating an average value of pixel change amounts of the object corresponding to the layer in a predetermined number of layer images that correspond to the layer and that are captured at different times within a second time period that corresponds to the distance from the camera; The specific behavior is detected by the detection method for the layer image captured at the predetermined time point corresponding to the layer whose average value is equal to or greater than a predetermined value. Monitoring support program.

Images