Surveillance system

Abstract

To make it possible to detect abnormality with high accuracy via an information communication network.SOLUTION: A monitoring system 40 includes a first image analysis apparatus 11 provided at a vehicle 1 serving as an edge E of an information communication network 15 together with a camera 8. This first image analysis apparatus 11 is configured to analyze a captured image Vd of a vehicle interior 6 imaged by the camera 8 so as to execute detection determination of a state change occurring in the vehicle interior 6. Moreover, this first image analysis apparatus 11 has a function as an image distribution apparatus 80 configured to execute distribution of the captured image Vd via the information communication network 15 when occurrence of the state change is detected in the vehicle interior 6 as a monitored space 41. Moreover, the monitoring system 40 includes a second image analysis device 22 configured to execute detection determination of an abnormality occurring in the vehicle interior 6 by analyzing the captured image Vd of the vehicle interior 6 distributed via this information communication network 15.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a monitoring system.

Background Art

[0002] Conventionally, there has been a monitoring system capable of externally checking a photographed image of a monitored space projected by a camera via an information communication network. For example, the vehicle condition management system described in Patent Document 1 includes a plurality of sensors that detect the occurrence of a recording event indicating an abnormality of the vehicle. And, it is configured such that the type of the generated recording event and the photographed image at the time of occurrence can be confirmed via a mobile phone line.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since the above conventional technology is configured to detect a state change that has occurred in a monitored space using a human presence sensor, the situation in which the occurrence of the state change can be detected and the detection range are limited. For this reason, there is a problem that it is difficult to perform highly accurate abnormality detection via an information communication network.

Means for Solving the Problems

[0005] The monitoring system for solving the above problems includes a first image analysis device provided at the edge of the information communication network together with a camera, which analyzes a captured image of a monitoring space captured by the camera to detect and determine a state change occurring in the monitoring space, an image distribution device that distributes the captured image via the information communication network when the occurrence of the state change is detected, and a second image analysis device that analyzes the distributed captured image to detect and determine an abnormality occurring in the monitoring space.

[0006] According to the above configuration, the first image analysis device provided at the edge of the information communication network can detect a state change occurring in the monitoring space reflected in the captured image. Then, by analyzing the captured image of the monitoring space distributed due to the detection of this state change with the second image analysis device, an abnormality occurring in the monitoring space reflected in the captured image can be detected with high precision via the information communication network.

[0007] In addition, the detection and determination of the state change by analyzing the captured image has a smaller computational load than the detection and determination of the abnormality by analyzing the captured image. For this reason, there is an advantage that the required computational processing ability of the information processing device on which the first image analysis device is mounted can be suppressed to be relatively small. And when a state change in the monitoring space is not detected, the communication load can be reduced by not distributing the captured image via the information communication network.

[0008] In the monitoring system for solving the above problems, the first image analysis device preferably includes a pixel difference value calculation unit that calculates a pixel difference value between a previous frame of the captured image acquired in the previous analysis period and a current frame of the captured image acquired in the current analysis period by acquiring the captured image for each analysis period, a history holding unit that holds a calculation history of the pixel difference value, a variance value calculation unit that calculates a variance value of the pixel difference value based on the calculation history, and a state change determination unit that determines that the state change has occurred in the monitoring space when the variance value is equal to or greater than a predetermined threshold value.

[0009] According to the above configuration, it is possible to perform detection and determination of state changes based on the analysis of the captured image with relatively low computational load and high accuracy. And thereby, it is possible to perform high-precision anomaly detection via an information communication network while ensuring a high degree of installation freedom.

[0010] In addition, by using the pixel difference value between the previous frame and the current frame of the captured image acquired for each analysis period, it is possible to suppress the influence of the external environment of the monitoring space, such as a change in the amount of light, to a small extent.

[0011] In the monitoring system for solving the above problems, it is preferable that the pixel difference value calculation unit calculates the pixel difference value only for a detection area set in advance in the monitoring space. According to the above configuration, it is possible to accurately detect a state change that has occurred in a detection area set in advance in the monitoring space. And thereby, it is possible to suppress the occurrence of false determination and perform anomaly detection via the information communication network with higher accuracy.

[0012] In the monitoring system for solving the above problems, it is preferable that the image distribution device executes distribution of the captured image to an administrator located outside the monitoring space when the occurrence of the anomaly is detected in the second image analysis device.

[0013] According to the above configuration, an administrator can quickly confirm an anomaly that has occurred in the monitoring space. Thereby, it is possible to ensure a prompt response to the anomaly that has occurred in the monitoring space. And by limiting the situation of distributing the captured image to the administrator, an increase in communication load can be suppressed.

[0014] In the monitoring system for solving the above problems, it is preferable that the image distribution device executes distribution of the captured image to an administrator located outside the monitoring space at a higher compression rate than the captured image to be distributed to the second image analysis device.

[0015] That is, even for a captured image with a low resolution and a high compression ratio, in many cases, the administrator can, by visually checking, without problem, grasp the situation of the monitored space reflected in the captured image. Therefore, according to the above configuration, it is possible to construct a layered system with monitoring by the administrator while suppressing an increase in communication load. And thereby, it is possible to perform anomaly detection via the information communication network with higher accuracy.

[0016] The monitoring system for solving the above problems is preferably configured to be able to change the compression ratio of the captured image distributed to the administrator based on the request of the administrator. According to the above configuration, while suppressing an increase in communication load, the administrator can check the captured image of the monitored space at an appropriate resolution. And thereby, it is possible to perform anomaly detection via the information communication network with higher accuracy.

[0017] In the monitoring system for solving the above problems, when it is determined that the detection determination cannot be made by the second image analysis device or the accuracy of the detection determination has decreased, it is preferable to execute distribution of the captured image to the administrator.

[0018] According to the above configuration, by the administrator checking the captured image of the monitored space, it is possible to ensure high-accuracy anomaly detection via the information communication network. And by limiting the situation of distributing the captured image to the administrator, an increase in communication load can be suppressed.

[0019] In the monitoring system for solving the above problems, the second image analysis device detects the skeletal points of a person included in the captured image and executes the detection determination based on the information of the person obtained by the detection of the skeletal points, and preferably, based on the detection state of the skeletal points, it is determined whether to execute distribution of the captured image to the administrator.

[0020] According to the above configuration, it is possible to accurately identify a state in which the second image analysis device cannot perform abnormality detection determination by analyzing its captured image or the accuracy of the detection determination has decreased. As a result, it is possible to appropriately determine whether to execute the distribution of the captured image to the administrator. As a result, it is possible to avoid a situation where the captured images of the monitoring space are frequently distributed to the administrator and suppress an increase in the communication load.

[0021] In addition, by detecting the skeleton points, it is possible to accurately obtain physical information such as a person's posture and physique. As a result, based on the obtained person information, it is possible to perform high-precision abnormality detection determination.

[0022] In the monitoring system for solving the above problems, it is preferable that the monitoring space is a vehicle cabin. According to the above configuration, it is possible to accurately detect an abnormality that has occurred in the vehicle cabin via the information communication network.

Effect of the Invention

[0023] According to the present invention, it is possible to perform high-precision abnormality detection via an information communication network.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Mode for Carrying Out the Invention

[0025] Hereinafter, an embodiment of the monitoring system will be described with reference to the drawings. As shown in FIGS. 1 to 3, the vehicle 1 of this embodiment has a substantially rectangular box-shaped vehicle body 2 extending in the longitudinal direction of the vehicle. Further, a door opening 3 serving as an entrance and exit for passengers is provided on the side surface of the vehicle body 2. In addition, a pair of slide doors 4, 4 that open and close in opposite directions in the longitudinal direction of the vehicle are provided at the door opening 3. And the passenger 5 of the vehicle 1 gets on this vehicle 1 in a "sitting posture" of sitting on a seat 7 provided in the passenger compartment 6, or in a "standing posture" of using, for example, a hanging strap or a handrail (not shown).

[0026] In addition, the vehicle 1 of this embodiment is provided with a camera 8 for photographing the interior of the passenger compartment 6. In the vehicle 1 of this embodiment, this camera 8 is provided near the ceiling portion 9 in the vicinity of the corner portion 6fa at the front position of the passenger compartment 6. In addition, for example, an infrared camera or the like is used for this camera 8. And the camera 8 of this embodiment is configured to photograph the passenger 5 of the vehicle 1 from a predetermined direction set in the passenger compartment 6.

[0027] As shown in FIGS. 4 and 5, in the vehicle 1 of this embodiment, the photographed image Vd in the passenger compartment 6 projected by the camera 8 is input to a first image analysis device 11 mounted on an in-vehicle information processing device 10. Further, this photographed image Vd is distributed via an information communication network 15 to a second image analysis device 22 mounted on an out-of-vehicle information processing device 20 that constitutes a cloud server 17, and to an operator 32 as an administrator 31 waiting at the operation center 30 of the vehicle 1. The information communication network 15 is constituted by a wireless communication network, the Internet, or the like. In addition, the distribution of the photographed image Vd to the operator 32 is performed on an information processing device 35 provided in the operation center 30 that constitutes a receiving device 33 for this photographed image Vd and an operation terminal 34 of the operator 32. And in the vehicle 1 of this embodiment, a multi-layered monitoring system 40 formed by the first image analysis device 11 and the second image analysis device 22 of the in-vehicle and out-of-vehicle information processing devices 10, 20 connected via the information communication network 15, and the administrator 31 is constructed.

[0028] That is, in the monitoring system 40 of the present embodiment, the passenger compartment 6 of the vehicle 1 imaged by the camera 8 is set as the monitoring space 41. Further, the first image analysis device 11 is provided on the vehicle 1 together with the camera 8, and monitors the passenger compartment 6 of the vehicle 1 reflected in the captured image Vd at the edge E of the information communication network 15. And the monitoring system 40 of the present embodiment is configured such that the captured image Vd of the passenger compartment 6 distributed via the information communication network 15 is monitored by the second image analysis device 22 and the operator 32 of the operation center 30 from outside the vehicle 1.

[0029] (First Image Analysis Device) First, the configuration and functions of the first image analysis device 11 will be described. As shown in FIG. 5, the first image analysis device 11 of the present embodiment includes a state change detection unit 50 that detects a state change that has occurred in the passenger compartment 6 of the vehicle 1 set as the monitoring space 41 by analyzing the captured image Vd of the passenger compartment 6 imaged by the camera 8.

[0030] Specifically, as shown in FIGS. 6 to 10, the state change detection unit 50 of the present embodiment periodically acquires the captured image Vd of the passenger compartment 6 imaged by the camera 8. Further, the state change detection unit 50 includes a pixel difference value calculation unit 51 that calculates a pixel difference value δ between the previous frame Fb of the captured image Vd acquired in the previous analysis cycle and the current frame Fc of the captured image Vd acquired in the current analysis cycle. Furthermore, the state change detection unit 50 includes a history holding unit 52 that holds the calculation history X of the calculated pixel difference value δ, and a variance value calculation unit 53 that calculates a variance value Y of the pixel difference value δ based on the calculation history X of the pixel difference value δ held in the history holding unit 52. And the state change detection unit 50 of the present embodiment includes a state change determination unit 54 that determines a state change that has occurred in the passenger compartment 6 reflected in the captured image Vd based on the variance value Y of the pixel difference value δ.

[0031] That is, the frame F of the captured image Vd acquired by the state change detection unit 50 for each analysis period can be represented by the value of each pixel with the minimum unit for dividing this frame F into a grid being a "pixel". Then, the pixel difference value calculation unit 51 of the present embodiment calculates the pixel difference value δ between the previous frame Fb and the current frame Fc by obtaining the difference between the value in the previous analysis period and the value in the current analysis period for the values of these respective pixels.

[0032] For example, in the examples shown in FIGS. 7 to 10, FIG. 7 is the current frame Fc of the captured image Vd acquired by the state change detection unit 50 in the current analysis period, and FIG. 8 is the previous frame Fb of the captured image Vd acquired by the state change detection unit 50 in the previous analysis period. And FIG. 9 visualizes the pixel difference value δ between the current frame Fc and the previous frame Fb in a form that reproduces the pixel arrangement in these respective frames F.

[0033] That is, when comparing the current frame Fc illustrated in FIG. 7 and the previous frame Fb illustrated in FIG. 8, the standing position of the occupant 5 reflected in the captured image Vd has changed. And in the pixel difference value δ shown in FIG. 9, the movement of the occupant 5 who has moved to this floor area A4 appears.

[0034] Further, the pixel difference value calculation unit 51 of the present embodiment executes the calculation of the pixel difference value δ only for the rear seat area A1, the front seat area A2, and the middle seat area A3 where the occupant 5 rides in a seated posture, and the floor area A4 where the occupant 5 rides in a standing posture (see FIG. 3). That is, the pixel difference value calculation unit 51 of the present embodiment sets the boarding area α0 of the occupant 5 set in the passenger compartment 6 as the preset detection area α, and for example, does not execute the calculation of the pixel difference value δ for the vehicle window or the like reflected in the captured image Vd. And in the state change detection unit 50 of the present embodiment, thereby, the behavior of the occupant 5 reflected in the captured image Vd is likely to appear as the pixel difference value δ calculated by this pixel difference value calculation unit 51 as the state change that has occurred in the passenger compartment 6.

[0035] More specifically, as shown in FIG. 10, the pixel difference value calculation unit 51 of the present embodiment executes the calculation of the pixel difference value δ in each analysis period in which the state change detection unit 50 acquires the captured image Vd. Further, in the state change detection unit 50 of the present embodiment, the previous value δb of the pixel difference value δ calculated by the pixel difference value calculation unit 51 is held in the history holding unit 52 as the calculation history X of the pixel difference value δ for a predetermined past period. In FIG. 10, the previous value δb1 indicates that it is the previous value δb of the pixel difference value δ calculated in the previous analysis period, and the previous value δb2 indicates that it is the previous value δb of the pixel difference value δ calculated in the analysis period two times before. Then, the state change detection unit 50 of the present embodiment calculates the variance value Y of the pixel difference value δ for each analysis period by reading out the calculation history X of the pixel difference value δ held by the history holding unit 52.

[0036] That is, the greater the state change that occurs in the passenger compartment 6 reflected in the captured image Vd, the greater the value of the variance value Y of the pixel difference value δ. Further, in the state change detection unit 50 of the present embodiment, the state change determination unit 54 holds a predetermined threshold value Yth for the variance value Y of the pixel difference value δ. Then, the state change determination unit 54 of the present embodiment is configured to determine that a state change has occurred in the passenger compartment 6 reflected in the captured image Vd when the variance value Y of the pixel difference value δ is equal to or greater than the threshold value Yth (Y≧Yth).

[0037] Also, as shown in FIG. 5, the first image analysis device 11 of the present embodiment includes a communication control unit 55 that executes information communication via the information communication network 15. Further, when the state change detection unit 50 detects the occurrence of a state change in the passenger compartment 6 reflected in the captured image Vd, the communication control unit 55 distributes the captured image Vd of the passenger compartment 6 captured by the camera 8 to the second image analysis device 22. Then, the monitoring system 40 of the present embodiment is configured such that the second image analysis device 22 disposed outside the vehicle 1 monitors the passenger compartment 6 by analyzing the captured image Vd.

[0038] That is, as shown in FIG. 11, in the first image analysis device 11 of the present embodiment, every predetermined analysis cycle, the state change detection unit 50 acquires the captured image Vd of the passenger compartment 6 (step 101). Next, the state change detection unit 50 calculates the pixel difference value δ between the previous frame Fb of the captured image Vd acquired in the previous analysis cycle and the current frame Fc of the captured image Vd acquired in the current analysis cycle (step 102). Subsequently, the state change detection unit 50 reads out the calculation history X of the pixel difference value δ calculated in the past analysis cycles (step 103), and calculates the variance value Y of the pixel difference value δ (step 104). Further, when the variance value Y of the pixel difference value δ is equal to or greater than the threshold value Yth (Y≧Yth, step 105: YES), the state change detection unit 50 determines that a state change has occurred in the passenger compartment 6 reflected in the captured image Vd (step 106). And the first image analysis device 11 of the present embodiment is configured such that the communication control unit 55 starts distributing the captured image Vd to the second image analysis device 22 accordingly (step 107).

[0039] In addition, in the state change detection unit 50 of the present embodiment, the calculation history X of the pixel difference value δ held by the history holding unit 52 is sequentially updated with the new pixel difference value δ calculated by the pixel difference value calculation unit 51 as the latest previous value δb. Also, when the variance value Y of the pixel difference value δ is less than the threshold value Yth (Y<Yth, step 105: NO), the state change determination unit 54 determines that no state change to be detected by the state change detection unit 50 has occurred in the passenger compartment 6 reflected in the captured image Vd (step 108). And in the first image analysis device 11 of the present embodiment, in this case, the communication control unit 55 is configured not to execute the distribution of the captured image Vd to the second image analysis device 22 (step 109).

[0040] More specifically, as shown in FIG. 5, the first image analysis device 11 of the present embodiment is provided with an image compression unit 56 that compresses the captured image Vd of the passenger compartment 6 captured by the camera 8. And the first image analysis device 11 of the present embodiment distributes the captured image VdL compressed by the image compression unit 56 to the second image analysis device 22 connected via the information communication network 15.

[0041] Specifically, in the first image analysis device 11 of the present embodiment, the image compression unit 56 compresses the captured image Vd of the passenger compartment 6 to be distributed to the second image analysis device 22 at a predetermined compression rate βL (β = βL). Then, the monitoring system 40 is configured to reduce the communication load caused by the distribution of the captured image Vd via the information communication network 15 in this way.

[0042] Further, the first image analysis device 11 of the present embodiment is provided with a preprocessing unit 57 that performs preprocessing on the captured image Vd input to the first image analysis device 11. Specifically, in the first image analysis device 11 of the present embodiment, the preprocessing unit 57 executes noise removal included in the captured image Vd, brightness adjustment and smoothing of the pixels constituting the captured image Vd, and the like. Then, the first image analysis device 11 of the present embodiment is configured to reduce the influence of so-called reflection, external light, etc. on the captured image Vd in this way.

[0043] (Second Image Analysis Device) Next, the configuration and functions of the second image analysis device 22 will be described. As shown in FIG. 5, in the monitoring system 40 of the present embodiment, the second image analysis device 22 includes a communication control unit 60 having a function of receiving the captured image Vd of the passenger compartment 6 distributed by the first image analysis device 11 as described above. Then, the second image analysis device 22 of the present embodiment has a function of performing detection and determination of an abnormality occurring in the passenger compartment 6 of the vehicle 1 by analyzing the received captured image Vd.

[0044] Specifically, the second image analysis device 22 of the present embodiment includes a person recognition unit 61 that recognizes a person H in the passenger compartment 6 shown in the captured image Vd, that is, a passenger 5 of the vehicle 1. In the second image analysis device 22 of the present embodiment, this person recognition unit 61 executes the recognition process of the person H by using an inference model generated by machine learning. And the second image analysis device 22 of the present embodiment includes an abnormality detection unit 62 that detects an abnormality occurring in the passenger compartment 6 shown by the camera 8 by monitoring the passenger 5 of the vehicle 1 recognized thereby.

[0045] Specifically, as shown in FIGS. 5 and 12, the second image analysis device 22 of the present embodiment includes a skeleton point detection unit 63 that detects skeleton points SP of a person H included in the captured image Vd. That is, the skeleton points SP are unique points that characterize the body of the person H, such as joints and points on the body surface. For example, the head, neck, shoulders, armpits, elbows, wrists, fingertips, waist, hip joints, buttocks, knees, ankles, etc. are applicable. And in the second image analysis device 22 of the present embodiment, this skeleton point detection unit 63 also executes the detection process of the skeleton points SP by using an inference model generated by machine learning.

[0046] Also, as shown in FIG. 5, the second image analysis device 22 of the present embodiment includes a feature quantity calculation unit 64 that calculates a feature quantity Vsp based on the detection of the skeleton points SP. Specifically, in the second image analysis device 22 of the present embodiment, this feature quantity calculation unit 64 calculates the feature quantity Vsp of the person H shown in the captured image Vd based on the position of the skeleton points SP in the two-dimensional coordinates in the captured image Vd. Further, this feature quantity calculation unit 64 calculates the feature quantity Vsp of the person H shown in the captured image Vd based on, for example, body dimensions shown by a plurality of the skeleton points SP, such as the shoulder width of the passenger 5. And the second image analysis device 22 of the present embodiment includes a person information acquisition unit 65 that acquires information Ih of the person H recognized in the monitoring space 41 shown by the camera 8 based on the feature quantity Vsp of the person H obtained by this series of analysis processes.

[0047] Specifically, as shown in FIG. 13, the human information acquisition unit 65 of the present embodiment includes a posture determination unit 66 that determines the posture of the person H reflected in the captured image Vd. The posture determination unit 66 of the present embodiment inputs the feature amount Vsp of the person H acquired from the feature amount calculation unit 64 into an inference model generated by machine learning. Then, based on the posture discrimination probability value obtained thereby, the posture of the person H reflected in the captured image Vd in the passenger compartment 6 is discriminated.

[0048] Specifically, the posture determination unit 66 of the present embodiment includes a standing position discrimination probability value calculation unit 66a that calculates the probability that the posture of the occupant 5 who is the target of posture discrimination is a "standing posture". Further, the posture determination unit 66 includes a sitting position discrimination probability value calculation unit 66b that calculates the probability that the posture of the occupant 5 who is the target is a "sitting posture". Then, the posture determination unit 66 of the present embodiment includes a fall discrimination probability value calculation unit 66c that calculates the probability that the posture of the occupant 5 who is the target is a "fallen posture".

[0049] That is, in the posture determination unit 66 of the present embodiment, as the posture discrimination probability value, the standing position discrimination probability value calculation unit 66a calculates the standing position discrimination probability value ZA, the sitting position discrimination probability value calculation unit 66b calculates the sitting position discrimination probability value ZB, and the fall discrimination probability value calculation unit 66c calculates the fall discrimination probability value ZC. Further, the posture determination unit 66 of the present embodiment executes the calculation of the posture discrimination probability value so that the total value of these standing position discrimination probability value ZA, sitting position discrimination probability value ZB, and fall discrimination probability value ZC becomes "1.0". Then, the posture determination unit 66 of the present embodiment can thereby determine the posture of the occupant 5 without contradiction based on the posture discrimination probability value.

[0050] In addition, the standing position discrimination probability value ZA calculated by the standing position discrimination probability value calculation unit 66a of the present embodiment is further classified into the probability that the occupant 5 in the "standing posture" is in a "moving state", the probability of being in a "stationary state", and the probability of being in a state of "using a suspension strap or handrail, etc.". Then, the posture determination unit 66 of the present embodiment has a configuration that can subdivide and discriminate the "standing posture".

[0051] In addition, in the second image analysis device 22 of the present embodiment, the human information acquisition unit 65 is provided with, in addition to the posture determination unit 66, an attribute determination unit 67 that determines the attributes of the person H reflected in the captured image Vd, a physique determination unit 68 that determines the physique of the person H, and the like. And the second image analysis device 22 of the present embodiment can thereby accurately detect the state of the person H reflected in the captured image Vd.

[0052] As shown in FIGS. 5 and 13, when the abnormality detection unit 62 of the present embodiment determines that the occupant 5 of the vehicle 1 has fallen by the posture determination unit 66 provided in the human information acquisition unit 65, it determines that an abnormality has occurred in the passenger compartment 6 reflected in the captured image Vd of the camera 8. Further, in this case, the second image analysis device 22 of the present embodiment transmits a notification to the vehicle 1 and the operation center 30 indicating that the occurrence of the abnormality has been detected. Note that, in the second image analysis device 22 of the present embodiment, the transmission of the abnormality detection signal S1 for notifying the occurrence of this abnormality is executed by the communication control unit 60. And the monitoring system 40 of the present embodiment is configured such that the operator 32 waiting at the operation center 30 of the vehicle 1 can promptly respond to the abnormality that has occurred in the passenger compartment 6.

[0053] Also, as shown in FIG. 5, the second image analysis device 22 of the present embodiment includes a detection state determination unit 69 that determines the detection state of the skeleton points SP by the skeleton point detection unit 63. Specifically, in the second image analysis device 22 of the present embodiment, the detection state determination unit 69 determines whether the skeleton points SP can be detected by analyzing the captured image Vd in the skeleton point detection unit 63. Further, the detection state determination unit 69 determines whether the main skeleton points SP used for information Ih of the person H reflected in the captured image Vd, that is, the passenger information Ich, such as posture determination, can be stably detected. Furthermore, in the second image analysis device 22 of the present embodiment, when the detection state determination unit 69 determines that the skeleton points SP cannot be detected, a notification to that effect is transmitted to the vehicle 1 and the operation center 30. And the second image analysis device 22 of the present embodiment is configured to transmit a notification to that effect to the vehicle 1 and the operation center 30 also when the detection state determination unit 69 determines that the detection accuracy has decreased to a state where the skeleton points SP cannot be stably detected.

[0054] That is, due to the influence of the light source, the positional relationship with the camera 8, etc., it may be impossible to detect the skeleton points SP by analyzing the captured image Vd, or the detection accuracy may decrease. In view of this point, the second image analysis device 22 of the present embodiment is provided with the detection state determination unit 69. Incidentally, in the second image analysis device 22 of the present embodiment, the transmission of the detection failure signal S2 notifying that the skeleton points SP cannot be detected and the detection accuracy decrease signal S3 notifying that the detection accuracy has decreased is also executed by the communication control unit 60. And the monitoring system 40 of the present embodiment can thereby enable the operator 32 waiting at the operation center 30 to promptly grasp the situation based on the detection failure notification and the detection accuracy decrease notification of the skeleton points SP executed by the second image analysis device 22. That is, it is configured to be able to recognize the fact that a situation has occurred in which the second image analysis device 22 cannot detect an abnormality or the accuracy of the detection determination has decreased.

[0055] Also, as described above, in the monitoring system 40 of the present embodiment, when an abnormality occurs in the passenger compartment 6 in the second image analysis device 22, the captured image Vd in the passenger compartment 6 captured by the camera 8 is distributed to the operator 32 waiting at the operation center 30. Further, in the monitoring system 40 of the present embodiment, even when a situation occurs in which the second image analysis device 22 cannot detect an abnormality or it is determined that the accuracy of the detection determination has decreased, the distribution of the captured image Vd to the operator 32 is executed. And the monitoring system 40 of the present embodiment is configured such that the operator 32 as the administrator 31 can confirm the captured image Vd in the passenger compartment 6 by using the image receiving device 33 provided in the operation center 30.

[0056] That is, as shown in FIG. 14, when the second image analysis device 22 of the present embodiment receives the captured image Vd of the passenger compartment 6 distributed by the first image analysis device 11 (step 201), it executes detection of the skeleton points of the passenger 5 included in this captured image Vd (step 202). And the second image analysis device 22 executes determination of the detection state of the skeleton points SP in this step 202 (step 203).

[0057] Next, the second image analysis device 22 determines whether the skeleton points SP can be detected based on the result of the detection state determination in the above step 203 (step 204). And when the second image analysis device 22 determines in this step 204 that the skeleton points SP can be detected (step 204: YES), subsequently, it determines whether the detection accuracy of the skeleton points SP has not decreased (step 205).

[0058] Furthermore, when the second image analysis device 22 of the present embodiment determines in step 205 that the detection accuracy of the skeleton point SP has not decreased (step 205: YES), it performs an abnormality detection determination based on the detection of the skeleton point SP (step 206). Then, in this abnormality detection determination, when an abnormality occurring in the passenger compartment 6 reflected in the captured image Vd is detected (step 207: YES), it transmits an abnormality detection signal S1 and determines to distribute the captured image Vd to the administrator 31 (step 208).

[0059] Also, when the second image analysis device 22 of the present embodiment determines in step 204 that the skeleton point SP cannot be detected (step 204: NO), it transmits a non-detection signal S2 indicating that fact (step 209). Furthermore, when the second image analysis device 22 determines in step 205 that the detection accuracy of the skeleton point SP has decreased (step 205: NO), it transmits a detection accuracy decrease signal S3 indicating that fact (step 210). And in these cases as well, the second image analysis device 22 of the present embodiment is configured to also determine to distribute the captured image Vd to the operator 32 as the administrator 31 (steps 209 and 210).

[0060] (Distribution control of captured image) Next, the distribution control of the captured image Vd in the monitoring system 40 of the present embodiment will be described.

[0061] As shown in FIGS. 4 and 5, in the monitoring system 40 of the present embodiment, the distribution of the captured image Vd to the operator 32 as the administrator 31 is executed by the first image analysis device 11 provided in the vehicle 1 together with the camera 8.

[0062] Specifically, as shown in FIG. 15, the first image analysis device 11 of the present embodiment determines whether it has received the above-described abnormality detection signal S1 transmitted by the second image analysis device 22 (step 301). Then, when the first image analysis device 11 receives this abnormality detection signal S1 (step 301: YES), it starts distributing the captured image Vd to the operator 32 of the operation center 30 (step 302).

[0063] Also, the first image analysis device 11 determines whether it has received the above-described undetectable signal S2 transmitted by the second image analysis device 22 (step 303). Then, even when the first image analysis device 11 receives this undetectable signal S2 (step 303: YES), by executing the above step 302, it starts distributing the captured image Vd to the operator 32 of the operation center 30.

[0064] Furthermore, the first image analysis device 11 determines whether it has received the above-described detection accuracy degradation signal S3 transmitted by the second image analysis device 22 (step 304). Then, even when the first image analysis device 11 receives this detection accuracy degradation signal S3 (step 304: YES), by executing the above step 302, it starts distributing the captured image Vd to the operator 32 of the operation center 30.

[0065] Also, in the first image analysis device 11 of the present embodiment, at this time, the image compression unit 56 compresses the captured image Vd of the passenger compartment 6 imaged by the camera 8 at a compression ratio βH higher than the compression ratio βL of the captured image VdL distributed to the second image analysis device 22 (βH>βL). And in the monitoring system 40 of the present embodiment, the captured image VdH of the passenger compartment 6 having this high compression ratio βH is configured to be distributed to the operator 32 waiting at the operation center 30 via the information communication network 15.

[0066] That is, even for a captured image Vd with a low resolution and a high compression ratio, in many cases, the situation in the passenger compartment 6 reflected in the captured image Vd can be grasped without problems by a human eye, that is, by an operator 32 as the administrator 31. Based on this point, in the monitoring system 40 of the present embodiment, as described above, a captured image VdH having a higher compression ratio βH is distributed to the operator 32 of the operation center 30. And the monitoring system 40 of the present embodiment is configured to reduce the communication load caused by the distribution of the captured image Vd via the information communication network 15 accordingly.

[0067] More specifically, in the monitoring system 40 of the present embodiment, when the operator 32 of the operation center 30 operates the operation terminal 34, the operator 32 can check the captured image Vd of the passenger compartment 6 reflected in the image receiving device 33 and respond to the situation in the passenger compartment 6. Specifically, in the monitoring system 40 of the present embodiment, by operating the operation terminal 34, the operator 32 of the operation center 30 can communicate with the passenger 5 in the passenger compartment 6 via the information communication network 15. And the monitoring system 40 of the present embodiment is configured such that, for example, the operator 32 can take actions such as speaking to the passenger 5 who has fallen in the passenger compartment 6 accordingly.

[0068] Specifically, as shown in FIG. 16, in the monitoring system 40 of the present embodiment, the information processing device 35 of the operation center 30 constituting the image receiving device 33 displays the captured image Vd of the passenger compartment 6 received from the vehicle 1 on the display 70 (see FIG. 4, step 401). And at this time, the information processing device 35 of the present embodiment also executes a notification output regarding the content indicated by the abnormality detection signal S1, the non-detection signal S2, or the detection accuracy reduction signal S3 received from the second image analysis device 22 (step 402).

[0069] Furthermore, the information processing device 35 as the operation terminal 34 determines whether there is an operation input requesting a call connection with the vehicle 1 (step 403). Further, when the information processing device 35 detects a call request from the operator 32 to the vehicle 1 as a result (step 403: YES), the information processing device 35 transmits a call request signal S4 requesting a call connection to the information processing device 10 mounted on the vehicle 1 (step 404). Then, the monitoring system 40 of the present embodiment is configured such that a call connection between the vehicle 1 and the operation center 30 via the information communication network 15 is established thereby (step 405).

[0070] Also, in the monitoring system 40 of the present embodiment, by operating the operation terminal 34 of the operation center 30, it is possible to change the compression ratio β of the captured image Vd distributed to the operator 32 in response to the request of the operator 32 who is the administrator 31. For example, when it is difficult to confirm the situation of the passenger compartment 6 from the captured image VdH having a high compression ratio βH, the operator 32 can lower the compression ratio β of the captured image VdH from the compression ratio βH which is the initial value by operating the operation terminal 34. Then, the monitoring system 40 of the present embodiment is configured such that the operator 32 at the operation center 30 can confirm the captured image Vd of the passenger compartment 6 with a higher resolution thereby.

[0071] Specifically, as shown in FIG. 17, the information processing apparatus 35 as the operation terminal 34 determines whether there is an operation input requesting a change in the compression rate β of the captured image Vd (step 501). Further, when the information processing apparatus 35 detects a request to change the compression rate β of the captured image Vd by the operator 32 as a result (step 501: YES), it transmits a compression rate change request signal S5 to the first image analysis apparatus 11 of the vehicle 1 (step 502). In the monitoring system 40 of the present embodiment, it is possible to input either an operation to lower the compression rate β of the captured image Vd or an operation to increase the compression rate β to the operation terminal 34 of the operation center 30. And in the monitoring system 40 of the present embodiment, when the first image analysis apparatus 11 receives this compression rate change request signal S5, the compression rate β of the captured image Vd distributed to the operator 32 is changed (step 503).

[0072] Also, in the monitoring system 40 of the present embodiment, when it is confirmed that there is no abnormality in the passenger compartment 6 of the vehicle 1 shown in the captured image Vd, the operator 32 inputs a normal confirmation operation to the operation terminal 34. And in the monitoring system 40 of the present embodiment, based on the input of this normal confirmation operation, the distribution of the captured image Vd to the operator 32 as the administrator 31 is stopped.

[0073] Specifically, as shown in FIG. 18, the information processing apparatus 35 as the operation terminal 34 determines whether there is an input of a normal confirmation operation indicating that there is no abnormality in the passenger compartment 6 of the vehicle 1 shown in the captured image Vd (step 601). Further, when the information processing apparatus 35 detects the input of a normal confirmation operation by the operator 32 as a result (step 601: YES), it transmits a normal confirmation signal S6 notifying that there is no abnormality in the passenger compartment 6 to the first image analysis apparatus 11 of the vehicle 1 (step 602). And in the monitoring system 40 of the present embodiment, when the first image analysis apparatus 11 receives this normal confirmation signal S6, the distribution of the captured image Vd to the operator 32 is stopped (step 603).

[0074] In addition, in the monitoring system 40 of the present embodiment, even when the captured image Vd is not distributed to the operator 32, when the operator 32 operates the operation terminal 34, the distribution of the captured image Vd to the operator 32 is executed.

[0075] Specifically, as shown in FIG. 19, the information processing device 35 as the operation terminal 34 determines whether there is an operation input for requesting the distribution of the captured image Vd (step 701). Further, when the information processing device 35 detects the input of the distribution request operation by the operator 32 as a result (step 701: YES), the information processing device 35 transmits a distribution request signal S7 based on the input of the distribution request operation to the first image analysis device 11 of the vehicle 1 (step 702). Then, in the monitoring system 40 of the present embodiment, when the first image analysis device 11 receives this distribution request signal S7, the distribution of the captured image Vd to the operator 32 is started (step 703).

[0076] Furthermore, in the monitoring system 40 of the present embodiment, the second image analysis device 22 also executes a normal confirmation determination that there is no abnormality in the passenger compartment 6 reflected in the captured image Vd. Note that the normal confirmation determination by the second image analysis device 22 is performed, for example, by determining whether a state in which no abnormality occurring in the passenger compartment 6 reflected in the captured image Vd is detected continues for a predetermined time in the abnormality detection determination (see FIG. 14, step 207: NO). Then, in the monitoring system 40 of the present embodiment, based on this normal confirmation determination, the distribution of the captured image Vd to the second image analysis device 22 mounted on the information processing device 20 outside the vehicle that constitutes the cloud server 17 is stopped.

[0077] That is, as shown in FIG. 20, when the second image analysis device 22 of the present embodiment executes the normal confirmation determination (step 801), in this normal confirmation determination, it is determined whether or not it is confirmed that there is no abnormality in the passenger compartment 6 shown in the captured image Vd (step 802). Further, when the second image analysis device 22 confirms that there is no abnormality in the passenger compartment 6 shown in the captured image Vd (step 802: YES), the second image analysis device 22 transmits a normal confirmation signal S8 notifying the first image analysis device 11 of the vehicle 1 that there is no abnormality in the passenger compartment 6 to the first image analysis device 11 of the vehicle 1 (step 803). Then, in the monitoring system 40 of the present embodiment, when the first image analysis device 11 receives this normal confirmation signal S8, the distribution of the captured image Vd to the second image analysis device 22 is stopped (step 803).

[0078] Next, the operation of the monitoring system 40 of the present embodiment configured as described above will be described. FIG. 21 is an example in the case where the state change that occurred in the passenger compartment 6 shown in the captured image Vd of the camera 8 is an event in the category determined to be "no abnormality", that is, "normal".

[0079] In this case, first, the state change that occurred in the passenger compartment 6 is detected by the first image analysis device 11 provided in the vehicle 1 together with the camera 8 (step 1101). Then, based on the detection of this state change, the first image analysis device 11 distributes the captured image Vd of the passenger compartment 6 to the second image analysis device 22 mounted on the information processing device 20 outside the vehicle that constitutes the cloud server 17 via the information communication network 15 (step 1102). As described above, at this time, the captured image Vd distributed to the second image analysis device 22 is the captured image VdL compressed at a relatively low compression rate βL.

[0080] Next, the second image analysis device 22 that has received the captured image Vd executes an abnormality detection determination for the passenger compartment 6 reflected in this captured image Vd (step 1103). As described above, this abnormality detection determination is performed by detecting the skeleton points SP of the occupant 5 reflected in the captured image Vd and acquiring the occupant information Ich. Further, in this example, by executing this abnormality detection determination, it is confirmed in the second image analysis device 22 that there is no abnormality in the passenger compartment 6 reflected in the captured image Vd, and thus the second image analysis device 22 transmits a normal confirmation signal S8 (step 1104). Then, when the first image analysis device 11 receives this normal confirmation signal S8, the distribution of the captured image Vd to the second image analysis device 22 is stopped (step 1105).

[0081] On the other hand, FIG. 22 shows an example in the case where the state change that has occurred in the passenger compartment 6 reflected in the captured image Vd is actually something that should be determined as "abnormal", such as the fall of the occupant 5. Also in this example, first, the first image analysis device 11 detects the state change that has occurred in the passenger compartment 6 (step 1201). Further, based on the detection of this state change, the first image analysis device 11 starts distributing the captured image Vd to the second image analysis device 22 (step 1202). Then, in this example, an abnormality that has occurred in the passenger compartment 6 reflected in the captured image Vd is detected by the abnormality detection determination executed by the second image analysis device 22 (step 1203).

[0082] Also, in this case, subsequently, the second image analysis device 22 transmits an abnormality detection signal S1 (step 1204). Further, when the first image analysis device 11 receives this abnormality detection signal S1, the distribution of the captured image Vd to the operator 32 as the administrator 31 waiting at the vehicle operation center 30 of the vehicle 1 via the information communication network 15 is started (step 1205). Then, thereby, the captured image Vd of the passenger compartment 6 is displayed on the display 70 of the image receiving device 33 provided in the operation center 30 (step 1206).

[0083] Still, as described above, at this time, the captured image Vd of the passenger compartment 6 distributed to the operator 32 is the captured image VdH compressed at a compression rate βH higher than the compression rate βL of the captured image VdL distributed to the second image analysis device 22 (βH > βL). Then, in conjunction with the display of this captured image Vd, the content of the abnormality detection signal S1, that is, the notification output indicating that an abnormality has been detected in the passenger compartment 6 of the vehicle 1, which is the monitoring space 41, is executed.

[0084] Also, in this example, the operator 32 performs an operation input requesting a call connection with the vehicle 1 to the operation terminal 34 provided in the operation center 30 (step 1207). Further, by inputting this call connection request, a call request signal S4 is transmitted from the information processing device 35 of the operation center 30, and when the vehicle 1 receives this call request signal S4, a call connection between the vehicle 1 and the operation center 30 is established (step 1208). And in this example, by the operator 32 "addressing" the passenger 5 shown in the captured image Vd of the passenger compartment 6 (step 1209), it is confirmed that there is no abnormality in the passenger compartment 6 of this vehicle 1.

[0085] Next, by the operator 32 inputting a normal confirmation operation to the operation terminal 34, a normal confirmation signal S6 is transmitted from the information processing device 35 of the operation center 30 (step 1210). And when the first image analysis device 11 of the vehicle 1 receives this normal confirmation signal S6, the distribution of the captured image Vd to the operator 32 is stopped (step 1211).

[0086] Next, the effects of this embodiment will be described. (1) The monitoring system 40 includes a first image analysis device 11 provided in the vehicle 1 that serves as the edge E of the information communication network 15 together with the camera 8. This first image analysis device 11 analyzes the captured image Vd of the passenger compartment 6 captured by the camera 8 to execute detection and determination of state changes in the passenger compartment 6. Further, when a state change is detected in the passenger compartment 6 as the monitoring space 41, the first image analysis device 11 has a function as an image distribution device 80 that distributes the captured image Vd via the information communication network 15. And the monitoring system 40 includes a second image analysis device 22 that analyzes the captured image Vd of the passenger compartment 6 distributed via the information communication network 15 to execute detection and determination of an abnormality that has occurred in the passenger compartment 6.

[0087] According to the above configuration, the first image analysis device 11 provided in the vehicle 1 can detect a state change that has occurred in the passenger compartment 6 reflected in the captured image Vd. Then, by the second image analysis device 22 analyzing the captured image Vd of the passenger compartment 6 distributed due to the detection of this state change, an abnormality that has occurred in the passenger compartment 6 reflected in the captured image Vd can be detected with high accuracy via the information communication network 15.

[0088] Also, the detection and determination of a state change by analyzing the captured image Vd has a smaller computational load than the detection and determination of an abnormality by analyzing the captured image Vd. For this reason, for the in-vehicle information processing device 10 in which the first image analysis device 11 is implemented, there is an advantage that the required computational processing ability can be suppressed to be relatively small. And when no state change in the passenger compartment 6 is detected, the communication load can be reduced by not distributing the captured image Vd via the information communication network 15.

[0089] (2) The first image analysis device 11 includes a pixel difference value calculation unit 51 that calculates a pixel difference value δ between a previous frame Fb acquired in the previous analysis period and a current frame Fc acquired in the current analysis period by acquiring the captured image Vd for each analysis period. The first image analysis device 11 also includes a history holding unit 52 that holds the calculation history X of the pixel difference value δ, and a variance value calculation unit 53 that calculates a variance value Y of the pixel difference value δ based on the calculation history X. Then, the first image analysis device 11 includes a state change determination unit 54 that determines that a state change has occurred in the passenger compartment 6 reflected in the captured image Vd when the variance value Y of the pixel difference value δ is equal to or greater than a predetermined threshold value Yth.

[0090] According to the above configuration, it is possible to detect and determine a state change based on the analysis of the captured image Vd with relatively light computational load and high accuracy. As a result, it is possible to perform high-precision abnormality detection via the information communication network 15 while ensuring excellent in-vehicle performance.

[0091] In addition, by using the pixel difference value δ between the previous frame Fb and the current frame Fc of the captured image Vd acquired for each analysis period, it is possible to significantly suppress the influence of the external environment of the passenger compartment 6, such as a change in the amount of light.

[0092] (3) The pixel difference value calculation unit 51 calculates the pixel difference value δ only for the boarding area α0 of the passenger 5 in the passenger compartment 6, with the boarding area α0 of the passenger 5 in the passenger compartment 6 being set in advance as the detection area α in the passenger compartment 6. According to the above configuration, it is possible to accurately detect a state change that has occurred in the boarding area α0 set in the detection area α, that is, a state change that has occurred in the passenger compartment 6 targeted at the passenger 5 in the passenger compartment 6 who has boarded the vehicle 1. As a result, it is possible to suppress the occurrence of misjudgment and perform abnormality detection via the information communication network 15 with higher precision.

[0093] (4) The first image analysis device 11 as the image distribution device 80 has a function of distributing the captured image Vd to the operator 32 as the administrator 31 located in the operation center 30 of the vehicle 1 via the information communication network 15. Then, the first image analysis device 11 distributes the captured image VdH to the operator 32 at a higher compression rate β than the captured image VdL distributed to the second image analysis device 22.

[0094] That is, even for a captured image Vd with a low resolution and a high compression rate, in many cases, the situation in the passenger compartment 6 reflected in the captured image Vd can be grasped without problems by the human eye, that is, by the operator 32 as the administrator 31. Therefore, according to the above configuration, a multi-layered system with monitoring by the administrator 31 can be constructed while suppressing an increase in the communication load. And thereby, abnormality detection via the information communication network 15 can be performed with higher accuracy.

[0095] (5) When an abnormality occurring in the passenger compartment 6 reflected in the captured image Vd is detected in the second image analysis device 22, the first image analysis device 11 as the image distribution device 80 distributes the captured image Vd to the operator 32.

[0096] According to the above configuration, the operator 32 as the administrator 31 can quickly confirm the abnormality occurring in the passenger compartment 6. Thereby, a prompt response to the abnormality occurring in the passenger compartment 6 can be ensured. And by limiting the situation where the captured image Vd is distributed to the operator 32, an increase in the communication load can be suppressed.

[0097] (6) When it is determined in the second image analysis device 22 that the detection determination of the abnormality occurring in the passenger compartment 6 cannot be made or the accuracy of the detection determination has decreased, the first image analysis device 11 as the image distribution device 80 distributes the captured image Vd to the operator 32.

[0098] According to the above configuration, by having the operator 32 as the administrator 31 check the captured image Vd of the passenger compartment 6, it is possible to ensure highly accurate anomaly detection via the information communication network 15. And by limiting the situation of distributing the captured image Vd to the operator 32, an increase in communication load can be suppressed.

[0099] (7) The second image analysis device 22 includes a skeleton point detection unit 63 that detects the skeleton points SP of the passenger 5 included in the captured image Vd. Further, the second image analysis device 22 includes an anomaly detection unit 62 that detects an anomaly occurring in the passenger compartment 6 based on the passenger information Ich obtained by detecting the skeleton points SP. Furthermore, the second image analysis device 22 includes a detection state determination unit 69 that determines the detection state of the skeleton points SP. And in the monitoring system 40, based on the detection state of the skeleton points SP, it is determined whether to execute the distribution of the captured image Vd to the operator 32.

[0100] According to the above configuration, it is possible to accurately identify a state where the second image analysis device 22 cannot perform the anomaly detection determination of the captured image Vd or the accuracy of the detection determination has decreased. And thereby, it is possible to appropriately determine whether to execute the distribution of the captured image Vd to the operator 32. As a result, it is possible to avoid a situation where the captured image Vd of the passenger compartment 6 is frequently distributed to the operator 32 and suppress an increase in communication load.

[0101] Also, by detecting the skeleton points SP, it is possible to accurately obtain physical passenger information Ich such as the posture and physique of the passenger 5. And thereby, based on the obtained passenger information Ich, it is possible to perform a highly accurate anomaly detection determination regarding the passenger compartment 6 in which the passenger 5 rides.

[0102] (8) The monitoring system 40 is configured to be able to change the compression ratio β of the captured image Vd distributed to the operator 32 based on the request of the operator 32. According to the above configuration, the operator 32 can confirm the captured image Vd of the passenger compartment 6 at an appropriate resolution while suppressing an increase in communication load. And thereby, abnormality detection via the information communication network 15 can be performed with higher accuracy.

[0103] Note that the above embodiment can be implemented with the following modifications. The above embodiment and the following modification examples can be implemented in combination with each other within a range where there is no technical contradiction.

[0104] ·In the above embodiment, an infrared camera is used as the camera 8, but its type can be arbitrarily changed. For example, a configuration using a visible light camera or the like may be employed. And a configuration in which a plurality of cameras 8 are used to capture the monitoring space 41 may also be adopted.

[0105] ·In the above embodiment, the boarding area α0 of the passenger 5 in the passenger compartment 6 is set as the detection area α set in advance in the passenger compartment 6, and the pixel difference value δ is calculated only for the boarding area α0. However, the present invention is not limited to this, and the setting of the detection area α for calculating the pixel difference value δ can be arbitrarily changed.

[0106] ·Regarding the compression ratio βL of the captured image VdL distributed to the second image analysis device 22 and the compression ratio βH of the captured image VdH distributed to the operator 32 as the administrator 31, they can be arbitrarily set. From the viewpoint of suppressing an increase in communication load, it is preferable that the compression ratio βH of the captured image VdH distributed to the operator 32 is higher than the compression ratio β of the captured image VdL distributed to the second image analysis device 22 (βH>βL). Also, for the captured image VdL distributed to the second image analysis device 22, it is desired to ensure a high resolution that enables accurate detection of the skeleton points SP. And for the captured image VdH distributed to the operator 32, it is desired that the communication amount is small enough to suppress an increase in communication load.

[0107] · Further, it may be configured to distribute the captured image Vd to the operator 32 at the same compression rate β as the captured image Vd distributed to the second image analysis device 22. And the captured image Vd distributed to the operator 32 may have a lower compression rate β than the captured image Vd distributed to the second image analysis device 22.

[0108] · In the above embodiment, it is determined whether to distribute the captured image Vd to the operator 32 based on the detection state of the skeleton point SP. However, the present invention is not limited to this. The determination of whether the captured image Vd can be distributed, that is, in the second image analysis device 22, the detection determination of the abnormality occurring in the passenger compartment 6 cannot be made or the accuracy of the detection determination has decreased, does not necessarily depend on the detection state of the skeleton point SP.

[0109] · In the above embodiment, the posture determination of the occupant 5 is executed based on the detection of the skeleton point SP. Then, by detecting the falling posture, the abnormality in the passenger compartment 6 reflected in the captured image Vd is detected. However, the present invention is not limited to this. For example, when the occupant 5 moving in the passenger compartment 6 is detected during the running of the vehicle 1, it may be determined that an abnormality has occurred in the passenger compartment 6. And a configuration may be adopted in which the abnormality detection determination is executed by using other occupant information Ich obtained by analyzing the captured image Vd.

[0110] · Further, the detection determination of the abnormality occurring in the passenger compartment 6 by the second image analysis device 22 does not necessarily have to be based on the detection of the skeleton point SP. As long as it is based on the analysis of the captured image Vd captured by the camera 8, the abnormality may be detected and determined by other methods using so-called AI technologies such as deep learning and machine learning. And it may be applied to a configuration for detecting an abnormality occurring in the passenger compartment 6 with a target other than the person H.

[0111] · Further, the detection determination of the state change that has occurred in the passenger compartment 6 by the first image analysis device 11 does not necessarily have to be based on the variance value Y of the pixel difference value δ between the previous frame Fb and the current frame Fc of the captured image Vd acquired for each analysis cycle. It may be based on the analysis of the captured image Vd captured by the camera 8. However, considering the optimal distribution of computing power, it is preferable that the detection determination of the state change by the first image analysis device 11 has a smaller computing load than the determination of abnormality detection by the second image analysis device 22.

[0112] · In the above embodiment, the operator 32 waiting at the operation center 30 of the vehicle 1 is used as the administrator 31, and the captured image VdH is distributed to the administrator 31 at a high compression rate βH. However, it is not limited to this, and the setting of the administrator 31 may be arbitrarily changed. For example, it does not necessarily have to wait at one location. Furthermore, the receiving device 33 for the captured image Vd may also be arbitrarily changed, and for example, it may be a device such as a mobile terminal. And the method by which the administrator 31 inputs the request may also be arbitrarily set, for example, voice input or the like.

[0113] · In the above embodiment, the first image analysis device 11 is used as the image distribution device 80, and the captured image Vd is distributed to the administrator 31. However, it is not limited to this, and the second image analysis device 22 may generate a captured image VdH with a high compression rate βH by compressing the received captured image VdL and transfer it to the information processing device 35 of the operation center 30. And the image distribution device 80 may have a configuration different from these first image analysis device 11 and second image analysis device 22.

[0114] · Also, in the above-described embodiment, when the second image analysis device 22 detects an abnormality, when the second image analysis device 22 cannot detect an abnormality, or when the accuracy of the detection determination has decreased, the distribution of the captured image Vd to the operator 32 is executed. However, the present invention is not limited to this, and the situation in which the captured image Vd is distributed to the operator 32 may be arbitrarily changed. Further, for example, a configuration may be adopted in which the captured image VdH having a compression rate βH higher than the compression rate β of the captured image VdL distributed to the second image analysis device 22 is always distributed to the administrator 31. And a configuration in which such distribution of the captured image Vd to the administrator 31 is not performed may also be adopted.

[0115] · In the above-described embodiment, the monitoring system 40 is embodied in which the passenger compartment 6 of the vehicle 1 imaged by the camera 8 is the monitoring space 41. However, the present invention is not limited to this, and a configuration in which the interior of a building is the monitoring space 41 may be adopted. And, for example, a configuration in which the monitoring space 41 is set outdoors may also be adopted.

[0116] Next, the technical idea that can be grasped from the above-described embodiment and modification example will be described. (A) The detection area is the boarding area of the passenger, which is characterized in that. Thereby, it is possible to accurately detect a state change that has occurred in the passenger compartment for the passengers in the passenger compartment who have boarded the vehicle.

Explanation of Signs

[0117] 1... Vehicle 6... Passenger compartment 8... Camera 11... First image analysis device 15... Information communication network 22... Second image analysis device 40... Monitoring system 41... Monitoring space 80... Image distribution device E... Edge Vd... Captured image

Claims

1. A first image analysis device provided at an edge of an information communication network together with a camera, and performing detection determination of a state change occurring in the monitoring space by analyzing a captured image of the monitoring space captured by the camera; An image distribution device that performs distribution of the captured image via the information communication network when occurrence of the state change is detected; A second image analysis device that performs detection determination of an abnormality occurring in the monitoring space by analyzing the distributed captured image, and comprising: The image distribution device performs distribution of the captured image to an administrator located outside the monitoring space when occurrence of the abnormality is detected by the second image analysis device; The image distribution device performs distribution of the captured image to the administrator when it receives a notification from the second image analysis device indicating that the detection determination cannot be made or that the accuracy of the detection determination has decreased, and a monitoring system characterized by this.

2. In the monitoring system according to Claim 1, The first image analysis device, A pixel difference value calculation unit that calculates a pixel difference value between a previous frame of the captured image acquired in the previous analysis period and a current frame of the captured image acquired in the current analysis period by acquiring the captured image for each analysis period; A history holding unit that holds a calculation history of the pixel difference value; A variance value calculation unit that calculates a variance value of the pixel difference value based on the calculation history; And a state change determination unit that determines that the state change has occurred in the monitoring space when the variance value is equal to or greater than a predetermined threshold value, and a monitoring system characterized by this.

3. In the monitoring system according to Claim 2, The pixel difference value calculation unit calculates the pixel difference value only for a detection area set in advance in the monitoring space, and a monitoring system characterized by this.

4. In the monitoring system according to any one of Claims 1 to 3, The image distribution device performs distribution of the captured image to the administrator located outside the monitoring space at a higher compression rate than the captured image to be distributed to the second image analysis device, and a monitoring system characterized by this.

5. In the monitoring system according to Claim 4, Based on a request from the administrator, the compression rate of the captured image to be distributed to the administrator can be changed, and a monitoring system characterized by this.

6. In the monitoring system according to Claim 1, The second image analysis device detects the skeletal points of a person included in the captured image and executes the detection determination based on the information of the person obtained by the detection of the skeletal points, and it is characterized in that whether to distribute the captured image to the administrator is determined based on the detection state of the skeletal points. A monitoring system.

7. In the monitoring system according to any one of Claims 1 to 6, the monitoring space is a passenger compartment of a vehicle, and the monitoring system is characterized by this.

Images