Abnormality detection device

Abstract

To provide an abnormality detection device which can simply and rapidly detect abnormality such as fall or a rapid squat of a person, that is, occurrence of a person's attitude abnormality.SOLUTION: The abnormality detection device detects the person's attitude abnormality through a position relation between a reference point PP1 of a total body image PE and an object point PP2 of a head image PH being an upper body image detected from a body in an image. Thus, by performing detection based on a relative position relation between the reference point PP1 of the total body image PE and the object point PP2 of the upper body image (head image PH), the person's attitude abnormality is detected, and further a dangerous state is determined.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an abnormality detection device that detects whether or not an abnormality has occurred in a passerby or the like. [Background technology]

[0002] For example, an abnormality determination device (see Patent Document 1) is known that determines whether a person's behavior is abnormal based on the person's movement path detected from images taken by multiple cameras discretely placed within a monitored area and the time the person spends in blind spots on that path.

[0003] However, for example, in Patent Document 1, it takes time to detect abnormal behavior, and there is a possibility that it may not be possible to quickly detect sudden abnormal behavior or states, such as a passerby falling or suddenly crouching down. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-109724 Summary of the Invention

[0005] The present invention has been made in consideration of the above points, and aims to provide an abnormality detection device that can easily and quickly detect abnormalities such as a person falling or suddenly crouching, i.e., the occurrence of abnormal human posture.

[0006] To achieve the above object, an abnormality detection device detects abnormal posture of a person from the positional relationship between a reference point in a whole-body image extracted from the person in the image and a target point in an upper-body image.

[0007] The above-mentioned abnormality detection device detects abnormalities in a person's posture quickly and easily through simple processing, by detecting them based on the relative positional relationship between a reference point in the whole-body image and a target point in the upper-body image, and ultimately can determine whether a dangerous situation exists.

[0008] In a specific aspect of the present invention, a target point is identified from a head image. In this case, by using the head image in identifying the target point, human posture detection can be performed quickly and reliably.

[0009] In another aspect of the present invention, based on the degree of overlap between the region of the full body image and the region of the upper body image, it is determined whether the full body image and the upper body image are human body images of the same person. In this case, by making a determination based on the degree of overlap, it is possible to avoid or suppress the occurrence of false detection in human detection.

[0010] In still another aspect of the present invention, based on a marker fixedly installed at a specific location within the imaging range for acquiring the full body image and the upper body image, the position in the depth direction is calculated. In this case, three-dimensional human position detection becomes possible.

[0011] In still another aspect of the present invention, the presence or absence of an abnormality is determined based on an abnormality determination map regarding the positional relationship between the reference point of the full body image and the target point of the upper body image. In this case, by referring to the abnormality determination map, simple and reliable abnormality determination becomes possible.

[0012] In still another aspect of the present invention, the abnormality determination map varies according to the detection position of the person. In this case, an appropriate one can be selected as the determination criterion according to the detection position of the person.

[0013] In still another aspect of the present invention, when the number of detection targets within the imaging range is equal to or less than a predetermined number, the process of abnormality detection based on the full body image and the upper body image is started. In this case, for example, abnormality detection can be performed in a situation where there are not many people around and there is a possibility that an abnormality may be overlooked even if an abnormality occurs.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0015] Hereinafter, with reference to FIG. 1 and the like, an example of an abnormality detection device according to an embodiment will be described. FIG. 1 is a perspective view showing an example of the installation state of the abnormality detection device 100 according to the present embodiment, and FIGS. 2(A) to 2(C) are image diagrams showing the state of detection images by the abnormality detection device 100.

[0016] As shown in FIG. 1, the abnormality detection device 100 according to the present embodiment includes a plurality of cameras 10 (10A, 10B, 10C,...), a PC including a CPU and various storage devices, and various circuits, etc., and an image processing device 11 connected to each camera 10. The abnormality detection device 100 performs various image processes on the images captured by the plurality of cameras 10 with the image processing device 11 to detect whether an abnormality has occurred in the imaging range SS. In an example shown in FIG. 1 and the like, the abnormality detection device 100 is installed at a station, uses the platform of the station as the target range for abnormality detection, extracts a person, that is, a user of the station, from the images of the platform captured by the plurality of cameras 10 in the image processing device 11, and detects whether there is an abnormality in the person. Each of the cameras 10A, 10B, 10C has its respective imaging range SS (SSA, SSB, SSC,...) arranged along the platform as the monitoring range. Also, in an example shown in the figure, as an example of installing on the platform of the station, a plurality of cameras 10 are installed along the platform, but the case where only one camera is used is also conceivable.

[0017] Regarding the state of the detected images by the abnormality detection device 100, it is exemplified with reference to the image diagrams conceptually shown as FIGS. 2(A) to 2(C). Among these, FIG. 2(A) shows an example of the state of the detection when there is no abnormality (a person is not fallen, etc.), and FIG. 2(B) shows an example of the state of the detection when there is an abnormality (a person is fallen, etc.). Also, the abnormality detection device 100 is not limited to the case of being installed on the platform of the station, and can be installed in various places such as being installed in an indoor facility as shown in an example in FIG. 2(C).

[0018] In this embodiment, as shown in FIG. 2(A) and the like, a full-body image PE extracted from a person in the image GG and a head image PH as an example of an upper-body image are extracted. Further, an overall frame FR1 surrounding the full-body image PE and a head frame FR2 surrounding the head image PH are extracted from the image GG. The abnormality detection device 100 simply and quickly detects the occurrence of a human posture abnormality by comparing these. Note that, for the person in the image GG illustrated in FIG. 2(A) and the like, the reference point PP1 is, for example, the center of gravity of the full-body image PE, and the target point PP2 is, for example, the center of the head image PH.

[0019] Hereinafter, with reference to FIG. 3 and the like, a specific example of image processing for abnormality detection based on a human posture in the abnormality detection device 100, that is, image processing in the image processing device 11, will be described. FIGS. 3(A) to 3(D) are conceptual image diagrams showing the procedure of image processing in the abnormality detection device 100. In the illustrated example, for the image GG, it is regarded as a dot display on the xy plane where the horizontal direction (left-right direction) is x and the vertical direction (up-down direction) is y. FIGS. 4(A) and 4(B) are block diagrams showing an example of the internal structure of the abnormality detection device 100 for performing a series of processes illustrated in FIG. 3.

[0020] As image processing, first, as shown in an example in FIG. 3(A), a full-body image PE and a head image PH of a person are extracted from the image GG. Further, as shown in FIG. 3(B), a rectangular overall frame FR1 surrounding the full-body image PE is drawn. At the same time, a rectangular head frame FR2 surrounding the head image PH is drawn. When there are a plurality of people in one image GG, a plurality of full-body images PE, head images PH, and further, a plurality of overall frames FR1 and head frames FR2 will be extracted.

[0021] Here, when displays for station staff and the like to perform monitoring are made on the display unit (not shown), the overall frame FR1 and the head frame FR2 are drawn as shown in the figure. However, in the pixel plane indicated by xy, it is sufficient if information on the positions (display dot positions) of the overall frame FR1 and the head frame FR2 in the image data can be obtained. Also, although details will be described later, in the present embodiment, based on the degree of overlap, that is, the degree of superimposition, between the overall frame FR1 corresponding to the region of the full-body image PE extracted as described above and the head frame FR2 corresponding to the region of the head image PH, it is determined whether the full-body image PE and the head image PH are human body images of the same person.

[0022] Next, as shown in FIG. 3(C), the reference point PP1 and the target point PP2 are extracted. Regarding these, as described above, it is conceivable to set the center of gravity of the full-body image PE as the reference point PP1 and the center of the head image PH as the target point PP2. That is, the reference point PP1 is a point representing the position of the full-body image PE, and the target point PP2 is a point representing the position of the head image PH. Hereinafter, the point at the center (center of gravity) position of the overall frame FR1 of the rectangle corresponding to the full-body image PE is set as the reference point PP1, and the point at the center (center of gravity) position of the head frame FR2 of the rectangle corresponding to the head image PH is set as the target point PP2. Here, in this example, since the head image PH is adopted as an example of the upper body image to be extracted, in the above, the target point PP2 is in a mode specified from the head image PH. Also, the reference point PP1 and the target point PP2 are not limited to the examples shown above, and various points representing the positions of the full-body image PE and the head image PH can be taken.

[0023] Next, as shown in FIG. 3(D), by comparing the reference point PP1 and the target point PP2 and calculating the positional relationship such as the distance between the two as indicated by the arrow AA1, the presence or absence of an abnormal human posture is detected. For example, if a person has not fallen or the like, when comparing the central position of the body and the central position of the head, in the vertical direction (y-direction) in the image, the central position of the body is higher, and in the horizontal direction (x-direction), it is considered that there is little difference. On the other hand, for example, in the case of a person who has fallen, it is considered that the up and down are reversed or the left and right are separated. Therefore, in the present embodiment, by detecting an abnormal human posture from the positional relationship between the reference point PP1 and the target point PP2 while taking the above into consideration, it is possible to simply, quickly, and accurately detect the occurrence of an abnormality related to a person.

[0024] Hereinafter, with reference to the block diagrams shown as FIGS. 4(A) and 4(B), an example of the internal structure of the abnormality detection device 100 for performing the above-described series of image processes exemplified with reference to FIGS. 3(A) to 3(D) will be described. FIG. 4(A) shows the overall functional configuration mainly related to the image processing device 11 in the abnormality detection device 100, and FIG. 4(B) shows various programs and data stored for exerting various functions.

[0025] First, with reference to FIG. 4(A), general matters of the abnormality detection device 100 that performs various processes including the above-described image processing will be described. As shown in FIG. 4(A), the image processing device 11 that constitutes the main part of the abnormality detection device 100 includes a control unit 110, a storage unit 111, an image acquisition unit 112, a timing unit 113, and an abnormal posture determination unit 114. In the present embodiment, the abnormal posture determination unit 114 mainly performs the various processes exemplified in FIG. 3. In addition to the above, the image processing device 11 includes an attribute specifying unit 115 and a notification unit 118. As described above, the image processing device 11 is composed of a PC, various circuits, etc., and the CPU and various storage devices provided in the PC, etc. cooperate to exhibit the functions as the above-described respective units.

[0026] The control unit 110 is composed of a CPU or the like and controls the operations of each unit. The storage unit 111 is composed of various storage devices or the like and stores various data and programs read according to instructions from the control unit 110. The image acquisition unit 112 acquires image data output from a plurality of cameras 10 according to instructions from the control unit 110 and stores it in the storage unit 111. The timing unit 113 continuously measures the current time and associates the measured current time when the image acquisition unit 112 acquires image data. That is, time information is added to the image data stored in the storage unit 111.

[0027] The abnormal posture determination unit 114 performs various image processes exemplified with reference to FIG. 3(A) or the like on the image data stored in the storage unit 111 and detects abnormalities based on the posture of a person. Details of this will be described with reference to FIG. 4(B).

[0028] In addition to the above image processing by the abnormal posture determination unit 114, the abnormality detection device 100 performs various image processes to monitor the station platform.

[0029] For example, the attribute identification unit 115 identifies the attributes of the people shown in the image from the images stored in the storage unit 111 by using known image recognition methods. Specifically, the attributes of the people identified by the attribute identification unit 115 are, for example, "child", "elderly person", "visually impaired person", "wheelchair user", etc. If the attribute identification unit 115 detects a schoolbag by object detection from a photographed image, it identifies the attribute of the schoolbag owner as "child", if it detects a handcart, it identifies the attribute of the handcart owner as "elderly person", if it detects a white cane, it identifies the attribute of the white cane holder as "visually impaired person", and if it detects a wheelchair, it identifies the attribute of the person owning the wheelchair as "wheelchair user". The attributes identified by the attribute identification unit 115 are stored in the storage unit 111.

[0030] The abnormality detection device 100 can detect, from the detection result of the attribute identification unit 115, for example, whether there is a user who requires special attention. Also, in such a case, in parallel with the abnormal posture determination unit 114, it is conceivable to perform monitoring, for example, on whether there is a possibility of falling for a white cane holder, by various methods. Furthermore, the abnormality detection device 100 may be configured to perform various processes in addition to the image processing by the attribute identification unit 115. On the other hand, it is considered that there is a very high requirement for the speed of detecting abnormalities such as a person falling. Therefore, there is a general requirement to simplify each process and minimize the load. Thus, in the present embodiment, in order to reduce the processing as the abnormal posture determination unit 114, the reference point PP1 of the full-body image PE and the target point PP2 of the head image PH (see FIG. 3) are used to simply and quickly detect an abnormality related to a person falling.

[0031] The notification unit 118 outputs information to the outside. Here, for example, when the control unit 110 determines that an abnormality should be notified based on the detection result of the abnormal posture determination unit 114, a notification prompting attention to the outside is transmitted via the notification unit 118. Various modes are assumed as the notification destination. For example, in the case of a manned station, it is conceivable to notify various notification means such as a display device and a speaker provided in the station office. Also, it may be configured to notify a portable terminal (such as a smartphone) held by a station staff. In the case of an unmanned station, it is conceivable to notify a nearby manned station. Furthermore, it is also conceivable to notify the driver of a train scheduled to arrive at the station.

[0032] Next, with reference to FIG. 4(B), a configuration example for performing abnormality detection based on the posture determination of a person by the abnormal posture determination unit 114 will be described. FIG. 4(B) shows various programs and data stored to exhibit various functions in order to perform the above-described series of image processes exemplified with reference to FIGS. 3(A) to 3(D). These data are stored, for example, in various storage devices such as the storage unit 111, and are appropriately read out according to the operation of the CPU, thereby functioning as the abnormal posture determination unit 114. More specifically, as shown in the figure, a program storage unit PM and a data storage unit DM are provided in a predetermined area of the storage unit 111. Using various programs stored in the program storage unit PM, various image processes are performed, and various data as processing results are stored in the data storage unit DM. As described above, in the image processing apparatus 11, the function as the abnormal posture determination unit 114 is exhibited.

[0033] First, as the program storage unit PM, it includes a person extraction unit HE, a frame extraction unit FE, an overlap degree determination unit OJ, a point extraction unit CE, and an abnormality determination unit EJ as programs for performing various extractions and determinations regarding a person from an image.

[0034] The person extraction unit HE has a full body image extraction unit HE1 and a head image extraction unit HE2. The full body image extraction unit HE1 extracts a full body image PE from the image GG, and the head image extraction unit HE2 extracts a head image PH from the image GG.

[0035] The frame extraction unit FE has an overall frame extraction unit FE1 and a head frame extraction unit FE2. The overall frame extraction unit FE1 extracts an overall frame FR1 from the full body image PE, and the head frame extraction unit FE2 extracts a head frame FR2 from the head image PH.

[0036] The overlap determination unit OJ determines whether the extracted overall frame FR1 and the head frame FR2 are of the same person based on the degree of overlap between the overall frame FR1 and the head frame FR2 extracted by the frame extraction unit FE. Also, when a single image GG is detected for a plurality of overall frames FR1 and head frames FR2, these are matched. That is, for the full-body image and the head image, those estimated to be of the same person from the degree of overlap are linked together.

[0037] The point extraction unit CE includes a reference point extraction unit CE1 and a target point extraction unit CE2. The reference point extraction unit CE1 extracts the reference point PP1 of the full-body image PE from the overall frame FR1, and the target point extraction unit CE2 extracts the target point PP2 of the head image PH from the head frame FR2.

[0038] The abnormality determination unit EJ performs an abnormality determination based on a comparison between the reference point PP1 and the target point PP2. Here, as an example, the abnormality determination unit EJ calculates the difference between the coordinates of the reference point PP1 and the target point PP2, and the image processing apparatus 11 (abnormal posture determination unit 114) makes a determination by referring to the abnormality determination map JM stored in the data storage unit DM regarding the calculation result. For details of the determination method using the abnormality determination map JM, refer to FIGS. 5 and 6, and an example will be described later.

[0039] Next, as the data storage unit DM, in addition to including an image data storage unit GP for storing the results of the above various image processes, it includes the aforementioned abnormality determination map JM.

[0040] The image data storage unit GP includes a person image data storage unit HP that stores image data related to the person extracted by the person extraction unit HE, a frame data storage unit FP that stores the frame data extracted by the frame extraction unit FE, and a point data storage unit CP that stores the point data extracted by the point extraction unit CE. Note that the determination result by the overlap determination unit OJ is stored in the frame data storage unit FP in association with the corresponding data. For example, when the association between data is made by the overlap determination unit OJ, the information of the association is stored together. Also, the determination result by the abnormality determination unit EJ is stored in the point data storage unit CP. Further, the above-mentioned image data, frame data, and point data are associated with each other among those estimated to be the same person and are overall tabularized.

[0041] As described above, the abnormality determination map JM is table data for determining whether there is an abnormality in the result of calculation (distance calculation) regarding the data stored in the point data storage unit CP in the calculation using the abnormality determination unit EJ. The abnormality determination unit EJ performs the above calculation and collates the calculated result with the abnormality determination map JM to determine whether there is an abnormality or whether there is a possibility of an abnormality. That is, the presence or absence of an abnormality is determined based on the abnormality determination map JM regarding the positional relationship between the reference point PP1 and the target point PP2.

[0042] FIG. 5(A) and FIG. 5(B) are conceptual image diagrams showing an example of a person image to be processed, and FIG. 6 is a conceptual diagram showing an example of the abnormality determination map JM and an example of the mapping state of the person image to be processed illustrated in FIG. 5(A) and FIG. 5(B) on the abnormality determination map JM. Here, FIG. 5(A) shows a typical example when a person is not fallen or the like (Case 1), that is, when there is no abnormality, and FIG. 5(B) shows a typical example when a person is fallen (Case 2), that is, when there is an abnormality. As shown in the figure, here, the xy coordinates of the reference point PP1 of the full body image PE are set as (X1, Y1), and the xy coordinates of the target point PP2 of the head image PH are set as (X2, Y2). Further, the difference between these is set as the determination index (X3, Y3). That is, X3 = X2 - X1 Y3 = Y2 - Y1 Let it be so. For the determination index (X3, Y3) defined as above, the reference abnormal determination map JM is defined as shown in FIG. 6. Specifically, when the determination index (X3, Y3) belongs to the first region DM1 in the abnormal determination map JM of FIG. 6, it is determined to be normal (no abnormality); when it belongs to the second region DM2, it is determined that there is a possibility of abnormality; and when it belongs to the third region DM3, it is determined to be abnormal.

[0043] Here, in FIG. 6, among the abnormal determination maps JM as two-dimensional maps (XY coordinates) showing the determination index (X3, Y3), the first region DM1 is a region where the value of Y3 is a positive region and the absolute value of Y3 is relatively large compared to the absolute value of X3. In comparison, the second region DM2 is a region where the value of Y3 is zero or more and the absolute value of Y3 is relatively large compared to the absolute value of X3, but there is no difference between the absolute value of Y3 and the absolute value of X3 as much as in the first region DM1. The third region DM3 is a region other than the above two, which is a region where the value of Y3 is a negative region or the difference between the absolute value of X3 and the absolute value of Y3 is not much or is large.

[0044] For example, as illustrated in FIG. 1, when the camera 10 is installed at a certain height (for example, higher than the height of a person) and imaging is performed in an overlooking state, for the person shown, if they are not falling or the like, the center of the head is above the center of the body to a certain extent or more in the y direction (+y side). That is, in this case, among the above-described determination index (X3, Y3), the value Y3 becomes a large positive value. On the other hand, in the left-right direction (x direction), it is considered that the person's posture is upright or close to it, and the value X3, which is the difference in the x direction, does not become a very large value.

[0045] From the above, when the determination index (X3, Y3) is in a region such as the first region DM1, it will be judged as normal.

[0046] On the other hand, when a person is about to fall or there is suspicion of a fall, it is considered that the value Y3 becomes smaller, and for the value X3, its absolute value becomes larger (a difference occurs in either the left or right direction).

[0047] From the above, when there are determination indices (X3, Y3) in an area such as the second area DM2, it is judged that there is a risk of abnormality.

[0048] Furthermore, when the value Y3 is negative, that is, when the center of the head is below the center of the body, or when the absolute value of the value X3 is larger than the value Y3, that is, when the head is significantly separated from the center of the body in the left - right direction, it is considered that the possibility of the person falling is very high.

[0049] From the above, when there are determination indices (X3, Y3) in an area such as the third area DM3, the abnormality determination unit EJ determines that there is an abnormality.

[0050] Regarding the above, for example, as shown in the mapping result in the abnormality determination map JM on the right side of FIG. 6, in the first case illustrated in FIG. 5(A), it belongs to the first area DM1 and is judged to be normal, while in the second case illustrated in FIG. 5(B), it belongs to the third area DM3 and is judged to be abnormal.

[0051] Furthermore, the mapping results for various cases are, for example, as illustrated in FIG. 7. As shown in the figure, those in a vertical or nearly vertical state belong to the first area DM1, those in a horizontal state or upside - down belong to the third area DM3, and those in an intermediate state belong to the second area DM2.

[0052] Next, an example of the installation preparation of the abnormality detection device 100 as illustrated in FIG. 1 will be described. In the series of explanations regarding the above operations, for the sake of simplification, only the left - right direction (x - direction) and the up - down direction (y - direction) are mentioned. At this time, the differences between the in - plane left - right / up - down directions on the image and the left - right / up - down directions in the actual three - dimensional space are treated equally and explained. Furthermore, the explanation about the depth direction has been omitted. In contrast, the imaging range when each camera 10 (see FIG. 1) performs monitoring is a three - dimensional space, and it is conceivable to consider the depth direction, that is, whether the subject is far or near from the camera 10. Regarding the depth direction, for example, if it can be considered that the shape of a human being projected onto the xy - plane on the image becomes proportionally smaller, it is also conceivable to cope with this by providing proportional indicators for the x - direction and the y - direction in the abnormality determination map JM illustrated in FIG. 6. However, for example, in the case of a person located very close to the camera 10, a situation where the overlooking state (diagonal viewing angle) becomes extremely large is also assumed. In such a case, it is considered that the central position of the head and the central position of the body on the image come very close, and if a uniform abnormality determination map JM is used, there is a possibility that accurate determination cannot be made.

[0053] In view of the above, here, with reference to FIG. 8, an example of the case of performing the installation preparation of the abnormality detection device 100 while considering the depth direction will be described. As shown in FIG. 8(A), for the directions of the three - dimensional space as the real space, orthogonal coordinates using the u - direction, the v - direction, and the w - direction are used. Among these, the u - direction corresponds to the x - direction (left - right direction) on the image, the w - direction corresponds to the y - direction (left - right direction) on the image, and the v - direction is the depth direction.

[0054] First, as shown in FIG. 8(A), in order to grasp the depth direction, here, among the floor surfaces FL parallel to the uv plane, four markers MK1 to MK4 are installed at specified positions included in the imaging range (monitoring range) SS of the camera 10. In this case, for the w direction, it is assumed to be the same and is omitted, and the uv coordinates indicating the positions of the markers MK1 to MK4 are set as the point (u1, v1), the point (u2, v2), the point (u3, v3), and the point (u4, v4).

[0055] Corresponding to the above markers MK1 to MK4, as shown in FIG. 8(B), in the image GG, marker images MKi1 to MKi4 are projected. Here, the xy coordinates indicating the positions of the marker images MKi1 to MKi4 on the image are set as the point (x1, y1), the point (x2, y2), the point (x3, y3), and the point (x4, y4). These points respectively correspond to the point (u1, v1), the point (u2, v2), the point (u3, v3), and the point (u4, v4) in FIG. 8(A). The position on the image in the depth direction (v direction) in the real space can be expressed by a linear transformation matrix based on these corresponding points.

[0056] Also, as described above, by setting the four markers MK1 to MK4 provided in the real space as points on the floor surface FL, when capturing the person HU in the real space illustrated in FIG. 8(A), the position of the feet part in the full body image PE of the person illustrated in FIG. 8(B), which is the corresponding image diagram, can be used as a reference for determining the position in the depth direction. That is, by using the above transformation matrix for the position coordinates of the feet of the full body image PE of the person in the image GG, the standing position (position in the depth direction) of the person HU in the real space corresponding to the full body image PE can be calculated.

[0057] As described above, based on the markers MK1 to MK4 fixedly installed at specific locations within the imaging range for acquiring the full body image PE and the upper body image (head image PH), by calculating the position in the depth direction (v direction), three-dimensional human position detection becomes possible.

[0058] Also, in the above cases, by using different abnormality determination maps JM according to the values in the depth direction, that is, by making the abnormality determination maps JM different according to the detection positions of the persons, it becomes possible to perform a more appropriate determination of the presence or absence of abnormalities.

[0059] Hereinafter, with reference to the flowchart of FIG. 9, an example of a series of processing operations by the above-described abnormality detection device 100 will be described.

[0060] First, when the camera 10 of the abnormality detection device 100 acquires image data (step S101), the image processing device 11 of the abnormality detection device 100 performs a series of processes related to object detection, that is, person detection (step S102). That is, as an operation in step S102, the image processing device 11 extracts a full-body image PE and a head image PH, and also extracts an overall frame FR1 and a head frame FR2 surrounding them.

[0061] Next, based on the overlap determination unit OJ, the image processing device 11 checks whether or not the overlap between the extracted overall frame FR1 and head frame FR2 is equal to or greater than a predetermined threshold (step S103). That is, the image processing device 11 determines whether or not the extracted overall frame FR1 and head frame FR2 are of the same person based on the degree of overlap between the overall frame FR1 and the head frame FR2.

[0062] In step S103, when it is determined that the value is equal to or greater than the threshold (step S103: Yes), the image processing device 11 associates the extraction result (person detection) of the full-body image PE corresponding to the overall frame FR1 with the extraction result (face detection) of the head image PH corresponding to the head frame FR2 (step S104).

[0063] Next, the image processing apparatus 11 performs an abnormality determination for the same person for whom the full-body image PE and the head image PH are associated in step S104 (step S105). That is, the image processing apparatus 11 based on the abnormality determination unit EJ calculates a determination index as regarding the positional relationship between the reference point PP1 of the full-body image PE and the target point PP2 of the head image PH, and collates the calculated determination index with the abnormality determination map JM (step S106).

[0064] As a result of the above collation, when it is determined that there is an abnormality (step S106: Yes), the image processing apparatus 11 based on the abnormality determination unit EJ transmits this fact to the storage unit 111 as a registration unit or an update unit in order to newly register or update it (step S107), returns to the first operation, and repeats the operation. For example, in the case where continuous imaging is performed by the camera 10 and image data is continuously input, the above operations are repeated, and data is accumulated or updated in the storage unit 111.

[0065] On the other hand, in step S106, when it is not determined that there is an abnormality (step S106: No), and when it is determined that there is a possibility of an abnormality (step S108: Yes), the image processing apparatus 11 based on the abnormality determination unit EJ transmits this fact to the storage unit 111 in order to newly register or update it (step S107), and returns to the first operation.

[0066] Furthermore, in step S107, when it is not determined that there is a possibility of an abnormality (step S108: No), that is, when it is determined to be normal (no abnormality), the abnormality determination unit EJ discards the data (step S109), transmits this fact to the storage unit 111 (step S107), and returns to the first operation. In this case, a series of various data regarding the corresponding target person stored in the storage unit 111 is discarded.

[0067] In step S103, when it is not determined that the value is equal to or greater than the threshold value (step S103: No), that is, even when the overlap between the overall frame FR1 and the head frame FR2 is less than the predetermined threshold value, the data is discarded (step S109), and this fact is transmitted to the storage unit 111 (step S107), and the process returns to the first operation. In this case, it is determined that the overall frame FR1 and the head frame FR2 are not of the same person, and thus a series of various data related to the target person stored in the storage unit 111 is discarded. Regarding this, for example, in addition to the case where the body and head of different persons are captured, the case where a non-head part is misrecognized as the head during person extraction may be considered. That is to say, here, a process for removing such misrecognition in advance is being performed.

[0068] The above operation is repeated until the abnormality detection by the abnormality detection device 100 is completed.

[0069] As described above, the abnormality detection device 100 according to the present embodiment detects an abnormal human posture from the positional relationship between the reference point PP1 of the full body image PE extracted for the person in the image and the target point PP2 of the head image PH which is the upper body image. Thereby, in the abnormality detection device 100, by performing detection based on the relative positional relationship between the reference point PP1 of the full body image PE and the target point PP2 of the upper body image (head image PH), it is possible to quickly detect an abnormal human posture with simple processing, and thus determine a dangerous state.

[0070] Hereinafter, with reference to the flowchart of FIG. 10, a modified example of a series of processing operations by the abnormality detection device 100 will be described. FIG. 10 is a flowchart corresponding to FIG. 9, and after performing a series of processes related to object detection, that is, person detection in step S102, it is different from the example shown in FIG. 9 in that it determines whether the number of person detections is less than or equal to a predetermined number (step SD). Specifically, as a result of person detection in step S102, if the number of detected persons is less than or equal to a predetermined number (step SD: Yes), the processes after step S103 are performed in the same manner as in the case of FIG. 9. In other words, when the number of detection targets within the imaging range by the camera 10 is less than or equal to a predetermined number, the abnormality detection process based on the full body image PE and the upper body image (head image PH) is started. On the other hand, if the number of detected persons exceeds the predetermined number (step SD: No), the operation ends without performing the subsequent operations and returns to the first operation. Note that the predetermined number (number of detected persons) specified in advance in step SD can be appropriately set according to the usage mode and the like.

[0071] Here, the above operation mode is based on the image processing ability in the abnormality detection device 100, the occurrence of detection errors, and further from the perspective of the necessity of abnormality detection. As described above, in the image processing device 11 of the abnormality detection device 100, in addition to the abnormality detection regarding the posture of the person in the above mode, various image processes such as the detection of the holder of a white cane are performed in parallel. For this reason, there is a demand to minimize the load caused by the image processing regarding the posture of the person. Also, when a large number of people are detected in one image, in addition to the increase in load according to the number of people, the possibility of errors such as misrecognition also increases, and as a result, the number of things that become useless processing also increases. On the other hand, the abnormality detection here is to quickly find out when a person has fallen, etc. However, in a situation where several people are shown in one image, it is considered extremely unlikely that no one around will notice even if there is a fallen person. On the contrary, in a situation where it is likely to be deserted, such as an unmanned station, and there are no station staff, it is considered that a person's fall, etc. is likely to be overlooked, and there is a strong demand to avoid or suppress such a situation. Considering such circumstances, by determining in advance a specified number within an appropriate range, it becomes possible to detect abnormalities at the optimal timing.

[0072] Also in the above-described modification example, by performing detection based on the relative positional relationship between the reference point PP1 of the full-body image PE and the target point PP2 of the upper-body image (head image PH), it is possible to quickly detect an abnormal posture of a person with simple processing, and thus determine a dangerous state. In particular, in this modification example, it can be configured to start abnormality detection at an appropriate timing.

[0073] 〔Others〕 The present invention is not limited to the above-described embodiments, and can be implemented in various modes without departing from the gist thereof.

[0074] First, regarding the resolution and type of the camera 10 to be used, various ones can be considered according to the required accuracy.

[0075] In addition, in one example of FIG. 1, the abnormality detection device 100 processes the image data from the plurality of cameras 10 collectively in one image processing device 11. However, the present invention is not limited to this. For example, it is also conceivable to install one image processing device 11 for each camera 10.

[0076] Further, in the above description, the head image PH is adopted as a representative of the upper body image. However, the present invention is not limited to this. As the upper body image, it is conceivable to adopt images of various parts of a human (partial images, partial pictures), or to include a part of the body following the head. Also, if accurate fall detection is possible, it is also conceivable to set a part other than the upper body as the detection target.

[0077] In addition, in the above description, for the reference point PP1 and the target point PP2, the center (center of gravity) positions of the entire frame FR1 and the head frame FR2, which are rectangles, are used. However, for the reference point PP1 and the target point PP2, as long as the above-described determination can be accurately made, the present invention is not limited to this, and various positions can be set. Also, the shapes of the entire frame FR1 and the head frame FR2 may be other than rectangles.

[0078] Regarding the abnormality determination map JM shown as an example in FIG. 6, various modes are possible. As described above, it is also possible to prepare a plurality of maps with different patterns depending on the position in the depth direction. Furthermore, regarding the shape of each region, it is conceivable to adopt various shapes, not limited to the case where the boundary is linear as in FIG. 6. In addition, in one example of FIG. 6, it is divided into three stages of the first region DM1 to the third region DM3. However, it may be further divided into more stages, or conversely, it may be determined in two stages of abnormal or normal.

[0079] Regarding the timing of notification when a fall is detected, various modes are conceivable. In addition to the mode of performing the notification immediately after the fall is detected, considering the case where there is a possibility of getting up immediately even after falling, for example, in the case of continuously processing consecutive images, the notification may be made when it is confirmed that the fallen state is maintained continuously to a certain extent.

Explanation of Reference Numerals

[0080] 10… Camera, 11… Image processing device, 100… Abnormality detection device, 110… Control unit, 111… Memory unit, 112… Image acquisition unit, 113… Timing unit, 114… Abnormal posture determination unit, 115… Attribute identification unit, 118… Notification unit, AA1… Arrow, CE… Point extraction unit, CE1… Reference point extraction unit, CE2… Target point extraction unit, CP… Point data storage unit, DM1… First region, DM2… Second region, DM3… Third region, DM… Data storage unit, EJ… Abnormality determination unit, FE… Frame extraction unit, FE1… Overall frame extraction unit, FE2… Head frame extraction unit, FL… Floor surface, FP… Frame data storage unit, FR1… Overall frame, FR2… Head frame, GG… Image, GP… Image data storage unit, HE… Person extraction unit, HE1… Whole body image extraction unit, HE2… Head image extraction unit, HP… Person image data storage unit, HU… Person, JM… Abnormality determination map, MK1~MK4… Marker, MKi1~MKi4… Marker image, OJ… Overlap degree determination unit, PE… Whole body image, PH… Head image, PH… Upper body image (head image, PM… Program memory unit, PP1… Reference point, PP2… Target point, SS… Imaging range (monitoring range), X1~X3… Value, Y1~Y3… Value

Claims

1. An abnormality detection device that detects an abnormal human posture based on the positional relationship between a reference point of a full-body image extracted for a person in an image and a target point of an upper-body image, The abnormality detection device determines the presence or absence of an abnormality based on a comparison of the magnitudes of the difference in one direction and the difference in the other direction between the reference point of the full-body image and the target point of the upper-body image on a plane coordinate.

2. The abnormality detection device according to claim 1, wherein the presence or absence of an abnormality is determined based on an abnormality determination map referred to for the difference in one direction and the difference in the other direction indicating the positional relationship between the reference point of the full-body image and the target point of the upper-body image.

3. The abnormality detection device according to claim 2, wherein the abnormality determination map varies according to the detection position of the person.

4. The abnormality detection device according to any one of claims 1 to 3, wherein the target point is specified from a head image.

5. The abnormality detection device according to any one of claims 1 to 4, wherein based on the degree of overlap between the area of the full-body image and the area of the upper-body image, it is determined whether the full-body image and the upper-body image are body images of the same person.

6. The abnormality detection device according to any one of claims 1 to 5, wherein the position in the depth direction is calculated based on a marker fixedly installed at a specific location within the imaging range targeted for acquiring the full-body image and the upper-body image.

7. The abnormality detection device according to any one of claims 1 to 6, wherein when the number of detection targets within the imaging range is equal to or less than a predetermined number, the abnormality detection process based on the full-body image and the upper-body image is started.

Images