Image processing device, image processing method, and program

Abstract

To provide an image processing device capable of detecting an intrusion of a person into a specific area with high accuracy even when an image is distorted.SOLUTION: An image processing device includes: detection means that detects a first person area indicating a position of a person in an image; acquisition means that acquires direction information indicating a predetermined direction in a plurality of areas in an image, body information related to a size of a body of the person, and area information indicating a range of a specific area in the image; specification means that specifies, on the basis of the first person area detected by the detection means and the direction information and the body information acquired by the acquisition means, a second person area indicating a position of a person in an image, the position being different from the position indicated by the first person area; and determination means that determines, on the basis of the second person area and the area information, whether a person intrudes the specific area.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a technology for detecting the intrusion of a person.

Background Art

[0002] In recent years, in imaging devices such as surveillance cameras, an intrusion detection system has been proposed that detects a person from an image and detects the intrusion of a person into a specific area.

[0003] In such an intrusion detection system, for example, when a predetermined person area (person position), such as the feet of a person, is included in a specific area, it is determined that a person has intruded. Therefore, if the person area is not detected appropriately, there is a risk that the intrusion may not be correctly determined. There are cases where a person detected using machine learning or the like is surrounded by a rectangle parallel to the vertical and horizontal sides of the image and displayed on the image. However, usually, since a surveillance camera images a person from above, a person located particularly far from the center of the field of view may be displayed obliquely on the image. For this reason, the rectangle surrounding the person may become wider than necessary in the horizontal direction, and there are cases where the intrusion of a person into a specific area cannot be correctly determined. Patent Document 1 describes a method of estimating the position of the feet of a person using the parameters of an imaging device based on the position of the head of the person detected from an image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] According to the method described in Patent Document 1, the position of the feet of an upright person in an image can be estimated. However, due to the distortion of the image caused by the influence of the lens of the imaging device, the person may be displayed tilted in different directions for each position of the person in the image. The method described in Patent Document 1 does not consider the influence of image distortion or the like. Therefore, even when the method described in Patent Document 1 is used in an intrusion detection system, it may not be possible to accurately estimate the person area (the position of the person's feet), and there may be a case where intrusion detection is not correctly performed. The present invention has been made in view of the above problems. The object is to enable accurate intrusion detection even when there is image distortion.

Means for Solving the Problems

[0006] In order to achieve the above object, an image processing apparatus according to one aspect of the present invention includes: detection means for detecting a first person area indicating the position of a person in an image; direction information indicating a predetermined direction in a plurality of areas in the image; body information regarding the size of a person's body; and area information indicating the range of the specific area in the image, acquisition means for acquiring the direction information, the body information, and the area information; specific means for specifying a second person area indicating a position different from the position indicated by the first person area, which is the position of the person in the image, based on the first person area detected by the detection means and the direction information and the body information acquired by the acquisition means; and determination means for determining intrusion of the person into the specific area based on the second person area and the area information.

Effects of the Invention

[0007] According to the present invention, accurate intrusion detection can be performed even when there is image distortion.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present invention will be described in detail based on the accompanying drawings. Note that the following embodiments do not limit the invention related to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are assigned reference numerals, and duplicate explanations are omitted.

[0010] <The First Embodiment> FIG. 1 is a diagram showing an example of the configuration of a system including an imaging device corresponding to Embodiment 1 of the invention. As shown in FIG. 1, the system 100 of Embodiment 1 is configured to include an imaging device 110, a client device 120, and a network 130. The configuration of the system 100 shown in FIG. 1 is merely an example, and other components may be further added. Hereinafter, each component will be described.

[0011] The imaging device 110 is an imaging device such as a network camera, which images the real space to generate an imaging image. Further, it determines the presence or absence of a change from the generated imaging image, and transmits the generated imaging image and the determination result described later to the client device 120 through the network 130. The imaging device 110 may be composed of not only one network camera but also a plurality of network cameras.

[0012] The client device 120 is a device that receives input from the user and outputs information to the user (for example, displays information). In the present embodiment, for example, the client device 120 is a computer such as a PC, and information input and output are performed by a browser or a native application installed on the computer. Note that in FIG. 1, the imaging device 110 and the client device 120 are shown as separate devices, but the configuration is not limited to this. For example, these devices may be integrated into one device. Further, for example, the client device 120 may be divided into an input device and an output device, or the imaging device 110 may be divided into an imaging device and an image processing device.

[0013] The network 130 enables the imaging device 110 and the client device 120 to communicate with each other It is a communication network that connects in a certain state. The network 130 is composed of a plurality of routers, switches, cables, etc. that satisfy a communication standard such as Ethernet (registered trademark). In this embodiment, the network 130 only needs to be able to communicate between the imaging device 110 and the client device 120, regardless of its communication standard, scale, and configuration. For example, the network 130 may be composed of the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), etc.

[0014] Next, with reference to FIG. 2, the hardware configuration of each device in the image processing system 100 will be described. FIG. 2(A) is a diagram showing the hardware configuration of the imaging device 110. The imaging device 110 includes a CPU 201, a RAM 202, a ROM 203, an HDD 204, an input unit 205, an output unit 206, a network I / F unit 207, a drive unit 208, an imaging unit 209, and a bus 210.

[0015] The CPU 201 reads out the control computer programs stored in the ROM 203 and the HDD 204 and loads them into the RAM 202 to execute various control processes. The RAM 202 is used as a storage area for temporarily storing the programs executed by the CPU 201 and work memories, etc. The ROM 203 stores data related to the settings of the imaging device 110 and boot programs, etc. The HDD 204 stores image data, setting parameters, and various programs, etc. It is assumed that the HDD 204 can receive data input from an external device via the network I / F unit 207.

[0016] The input unit 205 is an input device such as a button and a touch panel, for example. The output unit 206 is a device that outputs information, such as a display device such as a liquid crystal display and an audio output device such as a speaker. The input unit 205 and the output unit 206 are also connected to the bus 210. Note that the imaging device 110 may be configured not to have at least either the input unit 205 or the output unit 206. The network I / F unit 207 transmits and receives various information between the imaging device 110 and other devices via the Internet. Data received from other devices via the network I / F unit 207 is transmitted and received to and from the CPU 204, the RAM 205, the ROM 206, etc. via the bus 210.

[0017] The drive unit 208 drives the imaging unit 209 to control the posture (imaging direction), the angle of view, etc. of the imaging unit 209. The control target by the drive unit 208 is not limited to a specific target, and may be the posture and the angle of view of the imaging unit 209, or only either one of them, or other targets (for example, the position of the imaging unit 209). Also, at least any one of the position, the posture, and the angle of view of the imaging unit 209 may be fixed. Further, when the position, the posture, and the angle of view of the imaging unit 209 are fixed and there is no need to control them, the imaging device 110 may be configured not to have the drive unit 208. The imaging unit 209 has an image sensor and an optical system, and forms an image of a subject on the image sensor with the intersection of the optical axis of the optical system and the image sensor as the imaging center. Examples of the image sensor include a CMOS (Complementary Metal - Oxide Semiconductor) and a CCD (Charged Coupled Device). The imaging unit 209 is controlled by the drive unit 208 with respect to its position, posture, angle of view, etc.

[0018] FIG. 2(B) is a diagram showing the hardware configuration of the client device 120. The client device 120 includes a CPU 211, a RAM 212, a ROM 213, an HDD 214, an input unit 215, an output unit 216, a network I / F unit 217, and a bus 218. Since these functions are the same as those of the CPU 201, RAM 202, ROM 203, HDD 204, input unit 205, output unit 206, network I / F unit 207, and bus 210 in the imaging device 110, the description thereof will be omitted.

[0019] Next, with reference to FIG. 3, a functional configuration example of the imaging device 110 and the client device 120 will be described. As shown in FIG. 3(A), the imaging device 110 includes an imaging unit 311, an acquisition unit 312, a storage unit 313, a detection unit 314, a specifying unit 315, a determination unit 316, a notification unit 317, and a setting unit 318.

[0020] The imaging unit 311 controls the imaging operation of the imaging unit 215 and captures images at regular time intervals. The acquisition unit 312 acquires the captured images. In addition, based on an instruction from the client device 120, the acquisition unit 312 acquires direction information, body information, and area information from the client device 120 and stores them in the storage unit 313. Further, when the acquisition unit 312 receives a transmission instruction from the client device 120, it acquires the direction information, body information, and area information from the storage unit 313 and transmits them to the client device.

[0021] The direction information is information indicating a predetermined direction in a plurality of regions within an image. In the present embodiment, the predetermined direction is assumed to be the direction indicating the direction of gravity within the image, but it is not limited to this. For example, the direction from the head to the feet of a standing person or the direction of the body axis toward the feet of a standing person may be set as the predetermined direction. Specifically, the body information is assumed to be information indicating the ratio of the sizes of a person's body. As an example, the aspect ratio of a person's body or the ratio of the size of a person's body to the head can be used as the body information. The region information is information indicating the range of a specific region in the image. The specific region is a region that is the target of determination of the intrusion of a person. The specific region can be expressed by an arbitrary polygon in the image, for example, but it is not limited to this. For example, the range may be expressed by a mask image in which the inside of a rectangle or polygon is filled.

[0022] The storage unit 313 stores the direction information, body information, and region information acquired by the acquisition unit 312. The imaging device 110 sends an acquisition instruction for causing the direction information, body information, and region information to be transmitted to the imaging device 110 to the client device 120. The specifying unit 315 specifies a second person region indicating the position of a person in the image based on the first person region detected by the detecting unit 314, the body information, and the direction information acquired from the storage unit 313.

[0023] Next, with reference to FIG. 4, an example of the processing of the detection unit 314 and the specification unit 315 will be described. Hereinafter, two examples of processing, namely, the processing shown in FIG. 4(A) and the processing shown in FIG. 4(B), will be described.

[0024] First, with reference to FIG. 4(A), an example of the processing performed by the detection unit 314 and the specification unit 315 will be described. In FIG. 4(A), in an image of a person being imaged, the direction of gravity 411, which is a predetermined direction determined for each of the 12 regions divided within the image, is indicated by a white arrow. The direction information 410, which is the direction of gravity 411 for each region, is preset and stored in the storage unit 313.

[0025] The detection unit 314 detects a first person area 412 indicating the position of a person in the image based on the image acquired from the imaging unit 311 or the storage unit 313. In the first embodiment, the detection unit 314 detects the position of a first rectangle that surrounds the entire body of the person and is the first person area 412. Each side of the first rectangle, both vertically and horizontally, is parallel to each side of the image. For person detection, techniques such as machine learning or pattern matching may be used, for example.

[0026] The specifying unit 315 specifies the average in a predetermined direction within the first rectangle from the direction information 410 output from the storage unit 313. In this case, the specifying unit 315 weights and averages the gravitational direction, which is a predetermined direction defined for each area within the first rectangle, based on the area ratio of the overlapping area between the first rectangle and each area. Also, the specifying unit 315 acquires the "aspect ratio of a person's body", which is body information, from the storage unit 313. The specifying unit 315 specifies the position of a second rectangle, which is a second person area where at least one end of the lower side is inscribed in the first rectangle, based on the average in the predetermined direction, which is the gravitational direction within the first rectangle, and the aspect ratio of the body, which is the body information. Here, the lower side refers to the shorter side among the shorter sides of the second rectangle where the position of the midpoint of the side is located below in the image. In FIG. 4(A), the second person area 414 is the second rectangle represented by the solid line and is a rectangle where at least one end of the lower side is inscribed in the first rectangle, but this is not the only case. For example, the second person area may be an arbitrary polygon where at least one end of the lower side is inscribed in the first rectangle.

[0027] Next, with reference to FIG. 4(B), an example of the processing performed by the detection unit 314 and the specifying unit 315 will be described. Similar to FIG. 4(A) in FIG. 4(B), the gravitational direction 411, which is a predetermined direction defined for each of the 12 areas divided within the image, is indicated by white arrows. Also, the direction information 410, which is the gravitational direction 411 of each area, is preset and stored in the storage unit 313.

[0028] The detection unit 314 detects a first person area 421 indicating the position of the head of a person in the image based on the image output from the imaging unit 311 or the storage unit 313. In the example shown in FIG. 4(B), the detection unit 314 detects the position of the first rectangle that is the first person area and surrounds the head of the person. Each of the vertical and horizontal sides of the first rectangle is parallel to the vertical and horizontal sides of the image. For the detection of the head of the person, for example, techniques such as machine learning or pattern matching may be used.

[0029] The specifying unit 315 acquires the direction information 410 with a predetermined direction in the image from the storage unit 313. The specifying unit 315 acquires the "ratio of the size of a person's body to the size of the head", which is body information, from the storage unit 313. The specifying unit 315 specifies a reference length obtained by subtracting half of the length of the head from the length of the body based on the ratio of the size of the body to the size of the head and the size of the first rectangle. The specifying unit 315 extends a line along each of the predetermined directions starting from the position of the center of the first rectangle, and based on the position of the end point where the total length of the extended line is equal to the reference length and the position of the first rectangle, specifies the position of the second rectangle that is the second person area. Specifically, the specifying unit 315 extends a straight line along a predetermined direction corresponding to the area to which the position of the center of the first rectangle belongs, and extends the straight line in the corresponding predetermined direction every time the area to which the tip of the straight line belongs changes. The specifying unit 315 extends the line until the position of the end point where the total length of the extended line is equal to the reference length. Next, the specifying unit 315 specifies a rectangle surrounding the first rectangle, which has a long side parallel to the straight line connecting the start point and the end point and a short side whose one short side has the end point as the midpoint, as the second rectangle. In FIG. 4(B), the second person area 424 is shown as the second rectangle represented by a solid line, but this is not the only case. For example, the second person area 424 may be an arbitrary polygon surrounding at least the first person area and the end point.

[0030] As described above, two examples of the processing performed by the detection unit 314 and the specifying unit 315 have been described with reference to FIG. 4.

[0031] Returning to Fig. 3(A), the determination unit 316 determines whether a person has entered a specific area based on the second person area and the area information. The foot position of the person used for the determination is the lower side of the short sides of the second rectangle indicated by the second person area, where the midpoint is located below the image. As an example, in the image, when at least a part of the lower side is included in the specific area, the determination unit 316 determines that there is "entry" of the person into the specific area, and determines "no entry" in other cases. However, the determination method performed by the determination unit 316 is not limited to this. For example, the determination unit 316 may determine that there is "entry" when the entire area of the rectangle indicated by the second person area overlaps with the specific area. Also, the determination unit 316 may determine that there is "entry" when the boundary of the specific area intersects with the side indicating the head position, where the head position is the side among the four sides of the second rectangle that is the second person area and whose midpoint is located highest in the image. After the determination, the determination unit 316 outputs the second person area and the determination result to the notification unit 317.

[0032] The notification unit 317 transmits the second person area and the determination result output from the determination unit 316 to the client device 120. In the first embodiment, the notification unit 317 is assumed to transmit when the determination result is "entry", but this is not limited to this. For example, the notification unit 317 may transmit when the state of "entry" continues for a certain period of time or more, or may aggregate the determination results for a certain period of time or more and transmit based on the aggregated result.

[0033] The setting unit 318 sets the gravity direction in the image based on the direction information acquired from the client device 120. Further, the setting unit 316 can also set, by interpolation through calculation, the gravity direction in each region within the image based on the direction information. For example, the setting unit 318 can perform interpolation and setting of the predetermined direction in the second region sandwiched between a plurality of first regions each having a corresponding predetermined direction, based on the predetermined directions corresponding to each of the plurality of first regions. In addition to simple averaging, existing interpolation methods such as performing polynomial approximation for each of the predetermined directions corresponding to a plurality of regions arranged in the horizontal or vertical direction of the image may be applied for the interpolation. The setting unit 318 outputs the interpolated direction information to the storage unit 313. Note that by reducing the region for setting the direction information according to region information such as a specific region or the boundary portion of a specific region, it becomes possible to improve the accuracy of intrusion detection. Also, when the setting unit 318 acquires an instruction to change the direction information from the client device 120, it can change the set gravity direction based on the change instruction.

[0034] Next, a functional configuration example of the client device 120 corresponding to the present embodiment will be described using the block diagram of FIG. 3(B). Each functional unit shown in FIG. 3(B) may be implemented as hardware or as software (computer program).

[0035] The communication unit 321 is connected to the network 130, for example, and executes communication with an external device such as the imaging device 110 via the network 130. Note that this is merely an example. For example, the communication unit 321 may be configured to establish a direct connection with the imaging device 110 and communicate with the imaging device 110 without going through the network 130 or other devices. The communication unit 321 transmits an acquisition instruction to the imaging device 110 as needed at any time, and outputs the captured image, region information, body information, and direction information received from the imaging device 110 to the display unit 322. Further, when the communication unit 321 receives any one of a region information change instruction, a body information change instruction, and a direction information change instruction from the operation unit 323, the communication unit 321 transmits the received instruction to the imaging device 110. Furthermore, when the communication unit 321 receives either the second person region or the determination result from the imaging device 110, the communication unit 321 outputs the received second person region and determination result to the display unit 322.

[0036] The display unit 322 presents information to the user via, for example, a display. In the present embodiment, information is presented to the user by displaying the result rendered by the browser on the display. Note that information may be presented by means other than screen display such as sound or vibration. The display unit 322 displays the captured image and the specific region received from the communication unit 321. The display unit 322 also displays the second person region and the determination result received from the communication unit 321.

[0037] The operation unit 323 receives an operation from the user. In the present embodiment, the operation unit 323 is a mouse or a keyboard, and it is assumed that the user operates these to input a user operation to the browser. However, it is not limited to this. The operation unit 323 may be any device capable of detecting the intention of another user, such as a touch panel or a microphone, for example. When any one of a region information change instruction, a body information change instruction, and a direction information change instruction is input from the user, the operation unit 323 outputs the input instruction to the communication unit 321.

[0038] Next, with reference to FIG. 5, the display screen presented by the display unit 322 to the user will be described. For example, a user interface 500 is displayed on this display screen. FIG. 5(A) is an example of a result display screen, on which a determination result 511, a person area 512, a specific area 513, an area information setting button 514, a body information setting button 515, and a direction information setting button 516 are displayed. The determination result 511 is displayed according to the determination result received from the imaging device 110. In the present embodiment, "Intrusion detected" and "No intrusion" are displayed in characters.

[0039] The person area 512 is the second person area received from the imaging device 110 and is shown as a second rectangle surrounding the person. The specific area 513 is displayed as the range of a specific area on the image set by the user. The area information setting button 514 is a button for transitioning to the area information setting screen 520. The body information setting button 515 is a button for transitioning to the body information setting screen 530. The direction information setting button 516 is a button for transitioning to the direction information setting screen 540.

[0040] FIG. 5(B) is an example of the area information setting screen 520, on which a specific area 521, an edit button 522, a delete button 523, an area list 524, and a completion button 525 are displayed. The specific area 521 is the area to be the target of the determination of the intrusion of a person and is set by the user. The specific area 521 can be edited by moving the positions of the vertices of the polygon surrounding the specific area, but this is not the only way. For example, the specific area 521 may be editable by inputting numerical values for the positions. The edit button 522 is a button for editing the selected specific area. The delete button 523 is a button for deleting the selected specific area. The area list 524 displays the area information set by the user in a list. In this embodiment, one of the areas in the list can be selected, and the selected area can be edited and deleted by the edit button 522 and the delete button 523. The completion button 525 is a button for transitioning to the result display screen when the user has completed setting the area information. When the setting of the area information is completed, the client device 120 transmits an area information change instruction including the area information in the area list 524 to the imaging device 110.

[0041] FIG. 5(C) is an example of the body information setting screen 530, in which a head-to-body ratio setting bar 531, a head-to-body ratio 532, and a completion button 533 are displayed. The head-to-body ratio setting bar 531 is used when the user sets the head-to-body ratio. The head-to-body ratio setting bar 531 is composed of an adjustment bar extending from a circle representing the head and a circle representing the body. On the body information setting screen 530, the user can set the size of the head by the size of the circle, and can set the ratio of the length of the body to the size of the head by the length of the adjustment bar. Also, on the body information setting screen 530, the user can directly input and set the numerical value of the head-to-body ratio 532. The completion button 533 is a button that causes a transition to the result display screen when the user completes the body information setting. When the body information setting is completed, the client device 120 transmits a body information change instruction including the set head-to-body ratio to the imaging device 110.

[0042] FIG. 5(D) is an example of the direction information setting screen 540, on which a gravity direction 541, an edit button 542, a delete button 543, a direction information list 544, and a completion button 545 are displayed. The gravity direction 541 is set by the user. Although it is assumed that the gravity direction 541 can be edited by moving the position and direction, this is not the only case. For example, the gravity direction 541 may be editable by inputting the positions of the start point and end point of the direction. The edit button 542 is a button for editing the selected direction information. The delete button 543 is a button for deleting the selected direction information. The direction information list 544 displays the direction information set by the user in a list. In this embodiment, one of the direction information in the list can be selected, and the selected direction information can be edited or deleted by the edit button 522 and the delete button 523. The completion button 545 is a button for completing the direction information setting and transitioning to the result display screen. When the direction information setting is completed, the client device 120 transmits a direction information change instruction including the direction information in the direction information list 544 to the imaging device 110. Also, when the area for setting the direction information is set to be small at a specific area in the image or at the boundary of the specific area, the density of the direction information increases and the accuracy of the determination improves. In this case, the direction information setting screen may be displayed after the area information setting is completed. Also, it may be possible to prompt the user to display more areas for setting direction information at each specific area or at the boundary of each specific area than in other parts.

[0043] Next, with reference to FIG. 6, the flow of the determination process executed in the system of this embodiment will be described. In this process, the imaging device 110 acquires a captured image and obtains a determination result from the acquired captured image. Then, the client device 120 presents the determination result received via the imaging device 110 to the user.

[0044] In S601, the imaging unit 311 of the imaging device 110 captures an image. The captured image is output to the detection unit 314 via the acquisition unit 312 and the storage unit 313, or directly. In S602, the detection unit 314 of the imaging device 110 detects a first person area from the image captured by the imaging unit 311. In S603, the acquisition unit 312 of the imaging device 110 acquires from the storage unit 313 body information regarding the size of a person's body and direction information indicating a respective predetermined direction for a plurality of areas in the image. In S604, the specifying unit 315 of the imaging device 110 specifies a second person area from the first person area obtained in S602, the body information, and the direction information acquired in S603. In S605, the acquisition unit 312 of the imaging device 110 acquires from the storage unit 313 area information indicating the range of the specified area. In S606, the determination unit 316 of the imaging device 110 determines whether a person has entered the specified area based on the second person area specified in S604 and the area information acquired in S605. In S607, the notification unit 317 of the imaging device 110 notifies the client device 120 of the determination result determined as the second person area according to the determination result acquired from the determination unit 316. The display unit 322 of the client device 120 displays the second person area and the determination result received from the imaging device 110 via the communication unit 321 to the user.

[0045] The intrusion detection performed by the system in the present embodiment has been described above. According to the method of the present embodiment, even when a person is displayed tilted in different directions for each position in the image due to image distortion, intrusion detection can be accurately performed.

[0046] Note that part or all of the storage unit 313, the detection unit 314, the specifying unit 315, the determination unit 316, the notification unit 317, and the setting unit 318 may be executed by a device different from the imaging device 110. For example, part or all of the storage unit 313, the detection unit 314, the specifying unit 315, the determination unit 316, the notification unit 317, and the setting unit 318 may be executed by the client device 120. In this case, part or all of S602 to S607 can also be performed by the client device 120. Further, the determination result may be represented by a numerical value of three levels or more.

[0047] Further, for example, the determination result may be in three stages: presence of stay, presence of intrusion, and absence of intrusion. In this case, the duration of the determination state of presence of intrusion may be obtained, and if the duration is equal to or longer than a certain time, it may be notified as presence of stay.

[0048] Further, the setting unit 318 may use the distortion information of the lens distortion of the imaging unit for setting the direction information. In this case, the distortion information, which is the information of the lens distortion of the imaging unit, is obtained in advance using a calibration board or the like, and stored in the main storage device 212. Next, at least one or more acquired predetermined directions are converted based on the distortion information of the imaging unit, and the result of inverse conversion of the converted predetermined directions at the positions on all the images is set as the direction information. That is, the predetermined direction in each area in the image is obtained based on at least one or more predetermined directions and the distortion information of the imaging means, and used as the direction information.

[0049] Further, the direction information setting screen 540 may be provided with buttons for setting direction information, and the direction information may not be set on the screen. In this case, the user sets the direction information by moving the range of the captured image while keeping a rod with an image marker recognizable up and down standing vertically. The recognized marker positions are set as the start point and the end point in the gravity direction.

[0050] <Second Embodiment> In the second embodiment, an image processing apparatus that acquires an image from an imaging device and performs intrusion detection will be described. Here, the description will focus on the differences from the first embodiment, and the same components will be denoted by the same reference numerals and the description thereof will be omitted. FIG. 7 is a diagram showing a system 700. The system 700 includes an imaging device 150, a client device 120, and an image processing device 140. The hardware configuration of the imaging device 150 is the same as that in FIG. 2(A). The hardware configuration and functional configuration of the client device 120 are the same as those in the first embodiment. Also, the hardware configuration of the image processing device 140 is assumed to be the same as that in FIG. 2(B).

[0051] FIG. 8 is a diagram for explaining the functional configuration of the image processing apparatus 140. In the first embodiment, the imaging apparatus 110 executed the processes from the acquisition of the captured image to the intrusion determination. However, in this embodiment, the difference is that the image processing apparatus 140 executes the intrusion determination process. The acquisition unit 312 acquires a captured image from the imaging apparatus 150. Also, the acquisition unit 312 acquires various instructions from the client device 120. In addition, it is assumed that the processes performed by each processing unit included in the image processing apparatus 140 are the same as those in the first embodiment.

[0052] FIG. 9 is a diagram for explaining the processes performed by the image processing apparatus 140. The difference from FIG. 6 is S901. In S901, the image processing apparatus 140 acquires an image from the imaging apparatus 150.

[0053] As described above, the form in which intrusion detection is performed by another image processing apparatus 140 connected to the imaging apparatus 150 has been explained. Note that, in the above description, the imaging apparatus 150 has been described as having only an imaging function for acquiring a captured image, but it is not limited to this. For example, the imaging apparatus 150 may have a detection unit 314, and a configuration may be such that a person is detected by the imaging apparatus 150 and the detection result is transmitted to the image processing apparatus 140. In this way, any processing unit among the processing units included in the image processing apparatus 140 may be included in the imaging apparatus 150.

[0054] (Other Embodiments) In the first embodiment, a predetermined direction was set as the gravitational direction 411. However, the predetermined direction of the present invention is not limited to the "gravitational direction", and may be, for example, the direction from the head to the feet of a standing person, or the direction of the body axis toward the feet. Also, doors, windows, window frames, utility poles, traffic signals, and columns of road signs can be used as the predetermined direction after confirming the direction.

[0055] The present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that implements one or more functions. The invention is not limited to the above-described embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Accordingly, the claims are appended to disclose the scope of the invention.

[0056] The disclosure of this embodiment includes the following image processing apparatus, method, and program. (Configuration 1) Detection means for detecting a first person region indicating the position of a person in the image, Acquisition means for acquiring direction information indicating a predetermined direction in a plurality of regions in the image, body information regarding the size of a person's body, and region information indicating the range of the specific region in the image, Based on the first person region detected by the detection means, the direction information and the body information acquired by the acquisition means, specific means for specifying a second person region indicating a position in the image that is different from the position indicated by the first person region, An image processing apparatus, comprising: determination means for determining an intrusion of the person into the specific region based on the second person region and the region information. (Configuration 2) The image processing apparatus further includes a setting unit, The setting unit sets, by interpolation, the predetermined direction in a second region sandwiched between a plurality of first regions each having a corresponding predetermined direction, based on the predetermined directions corresponding to the plurality of first regions. The image processing apparatus according to Configuration 1, characterized in that. (Configuration 3) The detection means detects the position of a first rectangle that surrounds the entire body of the person and is the first person region. The image processing apparatus according to Configuration 1 or Configuration 2, characterized in that. (Configuration 4) The specific means specifies the position of a second rectangle, which is a second person area with at least one end of the lower side inscribed in the first rectangle, based on the average in the predetermined direction within the first rectangle and the aspect ratio of the body, which is the body information, in the image processing apparatus according to Configuration 3. (Configuration 5) The determination means determines that there is an intrusion of the person into the specific area when at least a part of the lower side is included in the specific area in the image, in the image processing apparatus according to Configuration 4. (Configuration 6) The detection means detects the position of a first rectangle that is the first person area and surrounds the head of the person, in the image processing apparatus according to Configuration 1 or Configuration 2. (Configuration 7) The specific means specifies a reference length obtained by subtracting half of the length of the head from the length of the body based on the ratio of the size of the body and the head, which is the body information, and the size of the first rectangle. The specific means specifies the position of a second rectangle, which is the second person area, based on the position of the end point where the total length of the lines extended along each of the predetermined directions from the position of the center of the first rectangle is equal to the reference length and the position of the first rectangle, in the image processing apparatus according to Configuration 6. (Configuration 8) The specific means has a long side parallel to the straight line connecting the start point and the end point and a short side that is the lower side with one end being the midpoint of the end point, and specifies a rectangle surrounding the first rectangle as the second rectangle, in the image processing apparatus according to Configuration 7. (Configuration 9) The determination means determines that there is an intrusion of the person into the specific area when at least a part of the lower side is included in the specific area in the image, in the image processing apparatus according to Configuration 8. (Configuration 10) Obtaining the predetermined direction in each region of the image based on at least one of the predetermined directions and the distortion information of the imaging means, and using the obtained direction as the direction information, the image processing apparatus according to any one of Configurations 1 to 9. (Configuration 11) An image processing method for determining the intrusion of a person into a specific region, comprising: A first acquisition step of acquiring an image; A detection step of detecting a first person region indicating the position of a person in the image; A second acquisition step of acquiring direction information indicating a predetermined direction for a plurality of regions in the image and body information related to the size of a person's body; A specifying step of specifying a second person region indicating the position of the person in the image based on the first person region, the direction information, and the body information; A third acquisition step of acquiring region information indicating the range of the specific region in the image; A determination step of determining the intrusion of the person into the specific region based on the second person region and the region information; A notification step of notifying the second person region and the determined result, the image processing method being characterized by comprising the above steps. (Configuration 12) A program for controlling each means of the image processing apparatus according to any one of Configurations 1 to 10 by a computer.

Explanation of Reference Numerals

[0057] 100: System, 110: Imaging Device, 120: Client Device, 130: Imaging Device, 140: Image Processing Device, 150: Imaging Device, 311: Imaging Unit, 312: Acquisition Unit, 313: Storage Unit, 314: Detection Unit, 315: Specification Unit, 316: Determination Unit, 317: Notification Unit, 318: Setting Unit, 322: Display Unit, 323: Operation Unit, 700: System

Claims

1. detection means for detecting a first person area indicating the position of a person in the image; acquisition means for acquiring direction information indicating a predetermined direction in a plurality of areas within the image, body information regarding the size of a person's body, and area information indicating the range of the specific area in the image; specification means for specifying a second person area indicating a position in the image that is different from the position indicated by the first person area based on the first person area detected by the detection means, the direction information and the body information acquired by the acquisition means; an image processing apparatus comprising determination means for determining an intrusion of the person into the specific area based on the second person area and the area information.

2. The image processing apparatus further comprises a setting unit, wherein the setting unit sets, by interpolation, the predetermined direction in a second area sandwiched between a plurality of first areas each having a corresponding predetermined direction, based on the predetermined direction corresponding to each of the plurality of first areas. The image processing apparatus according to claim 1.

3. The detection means detects the position of a first rectangle that surrounds the entire body of the person and is the first person area. The image processing apparatus according to claim 1.

4. The specification means specifies the position of a second rectangle that is the second person area and at least one end of the lower side of which is inscribed in the first rectangle, based on the average of the predetermined directions within the first rectangle and the aspect ratio of the body, which is the body information. The image processing apparatus according to claim 3.

5. The determination means determines that there is an intrusion of the person into the specific area when at least a part of the lower side is included in the specific area in the image. The image processing apparatus according to claim 4.

6. The detection means detects the position of a first rectangle that surrounds the head of the person and is the first person area. The image processing apparatus according to claim 1.

7. The specification means specifies a reference length obtained by subtracting half of the length of the head from the length of the body, based on the ratio of the size of the body and the head, which is the body information, and the size of the first rectangle. An image processing apparatus according to claim 6, characterized in that the position of the second rectangular region as the second person region is specified based on the position of the first rectangular region and the position of an end point where lines are extended along each of the predetermined directions starting from the position of the center of the first rectangle and the sum of the lengths of the extended lines is equal to the reference length.

8. The specifying means has a long side parallel to the straight line connecting the starting point and the end point, and a short side whose one short side is the lower side having the end point as the midpoint, and specifies a rectangle surrounding the first rectangle as the second rectangle. An image processing apparatus according to claim 7.

9. The determination means according to claim 8, wherein in the image, when at least a part of the lower side is included in the specific region, it is determined that there is an intrusion of the person into the specific region.

10. An image processing apparatus according to any one of claims 1 to 9, characterized in that the predetermined direction in each region in the image is obtained based on at least one or more of the predetermined directions and distortion information of the imaging means, and used as the direction information.

11. An image processing method for determining intrusion of a person into a specific region, comprising: a first acquisition step of acquiring an image; a detection step of detecting a first person region indicating the position of a person in the image; a second acquisition step of acquiring direction information indicating a predetermined direction and body information regarding the size of a person's body for a plurality of regions in the image; a specifying step of specifying a second person region indicating the position of the person in the image based on the first person region, the direction information, and the body information; a third acquisition step of acquiring region information indicating the range of the specific region in the image; a determination step of determining intrusion of the person into the specific region based on the second person region and the region information; and a notification step of notifying the second person region and the determined result.

12. A program for controlling each means of the image processing apparatus according to any one of claims 1 to 9 by a computer.

Images