Image recognition device, system, vehicle, program, and image recognition method

Abstract

To reduce the possibility of erroneously determining that a person has entered a specific area. [Solution] The image recognition device acquires an image 60 from an imaging device that captures an image 60 of a space 50 that includes a specific area, sets a frame 61 in the acquired image 60 that surrounds a person 70 in the space 50, identifies an area 63 corresponding to the person 70 at least within the frame 61 in the image 60, and has a control unit that outputs an area intrusion notification when it detects an overlap between the identified area 63 and the specific area in the image 60.

Description

[Technical Field]

[0001] The present disclosure relates to an image recognition device, a system, a vehicle, a program, and an image recognition method. [Background technology]

[0002] Patent Document 1 discloses a work vehicle that sets a boundary line indicating the boundary of a dangerous area in a photographed image of a work area, and identifies a person by surrounding it with a frame, and if it determines that part of the frame has crossed the boundary line into the dangerous area, stops the vehicle from carrying out work such as loading garbage or issues a warning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-088127 Summary of the Invention [Problem to be solved by the invention]

[0004] When capturing images from above using a fisheye lens, depending on the camera's installation location or type, the frame surrounding a person may occupy a large area in the image, and part of the frame may cross the boundary line into the danger area even though the person is not actually in the danger area. In such cases, conventional technology may erroneously determine that the person has entered the danger area.

[0005] An object of the present disclosure is to reduce the possibility of erroneously determining that a person has entered a specific area. [Means for solving the problem]

[0006] An image recognition device according to the present disclosure includes: Acquiring an image from an imaging device that captures an image of a space including a specific area; A frame is set to surround the person in the space in the acquired image; Identifying an area corresponding to the person within at least the frame of the image; When an overlap between the identified region and the specific area in the image is detected, an area intrusion notification is output. It has a control unit.

[0007] The image recognition method according to the present disclosure includes: An imaging device captures an image of a space including a specific area; an image recognition device setting a frame surrounding the person in the space in the image; The image recognition device identifies an area corresponding to the person within at least the frame of the image; When the image recognition device detects an overlap between the identified region and the specific area in the image, it outputs an area intrusion notification. Includes: [Effects of the Invention]

[0008] According to the present disclosure, it is possible to reduce the possibility of erroneously determining that a person has entered a specific area. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating a configuration of a vehicle according to an embodiment of the present disclosure. [Figure 2] FIG. 10 is a diagram showing an example in which a frame is set around a person in an image of a space including a specific area. [Figure 3] FIG. 3 is a diagram showing an example in which an area corresponding to a person is identified at least within the frame of the same image as in FIG. 2. [Figure 4] 1 is a block diagram illustrating a configuration of an image recognition device according to an embodiment of the present disclosure. [Figure 5] 1 is a flowchart illustrating an operation of an image recognition device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.

[0011] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0012] The configuration of a vehicle 10 according to this embodiment will be described with reference to FIG.

[0013] The vehicle 10 may be used for any purpose, but in this embodiment, it is used to transport passengers and is operated as, for example, a bus. The vehicle 10 may be any type of automobile, such as a gasoline vehicle, a diesel vehicle, a hydrogen vehicle, an HEV, a PHEV, a BEV, or an FCEV. "HEV" is an abbreviation for hybrid electric vehicle. "PHEV" is an abbreviation for plug-in hybrid electric vehicle. "BEV" is an abbreviation for battery electric vehicle. "FCEV" is an abbreviation for fuel cell electric vehicle. The vehicle 10 may be driven by a driver, or may be automated at any level. The level of automation may be, for example, any of levels 1 to 5 in the SAE classification. "SAE" is an abbreviation for Society of Automotive Engineers. The vehicle 10 may also be a dedicated MaaS vehicle. "MaaS" is an abbreviation for Mobility as a Service.

[0014] The vehicle 10 includes an in-vehicle system 11. The in-vehicle system 11 includes an image recognition device 20 and an imaging device 30. Although not essential, in this embodiment, the in-vehicle system 11 further includes an output device 40. The image recognition device 20 can communicate with the imaging device 30 and the output device 40 directly or via a network such as a LAN. "LAN" is an abbreviation for local area network.

[0015] The image recognition device 20 is a computer having an image recognition function. The image recognition device 20 may be installed at any location on the vehicle 10.

[0016] The imaging device 30 is, for example, an RGB camera or an infrared camera. In this embodiment, the imaging device 30 is a fisheye camera or a 360-degree camera. The imaging device 30 may be installed in any location where it can capture an image of the space 50. In this embodiment, the space 50 is the interior space of the vehicle 10, and the imaging device 30 is installed at a height equal to or higher than the top of the door 12 through which passengers get on and off. The height H from the floor of the vehicle 10 to the imaging device 30 is preferably a height that allows imaging from above the heads of standing passengers, and is, for example, 2.1 meters. The imaging device 30 may also be installed on a low ceiling inside the vehicle 10.

[0017] The output device 40 is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescent. If the output device 40 is a display, it may be installed in any location in the vehicle 10 that is visible to passengers. If the output device 40 is a speaker, it may be installed in any location in the vehicle 10 that can deliver sound to passengers.

[0018] The outline of this embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0019] The imaging device 30 captures an image 60 of a space 50. The space 50 includes a specific area 51. As shown in FIG. 2 , the image recognition device 20 sets a frame 61 surrounding a person 70 in the space 50 in the image 60. For example, a bounding box is set as the frame 61. As a specific example of setting a bounding box, a deep learning model such as YOLO or SSD may be used to identify the position of the person 70 in the image 60, and a rectangular frame defined by the coordinate points of the upper left and lower right of the person 70 may be drawn around the person 70 in the image 60. "YOLO" is an abbreviation for "you only look once." "SSD" is an abbreviation for "single shot multibox detector." The image recognition device 20 calculates the ratio of overlap 62 between the set frame 61 and the specific area 51 in the image 60. When the image recognition device 20 determines that the calculated ratio is less than the threshold, it identifies a region 63 corresponding to a person 70 within at least the frame 61 of the image 60, as shown in FIG. 3. The region 63 is identified, for example, by segmentation using deep learning. A specific example of segmentation is to use a deep learning model such as U-Net or DeepLab to predict to which class each pixel within at least the frame 61 of the image 60 belongs, and extract a group of pixels belonging to the "person" class as one segment. When the image recognition device 20 detects an overlap between the identified region 63 and the specific area 51 in the image 60, it outputs an area intrusion notification.

[0020] Depending on the installation location or type of the imaging device 30, such as when capturing an image from overhead using a fisheye lens, the area occupied by the frame 61 surrounding the person 70 in the image 60 may become large, resulting in the frame 61 overlapping the specific area 51 even when the person 70 is not present in the specific area 51. For example, when the image 60 is captured using a fisheye lens, the contour of the person 70 is distorted, which tends to make the frame 61 larger. In such a case, in this embodiment, if the ratio of overlap 62 is less than a threshold, a region 63 corresponding to the person 70 is identified by secondary processing, which is more accurate than the primary processing of setting the frame 61. Then, whether the person 70 has entered the specific area 51 is determined based on the overlap between the region 63 and the specific area 51. Therefore, this embodiment reduces the possibility of erroneously determining that the person 70 has entered the specific area 51. As a variation, the image recognition device 20 may identify the region 63 corresponding to the person 70 regardless of whether the ratio of overlap 62 is less than a threshold. In this modification, the image recognition device 20 does not need to calculate the ratio of the overlap 62.

[0021] In this embodiment, the region 63 is smaller than the frame 61. According to this embodiment, when the ratio of overlap 62 between the frame 61 and the specific area 51 is less than a threshold, an area smaller than the frame 61 is identified as the region 63, which makes it possible to prevent erroneous determination that the person 70 has entered the specific area 51 when the person 70 is not actually present in the specific area 51.

[0022] In this embodiment, the calculation time required to identify the region 63 is longer than the calculation time required to set the frame 61. According to this embodiment, when the ratio of overlap 62 between the frame 61 and the specific area 51 is equal to or greater than a threshold, it is not necessary to identify the region 63, and therefore the calculation time can be reduced.

[0023] In this embodiment, the computational cost required to identify the region 63 is greater than the computational cost required to set the frame 61. According to this embodiment, when the ratio of overlap 62 between the frame 61 and the specific area 51 is equal to or greater than a threshold, it is not necessary to identify the region 63, and therefore the computational cost can be reduced.

[0024] In this embodiment, when the image recognition device 20 determines that the ratio of overlap 62 between the frame 61 and the specific area 51 is equal to or greater than a threshold, it outputs an area intrusion notification without identifying the region 63. Therefore, according to this embodiment, when the ratio of overlap 62 between the frame 61 and the specific area 51 is equal to or greater than a threshold, it is possible to output an area intrusion notification earlier.

[0025] In this embodiment, when identifying the area 63, the image recognition device 20 sets the enlarged area 52 by enlarging the specific area 51 in the image 60, as shown in FIG. 3 . When the image recognition device 20 detects an overlap between the identified area 63 and the enlarged area 52 in the image 60, the image recognition device 20 outputs an area intrusion notification. In this embodiment, the specific area 51 is the area where the door 12 is located. For example, it is conceivable to include shoes 71 in the frame 61 but not in the area 63, and enlarge the enlarged area 52 by ankle height from the specific area 51. Alternatively, it is conceivable to include the lower body in the frame 61 but not in the area 63, and enlarge the enlarged area 52 by waist height from the specific area 51. According to these examples, by excluding parts that are difficult to detect, such as shoes 71, it is possible to prevent erroneous determination that the person 70 is not standing near the door 12 when in fact the person 70 is standing near the door 12. In this embodiment, a handrail 13 is provided around the door 12. For example, it is conceivable to set the specific area 51 so as not to include the handrail 13, with the frame 61 including the hand but the region 63 not including the handrail, and then expand the expanded area 52 to include the handrail 13. This example makes it possible to prevent the person 70 from being erroneously determined to be standing near the door 12, even though the person 70 is simply reaching out and grabbing the handrail 13 and is not actually standing near the door 12. The expanded area 52 may simply be expanded by a certain percentage from the specific area 51, or may be expanded in accordance with the position of the handrail 13.

[0026] The configuration of an image recognition device 20 according to this embodiment will be described with reference to FIG.

[0027] The image recognition device 20 includes a control unit 21, a storage unit 22, and a communication unit 23.

[0028] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. An example of the programmable circuit is an FPGA. "FPGA" is an abbreviation for field-programmable gate array. An example of the dedicated circuit is an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 21 executes processing related to the operation of the image recognition device 20 while controlling each part of the image recognition device 20.

[0029] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. Flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 22 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores information used in the operation of the image recognition device 20 and information obtained by the operation of the image recognition device 20.

[0030] The communication unit 23 includes at least one communication module. The communication module is, for example, an interface compatible with a wired LAN communication standard such as Ethernet (registered trademark), or an interface compatible with a wireless LAN communication standard such as IEEE802.11. "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. The communication module may be an interface compatible with other standards such as USB, HDMI (registered trademark), or Bluetooth (registered trademark). "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface. The communication unit 23 communicates with the imaging device 30 and the output device 40. The communication unit 23 may also communicate with the door 12. The communication unit 23 receives information used in the operation of the image recognition device 20 and transmits information obtained by the operation of the image recognition device 20.

[0031] The functions of the image recognition device 20 are realized by executing a program according to this embodiment on a processor serving as the control unit 21. That is, the functions of the image recognition device 20 are realized by software. The program causes a computer to execute the operations of the image recognition device 20, thereby causing the computer to function as the image recognition device 20. That is, the computer functions as the image recognition device 20 by executing the operations of the image recognition device 20 in accordance with the program.

[0032] The program can be stored on a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical disks, magneto-optical recording media, and ROMs. The program can be distributed by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs that store the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the storage of a server and transferring it from the server to another computer. The program can also be provided as a program product.

[0033] A computer temporarily stores a program stored on a portable medium or transferred from a server in its main storage device. The computer then reads the program stored in the main storage device using a processor and executes processing in accordance with the read program. The computer may also read the program directly from a portable medium and execute processing in accordance with the program. The computer may also execute processing in accordance with the received program each time a program is transferred from a server to the computer. Processing may also be executed through a so-called ASP-type service that achieves its functions by issuing execution instructions and obtaining results without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. A program is information used for processing by a computer and includes something equivalent to a program. For example, data that is not a direct instruction to a computer but has properties that specify computer processing falls under the category of "something equivalent to a program."

[0034] Some or all of the functions of the image recognition device 20 may be realized by a programmable circuit or a dedicated circuit as the control unit 21. In other words, some or all of the functions of the image recognition device 20 may be realized by hardware.

[0035] The operation of the image recognition device 20 according to this embodiment will be described with reference to Fig. 5. The operation described below corresponds to the image recognition method according to this embodiment. That is, the image recognition method according to this embodiment includes steps S1 to S8 shown in Fig. 5.

[0036] In S1, the control unit 21 acquires an image 60 of the space 50 including the specific area 51 from the imaging device 30. Specifically, the control unit 21 receives the image 60 from the imaging device 30 via the communication unit 23.

[0037] In S2, the control unit 21 sets a frame 61 surrounding the person 70 in the space 50 in the image 60 acquired in S1. The frame 61 can be set by a known method such as a bounding box.

[0038] In S3, the control unit 21 calculates the ratio of overlap 62 between the frame 61 set in S2 and the specific area 51 in the image 60 acquired in S1. The control unit 21 determines whether the calculated ratio is less than a threshold. The threshold is, for example, 1 / 9 of the area of ​​the frame 61, but may be adjusted depending on the area of ​​the specific area 51. Specifically, the threshold may be set to a smaller value as the area of ​​the specific area 51 becomes smaller. If it is determined that the ratio of overlap 62 between the frame 61 and the specific area 51 is less than the threshold, step S4 is performed. On the other hand, if it is determined that the ratio of overlap 62 between the frame 61 and the specific area 51 is equal to or greater than the threshold, step S8 is performed.

[0039] In S4, the control unit 21 identifies a region 63 corresponding to the person 70 within at least the frame 61 of the image 60 acquired in S1. Specifically, the control unit 21 identifies the region 63 by classifying a group of pixels within at least the frame 61 of the image 60 into the region 63 and one or more other types of regions. A known method such as segmentation can be used to classify the group of pixels.

[0040] In this embodiment, when specifying the region 63, the control unit 21 sets the enlarged area 52 by enlarging the specific area 51 in the image 60. Specifically, the control unit 21 sets the enlarged area 52 by enlarging the specific area 51 by a certain ratio. For example, the control unit 21 sets the enlarged area 52 by simply enlarging the specific area 51 by a certain percentage. Alternatively, the control unit 21 may set the enlarged area 52 by enlarging the specific area 51 by an amount corresponding to a height below a specific part of the person 70. In such an example, when setting the frame 61, the control unit 21 includes the specific part of the person 70 below the frame 61, and when specifying the region 63, the control unit 21 excludes the specific part of the person 70 below the frame 61 from the region 63. The specific part is, for example, the ankle or the waist. In other words, the control unit 21 may set the enlarged area 52 by including the shoes 71 in the frame 61 and excluding the shoes 71 from the region 63, and enlarging the specific area 51 by the height of the ankle. The control unit 21 may set the expanded area 52 by including the lower body in the frame 61 but not including the lower body in the region 63, and expanding the specific area 51 by waist height. Alternatively, the control unit 21 may set the expanded area 52 by expanding the specific area 51 in accordance with the position of the handrail 13. In such an example, the specific area 51 does not have to include the position of the handrail 13, and the expanded area 52 may include the position of the handrail 13. For example, the control unit 21 may set the specific area 51 to include the hands in the frame 61 but not include the hands in the region 63, and expand the specific area 51 so as to include the handrail 13, and then set the expanded area 52.

[0041] In S5, the control unit 21 determines whether or not the region 63 identified in S4 overlaps the specific area 51 in the image 60 acquired in S1. If no overlap between the region 63 and the specific area 51 is detected, step S6 is performed. On the other hand, if an overlap between the region 63 and the specific area 51 is detected, step S8 is performed.

[0042] In this embodiment, the control unit 21 determines whether or not there is an overlap between the region 63 in the image 60 and the enlarged area 52. If no overlap between the region 63 and the enlarged area 52 is detected, step S6 is performed. On the other hand, if an overlap between the region 63 and the enlarged area 52 is detected, step S8 is performed.

[0043] In S6, the control unit 21 determines whether or not the steps from S2 onward have been performed for all people in the space 50. If it is determined that the steps from S2 onward have not yet been performed for all people, the steps from S2 onward are performed again. On the other hand, if it is determined that the steps from S2 onward have already been performed for all people, step S7 is performed.

[0044] In S7, the control unit 21 permits the door 12 to be opened by outputting an area non-intrusion notification. Specifically, the control unit 21 permits the door 12 to be opened by transmitting a door control signal equivalent to the area non-intrusion notification via the communication unit 23. Alternatively, the control unit 21 may transmit a message equivalent to the area non-intrusion notification, such as "The door will open," via the communication unit 23, and cause the output device 40 to display the message on a screen or output the message as audio.

[0045] In S8, the control unit 21 prohibits the door 12 from being opened by outputting an area intrusion notification. Specifically, the control unit 21 prohibits the door 12 from being opened by transmitting a door control signal equivalent to the area intrusion notification via the communication unit 23. Alternatively, the control unit 21 may transmit a message equivalent to the area intrusion notification, such as "It is dangerous, please move away from the door," via the communication unit 23, and cause the output device 40 to display the message on a screen or output the message as audio.

[0046] In this embodiment, by performing the above-described operations, for example, using an imaging device 30 such as an RGB camera from a top-down position, it is possible to accurately determine whether a passenger on board the vehicle 10 is standing near the door 12 while holding onto the handrail 13, for example by sticking out only their arms, in a short time and at low cost. In this embodiment, the degree of overlap between the person detection rectangle and the target area is determined, and if the degree of overlap is low, the overlap between a more detailed segmentation region and the target area is determined. This makes it possible to perform high-precision area intrusion determination while reducing calculation time and calculation cost. According to this embodiment, even if it is difficult to install many devices on the ceiling of the vehicle 10, it is possible to improve determination accuracy without installing multiple cameras.

[0047] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagrams may be integrated, or one block may be divided. Two or more steps shown in the flowcharts may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure.

[0048] Some embodiments of the present disclosure will be described below as examples, however, it should be noted that the embodiments of the present disclosure are not limited to these examples. [Appendix 1] Acquiring an image from an imaging device that captures an image of a space including a specific area; A frame is set to surround the person in the space in the acquired image; Identifying an area corresponding to the person within at least the frame of the image; When an overlap between the identified region and the specific area in the image is detected, an area intrusion notification is output. An image recognition device having a control unit. [Appendix 2] 2. The image recognition device of claim 1, wherein the region is smaller than the frame. [Appendix 3] 3. The image recognition device according to claim 1, wherein the control unit identifies the region by classifying a group of pixels within at least the frame of the image into the region and one or more other types of regions. [Appendix 4] 4. The image recognition device according to claim 1, wherein the calculation time or calculation cost required to identify the region is greater than the calculation time or calculation cost required to set the frame. [Appendix 5] The control unit Calculating an overlap ratio between the set frame and the specific area in the image; If it is determined that the calculated ratio is less than a threshold value, a region corresponding to the person is identified within at least the frame of the image; 5. The image recognition device according to claim 1, wherein, when it is determined that the ratio is equal to or greater than the threshold, the area intrusion notification is output without identifying the area. [Appendix 6] The control unit When identifying the region, an enlarged area is set by enlarging the identified area in the image; 6. The image recognition device according to claim 1, wherein when an overlap between the region in the image and the enlarged area is detected, a notification of intrusion into the area is output. [Appendix 7] 7. The image recognition device according to claim 6, wherein the control unit sets the enlarged area by enlarging the specific area at a certain ratio. [Appendix 8] The control unit When setting the frame, include a specific part of the person and below in the frame, When identifying the region, excluding the specific part of the person and the part below it from the region; 7. The image recognition device of claim 6, wherein the enlarged area is set by enlarging the specific area by a height equivalent to or less than the specific part of the person. [Appendix 9] 9. The image recognition device according to claim 8, wherein the specific part is an ankle or a waist. [Appendix 10] the space being an interior space of a vehicle for transporting passengers; the specific area is an area where a door for the passengers to get on and off is located, 7. The image recognition device according to claim 6, wherein the control unit sets the enlarged area by enlarging the specific area according to the position of a handrail provided around the door. [Appendix 11] The specific area does not include the position of the handrail, 11. The image recognition device of claim 10, wherein the enlarged area includes the position of the handrail. [Appendix 12] the space being an interior space of a vehicle for transporting passengers; the specific area is an area where a door for the passengers to get on and off is located, 10. The image recognition device according to any one of Supplementary Note 1 to Supplementary Note 9, wherein the imaging device is installed at a height equal to or higher than the top end of the door. [Appendix 13] 13. The image recognition device according to claim 12, wherein the imaging device is a fisheye camera. [Appendix 14] the space being an interior space of a vehicle for transporting passengers; the specific area is an area where a door for the passengers to get on and off is located, 10. The image recognition device according to claim 1, wherein the control unit prohibits the door from being opened by outputting the area intrusion notification. [Appendix 15] An image recognition device according to any one of Supplementary Note 1 to Supplementary Note 14; the imaging device; A system comprising: [Appendix 16] 16. A vehicle comprising the system described in Appendix 15. [Appendix 17] A program that causes a computer to function as the image recognition device described in any one of Supplementary Note 1 to Supplementary Note 14. [Appendix 18] An imaging device captures an image of a space including a specific area; an image recognition device setting a frame surrounding the person in the space in the image; The image recognition device identifies an area corresponding to the person within at least the frame of the image; When the image recognition device detects an overlap between the identified region and the specific area in the image, it outputs an area intrusion notification. An image recognition method comprising: [Appendix 19] the image recognition device calculates an overlap ratio between the set frame and the specific area in the image; When the image recognition device determines that the calculated ratio is less than a threshold, it identifies an area corresponding to the person within at least the frame of the image; When the image recognition device determines that the ratio is equal to or greater than the threshold, the image recognition device outputs the area intrusion notification without specifying the area. 19. The image recognition method of claim 18, further comprising: [Appendix 20] The method further includes the step of, when the image recognition device identifies the region, setting an enlarged area by enlarging the identified area in the image; 20. The image recognition method according to claim 18, wherein the output of the area intrusion notification is performed when the image recognition device detects an overlap between the region in the image and the enlarged area. [Explanation of symbols]

[0049] 10 vehicles 11 In-vehicle systems 12 doors 13 Handrail 20 Image Recognition Device 21 Control section 22 Memory section 23 Communications Department 30 Imaging device 40 Output Devices 50 space 51 Specific Area 52 Expanded Area 60 images 61 slots 62 Overlap 63 areas 70 people 71 Shoes

Claims

1. Acquiring an image from an imaging device that captures an image of a space including a specific area; A frame is set to surround the person in the space in the acquired image; Identifying an area corresponding to the person within at least the frame of the image; When an overlap between the identified region and the specific area in the image is detected, an area intrusion notification is output. An image recognition device having a control unit.

2. The image recognition device according to claim 1 , wherein the region is smaller than the frame.

3. The image recognition device according to claim 1 , wherein the control unit identifies the region by classifying a group of pixels within at least the frame of the image into the region and one or more other types of regions.

4. The image recognition device according to claim 1 , wherein a calculation time or cost required to identify the region is greater than a calculation time or cost required to set the frame.

5. The control unit Calculating an overlap ratio between the set frame and the specific area in the image; If it is determined that the calculated ratio is less than a threshold value, a region corresponding to the person is identified within at least the frame of the image; The image recognition device according to claim 1 , wherein when it is determined that the ratio is equal to or greater than the threshold, the area intrusion notification is output without specifying the area.

6. The control unit When identifying the region, an enlarged area is set by enlarging the identified area in the image; The image recognition device according to claim 1 , wherein when an overlap between the region in the image and the enlarged area is detected, the area intrusion notification is output.

7. The image recognition device according to claim 6 , wherein the control unit sets the enlarged area by enlarging the specific area at a constant ratio.

8. The control unit When setting the frame, include a specific part of the person and below in the frame, When identifying the region, excluding the specific part of the person and the part below it from the region; The image recognition device according to claim 6 , wherein the enlarged area is set by enlarging the specific area by an amount corresponding to a height equal to or less than the specific part of the person.

9. The image recognition device according to claim 8 , wherein the specific part is an ankle or a waist.

10. the space being an interior space of a vehicle for transporting passengers; the specific area is an area where a door for the passengers to get on and off is located, The image recognition device according to claim 6 , wherein the control unit sets the enlarged area by enlarging the specific area in accordance with a position of a handrail provided around the door.

11. The specific area does not include the position of the handrail, The image recognition device according to claim 10 , wherein the enlarged area includes the position of the handrail.

12. the space being an interior space of a vehicle for transporting passengers; the specific area is an area where a door for the passengers to get on and off is located, The image recognition device according to claim 1 , wherein the imaging device is installed at a height equal to or higher than the upper end of the door.

13. The image recognition device according to claim 12 , wherein the imaging device is a fisheye camera.

14. the space being an interior space of a vehicle for transporting passengers; the specific area is an area where a door for the passengers to get on and off is located, The image recognition device according to claim 1 , wherein the control unit prohibits the door from being opened by outputting the area intrusion notification.

15. The image recognition device according to any one of claims 1 to 14; the imaging device; A system comprising:

16. A vehicle comprising the system of claim 15.

17. A program that causes a computer to function as the image recognition device according to any one of claims 1 to 14.

18. An imaging device captures an image of a space including a specific area; an image recognition device setting a frame surrounding the person in the space in the image; The image recognition device identifies an area corresponding to the person within at least the frame of the image; When the image recognition device detects an overlap between the identified region and the specific area in the image, it outputs an area intrusion notification. An image recognition method comprising:

19. the image recognition device calculates an overlap ratio between the set frame and the specific area in the image; When the image recognition device determines that the calculated ratio is less than a threshold, it identifies an area corresponding to the person within at least the frame of the image; When the image recognition device determines that the ratio is equal to or greater than the threshold, the image recognition device outputs the area intrusion notification without specifying the area. The image recognition method of claim 18, further comprising:

20. The method further includes the step of, when the image recognition device identifies the region, setting an enlarged area by enlarging the identified area in the image; The image recognition method according to claim 18 , wherein the output of the area intrusion notification is performed when the image recognition device detects an overlap between the region in the image and the enlarged area.

Images