Image recognition apparatus, system, vehicle, non-transitory computer readable medium and image recognition method

Abstract

The present disclosure relates to an image recognition device, system, vehicle, non-transitory computer readable medium, and image recognition method, which reduce the possibility of erroneously determining that a person intrudes into a specific area. An image recognition device is provided with a control unit that acquires an image (60) from an image capturing device that captures an image (60) of a space (50) including a specific region, sets a frame (61) surrounding a person (70) in the space (50) in the acquired image (60), specifies a field (63) corresponding to the person (70) in at least the frame (61) of the image (60), and recognizes the person (70) in the field (63). If an overlapping portion of the determined field (63) and the specific region in the image (60) is detected, an intrusion region notification is output.

Description

Technical Field

[0001] This disclosure relates to an image recognition device, system, vehicle, program, and image recognition method. Background Technology

[0002] Patent document 1 discloses a work vehicle that, in the image taken in the work area, sets a boundary line indicating the boundary of the danger zone, and surrounds and defines the image of a person with a frame. If it is determined that part of the frame crosses the boundary line to the danger zone side, the vehicle stops the execution of garbage loading and other operations or issues a warning.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2022-088127 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] When using a fisheye lens to film overhead, depending on the camera's location and type, the area occupied by the frame surrounding a person in the image may increase, even if the person is not actually in the danger zone. Part of the frame may still extend beyond the boundary line into the danger zone. In such cases, existing technology may misjudge a person as having intruded into the danger zone.

[0008] The purpose of this disclosure is to reduce the possibility of misjudging a person as having intruded into a specific area.

[0009] Technical solutions for solving the problem

[0010] The image recognition device disclosed herein includes a control unit.

[0011] The control unit acquires the image from a camera device that captures an image of a space containing a specific area, sets a frame surrounding a person in the space in the acquired image, determines an area corresponding to the person in at least the frame in the image, and outputs an intrusion area notification if an overlap between the determined area in the image and the specific area is detected.

[0012] The image recognition method disclosed herein includes:

[0013] The camera device captures images of a space containing a specific area;

[0014] The image recognition device sets a frame around a person in the space within the image;

[0015] The image recognition device determines, at least within the frame of the image, a region corresponding to the person; and

[0016] If the image recognition device detects an overlap between the determined area and the specific region in the image, it outputs an intrusion area notification.

[0017] Invention Effects

[0018] According to this disclosure, the possibility of misjudging a person as having intruded into a specific area can be reduced. Attached Figure Description

[0019] Figure 1 This is a diagram illustrating the structure of a vehicle according to an embodiment of the present disclosure.

[0020] Figure 2 This is an example of a frame around a person within a space that is defined as a specific area.

[0021] Figure 3 It is shown in relation to Figure 2 The same image, at least within a frame, defines an example of a human-like field.

[0022] Figure 4 This is a block diagram illustrating the structure of an image recognition device according to an embodiment of the present disclosure.

[0023] Figure 5 This is a flowchart illustrating the operation of an image recognition device according to an embodiment of the present disclosure.

[0024] (Symbol Explanation)

[0025] 10: Vehicle; 11: Vehicle system; 12: Door; 13: Handrail; 20: Image recognition device; 21: Control unit; 22: Storage unit; 23: Communication unit; 30: Camera device; 40: Output device; 50: Space; 51: Specific area; 52: Expanded area; 60: Image; 61: Frame; 62: Overlapping part; 63: Field; 70: Person; 71: Shoe. Detailed Implementation

[0026] The following description describes one embodiment of the present disclosure with reference to the accompanying drawings.

[0027] In each figure, the same or equivalent parts are marked with the same symbols. In the description of this embodiment, the description of the same or equivalent parts is appropriately omitted or simplified.

[0028] Reference Figure 1 This describes the structure of the vehicle 10 in this embodiment.

[0029] Vehicle 10 can be used for any purpose, but in this embodiment it is used to transport passengers, for example, to operate as a bus. Vehicle 10 can be any type of vehicle, such as a gasoline car, diesel car, hydrogen fuel cell vehicle, HEV, PHEV, BEV, or FCEV. "HEV" is an abbreviation for hybrid electric vehicle. "PHEV" is an abbreviation for plug-in hybrid electric vehicle. "BEV" is an abbreviation for battery electric vehicle. "FCEV" is an abbreviation for fuel cell electric vehicle. Vehicle 10 can be driven by a driver or can be automated at any level. The level of automation is, for example, one of levels 1 to 5 in the SAE classification. "SAE" is an abbreviation for Society of Automotive Engineers. Vehicle 10 can also be a MaaS (Mobility as a Service) vehicle. "MaaS" is an abbreviation for Mobility as a Service.

[0030] Vehicle 10 is equipped with an in-vehicle system 11. The in-vehicle system 11 includes an image recognition device 20 and a camera device 30. Although not strictly necessary, in this embodiment, the in-vehicle system 11 also includes an output device 40. The image recognition device 20 can communicate directly with the camera device 30 and the output device 40 or via a network such as a LAN. "LAN" is an abbreviation for local area network.

[0031] The image recognition device 20 is a computer with image recognition capabilities. The image recognition device 20 can be installed anywhere on the vehicle 10.

[0032] The camera device 30 is, for example, an RGB camera or an infrared camera. In this embodiment, the camera device 30 is a fisheye camera or a 360-degree camera. The camera device 30 can be installed anywhere that can capture images of the space 50. In this embodiment, the space 50 is the interior space of the vehicle 10, and the camera device 30 is installed at a height above the top of the door 12 used for passenger boarding and alighting. The height H from the floor of the vehicle 10 to the camera device 30 is preferably a height that allows for capturing images from above the heads of standing passengers, for example, 2.1 meters. The camera device 30 can also be installed in the low ceiling inside the vehicle 10.

[0033] The output device 40 is, for example, a display or a speaker. The display is, for example, an LCD or an OLED display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescent. If the output device 40 is a display, it can be placed anywhere in the vehicle 10 that passengers can see. If the output device 40 is a speaker, it can be placed anywhere in the vehicle 10 that the sound can reach the passengers.

[0034] Reference Figures 1 to 3 This section provides an overview of the implementation method.

[0035] Camera device 30 captures images 60 of space 50. Space 50 contains a specific area 51. (e.g., ...) Figure 2 As shown, the image recognition device 20 defines a bounding box 61 in the image 60 that surrounds a person 70 within the space 50. For example, the bounding box 61 is defined. As a specific example related to the definition of the bounding box, consider using a deep learning model such as YOLO or SSD to determine the position of the person 70 in the image 60. In the image 60, a rectangular bounding box defined by the coordinates of the upper left and lower right corners of the person 70 is drawn around the person 70. "YOLO" is an abbreviation for "you only look once." "SSD" is an abbreviation for "single-shot multibox detector." The image recognition device 20 calculates the ratio of the overlap 62 between the defined bounding box 61 and the specific region 51 in the image 60. If the image recognition device 20 determines that the calculated ratio is below a threshold, then... Figure 3 As shown, a region 63 corresponding to a person 70 is determined within at least one bounding box 61 of image 60. Region 63 is determined, for example, by segmentation using deep learning. As a specific example related to segmentation, consider using a deep learning model such as U-Net or DeepLab to predict which category each pixel in at least one bounding box 61 of image 60 belongs to, and extract the pixel group belonging to the "person" category as a segment. If the image recognition device 20 detects an overlap between the determined region 63 and a specific region 51 in image 60, it outputs an intrusion region notification.

[0036] In cases such as when using a fisheye lens to capture images from above, depending on the location or type of the camera device 30, the area occupied by the frame 61 surrounding the person 70 in the image 60 may increase, even if the person 70 is not present in the specific region 51, the frame 61 may still overlap with the specific region 51. For example, if the image 60 is captured using a fisheye lens, the outline of the person 70 is distorted, so the frame 61 tends to become larger. In such cases, in this embodiment, if the ratio of the overlapping portion 62 is lower than a threshold, a second processing step with higher precision than the setting of the frame 61 used in the first processing step is performed to determine the area 63 corresponding to the person 70. Then, based on the overlap between the area 63 and the specific region 51, it is determined whether the person 70 has intruded into the specific region 51. Therefore, according to this embodiment, the possibility of misjudging that the person 70 has intruded into the specific region 51 can be reduced. As a variation, the image recognition device 20 may also determine the area 63 corresponding to the person 70 regardless of whether the ratio of the overlapping portion 62 is lower than the threshold. In this variation, the image recognition device 20 may also not calculate the ratio of the overlapping portion 62.

[0037] In this embodiment, the area 63 is smaller than the frame 61. According to this embodiment, when the ratio of the overlap portion 62 between the frame 61 and the specific region 51 is less than a threshold, the area smaller than the frame 61 is defined as the area 63. Therefore, it is possible to suppress the misjudgment that the person 70 has invaded the specific region 51 even though the person 70 is not present in the specific region 51.

[0038] In this embodiment, the computation time consumed by the determined region 63 is longer than the computation time consumed by the set frame 61. According to this embodiment, when the ratio of the overlap portion 62 between the frame 61 and the specific region 51 is above a threshold, the region 63 can be left undetermined, thus suppressing computation time.

[0039] In this embodiment, the computational cost of determining the region 63 is greater than the computational cost of setting the frame 61. According to this embodiment, when the ratio of the overlap 62 between the frame 61 and the specific region 51 is above a threshold, the region 63 can be left undetermined, thus suppressing computational costs.

[0040] In this embodiment, if the image recognition device 20 determines that the ratio of the overlap portion 62 between the frame 61 and the specific region 51 is greater than or equal to a threshold, it determines that the region 63 is uncertain and outputs an intrusion region notification. Therefore, according to this embodiment, when the ratio of the overlap portion 62 between the frame 61 and the specific region 51 is greater than or equal to a threshold, an intrusion region notification can be output more early.

[0041] In this embodiment, when the image recognition device 20 determines the region 63, such as Figure 3As shown, the enlarged region 52 is defined by expanding a specific region 51 in image 60. If the image recognition device 20 detects an overlap between the determined area 63 in image 60 and the enlarged region 52, it outputs an intrusion area notification. In this embodiment, the specific region 51 is the area where the door 12 is located. For example, consider including a shoe 71 in frame 61 but not in area 63, and enlarging the enlarged region 52 by an amount corresponding to the height of the ankle compared to the specific region 51. Alternatively, consider including the lower body in frame 61 but not in area 63, and enlarging the enlarged region 52 by an amount corresponding to the height of the waist compared to the specific region 51. Based on these examples, by removing hard-to-detect parts such as shoes 71, it is possible to suppress the misjudgment that a person 70 is not standing near the door 12 even though the person 70 is standing near the door 12. In this embodiment, a handrail 13 is provided around the door 12. For example, consider including a hand in frame 61 but not in area 63. Set a specific area 51 to exclude the handrail 13, and expand area 52 to include the handrail 13. Based on this example, it is possible to suppress the misjudgment that person 70 is near the door 12 even if they only reach for the handrail 13 and are not actually standing near the door 12. The expanded area 52 can either simply increase the specific area 51 by a certain percentage, or it can be expanded according to the position of the handrail 13.

[0042] Reference Figure 4 The structure of the image recognition device 20 in this embodiment will be explained.

[0043] The image recognition device 20 includes a control unit 21, a storage unit 22, and a communication unit 23.

[0044] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specifically designed for particular processing. "CPU" is an abbreviation for Central Processing Unit. "GPU" is an abbreviation for Graphics Processing Unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for Field-Programmable Gate Array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for Application-Specific Integrated Circuit. The control unit 21 controls the various parts of the image recognition device 20 while performing processing related to the operation of the image recognition device 20.

[0045] Storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, RAM, ROM, or flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read-only memory. RAM is, for example, SRAM or DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read-only memory. Flash memory is, for example, SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, HDD. "HDD" is an abbreviation for hard disk drive. Storage unit 22 functions as, for example, a primary storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores information used in the operation of the image recognition device 20 and information obtained through the operation of the image recognition device 20.

[0046] The communication unit 23 includes at least one communication module. The communication module may be an interface corresponding to wired LAN communication standards such as Ethernet (registered trademark) or wireless LAN communication standards such as IEEE 802.11. "IEEE" is short for Institute of Electrical and Electronics Engineers. The communication module may also be an interface corresponding to other standards such as USB, HDMI (registered trademark), or Bluetooth (registered trademark). "USB" is short for Universal Serial Bus. "HDMI" is short for High-Definition Multimedia Interface. The communication unit 23 communicates with the camera device 30 and the output device 40. The communication unit 23 may also communicate with the gate 12. The communication unit 23 receives information used in the operation of the image recognition device 20 and also transmits information obtained through the operation of the image recognition device 20.

[0047] The functions of the image recognition device 20 are implemented by the processor, which serves as the control unit 21, executing the program of this embodiment. That is, the functions of the image recognition device 20 are implemented through software. The program causes the computer to execute the actions of the image recognition device 20, thereby enabling the computer to function as the image recognition device 20. In other words, the computer functions as the image recognition device 20 by executing the actions of the image recognition device 20 according to the program.

[0048] Programs can be stored on non-transitory computer-readable media. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical discs, optical-magnetic recording media, or ROM. Program distribution can occur, for example, through the sale, transfer, or lending of removable media such as SD cards, DVDs, or CD-ROMs containing the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read-only memory. Programs can also be distributed by storing them on a server's storage device and transferring them from the server to other computers. Programs can also be offered as program products.

[0049] Computers may temporarily store programs stored on removable media or transferred from servers in main storage. The computer then reads the program stored in main storage via its processor and executes the processing according to the read program. The computer can also directly read programs from removable media and execute the processing according to the program. Alternatively, the computer can execute the processing according to the received program each time a program is transferred from a server. It can also perform processing through so-called ASP-type services, which only instruct execution and obtain results, without transferring programs from a server to the computer. "ASP" is an abbreviation for Application Service Provider. A program contains program-based content containing information for processing performed by a computer. For example, data that is not a direct instruction to the computer but has the nature of specifying the computer's processing conforms to "program-based content."

[0050] Some or all of the functions of the image recognition device 20 can also be implemented by a programmable circuit or a dedicated circuit that serves as the control unit 21. That is, some or all of the functions of the image recognition device 20 can also be implemented by hardware.

[0051] Reference Figure 5The operation of the image recognition device 20 in this embodiment will be explained below. The operation described below corresponds to the image recognition method of this embodiment. That is, the image recognition method of this embodiment includes... Figure 5 The steps S1 to S8 are shown.

[0052] In S1, the control unit 21 acquires an image 60 of the space 50 containing a specific area 51 from the camera device 30. Specifically, the control unit 21 receives the image 60 from the camera device 30 via the communication unit 23.

[0053] In S2, the control unit 21 sets a frame 61 that surrounds the person 70 in the space 50 in the image 60 acquired by S1. As a method for setting the frame 61, known methods such as bounding boxes can be used.

[0054] In step S3, the control unit 21 calculates the ratio of the overlap 62 between the frame 61 set in step S2 and the specific region 51 in the image 60 acquired in step S1. The control unit 21 determines whether the calculated ratio is lower than a threshold. The threshold is, for example, 1 / 9 of the area of ​​the frame 61, but it can also be adjusted according to the area of ​​the specific region 51. Specifically, the smaller the area of ​​the specific region 51, the smaller the threshold value can be set. If it is determined that the ratio of the overlap 62 between the frame 61 and the specific region 51 is lower than the threshold, step S4 is performed. On the other hand, if it is determined that the ratio of the overlap 62 between the frame 61 and the specific region 51 is higher than the threshold, step S8 is performed.

[0055] In step S4, the control unit 21 determines a region 63 corresponding to the person 70 within at least one frame 61 of the image 60 acquired in step S1. Specifically, the control unit 21 determines the region 63 by classifying pixel groups within at least one frame 61 of the image 60 into region 63 and one or more other regions. Known methods such as segmentation can be used as a method for classifying pixel groups.

[0056] In this embodiment, when determining the region 63, the control unit 21 sets the enlarged region 52 by expanding the specific region 51 in the image 60. Specifically, the control unit 21 sets the enlarged region 52 by expanding the specific region 51 by a certain ratio. For example, the control unit 21 sets the enlarged region 52 by simply expanding the specific region 51 by a certain percentage. Alternatively, the control unit 21 may also set the enlarged region 52 by expanding the specific region 51 by an amount corresponding to the height below a specific part of the person 70. In such an example, when setting the frame 61, the control unit 21 includes the area below the specific part of the person 70 within the frame 61, and when determining the region 63, removes the area below the specific part of the person 70 from the region 63. The specific part is, for example, the ankle or waist. That is, the control unit 21 may also include the shoe 71 within the frame 61 and exclude the shoe 71 from the region 63 by expanding the specific region 51 by an amount corresponding to the height of the ankle. The control unit 21 can also include the lower body within the frame 61 but exclude it from the area 63 by expanding the specific area 51 by an amount corresponding to the waist height, thereby setting the expanded area 52. Alternatively, the control unit 21 can also set the expanded area 52 by expanding the specific area 51 according to the position of the armrest 13. In such an example, the specific area 51 may not include the position of the armrest 13, and the expanded area 52 may include the position of the armrest 13. For example, the control unit 21 can also include the hand within the frame 61 but exclude it from the area 63, setting the specific area 51 to exclude the armrest 13, and setting the expanded area 52 by expanding the specific area 51 to include the armrest 13.

[0057] In step S5, the control unit 21 determines whether there is an overlap between the region 63 identified in step S4 and the specific region 51 in the image 60 acquired in step S1. If no overlap between the region 63 and the specific region 51 is detected, step S6 is performed. On the other hand, if an overlap between the region 63 and the specific region 51 is detected, step S8 is performed.

[0058] In this embodiment, the control unit 21 determines whether there is an overlap between the region 63 and the expanded region 52 in the image 60. If no overlap is detected, step S6 is performed. On the other hand, if an overlap is detected, step S8 is performed.

[0059] In step S6, the control unit 21 determines whether the steps following S2 have been performed for all personnel within space 50. If it determines that the steps following S2 have not been performed for all personnel, the steps following S2 are performed again. On the other hand, if it determines that the steps following S2 have been performed for all personnel, step S7 is performed.

[0060] In S7, the control unit 21 authorizes the opening of the door 12 by outputting an unintruded area notification. Specifically, the control unit 21 authorizes the opening of the door 12 by sending a door control signal equivalent to an unintruded area notification via the communication unit 23. Alternatively, the control unit 21 may also send a message such as "open door" equivalent to an unintruded area notification via the communication unit 23, causing the output device 40 to display the message visually or output an audio message.

[0061] In S8, the control unit 21 prevents the door 12 from being opened by outputting an intrusion zone notification. Specifically, the control unit 21 prevents the door 12 from being opened by sending a door control signal equivalent to an intrusion zone notification via the communication unit 23. Alternatively, the control unit 21 may also send a message equivalent to an intrusion zone notification, such as "Because of danger, please stay away from the door," via the communication unit 23, causing the output device 40 to display the message visually or output an audio message.

[0062] In this embodiment, by performing the actions described above, for example, using a camera device 30 such as an RGB camera from a top-down position, it is possible to quickly, cost-effectively, and accurately determine whether a passenger boarding the vehicle 10 in a posture where they are only extending their arm to hold onto the handrail 13 is standing near the door 12. In this embodiment, the overlap between the person detection rectangle and the object region is determined. If the overlap is small, a more detailed segmentation area is determined to overlap with the object region, thereby enabling high-precision intrusion region determination while suppressing computation time and computational costs. According to this embodiment, even when it is difficult to install a large number of devices on the roof of the vehicle 10, it is not necessary to install multiple cameras to improve the determination accuracy.

[0063] This disclosure is not limited to the embodiments described above. For example, two or more blocks shown in the block diagram may be merged, or one block may be split. Alternatively, the two or more steps shown in the flowchart may not be executed sequentially according to the description, but may be executed in parallel or in a different order depending on the processing capacity of the device executing each step or as needed. Furthermore, modifications can be made without departing from the spirit of this disclosure.

[0064] Below, examples are given of some embodiments of the present disclosure. However, it should be noted that the embodiments of the present disclosure are not limited to these.

[0065] [Note 1]

[0066] An image recognition device includes a control unit.

[0067] The control unit acquires the image from a camera device that captures an image of a space containing a specific area, sets a frame surrounding a person in the space in the acquired image, determines an area corresponding to the person in at least the frame in the image, and outputs an intrusion area notification if an overlap between the determined area in the image and the specific area is detected.

[0068] [Note 2]

[0069] According to the image recognition device described in Note 1, wherein,

[0070] The field is smaller than the frame.

[0071] [Note 3]

[0072] According to the image recognition device described in note 1 or 2, wherein,

[0073] The control unit determines the region by classifying at least the pixel group in the frame of the image into the region and one or more other regions.

[0074] [Note 4]

[0075] The image recognition device according to any one of notes 1 to 3, wherein,

[0076] The computation time or cost consumed by the field is determined to be greater than the computation time or cost consumed by setting the frame.

[0077] [Note 5]

[0078] The image recognition device according to any one of notes 1 to 4, wherein,

[0079] The control unit calculates the ratio of the overlap between the set frame and the specific region in the image. If it determines that the calculated ratio is lower than a threshold, it determines an area equivalent to the person in at least the frame in the image. If it determines that the ratio is higher than the threshold, it does not determine the area and outputs an intrusion area notification.

[0080] [Note 6]

[0081] The image recognition device according to any one of notes 1 to 5, wherein,

[0082] When determining the area, the control unit sets an enlarged area by expanding the specific region in the image. If an overlap between the area in the image and the enlarged area is detected, an intrusion area notification is output.

[0083] [Note 7]

[0084] According to the image recognition device described in Note 6, wherein,

[0085] The control unit sets the expanded area by expanding the specific area at a certain ratio.

[0086] [Note 8]

[0087] According to the image recognition device described in Note 6, wherein,

[0088] When setting the frame, the control unit includes the area below the specific part of the person within the frame. When determining the field, the area below the specific part of the person is removed from the field. The expanded area is set by expanding the specific area by an amount corresponding to the height below the specific part of the person.

[0089] [Note 9]

[0090] According to the image recognition device described in Note 8, wherein,

[0091] The specific area mentioned is the ankle or the waist.

[0092] [Note 10]

[0093] According to the image recognition device described in Note 6, wherein,

[0094] The space in question is the interior space of a vehicle used to transport passengers.

[0095] The specific area refers to the area where the doors for passengers to get on and off the vehicle are located.

[0096] The control unit sets the expanded area by expanding the specific area according to the position of the handrails provided around the door.

[0097] [Note 11]

[0098] According to the image recognition device described in Note 10, wherein,

[0099] The specific area does not include the location of the handrail.

[0100] The enlarged area includes the location of the handrail.

[0101] [Note 12]

[0102] The image recognition device according to any one of notes 1 to 9, wherein,

[0103] The space in question is the interior space of a vehicle used to transport passengers.

[0104] The specific area refers to the area where the doors for passengers to get on and off the vehicle are located.

[0105] The camera device is positioned at a height above the top of the door.

[0106] [Note 13]

[0107] According to the image recognition device described in Note 12, wherein,

[0108] The camera device is a fisheye camera.

[0109] [Note 14]

[0110] The image recognition device according to any one of notes 1 to 9, wherein,

[0111] The space in question is the interior space of a vehicle used to transport passengers.

[0112] The specific area refers to the area where the doors for passengers to get on and off the vehicle are located.

[0113] The control unit prevents the door from being opened by outputting a notification of the intrusion area.

[0114] [Note 15]

[0115] A system that possesses:

[0116] The image recognition device described in any one of Notes 1 to 14; and

[0117] The camera device.

[0118] [Note 16]

[0119] A type of vehicle,

[0120] The vehicle is equipped with the system described in Note 15.

[0121] [Note 17]

[0122] A program,

[0123] To enable the computer to function as an image recognition device as described in any of Notes 1 to 14.

[0124] [Note 18]

[0125] An image recognition method, comprising:

[0126] The camera device captures images of a space containing a specific area;

[0127] The image recognition device sets a frame around a person in the space within the image;

[0128] The image recognition device determines, at least within the frame of the image, a region corresponding to the person; and

[0129] If the image recognition device detects an overlap between the determined area and the specific region in the image, it outputs an intrusion area notification.

[0130] [Note 19]

[0131] The image recognition method according to Note 18 further includes:

[0132] The image recognition device calculates the ratio of the overlap between the set frame and the specific region in the image;

[0133] If the image recognition device determines that the calculated ratio is below a threshold, it determines a region corresponding to the person within at least the frame of the image; and

[0134] If the image recognition device determines that the ratio is above the threshold, it will not determine the area and will output an intrusion area notification.

[0135] [Note 20]

[0136] According to the image recognition method described in Note 18 or 19, wherein,

[0137] It also includes the image recognition device defining an enlarged region by expanding the specific region in the image when determining the region.

[0138] If the image recognition device detects an overlap between the region and the expanded region in the image, it outputs an intrusion region notification.

Claims

1. An image recognition device, comprising a control unit, The control unit acquires the image from a camera device that captures an image of a space containing a specific area, sets a frame surrounding a person in the space in the acquired image, determines an area corresponding to the person in at least the frame in the image, and outputs an intrusion area notification if an overlap between the determined area in the image and the specific area is detected.

2. The image recognition device according to claim 1, wherein, The field is smaller than the frame.

3. The image recognition device according to claim 1, wherein, The control unit determines the region by classifying at least the pixel group in the frame of the image into the region and one or more other regions.

4. The image recognition device according to claim 1, wherein, The computation time or cost consumed by the field is determined to be greater than the computation time or cost consumed by setting the frame.

5. The image recognition device according to claim 1, wherein, The control unit calculates the ratio of the overlap between the set frame and the specific region in the image. If it determines that the calculated ratio is lower than a threshold, it determines an area equivalent to the person in at least the frame in the image. If it determines that the ratio is higher than the threshold, it does not determine the area and outputs an intrusion area notification.

6. The image recognition device according to claim 1, wherein, When determining the area, the control unit sets an enlarged area by expanding the specific region in the image. If an overlap between the area in the image and the enlarged area is detected, an intrusion area notification is output.

7. The image recognition device according to claim 6, wherein, The control unit sets the expanded area by expanding the specific area at a certain ratio.

8. The image recognition device according to claim 6, wherein, When setting the frame, the control unit includes the area below the specific part of the person within the frame. When determining the field, the area below the specific part of the person is removed from the field. The expanded area is set by expanding the specific area by an amount corresponding to the height below the specific part of the person.

9. The image recognition device according to claim 8, wherein, The specific area mentioned is the ankle or the waist.

10. The image recognition device according to claim 6, wherein, The space in question is the interior space of a vehicle used to transport passengers. The specific area refers to the area where the doors for passengers to get on and off the vehicle are located. The control unit sets the expanded area by expanding the specific area according to the position of the handrails provided around the door.

11. The image recognition device according to claim 10, wherein, The specific area does not include the location of the handrail. The enlarged area includes the location of the handrail.

12. The image recognition device according to claim 1, wherein, The space in question is the interior space of a vehicle used to transport passengers. The specific area refers to the area where the doors for passengers to get on and off the vehicle are located. The camera device is positioned at a height above the top of the door.

13. The image recognition device according to claim 12, wherein, The camera device is a fisheye camera.

14. The image recognition device according to claim 1, wherein, The space in question is the interior space of a vehicle used to transport passengers. The specific area refers to the area where the doors for passengers to get on and off the vehicle are located. The control unit prevents the door from being opened by outputting a notification of the intrusion area.

15. A system having: The image recognition device according to any one of claims 1 to 14; and The camera device.

16. A type of vehicle, The vehicle is equipped with the system described in claim 15.

17. A non-transitory computer-readable medium, The computer contains a program that enables it to function as an image recognition device according to any one of claims 1 to 14.

18. An image recognition method, comprising: The camera device captures images of a space containing a specific area; The image recognition device sets a frame around a person in the space within the image; The image recognition device determines, at least within the frame of the image, a region corresponding to the person; and If the image recognition device detects an overlap between the determined area and the specific region in the image, it outputs an intrusion area notification.

19. The image recognition method according to claim 18, wherein, Also includes: The image recognition device calculates the ratio of the overlap between the set frame and the specific region in the image; If the image recognition device determines that the calculated ratio is below a threshold, it determines a region corresponding to the person within at least the frame of the image; as well as If the image recognition device determines that the ratio is above the threshold, it will not determine the area and will output an intrusion area notification.

20. The image recognition method according to claim 18, wherein, It also includes the image recognition device defining an enlarged region by expanding the specific region in the image when determining the region. If the image recognition device detects an overlap between the region and the expanded region in the image, it outputs an intrusion region notification.

Images