Image recognition system, image recognition method and entry system

Abstract

To provide an image recognition system capable of determining whether or not a person has an object without previously learning an image of the object which may be a detection object of belongings.SOLUTION: An image recognition system comprises: an acquisition unit which acquires a captured image from a camera mounted on a vehicle; a person recognition unit which recognizes a person picked up in the captured image; and a determination unit which determines whether or not the person who is picked up in the captured image as an object holds baggage. The determination unit extracts a partial image of predetermined shape and size including the hand of the object picked up in the captured image from the captured image, and determines that the object holds the baggage when an image pattern different from a background image is recognized at the tip of the hand in the partial image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image recognition system, an image recognition method, and an entry system. [Background technology]

[0002] Patent Document 1 describes a vehicle door control device that captures an image of a person in a surrounding image near a vehicle door and automatically executes an operation to open the vehicle door when it is estimated based on the surrounding image that the person has both hands occupied. This vehicle door control device learns images of objects to be detected in advance, recognizes images of the learned objects included in the surrounding image, and determines that the person is carrying luggage when the learned object recognized in the surrounding image is in contact with part of the person. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-79692 Summary of the Invention [Problem to be solved by the invention]

[0004] The object of the present invention is to provide an image recognition system, an image recognition method, and an entry system that can determine whether a person is carrying an object (i.e., an object that can be a target of personal belongings detection) without having to previously learn images of the object. [Means for solving the problem]

[0005] One aspect of the present invention is an image recognition system that includes an acquisition unit that acquires an image from a camera mounted on a vehicle, a person recognition unit that recognizes people who appear in the image, and a judgment unit that determines from the image that the person in the image is a target and whether the target is carrying luggage, wherein the judgment unit extracts from the image from the image a partial image of a predetermined shape and size that includes the target's hand that appears in the image, and determines that the target is carrying luggage when it recognizes an image pattern at the tip of the hand in the partial image that differs from the background image of the partial image. Another aspect of the present invention is an image recognition method executed by a computer of an image recognition system, comprising: an acquisition step of acquiring an image from a camera mounted on a vehicle; a recognition step of recognizing a person appearing in the image; and a judgment step of determining from the image that the person appearing in the image is a subject and whether the subject is carrying luggage or not, wherein the judgment step extracts from the image the partial image of a predetermined shape and size including the subject's hand appearing in the image; and when an image pattern different from the background image of the partial image is recognized at the tip of the hand in the partial image, the image recognition method determines that the subject is carrying luggage. Yet another aspect of the present invention is an entry system comprising the image recognition system and a control unit that automatically opens and closes the vehicle doors, wherein the control unit automatically opens the door closest to the subject when the judgment unit of the image recognition system determines that the subject is carrying luggage. [Effects of the Invention]

[0006] According to the present invention, an image recognition system, an image recognition method, and an entry system can be provided that can determine whether a person is carrying an object even if images of objects that may be the subject of personal belongings detection have not been learned in advance. [Brief explanation of the drawings]

[0007] [Figure 1]FIG. 1 is a diagram showing an example of the configuration of a vehicle equipped with an entry system including an image recognition system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing the configuration of an image recognition system. [Figure 3] FIG. 3 is an explanatory diagram for explaining the process of determining whether or not a person is carrying luggage in the image recognition system. [Figure 4] FIG. 4 is a flowchart showing the procedure of the image recognition method executed by the image recognition system. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] [1. Vehicle configuration] 1 is a diagram showing the configuration of a vehicle 4 equipped with an entry system 3 including an image recognition system 1 according to one embodiment of the present invention. The entry system 3 includes the image recognition system 1 and an entry control device 2. The image recognition system 1 determines whether or not a target person P, who is any person approaching a vehicle 4, is carrying baggage. The entry control device 2 authenticates whether the subject P is a registered user who has been registered in advance as a person who has legitimate authority to use the vehicle 4. If the entry control device 2 can authenticate that the subject P is a registered user, and if the image recognition system 1 determines that the subject P is carrying luggage, the entry control device 2 executes an automatic opening operation to automatically open the door 8 of the vehicle 4.

[0010] In this embodiment, the vehicle 4 is, for example, a passenger car, and may be a shared car that can accommodate multiple people. However, the vehicle 4 is not limited to a passenger car and may be any vehicle. Such a vehicle may be a passenger car, a bus, a taxi, a train, etc.

[0011] On the right side of the vehicle 4, a right object detection sensor 5a that detects an object present on the right side of the vehicle 4 and a right camera 6a that takes an image of the right side of the vehicle 4 are provided.

[0012] On the left side of the vehicle 4, a left object detection sensor 5b that detects an object present on the left side of the vehicle 4 and a left camera 6b that takes an image of the left side of the vehicle 4 are provided.

[0013] At the rear of the vehicle 4, a rear object detection sensor 5c that detects an object present behind the vehicle 4 and a rear camera 6c that takes an image of the area behind the vehicle 4 are provided.

[0014] In the following, when there is no need to distinguish between the right object detection sensor 5a, the left object detection sensor 5b, and the rear object detection sensor 5c, they will be referred to as object detection sensors 5. Furthermore, when there is no need to distinguish between the right camera 6a, the left camera 6b, and the rear camera 6c, they will be referred to as cameras 6.

[0015] The camera 6 is, for example, a CCD camera. The object detection sensor 5 is, for example, a distance measurement sensor that can detect surrounding objects and measure the distance to the objects, and in this embodiment, is, for example, a millimeter-wave radar. The object detection sensor 5 is not limited to a radar, and can be any distance measurement sensor that can detect objects and measure distances. Such a sensor can be, in addition to radar, for example, a lidar, a sonar, or the like.

[0016] The vehicle 4 also includes a door opener 9 provided in each door 8 of the vehicle 4 for locking and unlocking the door 8 and opening and closing the door 8. The door opener 9 may include an actuator for locking and unlocking the corresponding door 8 and an actuator for opening and closing the door 8.

[0017] In this embodiment, the vehicle 4 is equipped with, as the doors 8 and door opening / closing devices 9, a right front door opening / closing device 9a provided on a driver's door 8a on the right front of the vehicle 4 and a right rear door opening / closing device 9b provided on a right rear door 8b on the right rear of the vehicle 4. The vehicle 4 also is equipped with a left front door opening / closing device 9c provided on a passenger's door 8c on the left front of the vehicle 4 and a left rear door opening / closing device 9d provided on a left rear door 8d on the left rear of the vehicle 4. The vehicle 4 is further equipped with a back door opening / closing device 9e provided on a back door (tailgate door) 8e of the vehicle 4.

[0018] [2. Entry system configuration] The configuration of the entry system 3 will be described. 2 is a diagram showing the configuration of the entry system 3. The entry system 3 includes an image recognition system 1 and an entry control device 2.

[0019] In the following, the term "target person" refers to a person who is the target of authentication by the entry system 3. Furthermore, the term "user" refers to a registered user who has been registered in advance as a person who has legitimate authority to use the vehicle 4 and who actually uses the vehicle 4. The target person P becomes a user when he or she is authenticated by the entry control device 2, which will be described later, and becomes able to use the vehicle 4.

[0020] The image recognition system 1 acquires an image of an object approaching the periphery of the vehicle 4, and performs image recognition processing on the acquired image. The image recognition system 1 determines whether the object is a person approaching the vehicle 4, i.e., whether the object is a target person P approaching the vehicle 4.

[0021] When the object is a target person P, the image recognition system 1 sends a facial image of the target person P to the entry control device 2. When the object is a target person P, the image recognition system 1 determines whether the target person P is carrying an object (luggage) based on an image of the target person P approaching the vehicle 4. The image recognition system 1 sends the result of the determination to the entry control device 2.

[0022] The entry control device 2 authenticates whether or not the subject P is a registered user who has been registered in advance as a person who has legitimate authority to use the vehicle 4, based on the facial image of the subject P sent from the image recognition system 1. When the entry control device 2 authenticates that the subject P is a registered user, it executes an automatic unlocking operation to automatically unlock the door 8 of the vehicle 4. Furthermore, when the entry control device 2 authenticates that the subject P is a registered user and the image recognition system 1 determines that the subject P is carrying luggage, it executes an automatic opening operation to automatically open the door 8 of the unlocked vehicle 4.

[0023] [2.1. Image Recognition System Configuration] The image recognition system 1 includes a first processor 20 and a first memory 21. The first memory 21 is configured by, for example, a volatile and / or non-volatile semiconductor memory and / or a hard disk device.

[0024] The first processor 20 is, for example, a computer including a CPU, etc. The first processor 20 may have a configuration including a ROM in which a program is written, a RAM for temporarily storing data, etc. The first processor 20 includes a detection unit 23, an acquisition unit 24, a person recognition unit 25, and a determination unit 26 as functional elements or functional units.

[0025] These functional elements of the first processor 20 are realized, for example, by the first processor 20, which is a computer, executing a first program 22 stored in a first memory 21. The first program 22 can be stored in any computer-readable storage medium. Alternatively, all or part of the functional elements of the first processor 20 can be configured by hardware each including one or more electronic circuit components.

[0026] The detection unit 23 determines whether or not an object has entered within a predetermined distance range from the vehicle 4 using the object detection sensor 5. When an object has entered within the predetermined distance range from the vehicle 4, the detection unit 23 notifies the acquisition unit 24 of this fact.

[0027] When the acquisition unit 24 receives a notification from the detection unit 23 that an object has entered within a predetermined distance from the vehicle 4, the acquisition unit 24 activates the camera 6 that is closest to the object detection sensor 5 that detected the object.

[0028] The acquisition unit 24 continuously acquires, at predetermined time intervals, captured images of the surroundings of the vehicle 4, including images of the above-mentioned object, using the activated camera 6. The acquisition unit 24 sends the acquired captured images to the person recognition unit 25 and the judgment unit 26.

[0029] The person recognition unit 25 recognizes people who appear in the captured image sent from the acquisition unit 24. For example, the person recognition unit 25 recognizes people who appear in the captured image by pattern matching with standard images of people stored in advance in the first memory 21, according to conventional technology. When the person recognition unit 25 recognizes a person who appears in the captured image, it notifies the judgment unit 26 to that effect.

[0030] Furthermore, when the person recognition unit 25 recognizes a person in a captured image, it determines whether or not the person is approaching the camera 6 (and therefore whether or not the person is approaching the vehicle 4) based on the time-series captured images sent from the acquisition unit 24. Then, when the person in the captured image is approaching, the person recognition unit 25 extracts a facial image of the person from the captured image and transmits it to the entry control device 2.

[0031] When the judgment unit 26 receives a notification from the person recognition unit 25 that it has recognized a person appearing in the captured image, it determines whether the person appearing in the captured image sent from the acquisition unit 24 is a subject P and, based on the captured image, determines whether the subject P is carrying a baggage (object).

[0032] 3 is an explanatory diagram for explaining the process of determining whether or not the subject P is carrying luggage, performed by the determination unit 26. In FIG. 3, an example of a captured image including the subject P, sent from the acquisition unit 24, is shown. 3, in this embodiment, in particular, the determination unit 26 extracts, from the captured image sent from the acquisition unit 24, each partial image PI of a predetermined shape and size, including each hand of the subject P captured in the captured image. Then, when the determination unit 26 recognizes an image pattern at the tip of the partial image PIa, which is the partial image PI of the right hand, that is different from the background image of the partial image PIa, the determination unit 26 determines that the subject P is holding a baggage in the right hand. On the other hand, when the determination unit 26 does not recognize an image pattern at the tip of the partial image PIa of the right hand that is different from the background image of the partial image PIa, the determination unit 26 determines that the subject P is not holding a baggage in the right hand.

[0033] Furthermore, when the determination unit 26 recognizes an image pattern at the tip of the partial image PIb, which is the partial image PI for the left hand, that is different from the background image of the partial image PIb, the determination unit 26 determines that the subject P is holding a baggage in the left hand. On the other hand, when the determination unit 26 does not recognize an image pattern at the tip of the partial image PIb for the left hand that is different from the background image of the partial image PIb, the determination unit 26 determines that the subject P is not holding a baggage in the left hand.

[0034] Here, whether or not there is an image pattern at the tip of the hand in the partial image PI that differs from the background image of the partial image PI can be determined, for example, by converting each of the partial image PI and the background image into a binary image and labeling the pixels, according to conventional technology. For example, if a labeling pattern that differs from the background image is recognized in the partial image PI, it can be determined that there is an image pattern that differs from the background image in the partial image PI. In the conversion to a binary image, the threshold luminance (black-and-white discrimination threshold luminance) may be adjusted based on the luminance distribution of the hand image, etc., so that an image area having similar features to the hand's features (e.g., the hue, saturation, brightness, and two-dimensional distribution thereof) can be effectively extracted by pixel labeling.

[0035] In addition, when the right hand or left hand to be judged is not captured in the captured image, the judgment unit 26 selects an image that captures the right hand of the subject P, an image that captures the left hand, and / or an image that captures both the left and right hands from among the multiple captured images captured at a predetermined time interval sent from the acquisition unit 24, and performs the above-mentioned processing on the selected captured image, thereby making it possible to judge whether the subject P is holding a baggage in his / her right and / or left hand.

[0036] The background image described above may be, for example, an image of a portion corresponding to the partial image PI in another captured image different from the captured image from which the partial image PI including the hand of the subject P was extracted. Furthermore, the other image may be an image captured by the acquisition unit 24 when the vehicle 4 is stopped, an image captured when an object is detected that has entered within a predetermined distance from the vehicle 4, or an image captured before the determination unit 26 determines whether the subject P is carrying luggage. For example, the acquisition unit 24 can detect that the vehicle 4 has stopped when the vehicle speed of the vehicle 4 obtained from an electronic control device (not shown) included in the vehicle 4 is less than a predetermined speed. Furthermore, the point in time when an object that has entered within a predetermined distance from the vehicle 4 is detected may be the point in time when the acquisition unit 24 receives a notification from the detection unit 23 that an object has been detected that has entered within a predetermined distance from the vehicle 4.

[0037] The determination unit 26 transmits to the entry control device 2 the results of the determination as to whether or not the subject person P is carrying baggage in each of his or her right and left hands.

[0038] [2.2. Entry control device configuration] The entry control device 2 includes a second processor 30 and a second memory 31. The second memory 31 is configured, for example, by a volatile and / or non-volatile semiconductor memory and / or a hard disk drive. The second memory 31 stores authentication data 33 for each of one or more registered users who are pre-registered as persons who have legitimate authority to use the vehicle 4. The authentication data 33 may be, for example, data on feature amounts in a facial image of the registered user that has been captured in advance.

[0039] The second processor 30 is, for example, a computer equipped with a CPU, etc. The second processor 30 may have a configuration including a ROM in which a program is written, a RAM for temporarily storing data, etc. The second processor 30 includes an authentication unit 34 and a control unit 35 as functional elements or units.

[0040] These functional elements of the second processor 30 are realized, for example, by the second processor 30, which is a computer, executing a second program 32 stored in a second memory 31. The second program 32 can be stored in any computer-readable storage medium. Alternatively, all or part of the functional elements of the second processor 30 can be configured by hardware including one or more electronic circuit components.

[0041] The authentication unit 34 authenticates whether or not the subject P is a registered user based on the facial image of the subject P received from the image recognition system 1. Specifically, the authentication unit 34 compares the received facial image of the subject P with the authentication data 33 of each registered user recorded in the second memory 31 according to conventional technology, and authenticates the subject P as a registered user when the degree of match between the facial image of the subject P and the authentication data 33 of any registered user is equal to or greater than a predetermined degree.

[0042] When the authentication unit 34 authenticates the target person P as a registered user, it allows the control unit 35 to allow the target person P to enter the vehicle 4.

[0043] The control unit 35 controls the operation of the door opener 9 to automatically lock and unlock the door 8 of the vehicle 4, and automatically open and close the door 8 of the vehicle 4. Specifically, when the control unit 35 receives permission for the target person P to enter the vehicle 4 from the authentication unit 34, it instructs the door opener 9 to automatically unlock the door 8 of the vehicle 4 (automatic unlocking operation). Furthermore, when the authentication unit 34 authenticates that the target person P is a registered user, and the control unit 35 receives a notification from the image recognition system 1 that it has determined that the target person P is carrying luggage, it automatically opens the unlocked door 8 of the vehicle 4 (automatic opening operation).

[0044] Alternatively, when the authentication unit 34 authenticates that the subject P is a registered user, the control unit 35 may automatically open the unlocked door 8 of the vehicle 4 when it receives a notification from the image recognition system 1 that it has determined that the subject P is carrying luggage in both hands.

[0045] The door 8 of the vehicle 4 that is the target of the automatic unlocking operation and the automatic opening operation can be the door 8 that is closest to the position of the subject P. The door 8 that is closest to the subject P can be determined by the control unit 35, for example, from information that identifies the camera 6 used by the acquisition unit 24 to acquire the captured image and information about the position of the subject P in the captured image (for example, whether the subject P is in the image on the left or right side of the center line of the captured image, etc.). The information that identifies the camera 6 and the information about the subject P in the captured image can be collected by the determination unit 26 of the image recognition system 1 and transmitted to the entry control device 2.

[0046] [3. Operation of image recognition system] Next, the operation procedure of the image recognition system 1 will be described. FIG. 4 is a flowchart showing the steps of an image recognition method executed by the first processor 20, which is a computer in the image recognition system 1.

[0047] The process shown in FIG. 4 starts when the image recognition system 1 is powered on, and is repeatedly executed. When the process starts, first, the detection unit 23 determines whether any object has been detected by any of the object detection sensors 5 within a predetermined distance from the vehicle 4 (S100). If no object has been detected (S100, NO), the detection unit 23 returns to step S100 and repeats the process, waiting for an object to be detected within the predetermined distance from the vehicle 4.

[0048] On the other hand, when an object is detected within a predetermined distance from the vehicle 4 (S100, YES), the acquisition unit 24 starts acquiring captured images of the surroundings of the vehicle 4, including an image of the object, using the camera 6 closest to the object detection sensor 5 that detected the object (S102). The acquisition unit 24 continuously acquires captured images of the surroundings of the vehicle 4, including an image of the object, at predetermined time intervals, and sends the acquired captured images to the person recognition unit 25 and the judgment unit 26.

[0049] The person recognition unit 25 attempts to recognize a person appearing in the captured image sent from the acquisition unit 24 (S104), and determines whether or not the person appearing in the captured image has been recognized (S106). If the person appearing in the captured image cannot be recognized (S106, NO), the person recognition unit 25 returns the process to step S100.

[0050] On the other hand, if the judgment unit 26 can recognize the person in the captured image (S106, YES), it executes steps S108 to S114, and determines whether the person in the captured image sent from the acquisition unit 24 is the subject P, based on the captured image, whether the subject P is carrying luggage.

[0051] First, the determination unit 26 extracts a partial image PI of a predetermined shape and size, including the hand of the subject P shown in the captured image, from the captured image sent from the acquisition unit 24 (S108). Next, the determination unit 26 determines whether an image pattern different from the background image of the partial image PI can be recognized at the tip of the hand in the extracted partial image PI (S110). If an image pattern different from the background image can be recognized in the partial image PI (YES in S110), the determination unit 26 determines that the subject P is carrying luggage (S112) and ends this processing. On the other hand, if an image pattern different from the background image cannot be recognized in the partial image PI (NO in S110), the determination unit 26 determines that the subject P is not carrying luggage (S114) and ends this processing. When this processing ends, the determination unit 26 transmits the result of the determination made in step S112 or S114 to the entry control device 2.

[0052] 4, step S102 corresponds to the acquisition step in the present disclosure, steps S104 and S106 correspond to the recognition step in the present disclosure, and steps S108 to S114 correspond to the determination step in the present disclosure.

[0053] 4. Other Embodiments Although the image recognition system 1 is configured as a single device in the above-described embodiment, it may be configured as a multiple device. For example, the multiple devices may have the functional elements of the above-described first processor 20 distributed among them, and the multiple devices may cooperate to configure the image recognition system 1.

[0054] Similarly, although the entry control device 2 is configured as a single device in the above-described embodiment, it may be configured as a plurality of devices. For example, the functional elements of the second processor 30 described above may be distributed among the plurality of devices, and the functions of the entry control device 2 may be realized as a system in which the plurality of devices cooperate with each other.

[0055] The present invention is not limited to the configurations of the above-described embodiments, and can be implemented in various forms without departing from the spirit of the present invention.

[0056] 5. Configurations supported by the above embodiments The above-described embodiment supports the following configurations.

[0057] (Configuration 1) An image recognition system comprising an acquisition unit that acquires an image from a camera mounted on a vehicle, a person recognition unit that recognizes people in the image, and a judgment unit that determines from the image that the person in the image is a target and whether the target is carrying luggage, wherein the judgment unit extracts a partial image of a predetermined shape and size from the image that includes the target's hand that is captured in the image, and determines that the target is carrying luggage when it recognizes an image pattern at the tip of the hand in the partial image that differs from the background image of the partial image. According to the image recognition system of configuration 1, it is possible to determine whether a person is carrying an object without having to learn in advance images of objects that may be the subject of belonging detection.

[0058] (Configuration 2) The image recognition system described in Configuration 1, wherein the judgment unit determines that the subject is not carrying any luggage when it does not recognize an image pattern at the tip of the hand in the partial image that is different from the background image of the partial image. According to the image recognition system of configuration 2, it is possible to determine whether a person is carrying no object without learning in advance images of objects that may be the subject of personal belongings detection.

[0059] (Configuration 3) The image recognition system according to configuration 1 or 2, wherein the background image is an image of a portion corresponding to the partial image in another captured image different from the captured image from which the partial image was extracted. According to the image recognition system of configuration 3, it is possible to determine whether a person is carrying an object by comparing a partial image including the person's hand with a background image, without having to learn images of objects that may be the subject of personal belongings detection in advance.

[0060] (Configuration 4) The image recognition system described in Configuration 3, wherein the other captured image is an image captured by the acquisition unit when the vehicle is stopped, an image captured when an object approaches within a predetermined distance from the vehicle, or an image captured before the judgment unit determines whether the subject is carrying luggage. According to the image recognition system of configuration 4, a background image is obtained from an image captured at a time when there is little possibility that a person will appear in the captured image, and a partial image including the person's hand is compared with the background image to determine whether the person is holding an object.

[0061] (Configuration 5) An image recognition method executed by a computer of an image recognition system, comprising: an acquisition step of acquiring an image from a camera mounted on a vehicle; a recognition step of recognizing a person appearing in the image; and a judgment step of determining whether the person appearing in the image is a subject and whether the subject is carrying luggage from the image; wherein the judgment step extracts a partial image of a predetermined shape and size including the subject's hand appearing in the image from the image; and when an image pattern different from the background image of the partial image is recognized at the tip of the hand in the partial image, the image recognition method determines that the subject is carrying luggage. According to the image recognition method of the fifth aspect, the same effects as those of the first aspect can be achieved.

[0062] (Configuration 6) An entry system comprising an image recognition system according to any one of configurations 1 to 4 and a control unit that automatically opens and closes the vehicle doors, wherein the control unit automatically opens the door closest to the subject when the judgment unit of the image recognition system determines that the subject is carrying luggage. According to the entry system of configuration 6, it is possible to appropriately determine whether a person attempting to enter a vehicle is carrying an object without having to learn images of objects that may be subject to personal belongings detection in advance, and when authentication of the person is successful, the vehicle door is automatically opened, enabling the person to smoothly enter the vehicle.

[0063] (Configuration 7) An entry system as described in Configuration 6, wherein the control unit automatically opens the door closest to the subject when the judgment unit of the image recognition system determines that the subject is carrying luggage in each hand. According to the entry system of configuration 7, it is possible to appropriately determine whether a person attempting to enter a vehicle is holding an object in both hands, and when authentication of the person is successful, the vehicle door is automatically opened, thereby enabling the person to smoothly enter the vehicle. [Explanation of symbols]

[0064] 1...image recognition system, 2...entry control device, 3...entry system, 4...vehicle, 5...object detection sensor, 5a...right object detection sensor, 5b...left object detection sensor, 5c...rear object detection sensor, 6...camera, 6a...right camera, 6b...left camera, 6c...rear camera, 8...door, 8a...driver's door, 8b...right rear door, 8c...passenger's door, 8d...left rear door, 8e...back door, 9...door opening / closing device, 9a...right front door opener, 9b...right rear door opener, 9c...left front door opener, 9d...left rear door opener, 9e...back door opener, 20...first processor, 21...first memory, 22...first program, 23...detection unit, 24...acquisition unit, 25...person recognition unit, 26...judgment unit, 30...second processor, 31...second memory, 32...second program, 33...authentication data, 34...authentication unit, 35...control unit.

Claims

1. an acquisition unit that acquires captured images from a camera mounted on the vehicle; a person recognition unit that recognizes people in the captured image; a determination unit that determines whether a person appearing in the captured image is a target person and determines whether the target person is carrying luggage from the captured image; Equipped with The determination unit extracting a partial image of a predetermined shape and size including the subject's hand captured in the captured image from the captured image; When an image pattern different from the background image of the partial image is recognized at the tip of the hand of the partial image, it is determined that the subject is carrying luggage. Image recognition system.

2. The determination unit When an image pattern different from the background image of the partial image is not recognized at the tip of the hand of the partial image, it is determined that the subject is not carrying any luggage. The image recognition system according to claim 1 .

3. the background image is an image of a portion of the captured image other than the captured image from which the partial image is extracted, the portion corresponding to the partial image; The image recognition system according to claim 1 .

4. The other captured image is an image captured by the acquisition unit when the vehicle is stopped, an image captured when an object approaches within a predetermined distance from the vehicle, or an image captured before the determination unit determines whether the subject is carrying luggage. The image recognition system according to claim 3 .

5. An image recognition method executed by a computer of an image recognition system, comprising: an acquisition step of acquiring a captured image from a camera mounted on the vehicle; a recognition step of recognizing a person appearing in the captured image; a determination step of determining whether a person appearing in the captured image is a target person and whether the target person is carrying luggage from the captured image; and In the determining step, extracting a partial image of a predetermined shape and size including the subject's hand captured in the captured image from the captured image; When an image pattern different from the background image of the partial image is recognized at the tip of the hand of the partial image, it is determined that the subject is carrying luggage. Image recognition methods.

6. The image recognition system according to any one of claims 1 to 4; a control unit that automatically opens and closes the vehicle door; Equipped with the control unit automatically opens the door closest to the target person when the determination unit of the image recognition system determines that the target person is carrying luggage. Entry system.

7. the control unit automatically opens the door closest to the target person when the determination unit of the image recognition system determines that the target person has luggage in each of their left and right hands.

7. The entry system according to claim 6.

Images