Determination device, determination method, and computer program

Abstract

To detect with higher accuracy the presence of an article left behind in a vehicle before passengers get off. [Solution] A judgment device comprising: a judgment unit that detects an object in an interior image, which is an image of the inside of the vehicle being judged, judges whether a person in the vehicle has disembarked, and judges that an item has been left behind if an object is detected at a position more than a predetermined distance away from the person and the person has disembarked; and an information control unit that produces a predetermined output from an output unit when the judgment unit judges that an item has been left behind.

Description

[Technical Field]

[0001] The present invention relates to a determination device, a determination method, and a computer program. [Background technology]

[0002] In passenger automobile transportation businesses such as taxi businesses and hire cars, there has been a problem of passengers leaving items inside the vehicle. To address this problem, cameras are installed inside passenger automobiles, and the images captured are analyzed to detect items left behind (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-41344 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional lost item detection technology only detects lost items after all passengers have disembarked, and even if a lost item is detected, it can be difficult to return it to the owner. On the other hand, if lost items are detected at an earlier stage, they may be mistakenly detected as lost, resulting in excessive warnings to owners who did not forget their lost items in the first place, which could reduce the quality of customer service. The present invention has been made in consideration of the above-mentioned circumstances, and provides a technology that makes it possible to detect with higher accuracy the presence of items left behind on board a vehicle before all passengers have disembarked. [Means for solving the problem]

[0005] One aspect of the present invention is a determination device that includes a determination unit that detects an object in an interior image, which is an image of the inside of the vehicle being determined, determines whether a person in the vehicle has alighted, and determines that an item has been left behind if an object is detected at a position more than a predetermined distance away from the person and the person has alighted, and an information control unit that produces a predetermined output from an output unit if the determination unit determines that an item has been left behind.

[0006] One aspect of the present invention is the above-mentioned determination device, wherein the determination unit determines whether a person in the vehicle to be determined has dismounted using a trained model obtained by performing a learning process using an image of a person who has dismounted as a teacher image.

[0007] One aspect of the present invention is the above-mentioned determination device, wherein the determination unit determines whether the person in the vehicle being determined has disembarked based on the area of ​​the region detected as the person's body part.

[0008] One aspect of the present invention is a determination method having a determination step of detecting an object in an interior image, which is an image of the inside of the vehicle to be determined, determining whether a person in the vehicle has alighted, and determining that an item has been left behind if an object is detected at a position more than a predetermined distance away from the person and the person has alighted, and an information control step of producing a predetermined output from an output unit if it is determined in the determination step that an item has been left behind.

[0009] One aspect of the present invention is a computer program for causing a computer to function as a judgment device that includes a judgment unit that detects an object in an interior image of the vehicle being judged, judges whether a person in the vehicle has alighted, and judges that an item has been left behind if an object is detected at a position more than a predetermined distance away from the person and the person has alighted, and an information control unit that produces a predetermined output from an output unit if the judgment unit judges that an item has been left behind. [Effects of the Invention]

[0010] The present invention makes it possible to detect with higher accuracy the presence of items left behind inside a vehicle before all passengers have disembarked. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic block diagram showing the system configuration of a determination system 100 according to the present invention. [Figure 2] 10A and 10B are diagrams showing specific examples of images used in the determination process of the determination device 20. FIG. [Figure 3] 10A and 10B are diagrams showing specific examples of images of the interior of a vehicle used in the determination process of the determination device 20. FIG. [Figure 4] 10A and 10B are diagrams showing specific examples of images of the interior of a vehicle used in the determination process of the determination device 20. FIG. [Figure 5] 10A and 10B are diagrams showing specific examples of images of the interior of a vehicle used in the determination process of the determination device 20. FIG. [Figure 6] 1 is a schematic block diagram showing a specific example of the functional configuration of a learning device 10. FIG. [Figure 7] 2 is a schematic block diagram showing a specific example of the functional configuration of a determination device 20. FIG. [Figure 8] FIG. 2 is a diagram illustrating an outline of an example of the hardware configuration of an information processing device 90 applied to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] FIG. 1 is a schematic block diagram showing the system configuration of a determination system 100 of the present invention. The determination system 100 includes a learning device 10 and a determination device 20. The learning device 10 generates a trained model used in the determination device 20. The determination device 20 is provided in a vehicle 80. The determination device 20 stores the trained model generated by the learning device 10 and performs a determination process using an image obtained by capturing an image of the interior of the vehicle 80 and the trained model. The determination device 20 stores multiple types of trained models, including models for detecting objects in an image, models for detecting people in an image, and models for determining whether a person in an image has alighted from the vehicle. By performing the determination process, the determination device 20 determines whether a predetermined condition (hereinafter referred to as a "lost item condition") indicating a high possibility that a passenger in the vehicle 80 has left something behind is met, and if the lost item condition is met, it issues a predetermined output.

[0013] Fig. 2 is a diagram showing a specific example of an image used in the determination process of the determination device 20. The image shown in Fig. 2 is an image (hereinafter referred to as an "in-vehicle image") obtained by capturing an image of the interior of the vehicle 80 to be determined using a camera installed near the ceiling in front of the driver's seat with its lens facing backward. The vehicle 80 to be determined is a vehicle 80 in which the determination device 20 is installed. The in-vehicle image shown in Fig. 2 shows a driver's seat 81, a passenger seat 82, and a rear seat 83 as equipment inside the vehicle 80.

[0014] FIG. 3 is a diagram showing a specific example of an interior image used in the determination process of the determination device 20. The interior image shown in FIG. 3 is an interior image taken by the same camera as the interior image shown in FIG. 2. In the interior image of FIG. 3, a driver is sitting in the driver's seat 81 and driving the vehicle 80. One passenger 85 is sitting in the back seat 83. An object 84 is placed on the seat of the back seat 83 next to the passenger 85. The object 84 is, for example, a mobile phone.

[0015] Fig. 4 is a diagram showing a specific example of an interior image used in the determination process of the determination device 20. The interior image shown in Fig. 4 is an interior image taken by the same camera as the interior image shown in Fig. 2. In the interior image of Fig. 4, a driver is sitting in the driver's seat 81, but is not holding the steering wheel and is not driving the vehicle 80. In the back seat 83, one passenger 85 is looking into his bag. An object 84 is placed on the seat of the back seat 83 next to him.

[0016] Fig. 5 is a diagram showing a specific example of an interior image used in the determination process of the determination device 20. The interior image shown in Fig. 5 is an interior image taken by the same camera as the interior image shown in Fig. 2. In the interior image of Fig. 5, a driver is sitting in the driver's seat 81, but is not holding the steering wheel and is not driving the vehicle 80. In the back seat 83, a passenger 85 is facing the door and preparing to get out of the vehicle 80. An object 84 is placed on the seat of the back seat 83 next to him.

[0017] The determination device 20 processes vehicle interior images such as those shown in Figs. 2 to 5. The determination device 20 may be configured to perform the determination process only when the vehicle in which the device (determination device 20) is installed is stopped, for example. In this case, the determination process is not performed while the driver in the driver's seat 81 is driving the vehicle 80 as shown in Fig. 3, but is performed while the driver has stopped the vehicle 80 as shown in Figs. 4 and 5. The determination process mainly involves object detection and a dismounting state determination. If an object is detected by object detection and it is determined that the driver is dismounting, it is determined that the lost-item condition is met.

[0018] The determination device 20 continuously performs object detection in a specific area (for example, the area of ​​the rear seat 83) of the captured image of the interior of the vehicle. The object detected by the object detection may be an unspecified object or a specific type of object. In FIGS. 4 and 5, a mobile phone is placed on the seat of the rear seat 83, and therefore the mobile phone is detected as the object 84.

[0019] The determination device 20 detects a person (passenger 85) in the captured image of the interior of the vehicle and determines whether the detected person is in an alighting state. The alighting state indicates that the person is about to alight. The determination device 20 may determine whether the person is in an alighting state, for example, using a trained model obtained by performing a learning process using an image of a person in an alighting state as a training image. In the cases of FIGS. 3 and 4, the passenger 85 is not about to alight in either case, so it is not determined that the person is in an alighting state. On the other hand, in the case of FIG. 5, the passenger 85 is about to alight and is determined to be in an alighting state. In the case of FIG. 5, an object is detected and it is determined that the person is in an alighting state, so it is determined that the lost-item condition is met. In this case, the determination device 20 issues a predetermined output to notify the driver or passenger 85 that there is a left-behind item.

[0020] This operation makes it possible to prevent erroneous determinations such as determining that an item has been left behind even when the vehicle 80 is not moving and the object 84 has been detected. In addition, since it is determined that an item has been left behind when the passenger 85 gets off the vehicle, it is determined that an item has been left behind when the passenger 85 is still inside the vehicle 80 or is about to get off. Therefore, it is possible to detect the presence of an item left behind inside the vehicle before the passenger has completely gotten off.

[0021] Such a determination system 100 will be described in detail below. Fig. 6 is a schematic block diagram showing a specific example of the functional configuration of a learning device 10. The learning device 10 is configured using an information processing device such as a personal computer or a server device. The learning device 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0022] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 communicates data with other devices via a network in accordance with the control of the control unit 13. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.

[0023] The storage unit 12 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 12 stores data used by the control unit 13. The storage unit 12 may function as, for example, a teacher data storage unit 121 and a trained model storage unit 122.

[0024] The teacher data storage unit 121 stores teacher data used in the learning process executed in the learning device 10. The teacher data storage unit 121 may store first teacher data, second teacher data, and third teacher data.

[0025] The first training data is an image of an object to be determined as a lost item. The first training data may be configured using images of an unspecified number of types of objects, or may be configured using images of a specific type of object. When the first training data is configured using images of a specific type of object, the first training data may be configured using, for example, only images of objects that are frequently turned in as lost items.

[0026] Specific examples of such objects include mobile phones (smartphones), wallets, keys, bags, headphones, glasses, and umbrellas. Narrowing down the types of objects to be detected in advance in this way enables more accurate object detection. In this case, to reduce the adverse effects of narrowing down the types of objects, only objects that are frequently left behind in the vehicle 80 are used as detection targets. The types of objects described above are merely specific examples. It is thought that the things that are frequently left behind in the vehicle 80 change depending on the characteristics of the era, region, etc. Therefore, the types of objects to be detected may be selected appropriately depending on the characteristics of the era, region, etc.

[0027] The second training data is an image of a person to be determined as a passenger 85. The second training data may be an image taken inside a vehicle, or an image taken at an unspecified number of locations. The use of an image taken inside a vehicle can improve detection accuracy.

[0028] The third training data is an image of a passenger 85 in the dismounting state. The third training data may be generated, for example, by capturing an image of multiple people consciously assuming a posture to dismount. The third training data may be configured, for example, using a frame image captured in a video captured when a passenger actually dismounts from the vehicle 80, taken a predetermined time before the passenger disappears from the video. The time span to go back may be defined as multiple lengths. For example, images taken 1 second, 2 seconds, and 3 seconds before may be used as the third training data.

[0029] The trained model storage unit 122 stores a trained model obtained by a learning process using the training data stored in the training data storage unit 121.

[0030] The control unit 13 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The control unit 13 functions as an information control unit 131 and a learning control unit 132 by the processor executing a program. Note that all or part of the functions of the control unit 13 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor storage devices (e.g., SSDs: Solid State Drives), as well as storage devices such as hard disks and semiconductor storage devices built into computer systems. The above programs may be transmitted via telecommunications lines.

[0031] The information control unit 131 controls the input and output of information. For example, the information control unit 131 acquires training data from other devices (information processing devices or storage media) and records it in the training data storage unit 121. For example, the information control unit 131 transmits the trained model stored in the trained model storage unit 122 to another device (for example, the determination device 20).

[0032] The learning control unit 132 executes a learning process using the training data stored in the training data storage unit 121. Specific examples of such learning processes include supervised learning for classification, such as support vector machines, random forests, and neural networks. The learning control unit 132 generates a first trained model for detecting objects in an input image by performing supervised learning using, for example, first training data. The learning control unit 132 generates a second trained model for detecting people in an input image by performing supervised learning using, for example, second training data. The learning control unit 132 generates a third trained model for determining whether a person in an input image has exited a vehicle by performing supervised learning using, for example, third training data. The learning control unit 132 records the generated trained model in the trained model storage unit 122. The trained model obtained by the learning control unit 132 may be transmitted to the determination device 20 and recorded in the storage unit 25 of the determination device 20.

[0033] 7 is a schematic block diagram showing a specific example of the functional configuration of the determination device 20. The determination device 20 is configured using an information device such as a smartphone, a tablet, or a dedicated device. The determination device 20 includes a communication unit 21, an input unit 22, an output unit 23, an image input unit 24, a storage unit 25, and a control unit 26.

[0034] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 communicates data with other devices via a network in accordance with the control of the control unit 26. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication. The communication unit 21 may acquire information about the state of the vehicle 80, for example, by communicating with a control device of the vehicle 80 in which the communication unit 21 (the determination device 20) is installed. A specific example of the information about the state of the vehicle 80 is information indicating whether the vehicle 80 is in a running state or a stopped state. A specific example of the information about the state of the vehicle 80 may further include information indicating a disembarking state. The disembarking state may be, for example, a state in which a passenger vehicle has finished renting and is waiting for passengers to disembark. The communication unit 21 may communicate with an information processing device located physically distant from the vehicle 80 by communicating via the Internet or a mobile communication network.

[0035] The input unit 22 is configured using existing input devices such as a keyboard, a pointing device (such as a mouse or tablet), buttons, or a touch panel. The input unit 22 is operated by a user (e.g., a driver) when inputting the user's instructions to the determination device 20. The input unit 22 may be an interface for connecting the input device to the determination device 20. In this case, the input unit 22 inputs an input signal generated in the input device in response to the user's input to the determination device 20. The input unit 22 may be configured using a microphone and a voice recognition device. In this case, the input unit 22 acquires an acoustic signal generated by the user's speech, performs voice recognition on the words spoken by the user, and inputs character string information of the recognition result to the determination device 20. The voice recognition process may be performed by the control unit 26. The input unit 22 may be configured in any way as long as it is capable of inputting user's instructions to the determination device 20.

[0036] The output unit 23 outputs information in a form that can be recognized by a user (e.g., a driver or a passenger). The output unit 23 may be an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 23 may be an interface for connecting an image display device to the determination device 20. In this case, the output unit 23 generates a video signal for displaying image data and outputs the video signal to an image display device connected to the output unit 23. The output unit 23 may be a device for outputting sound, such as a speaker. The output unit 23 may be an interface for connecting an audio output device, such as a speaker, to the determination device 20. In this case, the output unit 23 generates an audio signal for reproducing audio data and outputs the audio signal to an audio output device connected to the output unit 23. The output unit 23 may be configured as a touch panel integrated with the input unit 22.

[0037] The image input unit 24 accepts image data to be input to the determination device 20. The image input by the image input unit 24 is an interior image of the vehicle 80 in which the determination device 20 is installed. The image input unit 24 may acquire interior images captured by, for example, a still camera or video camera installed in the vehicle 80. When the interior images are acquired from the camera, wired communication via a communication cable such as a USB cable or a LAN cable, or wireless communication such as wireless LAN or Bluetooth (registered trademark), may be performed. The image input unit 24 may be configured as an imaging device such as a still camera or a video camera. The image input unit 24 may be configured in a different manner as long as it is capable of receiving input data of interior images. The camera used in the image input unit 24 may be, for example, a camera that captures visible light images, or a full-color camera that can be used in low-light conditions such as at night or under a fog gate.

[0038] The storage unit 25 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 25 stores data used by the control unit 26. The storage unit 25 stores data required when the control unit 26 performs processing. The storage unit 25 functions as, for example, an object determination model storage unit 251, a person determination model storage unit 252, and an off-board state determination model storage unit 253.

[0039] The object determination model storage unit 251 stores an object determination model. The object determination model is a determination model for detecting an object from an image. The object determination model may be, for example, a first trained model generated by the learning device 10. The object determination model may be, for example, an image pattern used when detecting an object from an image by pattern matching.

[0040] The person determination model storage unit 252 stores a person determination model. The person determination model is a determination model for detecting a person from an image. The person determination model may be, for example, a second trained model generated by the learning device 10. The person determination model may be, for example, an image pattern used when detecting a person from an image by pattern matching.

[0041] The alighting state determination model storage unit 253 stores an alighting state determination model. The alighting state determination model is a determination model for determining whether a person in an image is in an alighting state. The alighting state determination model may be configured as a determination model for detecting a person in an alighting state from an image. The person determination model may be, for example, a third trained model generated by the learning device 10.

[0042] The control unit 26 is configured using a processor such as a CPU and a memory (main storage device). The control unit 13 functions when the processor executes a program. All or part of the functions of the control unit 13 may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor storage devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor storage devices built into a computer system. The above program may be transmitted via a telecommunications line.

[0043] The control unit 26 may execute, for example, an application installed in its own device (the determination device 20). A specific example of such an application is an application provided to the determination device 20 as a dedicated application for the determination system 100. It is desirable that such an application be installed in the determination device 20 in advance.

[0044] The control unit 26 is configured using a processor such as a CPU and a memory. The control unit 26 functions as an information control unit 261 and a determination unit 262 by the processor executing a program. All or part of the functions of the control unit 26 may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., an SSD), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0045] The information control unit 261 transmits and receives data to and from other devices via the communication unit 21. The information control unit 261 acquires images of the interior of the vehicle from the image input unit 24. The information control unit 261 performs a predetermined output via the output unit 23, for example, depending on the determination result of the determination unit 262. The predetermined output may be, for example, displaying text or an image on an image display device indicating that an item has been left behind, or outputting a voice or warning sound on an audio output device indicating that an item has been left behind, or may be an output of another type. The predetermined output may be configured in any way as long as it is an output that notifies the driver or passengers that an item has been left behind.

[0046] The determination unit 262 determines whether the condition for lost property is satisfied in the vehicle interior image. For example, the determination unit 262 may perform the determination process as follows: The determination unit 262 repeatedly acquires vehicle interior images from the image input unit 24 at a predetermined timing. The predetermined timing may be determined as a cycle such as every 0.5 seconds, every 1 second, or every 2 seconds.

[0047] The determination unit 262 detects an object in the acquired vehicle interior image using an object determination model. At this time, the determination unit 262 may detect the object in a specific area of ​​the vehicle interior image. The specific area is an area where passengers are expected to be seated. Specific examples of such a specific area include the area of ​​the rear seat 83 and the area of ​​the passenger seat. For example, when the camera parameters (angle of view, viewpoint position, etc.) of the camera that captures the vehicle interior image are fixed, the specific area may be defined as a predetermined area represented by the image coordinate system of the captured vehicle interior image. The determination unit 262 may detect only a specific type of object that is frequently turned in as lost property, rather than an unspecified number of objects.

[0048] The determination unit 262 detects people in the acquired vehicle interior image using the person determination model. At this time, the determination unit 262 may detect people in a specific area where passengers are expected to be seated.

[0049] The determination unit 262 determines whether a person detected in the acquired in-vehicle image is in an alighting state using an alighting state determination model. If an object is detected and the person is in an alighting state, the determination unit 262 determines that the lost-item condition is met. If the determination unit 262 determines that the lost-item condition is met, the information control unit 261 controls the output unit 23 to perform a predetermined output. The determination unit 262 may determine that the lost-item condition is met if an object is detected at a position at least a predetermined distance away from the detected person and the person is in an alighting state.

[0050] The determination unit 262 may perform the determination process using the vehicle interior image only when the vehicle is stopped. The determination unit 262 may perform the determination process using the vehicle interior image only when the vehicle is in an alighting state.

[0051] The determination system 100 configured in this way determines that an item has been left behind only if the person has alighted from the vehicle, which makes it possible to more accurately detect the presence of an item left behind inside the vehicle before the passenger has completely alighted.

[0052] FIG. 8 is a diagram illustrating an outline of an example of the hardware configuration of an information processing device 90 applied to this embodiment. The information processing device 90 includes a processor 91, a main memory device 92, a communication interface 93, an auxiliary memory device 94, an input / output interface 95, and an internal bus 96. The processor 91, the main memory device 92, the communication interface 93, the auxiliary memory device 94, and the input / output interface 95 are communicably connected to each other via the internal bus 96. The information processing device 90 may be applied to, for example, the learning device 10 and the determination device 20. In this case, for example, the communication units 11 and 21 may be configured using the communication interface 93. For example, the memory units 12 and 25 may be configured using the auxiliary memory device 94. Furthermore, the control units 13 and 26 may be configured using the processor 91 and the main memory device 92.

[0053] (Variation) Learning device 10 may be implemented using multiple information processing devices. For example, learning device 10 may be implemented using a device such as a cloud. For example, in learning device 10, memory unit 12 and control unit 13 may be implemented in different information processing devices. For example, memory unit 12 of learning device 10 may be distributed and implemented in multiple information processing devices. For example, control unit 13 of learning device 10 may be distributed and implemented in multiple information processing devices.

[0054] The determination device 20 may be implemented using multiple information processing devices. For example, part of the processing of the determination device 20 may be performed by a device such as a cloud. More specifically, the storage unit 25 and the determination unit 262 may be implemented on a cloud or another information processing device. In this case, the information control unit 261 of the determination device 20 may transmit the vehicle interior image acquired from the image input unit 24 to the cloud via the communication unit 21. The cloud operates as the determination unit 262 to determine whether the received vehicle interior image satisfies the lost-item condition. The determination unit 262 transmits the determination result to the determination device 20. If the determination result indicates that the lost-item condition is satisfied, the information control unit 261 of the determination device 20 performs a predetermined output from the output unit 23. In this configuration, the information control unit 261 may transmit the vehicle interior image to the cloud only when the vehicle is stopped, or may transmit the vehicle interior image to the cloud only when the vehicle is in an alighted state.

[0055] The determination unit 262 may determine the alighting state without detecting a person using a person determination model. More specifically, the determination unit 262 may be configured to detect a person in the alighting state in the entire in-vehicle image or a specific area thereof.

[0056] The determination unit 262 may determine the dismounting state without using a trained model obtained by a learning process using an image of a person in the dismounting state as a teacher image. For example, the determination unit 262 detects the torso of a person in the in-vehicle image and obtains a value representing the size of the area of ​​the detected torso of the person (hereinafter referred to as "torso size"). Any existing technology may be applied to detect the torso of a person. For example, the person determination model stored in the person determination model storage unit 252 may be configured using a determination model for detecting the torso of a person. For example, a value representing the number of pixels in an area divided by performing segmentation of the torso may be obtained as the torso size. The torso size may be compared with statistical values ​​(e.g., maximum, average, or mode) of the torso size obtained when the vehicle 80 is in a traveling state, and it may be determined that the passenger has dismounted if a predetermined condition indicating that the torso size has become smaller when the vehicle 80 is in a stopped state or a dismounting state is satisfied. Specific examples of such conditions may be defined as a difference in torso size exceeding a threshold, or a ratio of torso sizes exceeding a threshold. Although the above description uses torso size, it does not have to be limited to the torso as long as it is a human body part.

[0057] When multiple people are detected as passengers, the lost-item condition may be defined as the condition that all of the detected people are determined to have alighted. In this case, the distance between the detected people and the object may be determined based on the shortest distance. In other words, the lost-item condition may be determined to be met when an object is detected at a position that is a predetermined distance or more away from all of the detected people and all of the people are alighting.

[0058] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0059] 100...Determination system, 10...Learning device, 11...Communication unit, 12...Memory unit, 121...Teacher data memory unit, 122...Learned model memory unit, 13...Control unit, 131...Information control unit, 132...Learning control unit, 20...Determination device, 21...Communication unit, 22...Input unit, 23...Output unit, 24...Image input unit, 25...Memory unit, 251...Object determination model memory unit, 252...Person determination model memory unit, 253...Get-off state determination model memory unit, 26...Control unit, 261...Information control unit, 262...Determination unit

Claims

1. a determination unit that detects an object in an interior image of a vehicle to be determined, determines whether a person in the vehicle has exited the vehicle, and determines that an item has been left behind if an object is detected at a position that is at least a predetermined distance away from the person and the person has exited the vehicle; an information control unit that outputs a predetermined output from an output unit when the determination unit determines that an item has been left behind; Equipped with the determination unit determines whether the person in the target vehicle has exited the vehicle based on an area of ​​the region detected as the person's body part; The determination unit is a determination device that determines that the person has dismounted when a predetermined condition indicating that the area has become smaller is met compared to the statistical value of the area size obtained when the vehicle is in a traveling state.

2. 2. The determination device according to claim 1, wherein the determination unit determines whether a person in the vehicle to be determined is in an alighting state by using a trained model obtained by performing a learning process using an image of a person in an alighting state as a teacher image.

3. a determination step of detecting an object in an interior image of the vehicle to be determined, determining whether a person in the vehicle has exited the vehicle, and determining that an item has been left behind if an object is detected at a position at least a predetermined distance away from the person and the person has exited the vehicle; an information control step of outputting a predetermined output from an output unit when it is determined in the determination step that there is an item left behind; and In the determining step, it is determined whether or not the person in the vehicle to be determined has exited the vehicle based on an area of ​​the region detected as the body part of the person; In the determination step, a determination method is provided in which the person is determined to be in a dismounted state if a predetermined condition indicating that the area has become smaller is met with respect to the statistical value of the area size obtained when the vehicle is in a moving state.

4. a determination unit that detects an object in an interior image of a vehicle to be determined, determines whether a person in the vehicle has exited the vehicle, and determines that an item has been left behind if an object is detected at a position that is at least a predetermined distance away from the person and the person has exited the vehicle; an information control unit that outputs a predetermined output from an output unit when the determination unit determines that there is an item left behind; the determination unit determines whether the person in the target vehicle has exited the vehicle based on an area of ​​the region detected as the person's body part; The judgment unit is a computer program for causing a computer to function as a judgment device that determines that the person has dismounted from the vehicle when a predetermined condition indicating that the area has become smaller is met compared to the statistical value of the area size obtained when the vehicle is in a moving state.

Images