Payment system, in-vehicle system, method, and program

Abstract

This payment system includes an in-vehicle system installed in an automobile, and a payment device. The in-vehicle system generates a first captured image of a user sitting in a seat of the automobile, executes facial authentication processing using the generated first captured image of the user and an image of the face of the user, and determines whether or not the facial authentication of the user was successful. Next, when it is determined that the facial authentication of the user was successful, the in-vehicle system generates a second captured image of the seat, executes person detection processing for detecting a person in the second captured image, and determines, on the basis of the result of the person detection processing, whether or not the person in the second captured image was detected. The in-vehicle system maintains the authentication state of the user when it is determined that the person in the second captured image was detected, and cancels the authentication state of the user when it is determined that the person in the second captured image was not detected. When the authentication state of the user is maintained, the in-vehicle system executes payment processing for the user.

Description

Payment system, in-vehicle system, method, and program

[0001] The present disclosure relates to a payment system, an in-vehicle system, a method, and a program for enabling a vehicle user to make a payment.

[0002] Conventionally, various technologies have been proposed for allowing vehicle users to make payments. As an example of such a technology, the in-vehicle system disclosed in Patent Document 1 identifies an occupant by face authentication using a captured image generated by photographing the occupant located around or inside the vehicle, determines whether the identified occupant is a registered user of the vehicle, and if it is determined that the occupant is a registered user of the vehicle, permits transmission of a user ID corresponding to the occupant to an external device. Then, a store terminal executes payment processing with the person identified by the user ID as the payee.

[0003] International Publication No. 2023 / 047559

[0004] However, in the in-vehicle system disclosed in Patent Document 1, once face authentication is performed, there is a possibility that payment processing is performed with the person identified by the user ID (Identification) as the payee even if the occupant subsequently changes. For this reason, there has been a problem of low security.

[0005] One object of the present disclosure, in view of the above-described problem, is to provide a payment system, an in-vehicle system, a method, and a program that can improve convenience and security when a user riding in a vehicle makes a payment.

[0006] The payment system according to the first embodiment includes an in-vehicle system installed in a vehicle and a payment device, the in-vehicle system includes a shooting control unit that causes a shooting device that photographs the seats of the vehicle to perform a shooting process to generate a first captured image of a user sitting in a seat, a face recognition unit that performs face recognition processing using the generated first captured image of the user and an image of the user's face, and an authentication result determination unit that determines whether or not the user's face recognition was successful, the shooting control unit, if it determines that the user's face recognition was successful, causes the shooting device to perform a shooting process to generate a second captured image of the seat, the in-vehicle system further includes a person detection unit that performs a person detection process to detect a person in the second captured image, a detection determination unit that determines whether or not a person was detected in the second captured image based on the result of the person detection process, an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second captured image, and releases the user's authentication state if it is determined that a person was not detected in the second captured image, and a payment processing unit that performs payment processing for the user if the user's authentication state is maintained. The payment device includes a transmission unit that transmits billing amount information about the user to the in-vehicle system, and the payment processing unit performs payment processing based on the billing amount information.

[0007] The payment system according to the second embodiment includes an in-vehicle system installed in a vehicle and a service provision device, the in-vehicle system includes a shooting control unit that causes a shooting device that photographs the seats of the vehicle to perform a shooting process to generate a first captured image of a user sitting in a seat, a face recognition unit that performs face recognition processing using the generated first captured image of the user and an image of the user's face, and an authentication result determination unit that determines whether or not the user's face recognition was successful, the shooting control unit, if it determines that the user's face recognition was successful, causes the shooting device to perform a shooting process to generate a second captured image of the seat, the in-vehicle system further includes a person detection unit that performs a person detection process to detect a person in the second captured image, a detection determination unit that determines whether or not a person was detected in the second captured image based on the result of the person detection process, an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second captured image, and releases the user's authentication state if it is determined that a person was not detected in the second captured image, and a transmission unit that transmits an authentication state notification to the service provision device to notify the user of the user's authentication state, The service provider includes a payment processing unit that performs payment processing for users, a notification determination unit that determines an authentication status notification sent by the in-vehicle system, a transmission unit that transmits the content subject to the service to the in-vehicle system if it is determined that the authentication status notification sent by the in-vehicle system indicates that the user's authentication status is maintained, and a settlement processing unit that performs settlement processing to calculate the billing amount for the service, the transmission unit of the service provider transmits billing amount information indicating the calculated billing amount to the in-vehicle system, and the payment processing unit performs payment processing based on the billing amount information.

[0008] The in-vehicle system according to the third embodiment includes: an imaging control unit that causes an imaging device to perform imaging processing to photograph the seats of a vehicle and generates a first image of a user sitting in the seat; a face recognition unit that performs face recognition processing using the generated first image of the user and an image of the user's face; and an authentication result determination unit that determines whether or not the user's face recognition was successful. The imaging control unit, if it determines that the user's face recognition was successful, causes the imaging device to perform imaging processing to generate a second image of the seat and includes a person detection unit that performs person detection processing to detect a person in the second image; a detection determination unit that determines whether or not a person was detected in the second image based on the result of the person detection processing; an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second image, and releases the user's authentication state if it is determined that a person was not detected in the second image; and a payment processing unit that performs payment processing for the user if the user's authentication state is maintained.

[0009] The method according to the fourth aspect involves an in-vehicle system installed in a vehicle causing a camera that photographs the seats of the vehicle to perform a photographic process to generate a first photographic image of a user sitting in the seat; the in-vehicle system performing a facial recognition process using the generated first photographic image of the user and an image of the user's face; the in-vehicle system determining whether or not the facial recognition of the user was successful; if the in-vehicle system determines that the facial recognition of the user was successful, causing the camera to perform a photographic process to generate a second photographic image of the seat; the in-vehicle system performing a person detection process to detect a person in the second photographic image; the in-vehicle system determining whether or not a person was detected in the second photographic image based on the results of the person detection process; if the in-vehicle system determines that a person was detected in the second photographic image, it maintains the user's authentication state; if it determines that no person was detected in the second photographic image, it releases the user's authentication state; the payment device transmits billing amount information regarding the user to the in-vehicle system; and if the user's authentication state is maintained, the in-vehicle system performs payment processing based on the billing amount information.

[0010] The method relating to the fifth aspect is as follows: An in-vehicle system installed in a vehicle causes a camera that photographs the seats of the vehicle to perform a shooting process to generate a first captured image of a user sitting in the seat; the in-vehicle system performs a facial recognition process using the generated first captured image of the user and an image of the user's face; the in-vehicle system determines whether or not the user's facial recognition was successful; if the in-vehicle system determines that the user's facial recognition was successful, it causes the camera to perform a shooting process to generate a second captured image of the seat; the in-vehicle system performs a person detection process to detect a person in the second captured image; based on the result of the person detection process, the in-vehicle system determines whether or not a person was detected in the second captured image; if the in-vehicle system determines that a person was detected in the second captured image, it maintains the user's authentication state; if it determines that no person was detected in the second captured image, it releases the user's authentication state; the in-vehicle system transmits an authentication status notification to a service provider notifying the user of the user's authentication state; the service provider evaluates the authentication status notification transmitted by the in-vehicle system. If the service provider determines that the authentication status notification sent by the in-vehicle system indicates that the user's authentication status is maintained, it sends the content subject to the service to the in-vehicle system. The service provider then performs a settlement process to calculate the billing amount for the service. The service provider then sends billing information indicating the calculated billing amount to the in-vehicle system. The in-vehicle system then performs a payment process based on the billing information.

[0011] The program according to the fifth aspect causes a camera that photographs the seats of a car to perform a shooting process to generate a first image of the user sitting in the seat, to perform a face recognition process using the generated first image of the user and an image of the user's face, to determine whether or not the user's face recognition was successful, if it is determined that the user's face recognition was successful, to cause the camera to perform a shooting process to generate a second image of the seat, to perform a person detection process to detect a person in the second image, to determine whether or not a person was detected in the second image based on the result of the person detection process, if it is determined that a person was detected in the second image, to maintain the user's authentication state, if it is determined that no person was detected in the second image, to release the user's authentication state, and if the user's authentication state is maintained, to perform a payment process for the user.

[0012] This disclosure makes it possible to provide a payment system, in-vehicle system, method, and program that can improve the convenience and security of payments made by users riding in a vehicle.

[0013] This is a schematic diagram showing an example of the payment system related to this disclosure. This is a diagram showing an example of the hardware configuration of the in-vehicle system related to this disclosure. This is a diagram showing an example of the hardware configuration of the ECU (Electronic Control Unit) included in the in-vehicle system related to this disclosure. This is a diagram showing an example of the hardware configuration of the settlement device related to this disclosure. This is a diagram showing an example of the functions of the ECU related to this disclosure. This is a diagram showing an example of the user information table. This is a diagram showing an example of the functions of the settlement device related to this disclosure. This is a sequence diagram showing an example of the processing performed in the payment system related to this disclosure. This is a diagram showing an example of the processing performed by the in-vehicle system related to this disclosure. This is a diagram showing another example of the processing performed by the in-vehicle system related to this disclosure. This is a diagram showing an example of the main components of the payment system related to this disclosure. This is a diagram showing an example of the main processing performed in the payment system related to this disclosure. This is a diagram showing another example of the functions of the ECU of the in-vehicle system related to this disclosure. This is a flowchart showing another example of the processing performed by the in-vehicle system related to this disclosure. This is a diagram showing an example of the functions of the service provider related to this disclosure. This is a sequence diagram showing an example of the processing performed in the payment system including the service provider related to this disclosure. This is a diagram showing another example of the main components of the payment system related to this disclosure. This is a diagram showing another example of the main processing performed in the payment system related to this disclosure.

[0014] <First Embodiment> Hereinafter, an exemplary embodiment will be described with reference to the drawings. Figure 1 is a schematic diagram showing an example of a payment system 1 according to the present disclosure. The payment system 1 includes an in-vehicle system 11 installed in an automobile 10 and a payment device 20 included in a store system. The store system is a system introduced in commercial facilities such as drive-ins. The in-vehicle system 11 and the payment device 20 communicate data with each other via a network 40. The network 40 may be configured to include various networks such as a wireless LAN (Local Area Network). Although Figure 1 shows one automobile 10, the payment system 1 may include multiple automobiles 10.

[0015] Figure 2 shows an example of the hardware configuration of the in-vehicle system 11 according to this disclosure. The in-vehicle system 11 comprises an ECU 12, a camera 13, a display 14, and a communication interface (I / F) 15. The ECU 12 exchanges data with the camera 13, the display 14, and the communication interface 15 via a communication cable 16.

[0016] The ECU 12 controls the in-vehicle system 11. Details of the configuration and functions of the ECU 12 will be described later.

[0017] Camera 13 is a camera that photographs the interior of the automobile 10. Camera 13 is positioned to photograph people sitting in the seats of the automobile 10. The seats include the driver's seat, passenger seat, and rear seats.

[0018] The display 14 outputs various images and information under the control of the ECU 12. The display 14 corresponds to a notification unit. The communication interface (I / F) 15 is an interface that performs data communication with other devices via an antenna (not shown).

[0019] Figure 3 shows an example of the hardware configuration of an ECU 12 included in the in-vehicle system 11 according to this disclosure. The ECU 12 comprises a processor 120, a memory 121, a storage device 122, a communication interface (I / F) 123, and a bus 124. These devices exchange data via the bus 124.

[0020] The processor 120 is a device that performs overall control of the ECU 12. Specific examples of the processor 120 include processors such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and MPU (Micro Processing Unit). The processor 120 loads instructions contained in the program stored in the storage device 122 into the memory 121 and executes them. In other embodiments, at least a portion of the functions performed by the processor 120 may also be performed by integrated circuits such as FPGAs (Field-Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits). These devices are equivalent to computers. Furthermore, the ECU 12 can be equipped with multiple processors 120.

[0021] The storage device 122 is a storage device that stores various data, such as programs executed by the processor 120 and data processed by the processor 120. The communication interface 123 is an interface for data communication between the ECU 12 and other devices.

[0022] Figure 4 shows an example of the hardware configuration of the settlement device 20 according to this disclosure. The settlement device 20 comprises a processor 21, a memory 22, a storage device 23, a communication interface (I / F) 24, and a bus 25. These devices exchange data via the bus 25.

[0023] The processor 21 is a device that performs overall control of the settlement device 20. Specific examples of the processor 21 include CPUs, GPUs, MPUs, and other processors. The processor 21 loads instructions contained in the program stored in the storage device 23 into the memory 22 and executes them. In other embodiments, at least a portion of the functions performed by the processor 21 may also be performed by integrated circuits such as FPGAs and ASICs. These devices are equivalent to computers. Furthermore, the settlement device 20 may be equipped with multiple processors 21.

[0024] The storage device 23 is a storage device that stores various data, such as programs executed by the processor 21 and data processed by the processor 21. The communication interface 24 is an interface for data communication between the settlement device 20 and other devices.

[0025] Figure 5 shows an example of the functions of the ECU 12 according to this disclosure. The ECU 12 includes a receiving unit 100, a shooting control unit 101, a face recognition unit 102, an authentication result determination unit 103, a person detection unit 104, a detection determination unit 105, a location identification unit 106, a location determination unit 107, an authentication state control unit 108, a payment method acquisition unit 109, a display control unit 110, an instruction reception unit 111, a payment processing unit 112, and a transmission unit 113. These functions can be implemented by a program.

[0026] The receiving unit 100 receives various information from other devices. The shooting control unit 101 causes the camera 13 to perform shooting processing and generates images of the seats of the automobile 10. The captured images include a first image generated before the user is authenticated and a second image generated after the user is authenticated.

[0027] The facial recognition unit 102 performs facial recognition processing using the first captured image generated by the shooting control unit 101 and the user's face image. The facial recognition unit 102 refers to a user information table as shown in Figure 6, identifies the image identification information (image ID) associated with the user's identification information (user ID), and uses the user's face image indicated by the image ID for facial recognition processing. The user information table associates the user's user ID, the image ID of the user's face image, and payment method information indicating the payment methods that the user can select.

[0028] The facial recognition unit 102 can perform facial recognition processing using a facial recognition model, which is a machine learning model. When a pair of images showing a person's face is input to the facial recognition model, it calculates a score indicating the likelihood that the person in these images is the same person. The facial recognition model is trained to output a high score when a pair of images showing the same person's face is input. Conversely, the facial recognition model is trained to output a low score when a pair of images showing different people's faces is input. The face used for facial recognition may be a person's frontal face and / or profile. If a person is authenticated using only a person's frontal face, security can be enhanced. On the other hand, if a person is authenticated using either a person's frontal face or a profile, convenience can be enhanced.

[0029] The authentication result determination unit 103 determines whether or not facial recognition by the facial recognition process was successful. Specifically, the authentication result determination unit 103 determines that facial recognition was successful if the score output by the facial recognition model is equal to or greater than a predetermined threshold. On the other hand, if the score output by the facial recognition model is less than a predetermined threshold, the authentication result determination unit 103 determines that facial recognition failed. The predetermined threshold can be any value.

[0030] The person detection unit 104 performs a person detection process using the second captured image. In the person detection process, the person detection unit 104 detects a person based on the silhouette of the person in the second captured image.

[0031] The detection and determination unit 105 determines whether or not a person has been detected in the second captured image based on the results of the person detection process.

[0032] The position identification unit 106 identifies the position of the detected person in the second captured image through person detection processing. Specifically, the position identification unit 106 identifies the image region in the second captured image defined by the silhouette of the detected person.

[0033] The position determination unit 107 determines whether the position of the identified person is within a predetermined range in the second captured image. Specifically, the position determination unit 107 determines whether the image area defined by the detected person's silhouette is within a predetermined range in the second captured image. The predetermined range is the image area in the second captured image corresponding to the seat position. If the authenticated user moves from the seat of the car 10, the image area defined by the detected person's silhouette will move out of the predetermined range. Therefore, the position determination unit 107 can detect that the authenticated user has moved from the seat of the car 10.

[0034] The authentication state control unit 108 controls the user's authentication state. Specifically, when the authentication state control unit 108 determines that a person has been detected in the second captured image, and that the location of the identified person is within a predetermined range within the second captured image, it maintains the user's authentication state. On the other hand, when the authentication state control unit 108 determines that no person has been detected in the second captured image, it deactivates the user's authentication state. Furthermore, when the authentication state control unit 108 determines that the location of the identified person is not within a predetermined range within the second captured image, it deactivates the user's authentication state.

[0035] The payment method acquisition unit 109 acquires payment method information indicating the payment method associated with a user whose authentication status is maintained. More specifically, the payment method acquisition unit 109 refers to a user information table as shown in Figure 6 and acquires payment method information indicating the payment method associated with the user ID of the user in question.

[0036] The display control unit 110 displays various images and / or information on the display 14.

[0037] The instruction receiving unit 111 determines whether or not it has received a payment instruction from the user. The instruction receiving unit 111 can receive payment instructions entered by the user using an input device such as a console panel or a touch panel type display 14 installed in the automobile 10.

[0038] The payment processing unit 112 executes the payment process based on the payment method specified by the user, provided that the user's authentication status is maintained.

[0039] The transmitting unit 113 transmits various types of information to other devices.

[0040] FIG. 7 is a diagram illustrating an example of functions of the settlement apparatus 20 according to the present disclosure. The settlement apparatus 20 includes a receiving unit 200, a settlement processing unit 201, and a transmitting unit 202. These functions can be implemented by a program.

[0041] The receiving unit 200 receives various types of information from other devices. The settlement processing unit 201 executes settlement processing for purchase of goods and / or provision of services related to a user. In the settlement processing, the settlement processing unit 201 generates charge amount information indicating a charge amount for the purchase of the goods and / or the provision of the service.

[0042] The transmitting unit 202 transmits various types of information to other devices.

[0043] FIG. 8 is a sequence diagram illustrating an example of processing executed in the payment system 1. In step S1, the processing of FIG. 9 is executed. The processing of FIG. 9 can be executed at a timing when a user gets on the automobile 10, a timing when the automobile 10 enters a predetermined area in a commercial facility, or a timing when the in-vehicle system 11 receives charge amount information from the settlement apparatus 20.

[0044] FIG. 9 is a diagram illustrating an example of processing executed by the in-vehicle system 11. In step S10, an imaging control unit 101 of the in-vehicle system 11 causes a camera 13 to execute imaging processing, and generates a first captured image of the user's face. In step S11, a face authentication unit 102 of the in-vehicle system 11 executes face authentication processing using the generated first captured image and a user's face image registered in a storage device 122 of the in-vehicle system 11.

[0045] In step S12, the authentication result determination unit 103 of the in-vehicle system 11 determines whether or not the user's authentication was successful based on the result of the facial recognition process. If it is determined that the user's facial recognition failed (NO), the process returns to step S10. In other embodiments, if it is determined that the user's facial recognition has failed, the user may be authenticated by another authentication method, such as the user entering a passcode or password, instead of facial recognition.

[0046] On the other hand, if it is determined that the user's facial recognition was successful (YES), the process branches to step S13. In step S13, the shooting control unit 101 of the in-vehicle system 11 causes the camera 13 to perform shooting processing and generates a second captured image of the user. In step S14, the person detection unit 104 of the in-vehicle system 11 performs person detection processing using the second captured image generated in step S13. In step S15, the detection determination unit 105 determines whether or not a person has been detected in the second captured image by the person detection processing. If it is determined that no person has been detected in the second captured image (NO), the process branches to step S18. On the other hand, if it is determined that a person has been detected in the second captured image (YES), the process branches to step S16.

[0047] In step S16, the position identification unit 106 identifies the position of the detected person in the second captured image. In step S17, the position determination unit 107 determines whether the identified person's position is within a predetermined range in the second captured image. If it is determined that the identified person's position is within the predetermined range in the second captured image (YES), the process returns to step S13. In this case, the user's authentication state is maintained. When the process returns to step S13, the second captured image is generated again. Therefore, the second captured image is repeatedly generated, and the processes in steps S14 to S17 are executed for each second captured image. In this embodiment, the interval at which the processes in steps S13 to S17 are repeatedly executed can be a predetermined interval, such as 0.1 to 1 second.

[0048] On the other hand, if it is determined that the position of the identified person is not within a predetermined range in the second captured image (NO), the process branches to step S18. In step S18, the authentication state control unit 108 cancels the authentication state of the user, and the process of FIG. 9 ends.

[0049] In step S2 of FIG. 8, the settlement processing unit 201 of the settlement apparatus 20 executes settlement processing for product purchase and / or service provision related to the user. In step S3, the transmitting unit 202 of the settlement apparatus 20 transmits billing amount information indicating the amount calculated through the settlement processing to the in-vehicle system 11 of the vehicle 10.

[0050] When the in-vehicle system 11 receives the billing amount information transmitted from the settlement apparatus 20, in step S4, the display control unit 110 displays the billing amount indicated by the billing amount information and the payment method associated with the user whose authentication state is maintained on the display 14 of the in-vehicle system 11. The user can use the input device of the in-vehicle system 11 to specify a payment method and instruct payment.

[0051] When the instruction receiving unit 111 of the in-vehicle system 11 receives a payment instruction from the user, in step S5, the payment processing unit 112 of the in-vehicle system 11 executes payment processing based on the payment method instructed by the user. In step S6, the transmitting unit 113 of the in-vehicle system 11 transmits a payment completion notification to the settlement apparatus 20 of the store system. In step S7, the display control unit 110 of the in-vehicle system 11 displays the payment completion notification on the display 14 of the in-vehicle system 11.

[0052] FIG. 10 is a diagram showing another example of processing executed by the in-vehicle system 11. The processing shown in FIG. 10 is executed when the in-vehicle system 11 receives the billing amount information transmitted from the settlement apparatus 20.

[0053] In step S20, the payment method acquisition unit 109 of the in-vehicle system 11 acquires payment method information indicating the payment method associated with the user whose authentication state is maintained. In step S21, the display control unit 110 of the in-vehicle system 11 displays the received billing amount information and the payment method indicated by the acquired payment method information on the display 14 of the in-vehicle system 11.

[0054] In step S22, the instruction receiving unit 111 of the in-vehicle system 11 determines whether or not it has received a payment instruction from the user. If it is determined that no payment instruction has been received (NO), the process in step S22 is executed again. On the other hand, if it is determined that a payment instruction has been received (YES), in step S23, the payment processing unit 112 of the in-vehicle system 11 executes the payment processing according to the payment method specified by the user. In step S24, the transmission unit 113 of the in-vehicle system 11 sends a payment completion notification to the settlement device 20. In step S25, the display control unit 110 of the in-vehicle system 11 displays the payment completion notification on the display 14 of the in-vehicle system 11, and the process in Figure 10 is completed.

[0055] Figure 11 shows an example of the main components of the payment system 1 according to this disclosure. Figure 12 shows an example of the main processes performed in the payment system 1 according to this disclosure.

[0056] The shooting control unit 101 of the in-vehicle system 11 causes the shooting device 13, which photographs the seats of the automobile 10, to perform a shooting process to generate a first captured image of the user sitting in the seat (step S30). The face recognition unit 102 of the in-vehicle system 11 performs a face recognition process using the generated first captured image of the user and an image of the user's face (step S31). The authentication result determination unit 103 of the in-vehicle system 11 determines the result of the user's face recognition (step S32). If the shooting control unit 101 of the in-vehicle system 11 determines that the user's face recognition was successful, it causes the shooting device 13 to perform a shooting process to generate a second captured image of the seat (step S33). The person detection unit 104 of the in-vehicle system 11 performs a person detection process to detect a person in the second captured image (step S34). The detection determination unit 105 of the in-vehicle system 11 determines whether or not a person was detected in the second captured image based on the result of the person detection process (step S35). The authentication state control unit 108 of the in-vehicle system 11 maintains the user's authentication state if it determines that a person has been detected in the second captured image, and releases the user's authentication state if it determines that no person has been detected in the second captured image (step S36). The transmission unit 202 of the payment device 20 transmits billing amount information for the user to the in-vehicle system 11 (step S37). The payment processing unit 112 of the in-vehicle system 11 performs payment processing for the user based on the billing amount information transmitted by the payment device 20, if the user's authentication state is maintained (step S38).

[0057] By adopting this configuration, the in-vehicle system 11 performs payment processing for the user when the user's authentication status is maintained, thereby improving convenience and security when a user riding in a vehicle makes a payment.

[0058] Furthermore, in the above-described embodiment, the position identification unit 106 of the in-vehicle system 11 determines that a person has been detected in the second captured image, and identifies the position of the person in the second captured image. The position determination unit 107 of the in-vehicle system 11 determines whether the position of the identified person is within a predetermined range in the second captured image. The authentication state control unit 108 of the in-vehicle system 11 maintains the user's authentication state if it determines that the position of the identified person is not within the predetermined range in the second captured image, and releases the user's authentication state if it determines that the position of the identified person is not within the predetermined range in the second captured image.

[0059] By adopting this configuration, the user's authentication status is maintained if the detected person is in a designated location, such as a seat. On the other hand, if the detected person is not in a designated location, the user's authentication status is deactivated. Therefore, security can be improved when a user makes a payment while riding in a vehicle.

[0060] Furthermore, in the above-described embodiment, the person detection unit 104 of the in-vehicle system 11 detects a person based on the silhouette of the person in the second captured image during the person detection process. This reduces the processing load on the in-vehicle system 11 compared to a configuration in which facial recognition is performed again to control the authentication state.

[0061] Furthermore, in the embodiment described above, the notification unit 110 of the in-vehicle system 11 notifies the user of the payment method associated with that method. The payment processing unit 112 of the in-vehicle system 11 executes the payment process based on the payment method specified by the user. This improves the convenience for users riding in a vehicle when making payments.

[0062] <Second Embodiment> The second embodiment will now be described, focusing on the differences from the first embodiment. The second embodiment is an example of a payment system applicable to energy supply stations such as gas stations, charging stations, and hydrogen stations. Figure 13 shows another example of the functions of the ECU 12 of the in-vehicle system 11. The ECU 12 further includes an amount calculation unit 114 and a billing amount determination unit 115.

[0063] The amount calculation unit 114 calculates the amount of fuel or electricity supplied to the vehicle 10 based on the amount of fuel or electricity supplied to the vehicle and the unit price of fuel or electricity.

[0064] The billing amount determination unit 115 compares the amount calculated by the amount calculation unit 114 with the billing amount indicated by the billing amount information provided by the settlement device 20, and determines whether the billing amount is correct or not.

[0065] Figure 14 is a sequence diagram showing another example of the processing performed in the payment system 1. In step S40, the processing shown in Figure 9 is performed. In step S41, the settlement processing unit 201 of the settlement device 20 performs settlement processing for the purchase of fuel or energy supplied to the automobile 10. In step S42, the transmission unit 202 of the settlement device 20 transmits billing amount information indicating the amount calculated by the settlement processing and unit price information indicating the unit price of the fuel or energy to the in-vehicle system 11.

[0066] When the in-vehicle system 11 receives the billing amount information and unit price information transmitted by the payment device 20, in step S43, the amount calculation unit 114 of the in-vehicle system 11 calculates the amount of fuel or electricity supplied to the vehicle 10 based on the amount of fuel or electricity supplied to the vehicle 10 and the unit price of fuel or electricity indicated by the received unit price information. In step S44, the billing amount determination unit 115 compares the amount calculated by the amount calculation unit 114 with the billing amount indicated by the received billing amount information and determines whether the billing amount is correct or not. The following describes the case when the billing amount is determined to be correct.

[0067] In step S45, the display control unit 110 displays the billing amount indicated by the billing amount information and the payment method associated with the user whose authentication status is maintained on the display 14 of the in-vehicle system 11. When the instruction receiving unit 111 of the in-vehicle system 11 receives a payment instruction from the user, in step S46, the payment processing unit 112 of the in-vehicle system 11 executes the payment process based on the payment method instructed by the user. In step S47, the transmission unit 113 of the in-vehicle system 11 transmits a payment completion notification to the settlement device 20 of the store system. In step S48, the display control unit 110 of the in-vehicle system 11 displays the payment completion notification on the display 14 of the in-vehicle system 11.

[0068] Figure 15 is a flowchart showing another example of a process performed by the in-vehicle system 11. The process shown in Figure 15 is performed when the in-vehicle system 11 receives billing amount information and unit price information transmitted by the payment device 20.

[0069] In step S50, the amount calculation unit 114 calculates the increase in fuel or the percentage increase in electricity for the vehicle 10 based on the amount of fuel or electricity supplied to the vehicle 10 before and after the supply of fuel or electricity. In step S51, the amount calculation unit 114 calculates the amount of fuel or electricity supplied to the vehicle 10. More specifically, the amount calculation unit 114 calculates the amount of fuel supplied to the vehicle 10 based on the calculated increase in fuel and the unit price of fuel indicated by the unit price information. The amount calculation unit 114 also calculates the amount of electricity supplied to the vehicle 10 based on the calculated percentage increase in electricity, the unit price of electricity indicated by the unit price information, and the capacity of the supplied battery.

[0070] In step S52, the billing amount determination unit 115 compares the amount calculated by the amount calculation unit 114 with the billing amount indicated by the billing amount information provided by the settlement device 20, and determines whether the billing amount is correct or not. If it is determined that the billing amount is incorrect (NO), in step S53, the transmission unit 113 sends a notification to the settlement device 20 that there is an error in the billing amount. On the other hand, if it is determined that the billing amount is correct (YES), the process branches to step S54. Steps S54 to S59 are the same as steps S20 to S25 in Figure 10.

[0071] As described above, the amount calculation unit 114 calculates the amount of fuel or electricity supplied to the vehicle 10 based on the amount of fuel or electricity supplied to the vehicle 10 and the unit price of fuel or electricity. The billing amount determination unit 115 compares the amount calculated by the amount calculation unit 114 with the billing amount indicated by the billing amount information transmitted by the settlement device 20 and determines whether the billing amount is correct. This makes it possible to verify the validity of the billing amount calculated by the settlement device 20.

[0072] <Third Embodiment> The third embodiment will now be described, focusing on the differences from the first embodiment. In the third embodiment, the payment system 1 may include a service provision device 30 instead of the settlement device 20. The service provision device 30 has a hardware configuration similar to that of the settlement device 20.

[0073] Figure 16 shows an example of the functions of the service provider 30. Specific examples of services provided by the service provider 30 include video streaming services and music streaming services. The service provider 30 includes a receiving unit 300, a notification determination unit 301, a settlement processing unit 302, and a transmission unit 303. These functions can be implemented by a program.

[0074] The receiving unit 300 receives various information from other devices. The notification determination unit 301 determines whether the authentication status notification transmitted by the in-vehicle system 11 is an authentication status notification. Specifically, the notification determination unit 301 determines whether the authentication status notification is a notification to maintain the authentication status or a notification to cancel the authentication status.

[0075] The settlement processing unit 302 performs settlement processing for the provision of services to the user. In the settlement processing, the settlement processing unit 302 generates billing amount information indicating the amount billed for the provision of the service.

[0076] The transmitting unit 303 transmits various information to other devices. Specifically, the transmitting unit 303 transmits content that is the subject of the service provided by the service provider 30 to the in-vehicle system 11. The content includes video and / or music.

[0077] Figure 17 is a sequence diagram showing an example of processing performed in the payment system 1, which includes the service provision device 30. In step S60, the processing shown in Figure 9 is performed. Here, it is assumed that the user of the automobile 10 has been authenticated and that the authentication status is maintained. If the user's authentication status is maintained, in step S61, the transmission unit 113 of the in-vehicle system 11 transmits an authentication status notification to the service provision device 30 indicating that the authentication status is maintained.

[0078] When the service provider 30 receives an authentication status notification, in step S62, the notification determination unit 301 of the service provider 30 determines the received authentication status notification. If it is determined that the authentication status notification is an authentication status notification indicating that the authentication status will be maintained, in step S63, the transmission unit 303 of the service provider 30 transmits the content to the in-vehicle system 11. The type of content can be specified by the user.

[0079] While the service provider 30 is providing content, if the authentication status control unit 108 of the in-vehicle system 11 cancels the user's authentication status, in step S64, the transmission unit 113 of the in-vehicle system 11 sends an authentication status notification to the service provider 30 indicating that the authentication status has been canceled.

[0080] When the service provider 30 receives an authentication status notification, in step S65, the notification determination unit 301 of the service provider 30 determines the received authentication status notification. If it is determined that the received authentication status notification is an authentication status notification indicating that the authentication status has been canceled, in step S66, the settlement processing unit 302 of the service provider 30 performs settlement processing for the service provided by the service provider 30.

[0081] Next, we will explain the case where the process shown in Figure 9 is performed. Assume that in step S67 the process shown in Figure 9 is performed, the user of the vehicle 10 is authenticated again, and the authentication status is maintained. In this case, in step S68 the transmission unit 113 of the in-vehicle system 11 transmits an authentication status notification to the service provider 30 indicating that the authentication status is maintained.

[0082] When the service provider 30 receives an authentication status notification, in step S69, the notification determination unit 301 of the service provider 30 determines the received authentication status notification. If it is determined that the received authentication status notification is an authentication status notification indicating that the authentication status will be maintained, in step S70, the transmission unit 303 of the service provider 30 transmits billing amount information indicating the amount calculated by the settlement process in step S66 to the in-vehicle system 11.

[0083] In step S71, the display control unit 110 of the in-vehicle system 11 displays the billing amount indicated by the billing amount information and the payment method associated with the user whose authentication status is maintained on the display 14 of the in-vehicle system 11. When the instruction receiving unit 111 of the in-vehicle system 11 receives a payment instruction from the user, in step S72, the payment processing unit 112 of the in-vehicle system 11 executes the payment process based on the payment method instructed by the user. In step S73, the transmission unit 113 of the in-vehicle system 11 transmits a payment completion notification to the service provision device 30. In step S74, the display control unit 110 of the in-vehicle system 11 displays the payment completion notification on the display 14 of the in-vehicle system 11.

[0084] Figure 18 shows another example of the main components of Payment System 1 according to this Disclosure. Figure 19 shows another example of the main processes performed in Payment System 1 according to this Disclosure.

[0085] The shooting control unit 101 of the in-vehicle system 11 causes the shooting device 13, which photographs the seats of the automobile 10, to perform a shooting process to generate a first captured image of the user sitting in the seat (step S80). The face recognition unit 102 of the in-vehicle system 11 performs a face recognition process using the generated first captured image of the user and an image of the user's face (step S81). The authentication result determination unit 103 of the in-vehicle system 11 determines the result of the user's face recognition (step S82). If the shooting control unit 101 of the in-vehicle system 11 determines that the user's face recognition was successful, it causes the shooting device 13 to perform a shooting process to generate a second captured image of the seat (step S83). The person detection unit 104 of the in-vehicle system 11 performs a person detection process to detect a person in the second captured image (step S84). The detection determination unit 105 of the in-vehicle system 11 determines whether or not a person was detected in the second captured image based on the result of the person detection process (step S85). The authentication status control unit 108 of the in-vehicle system 11 maintains the user's authentication status if it determines that a person has been detected in the second captured image, and releases the user's authentication status if it determines that no person has been detected in the second captured image (step S86). The transmission unit 113 of the in-vehicle system 11 transmits an authentication status notification to the service provision device 30, informing it of the user's authentication status (step S86).

[0086] The notification determination unit 301 of the service provision device 30 determines the authentication status notification transmitted by the in-vehicle system 11 (step S87). If it is determined that the authentication status notification indicates that the user's authentication status is maintained, the transmission unit 303 of the service provision device 30 transmits the content that is the subject of the service to the in-vehicle system 11 (step S88). The settlement processing unit 302 of the service provision device 30 performs settlement processing to calculate the amount billed for the service (step S89). The transmission unit 303 of the service provision device 30 transmits billing amount information indicating the calculated billing amount to the in-vehicle system 11 (step S90). The payment processing unit 112 of the in-vehicle system 11 performs payment processing based on the billing amount information (step S91).

[0087] By adopting this configuration, the service provider 30 can provide services as long as the user's authentication status is maintained. As a result, the user can receive services as long as their authentication status is maintained.

[0088] Furthermore, the service provider 30 can calculate the invoice amount for services provided while the user's authentication status is maintained. This allows the in-vehicle system 11 to pay the invoice amount for services provided while the user's authentication status is maintained.

[0089] In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

[0090] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0091] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments, rather than being associated with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0092] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) The in-vehicle system includes an in-vehicle system installed in a vehicle and a payment device, the in-vehicle system includes a shooting control unit that causes a shooting device to perform a shooting process to photograph the seats of the vehicle and generate a first photographed image of a user sitting in the seats, a face recognition unit that performs face recognition processing using the generated first photographed image of the user and an image of the user's face, and an authentication result determination unit that determines whether or not the user's face recognition was successful, the shooting control unit, if it determines that the user's face recognition was successful, causes the shooting device to perform a shooting process to generate a second photographed image of the seats, the in-vehicle system further includes a person detection unit that performs a person detection process to detect a person in the second photographed image, a detection determination unit that determines whether or not a person was detected in the second photographed image based on the result of the person detection process, an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second photographed image, and releases the user's authentication state if it is determined that no person was detected in the second photographed image, A payment system comprising: a payment processing unit that performs payment processing for the user when the user's authentication status is maintained; a transmission unit that transmits billing amount information for the user to the in-vehicle system; and the payment processing unit that performs payment processing based on the billing amount information. (Note 2) The payment system according to Note 1, wherein the in-vehicle system comprises: a position identification unit that identifies the position of a person in the second captured image when it is determined that a person has been detected in the second captured image; and a position determination unit that determines whether the identified position of the person is within a predetermined range in the second captured image; and the authentication status control unit maintains the user's authentication status when it is determined that the identified position of the person is within the predetermined range, and releases the user's authentication status when it is determined that the identified position of the person is not within the predetermined range. (Note 3) The payment system according to Note 1 or 2, wherein the person detection unit detects the person based on the silhouette of the person in the second captured image during the person detection process.(Note 4) The in-vehicle system further includes a notification unit that notifies the user of a payment method associated with the user, and the payment processing unit executes the payment processing based on the payment method specified by the user, as described in any one of Notes 1 to 3. (Note 5) The in-vehicle system further includes an amount calculation unit that calculates the amount of fuel or electricity supplied to the vehicle based on the amount of fuel or electricity supplied to the vehicle and the unit price of the fuel or electricity, and an amount determination unit that compares the calculated amount with the amount of billing information transmitted by the settlement device and determines whether the amount of billing is correct, as described in any one of Notes 1 to 4.(Note 6) The in-vehicle system includes an in-vehicle system installed in a vehicle and a service provision device, the in-vehicle system includes a shooting control unit that causes a shooting device to perform a shooting process to photograph the seats of the vehicle and generate a first photographed image of a user sitting in the seats, a face recognition unit that performs face recognition processing using the generated first photographed image of the user and an image of the user's face, and an authentication result determination unit that determines whether or not the user's face recognition was successful, the shooting control unit, if it determines that the user's face recognition was successful, causes the shooting device to perform a shooting process to generate a second photographed image of the seats, the in-vehicle system further includes a person detection unit that performs a person detection process to detect a person in the second photographed image, a detection determination unit that determines whether or not a person was detected in the second photographed image based on the result of the person detection process, an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second photographed image, and releases the user's authentication state if it is determined that no person was detected in the second photographed image, A payment system comprising: a transmission unit that transmits an authentication status notification indicating the authentication status of the user to the service provider; and a payment processing unit that performs payment processing for the user, wherein the service provider includes a notification determination unit that determines the authentication status notification transmitted by the in-vehicle system; a transmission unit that transmits content that is the subject of the service to the in-vehicle system when it is determined that the authentication status notification transmitted by the in-vehicle system indicates that the user's authentication status is maintained; and a settlement processing unit that performs settlement processing to calculate the billing amount for the service, wherein the transmission unit of the service provider transmits billing amount information indicating the calculated billing amount to the in-vehicle system; and the payment processing unit performs payment processing based on the billing amount information.(Note 7) An in-vehicle system comprising: an imaging control unit that causes an imaging device for imaging the seats of a car to perform imaging processing to generate a first image of a user sitting in the seats; a face recognition unit that performs face recognition processing using the generated first image of the user and an image of the user's face; and an authentication result determination unit that determines whether or not the user's face recognition was successful, wherein if the imaging control unit determines that the user's face recognition was successful, it causes the imaging device to perform imaging processing to generate a second image of the seats; a person detection unit that performs person detection processing to detect a person in the second image; a detection determination unit that determines whether or not a person was detected in the second image based on the result of the person detection processing; an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second image, and releases the user's authentication state if it is determined that a person was not detected in the second image; and a payment processing unit that performs payment processing for the user if the user's authentication state is maintained.(Note 8) An in-vehicle system installed in a vehicle causes a camera that photographs the seats of the vehicle to perform a photographic process to generate a first photographic image of the user sitting in the seat; the in-vehicle system performs a facial recognition process using the first photographic image of the user and an image of the user's face; the in-vehicle system determines whether or not the facial recognition of the user was successful; if the in-vehicle system determines that the facial recognition of the user was successful, it causes the camera to perform a photographic process to generate a second photographic image of the seat; the in-vehicle system performs a person detection process to detect a person in the second photographic image; based on the result of the person detection process, the in-vehicle system determines whether or not a person was detected in the second photographic image; if the in-vehicle system determines that a person was detected in the second photographic image, it maintains the user's authentication state; if it determines that no person was detected in the second photographic image, it releases the user's authentication state; the payment device transmits billing amount information for the user to the in-vehicle system. The method according to Appendix 8, wherein the in-vehicle system performs payment processing based on the billing amount information when the user's authentication status is maintained. (Note 9) The method according to Appendix 8, wherein the in-vehicle system determines that a person has been detected in the second captured image, identifies the location of the person in the second captured image, determines whether the identified location of the person is within a predetermined range in the second captured image, maintains the user's authentication status if the identified location of the person is within the predetermined range, and releases the user's authentication status if the identified location of the person is not within the predetermined range. (Note 10) The method according to Appendix 8 or 9, wherein the in-vehicle system detects the person based on the silhouette of the person in the second captured image during the person detection process. (Note 11) The method according to any one of Appendix 8 to 10, wherein the in-vehicle system notifies the user of a payment method associated with the user and performs payment processing based on the payment method specified by the user.(Note 12) The method according to any one of Notes 8 to 11, wherein the in-vehicle system calculates the amount of fuel or electricity supplied to the vehicle based on the amount of fuel or electricity supplied to the vehicle and the unit price of the fuel or electricity, compares the calculated amount with the billing amount indicated by the billing amount information transmitted by the settlement device, and determines whether the billing amount is correct. (Note 13) An in-vehicle system installed in a vehicle causes a camera that photographs the seats of the vehicle to perform a photographic process to generate a first photographic image of the user sitting in the seat; the in-vehicle system performs a facial recognition process using the first photographic image of the user and an image of the user's face; the in-vehicle system determines whether or not the facial recognition of the user was successful; if the in-vehicle system determines that the facial recognition of the user was successful, it causes the camera to perform a photographic process to generate a second photographic image of the seat; the in-vehicle system performs a person detection process to detect a person in the second photographic image; based on the result of the person detection process, the in-vehicle system determines whether or not a person was detected in the second photographic image; if the in-vehicle system determines that a person was detected in the second photographic image, it maintains the user's authentication state; if it determines that no person was detected in the second photographic image, it releases the user's authentication state; the in-vehicle system transmits an authentication state notification to the service provider informing it of the user's authentication state. A method comprising: a service provider determining the authentication status notification transmitted by the in-vehicle system; if the service provider determines that the authentication status notification transmitted by the in-vehicle system indicates that the user's authentication status is maintained, the service provider transmitting the content that is the subject of the service to the in-vehicle system; the service provider performing a settlement process to calculate the billing amount for the service; the service provider transmitting billing amount information indicating the calculated billing amount to the in-vehicle system; and the in-vehicle system performing a payment process based on the billing amount information.(Note 14) A program that causes a computer to perform a shooting process on a camera that photographs the seats of a car, thereby generating a first photographic image of a user sitting in the seats; to perform a facial recognition process using the first photographic image of the user and an image of the user's face; to determine whether or not the facial recognition of the user was successful; if it is determined that the facial recognition of the user was successful, to cause the camera to perform a shooting process on the camera to generate a second photographic image of the seats; to perform a person detection process to detect a person in the second photographic image; to determine whether or not a person was detected in the second photographic image based on the result of the person detection process; if it is determined that a person was detected in the second photographic image, to maintain the user's authentication state; if it is determined that no person was detected in the second photographic image, to release the user's authentication state; and if the user's authentication state is maintained, to perform a payment process related to the user. (Note 15) The program described in Note 14, which, when it is determined that a person has been detected in the second captured image, causes the computer to identify the location of the person in the second captured image, determines whether the identified location of the person is within a predetermined range in the second captured image, maintains the user's authentication state if it is determined that the identified location of the person is within the predetermined range, and cancels the user's authentication state if it is determined that the identified location of the person is not within the predetermined range. (Note 16) The program described in Note 14 or 15, which causes the computer to detect the person based on the silhouette of the person in the second captured image during the person detection process. (Note 17) The program described in any one of Notes 14 to 16, which causes the computer to notify the user of a payment method associated with the user, and executes the payment process based on the payment method specified by the user.

[0093] Some or all of the elements described in Appendices 2 to 5 that are subordinate to Appendice 1 may also be subordinate to Appendice 7 in the same manner as those described in Appendices 2 to 5. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0094] 1: Payment system 10: Automobile 11: In-vehicle system 12: ECU 13: Camera, shooting device 14: Display 15: Communication interface 16: Communication cable 20: Settlement device 21: Processor 22: Memory 23: Storage device 24: Communication interface 25: Bus 30: Service provision device 40: Network 100: Receiving unit 101: Shooting control unit 102: Face recognition unit 103: Authentication result determination unit 104: Person detection unit 105: Detection determination unit 106: Location identification unit 107: Location determination unit 108: Authentication status control unit 109: Payment method acquisition unit 110: Display control unit, notification unit 111: Instruction reception unit 112: Payment processing unit 113: Transmission unit 114: Amount calculation unit 115: Billing Amount Determination Unit 120: Processor 121: Memory 122: Storage Device 123: Communication Interface 124: Bus 200: Receiving Unit 201: Settlement Processing Unit 202: Transmitting Unit 300: Receiving Unit 301: Notification Determination Unit 302: Settlement Processing Unit 303: Transmitting Unit

Claims

1. The in-vehicle system includes an in-vehicle system installed in a vehicle and a payment device, the in-vehicle system includes a shooting control unit that causes a shooting device to perform a shooting process to photograph the seats of the vehicle and generate a first photographic image of a user sitting in the seats, a face recognition unit that performs face recognition processing using the generated first photographic image of the user and an image of the user's face, and an authentication result determination unit that determines whether or not the user's face recognition was successful, the shooting control unit, if it determines that the user's face recognition was successful, causes the shooting device to perform a shooting process to generate a second photographic image of the seats, the in-vehicle system further includes a person detection unit that performs a person detection process to detect a person in the second photographic image, a detection determination unit that determines whether or not a person was detected in the second photographic image based on the result of the person detection process, an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second photographic image, and releases the user's authentication state if it is determined that no person was detected in the second photographic image, A payment system comprising a payment processing unit that performs payment processing for the user when the user's authentication status is maintained, wherein the settlement device includes a transmission unit that transmits billing amount information for the user to the in-vehicle system, and the payment processing unit performs payment processing based on the billing amount information.

2. The payment system according to claim 1, wherein the in-vehicle system includes a position identification unit that identifies the position of a person in the second captured image when it is determined that a person has been detected in the second captured image, and a position determination unit that determines whether the identified position of the person is within a predetermined range in the second captured image, and the authentication state control unit maintains the user's authentication state when it is determined that the identified position of the person is within the predetermined range, and releases the user's authentication state when it is determined that the identified position of the person is not within the predetermined range.

3. The payment system according to claim 1 or 2, wherein the person detection unit detects the person based on the silhouette of the person in the second captured image during the person detection process.

4. The in-vehicle system further includes a notification unit that notifies the user of a payment method associated with the user, and the payment processing unit performs the payment processing based on the payment method specified by the user, according to any one of claims 1 to 3.

5. The payment system according to any one of claims 1 to 4, further comprising: an amount calculation unit that calculates the amount of fuel or electricity supplied to the vehicle based on the amount of fuel or electricity supplied to the vehicle and the unit price of the fuel or electricity; and an amount determination unit that compares the calculated amount with the amount indicated by the billing information transmitted by the settlement device and determines whether the billing amount is correct.

6. The in-vehicle system includes an in-vehicle system installed in a vehicle and a service provision device, the in-vehicle system includes a shooting control unit that causes a shooting device to perform a shooting process to photograph the seats of the vehicle and generate a first photographic image of a user sitting in the seats, a face recognition unit that performs face recognition processing using the generated first photographic image of the user and an image of the user's face, and an authentication result determination unit that determines whether or not the user's face recognition was successful, the shooting control unit, if it determines that the user's face recognition was successful, causes the shooting device to perform a shooting process to generate a second photographic image of the seats, the in-vehicle system further includes a person detection unit that performs a person detection process to detect a person in the second photographic image, a detection determination unit that determines whether or not a person was detected in the second photographic image based on the result of the person detection process, an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second photographic image, and releases the user's authentication state if it is determined that no person was detected in the second photographic image, A payment system comprising: a transmission unit that transmits an authentication status notification indicating the authentication status of the user to the service provider; and a payment processing unit that performs payment processing for the user, wherein the service provider includes a notification determination unit that determines the authentication status notification transmitted by the in-vehicle system; a transmission unit that transmits content that is the subject of the service to the in-vehicle system when it is determined that the authentication status notification transmitted by the in-vehicle system indicates that the user's authentication status is maintained; and a settlement processing unit that performs settlement processing to calculate the billing amount for the service, wherein the transmission unit of the service provider transmits billing amount information indicating the calculated billing amount to the in-vehicle system; and the payment processing unit performs payment processing based on the billing amount information.

7. An in-vehicle system comprising: a shooting control unit that causes a shooting device to perform a shooting process to photograph the seats of a car and generates a first photographic image of a user sitting in the seats; a face recognition unit that performs face recognition processing using the generated first photographic image of the user and an image of the user's face; and an authentication result determination unit that determines whether or not the user's face recognition was successful, wherein if the shooting control unit determines that the user's face recognition was successful, it causes the shooting device to perform a shooting process to generate a second photographic image of the seats; a person detection unit that performs a person detection process to detect a person in the second photographic image; a detection determination unit that determines whether or not a person was detected in the second photographic image based on the result of the person detection process; an authentication state control unit that maintains the user's authentication state if it is determined that a person was detected in the second photographic image, and releases the user's authentication state if it is determined that a person was not detected in the second photographic image; and a payment processing unit that performs payment processing for the user if the user's authentication state is maintained.

8. An in-vehicle system installed in a vehicle causes a camera to perform a shooting process to photograph the seats of the vehicle, thereby generating a first image of the user sitting in the seat; the in-vehicle system performs a facial recognition process using the generated first image of the user and an image of the user's face; the in-vehicle system determines whether or not the facial recognition of the user was successful; if the in-vehicle system determines that the facial recognition of the user was successful, it causes the camera to perform a shooting process to generate a second image of the seat; the in-vehicle system performs a person detection process to detect a person in the second image; based on the results of the person detection process, the in-vehicle system determines whether or not a person was detected in the second image; if the in-vehicle system determines that a person was detected in the second image, it maintains the user's authentication state; if it determines that no person was detected in the second image, it releases the user's authentication state; and the payment device transmits billing amount information for the user to the in-vehicle system. The in-vehicle system, when the user's authentication status is maintained, performs payment processing based on the billing amount information.

9. The method according to claim 8, wherein the in-vehicle system determines that a person has been detected in the second captured image, identifies the position of the person in the second captured image, determines whether the identified position of the person is within a predetermined range in the second captured image, maintains the user's authentication state if it is determined that the identified position of the person is within the predetermined range, and releases the user's authentication state if it is determined that the identified position of the person is not within the predetermined range.

10. The method according to claim 8 or 9, wherein the in-vehicle system detects the person based on the silhouette of the person in the second captured image during the person detection process.

11. The method according to any one of claims 8 to 10, wherein the in-vehicle system notifies the user of a payment method associated with the user and performs the payment processing based on the payment method specified by the user.

12. The method according to any one of claims 8 to 11, wherein the in-vehicle system calculates the amount of fuel or electricity supplied to the vehicle based on the amount of fuel or electricity supplied to the vehicle and the unit price of the fuel or electricity, compares the calculated amount with the billing amount indicated by the billing amount information transmitted by the settlement device, and determines whether the billing amount is correct.

13. An in-vehicle system installed in a vehicle causes a camera to perform a shooting process to photograph the seats of the vehicle, thereby generating a first photographic image of the user sitting in the seat; the in-vehicle system performs a facial recognition process using the generated first photographic image of the user and an image of the user's face; the in-vehicle system determines whether or not the facial recognition of the user was successful; if the in-vehicle system determines that the facial recognition of the user was successful, it causes the camera to perform a shooting process to generate a second photographic image of the seat; the in-vehicle system performs a person detection process to detect a person in the second photographic image; based on the results of the person detection process, the in-vehicle system determines whether or not a person was detected in the second photographic image; if the in-vehicle system determines that a person was detected in the second photographic image, it maintains the user's authentication state; if it determines that no person was detected in the second photographic image, it releases the user's authentication state; the in-vehicle system transmits an authentication state notification to the service provider informing it of the user's authentication state. A method comprising: a service provider determining the authentication status notification transmitted by the in-vehicle system; if the service provider determines that the authentication status notification transmitted by the in-vehicle system indicates that the user's authentication status is maintained, the service provider transmitting the content that is the subject of the service to the in-vehicle system; the service provider performing a settlement process to calculate the billing amount for the service; the service provider transmitting billing amount information indicating the calculated billing amount to the in-vehicle system; and the in-vehicle system performing a payment process based on the billing amount information.

14. A program that causes a computer to perform a shooting process on a camera that photographs the seats of a car, thereby generating a first image of a user sitting in the seats; to perform a face recognition process using the first image of the user and an image of the user's face; to determine whether or not the user's face recognition was successful; if it is determined that the user's face recognition was successful, to cause the camera to perform a shooting process on the camera to generate a second image of the seats; to perform a person detection process to detect a person in the second image; to determine whether or not a person was detected in the second image based on the result of the person detection process; if it is determined that a person was detected in the second image, to maintain the user's authentication state; if it is determined that no person was detected in the second image, to release the user's authentication state; and if the user's authentication state is maintained, to perform a payment process related to the user.

15. The program according to claim 14, which, when it is determined that a person has been detected in the second captured image, causes the computer to identify the location of the person in the second captured image, determines whether the identified location of the person is within a predetermined range in the second captured image, maintains the user's authentication state if it is determined that the identified location of the person is within the predetermined range, and cancels the user's authentication state if it is determined that the identified location of the person is not within the predetermined range.

16. The program according to claim 14 or 15, which causes the computer to detect a person based on the silhouette of a person in the second captured image during the person detection process.

17. The program according to any one of claims 14 to 16, which causes the computer to notify the user of a payment method associated with it, and to execute the payment process based on the payment method specified by the user.

Images