Accounting device and program

Abstract

To provide accounting devices and programs that can be used by customers through intuitive voice-activated operation, thereby improving customer convenience. [Solution] The accounting device of this embodiment comprises a voice reception unit, a voice recognition unit, a keyword detection unit, a display control unit, and an accounting processing unit. The voice reception unit receives voice input from the user. The voice recognition unit recognizes the input voice and converts it into text. The keyword detection unit detects keywords from the text. The display control unit displays the text on the display unit. The accounting processing unit performs accounting-related processing based on the detected keywords.

Description

Technical Field

[0001] Embodiments of the present invention relate to an accounting device and a program.

Background Art

[0002] In recent years, the introduction of self-POS (Point of Sales) devices operated by customers themselves has been progressing. In such self-POS devices, it is common for customers to operate the self-POS device by selecting buttons displayed on a touch panel. However, since self-POS devices have various models and applications depending on each manufacturer, users may feel inconvenient due to differences in screen configurations and operation procedures.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The problem to be solved by the present invention is to provide an accounting device and a program that can be used by customers through intuitive voice operations and improve customer convenience.

Means for Solving the Problems

[0004] The accounting device according to the embodiment includes a voice reception unit, a voice recognition unit, a keyword detection unit, a display control unit, and an accounting processing unit. The voice reception unit receives an input of a user's voice. The voice recognition unit recognizes the input voice and converts it into text. The keyword detection unit detects keywords from the text. The display control unit causes the text to be displayed on the display unit. The accounting processing unit executes processing related to accounting based on the detected keywords.

Brief Description of the Drawings

[0005] [Figure 1] FIG. 1 is a front view showing an example of the appearance of a self-POS device according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the hardware configuration of a self-POS device according to an embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the functional configuration of a self-service POS device according to this embodiment. [Figure 4A] Figure 4A is a flowchart showing an example of the payment method confirmation process according to the embodiment. [Figure 4B] Figure 4B is a continuation of the flowchart in Figure 4A. [Figure 5] Figure 5 shows an example of a payment method selection screen according to the embodiment. [Figure 6] Figure 6 shows an example of a payment method selection screen displaying payment method candidates based on the user's voice according to the embodiment. [Figure 7] Figure 7 shows an example of the payment method selection screen after a user has selected one of the payment method options in the payment method selection screen shown in Figure 6. [Figure 8] Figure 8 shows an example of the selection of payment method candidates based on member registration information or shopping history according to the embodiment. [Figure 9] Figure 9 shows an example of a video call screen with an attendant according to this embodiment. [Modes for carrying out the invention]

[0006] The embodiments will be described in detail below with reference to the drawings. However, the invention is not limited to the embodiments described below.

[0007] Figure 1 is a front view showing an example of the external appearance of a self-service POS (Point of Sales) device 1 according to this embodiment. The self-service POS device 1 is installed in stores such as supermarkets and convenience stores. The self-service POS device 1 is an example of an accounting device in this embodiment. The self-service POS device 1 performs product registration processing and accounting processing in response to operations by the user, the customer.

[0008] As shown in Figure 1, the self-service POS device 1 has, for example, a vertically elongated, roughly rectangular parallelepiped shape and comprises a first housing 10 and a second housing 11. The first housing 10 is located on the lower side of the roughly rectangular parallelepiped shape and houses the cash deposit and withdrawal section 107 inside.

[0009] The deposit and withdrawal unit 107, under the control of the control unit described later, deposits money received from customers and withdraws change to be paid to customers.

[0010] The second housing 11 is located on the upper side of a roughly rectangular parallelepiped shape and houses the control unit, which will be described later. The second housing 11 also includes a code reading unit 108, a card reading unit 109, a printing unit 110, and an audio input unit 114 as user-operated interfaces.

[0011] The code reading unit 108 is a reading device that reads information held in code symbols such as barcodes and two-dimensional codes. The code reading unit 108 reads code symbols through a reading window 13 formed in the second housing 11. For example, the code reading unit 108 reads code symbols written on products or code symbols displayed on a user's mobile device.

[0012] The card reader 109 is a reading device that reads information held on a card-type medium. The card reader 109 reads information from a card medium inserted into the card slot 14. The card reader 109 may also support touch reading. In this case, the card reader 109 reads information from the card medium it makes contact with. For example, the card reader 109 reads information related to electronic payments from an inserted or contacted credit card or electronic money card. The card reader 109 may also read point cards, membership cards, etc., that contain customer identification information to obtain customer identification information.

[0013] The printing unit 110 is a printing device such as a thermal printer. The printing unit 110 prints a receipt for the product purchased by the user and outputs (issues) the printed receipt from the receipt issuing slot 24 formed in the second housing 11.

[0014] The voice input unit 114 is a device such as a microphone that can input the user's voice. The voice input unit 114 sends the input voice as a signal to a control unit described later.

[0015] Also, the self-POS device 1 includes a display operation unit 15. The display operation unit 15 is provided on the upper surface of the second housing 11 or the like. The display operation unit 15 is, for example, a touch screen having a display unit 111 composed of a display device such as an LCD (Liquid Crystal Display) and an operation unit 112 composed of a touch panel or the like. The display unit 111 displays various types of information and images under the control of a control unit described later.

[0016] The operation unit 112 is, for example, a touch panel provided so as to overlap the display unit 111. The operation unit 112 outputs information based on the position touched by the user on the display operation unit 15 to the control unit.

[0017] Also, a basket placement table 12 may be provided on one side or the side of the self-POS device 1. The basket placement table 12 is a table for placing baskets and is used, for example, when placing a basket containing goods.

[0018] Also, the self-POS device 1 includes a camera 113. The camera 113 is provided at a position where it can photograph the face of a customer operating the self-POS device 1. The camera 113 is an example of a photographing device in the present embodiment.

[0019] Note that the configuration of the self-POS device 1 is not limited to that shown in FIG. 1. Also, the self-POS device 1 does not necessarily need to include all of the configurations shown in FIG. 1. For example, when the self-POS device 1 is dedicated to cashless settlement, the self-POS device 1 does not necessarily need to include the deposit / withdrawal unit 107.

[0020] FIG. 2 is a diagram showing an example of the hardware configuration of the self-POS device 1 according to the embodiment. As shown in FIG. 2, the self-POS device 1 includes the above-described deposit / withdrawal unit 107, code reading unit 108, card reading unit 109, printing unit 110, display unit 111, operation unit 112, camera 113, and voice input unit 114, etc.

[0021] Further, the self-POS device 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage unit 104, a voice output unit 105, and a communication unit 106, etc. The CPU 101 is connected to each of the hardware configurations shown in FIG. 2 via a bus or the like.

[0022] The CPU 101 is an example of a processor and comprehensively controls each part of the self-POS device 1. The ROM 102 stores various programs. The RAM 103 is a workspace for developing programs and various data. Here, the CPU 101, the ROM 102, and the RAM 103 constitute a control unit 100 which is an example of a computer.

[0023] The storage unit 104 has a storage medium such as an HDD (Hard Disk Drive) or flash memory, and maintains its contents even when the power is cut off. The storage unit 104 stores various programs and setting information that the CPU 101 can execute. For example, the storage unit 104 may store product information, keywords that are the detection target of the speech recognition processing described later, customer membership registration information, etc. Product information is information that associates identification information (product code) such as a JAN (Japanese Article Number) code, product name, and product price (selling price). Membership registration information includes, for example, customer identification information and the payment method registered by the customer. In addition, membership registration information may also include the customer's past shopping history, customer personal information, points, etc. The customer's past shopping history includes, for example, the purchase date of the product, product name, payment method, etc. Furthermore, the storage unit 104 may store trained models used in various processes executed by the control unit 100 described later.

[0024] The audio output unit 105 is an audio output device such as a speaker capable of outputting sound. The audio output unit 105 outputs voice messages, etc., according to the control of the control unit 100.

[0025] The communication unit 106 is a communication interface that can connect to a network such as a LAN (Local Area Network) (not shown). The communication unit 106 communicates with external devices such as a store server located inside the store, an information processing terminal held by a store employee, an information processing device at a center located outside the store, and an electronic payment server that provides electronic payment services, for example, via the network.

[0026] Next, the functional configuration of the self-service POS device 1 will be described with reference to Figure 3. Figure 3 is a diagram showing an example of the functional configuration of the self-service POS device 1 according to the embodiment. As shown in Figure 3, the self-service POS device 1 is functionally configured to include a reception unit 201, a product registration unit 202, an image acquisition unit 203, a face direction detection unit 204, a voice reception unit 205, a voice recognition unit 206, a keyword detection unit 207, an answer generation unit 208, a display control unit 209, a voice output control unit 210, a specification unit 211, an accounting processing unit 212, and a transmission / reception control unit 213.

[0027] Some or all of the functional configuration of the self-service POS device 1 may be a software configuration realized through the cooperation of the processor (e.g., CPU 101) and a program stored in the memory (e.g., ROM 102, storage unit 104) of the self-service POS device 1. Alternatively, some or all of the functional configuration of the self-service POS device 1 may be a hardware configuration realized by a dedicated circuit or the like mounted on the self-service POS device 1.

[0028] The reception unit 201 receives user input via the operation unit 112. The user of the self-service POS device 1 is, for example, a customer purchasing goods.

[0029] The product registration unit 202 registers the products that the customer intends to purchase. More specifically, the product registration unit 202 obtains the product code read by the code reading unit 108 from the code symbol installed on the product, and registers that product code as the product code of the product to be purchased in the storage unit 104, etc.

[0030] The image acquisition unit 203 acquires an image of the user's face from the camera 113.

[0031] The face orientation detection unit 204 detects the orientation of the user's face from the image acquired by the image acquisition unit 203. The method for detecting face orientation is not particularly limited; for example, image recognition techniques using deep learning such as CNN (Convolutional Neural Network) can be employed.

[0032] As explained in Figure 1, in the self-POS device 1 of this embodiment, the camera 113 is located in front of the self-POS device 1. Therefore, if the user's face is depicted facing forward in the captured image, the user's face is facing towards the self-POS device 1. The face direction detection unit 204 may determine whether or not the user's face is facing towards the self-POS device 1 depending on the installation position of the camera 113.

[0033] The voice reception unit 205 receives voice input from the user via the voice input unit 114.

[0034] The speech recognition unit 206 recognizes the user's voice input received by the voice reception unit 205 and converts it into text. The speech recognition method is not particularly limited; for example, speech recognition technology using deep learning can be employed.

[0035] More specifically, the speech recognition unit 206 converts the input speech into text if the user's face direction, as detected by the face direction detection unit 204, is facing towards the self-POS device 1. However, the speech recognition unit 206 does not convert the input speech into text if the user's face direction is not facing towards the self-POS device 1. This is because, if the user's face direction is not facing towards the self-POS device 1, the speech emitted by the user may not be intended as input to the self-POS device 1, but rather a monologue or a statement directed at a companion. Alternatively, if the user's face direction is not facing towards the self-POS device 1, the speech input unit 114 may not be the user's voice, but rather the voice of another person in the vicinity. To reduce the recognition of such unwanted speech, the speech recognition unit 206 limits the recognition target based on the user's face direction.

[0036] The keyword detection unit 207 detects keywords from the text converted from speech by the speech recognition unit 206.

[0037] Keywords are, for example, words related to the answers that Self-POS device 1 asks the user for, the user's questions, or requests for assistance. The words related to the answers that Self-POS device 1 asks the user for will differ depending on the processing that Self-POS device 1 is performing when the user inputs voice information. For example, if Self-POS device 1 is accepting the selection of a payment method, the answers that Self-POS device 1 asks the user for will be the names of the payment methods the customer will select, such as "cash," "credit card payment," "transportation IC (Integrated Circuit) card," or "○○Pay." Note that keywords do not have to be the names of the payment methods themselves, but words related to any of the payment methods may also be included. For example, the word "card" may be defined as a keyword related to "credit card payment." Alternatively, keywords may include words that do not explicitly state the name of the payment method, such as "the usual," "the usual payment method," or "the same payment method as last time," but which, in combination with the customer's membership registration information, allow the customer to identify the payment method they will select.

[0038] Keywords may be stored in the memory unit 104 beforehand. Alternatively, the keyword detection unit 207 may detect words that could be keywords by inputting text converted from the user's speech into a Large Language Model (LLM) or other pre-trained model using deep learning.

[0039] The type of response that the self-service POS device 1 is requesting from the user corresponds to the type of processing screen that the display control unit 209 displays on the display unit 111, as described later. For example, when the self-service POS device 1 is accepting the selection of a payment method, the display control unit 209 displays the payment method selection screen on the display unit 111. Keywords may be stored in the storage unit 104, for example, in association with the screen name.

[0040] The response generation unit 208 generates a response to the voice input by the user. For example, the response generation unit 208 may input the text converted from the voice by the speech recognition unit 206 into a large-scale language model and obtain the response output from the large-scale language model. However, the method of generating the response is not limited to this, and the response generation unit 208 may generate the response based on predetermined rules.

[0041] The display control unit 209 controls the display unit 111 to display various screens. More specifically, the display control unit 209 displays text converted from speech by the speech recognition unit 206 on the display unit 111. When displaying the text on the display unit 111, the display control unit 209 also highlights keywords contained in the text that have been detected by the keyword detection unit 207.

[0042] Furthermore, the display control unit 209 causes the text generated by the answer generation unit 208 to be displayed on the display unit 111.

[0043] The voice output control unit 210 controls the voice output unit 105 to output sound. For example, the voice output control unit 210 may output the response generated by the response generation unit 208 as a voice message.

[0044] The identification unit 211 identifies the payment method to be used for accounting settlement based on the keywords detected by the keyword detection unit 207.

[0045] One method for identifying payment method candidates from detected keywords is to pre-register the names of payment methods as keywords in the storage unit 104, and then identify them as payment method candidates when the names of those payment methods are detected from the user's voice. Alternatively, even if it is not the names of the payment methods themselves, words related to any of the payment methods may be registered as keywords in the storage unit 104 and associated with the payment methods.

[0046] Furthermore, the identification unit 211 may identify candidate payment methods based on keywords detected by the keyword detection unit 207 and member registration information or shopping history. For example, if words such as "the usual," "usual payment method," or "same payment method as last time" are detected as keywords, the identification unit 211 may identify registered payment methods or payment methods used in the previous transaction from member registration information or shopping history stored in the storage unit 104, based on customer identification information obtained from the member card, etc., by the card reading unit 109.

[0047] Alternatively, the specific unit 211 may identify the payment method to be used for accounting settlement by inputting the text converted from the user's speech into a large-scale language model or other pre-trained model obtained by deep learning.

[0048] The accounting processing unit 212 performs accounting-related processing based on keywords detected from the user's voice. Accounting-related processing includes, for example, settlement processing. The accounting processing unit 212 performs settlement using the settlement method identified by the identification unit 211. More specifically, the accounting processing unit 212 performs settlement using the settlement method confirmed by the user from among the candidate settlement methods identified by the identification unit 211.

[0049] As a method of user confirmation, for example, the reception unit 201 may receive a confirmation operation from the user on the operation unit 112. Alternatively, the voice reception unit 205 may receive a voice message from the user indicating their intention to confirm, and the voice recognition unit 206 may recognize it.

[0050] The transmission / reception control unit 213 will call an attendant if the keyword detected by the keyword detection unit 207 is related to a point of confusion or a request for assistance from the user. The transmission / reception control unit 213 may also call an attendant based on the user's actions.

[0051] In this embodiment, the attendant is, for example, an operator waiting at a center located outside the store. In this case, the transmission / reception control unit 213 transmits a signal to the information processing device at the center to call the operator via the communication unit 106. At the same time, the transmission / reception control unit 213 transmits the text converted from the user's input voice by the voice recognition unit 206 to the information processing device at the center. This allows the user's statements to be confirmed until an operator is called. In addition to the text converted from the user's voice, the transmission / reception control unit 213 may also transmit the response generated by the response generation unit 208 to the information processing device at the center. Subsequently, the transmission / reception control unit 213 initiates a video call with the information processing device at the center. The attendant may be located in a separate room within the store where the self-POS device 1 is installed, rather than at the center. The transmission / reception control unit 213 is an example of a calling unit in this embodiment.

[0052] Next, we will explain the processing flow performed by the self-service POS device 1 configured as described above. Here, we will use the selection of a payment method as an example of a situation in which voice input is used in the self-service POS device 1.

[0053] Figures 4A and 4B are flowcharts illustrating an example of the payment method confirmation process according to the embodiment. Prior to this process, for example, the product registration unit 202 performs a product registration process for the customer's purchase.

[0054] If product registration is complete (S1 "Yes"), the display control unit 209 displays the payment method selection screen on the display unit 111 (S2). Product registration is complete when, for example, the reception unit 201 receives the user's operation to complete product registration (or start the accounting process). If product registration is not complete (S1 "No"), the display control unit 209 does not display the payment method selection screen and repeats the process in S1.

[0055] Figure 5 shows an example of a payment method selection screen 90 according to the embodiment. As shown in Figure 5, the display control unit 209 causes the display unit 111 to display the payment method selection screen 90. The payment method selection screen 90 includes, for example, the number of registered items, the total price of the registered items, a note indicating that the payment method can be entered by voice, a button 811 to display all payment methods, and a button 812 to call a store clerk.

[0056] The message indicating that payment methods can be entered by voice is, for example, a text message such as "Please tell me your payment method by voice," as shown in Figure 5. Alternatively, the voice output control unit 210 may output a voice message with similar content from the voice output unit 105.

[0057] The "Display All Payment Methods" button 811 is an image button on which the user can input an operation to display all payment methods supported by the self-service POS device 1. When the user presses the "Display All Payment Methods" button 811, the display control unit 209 may display on the display unit 111 an image button on each payment method that the user can select, for each payment method supported by the self-service POS device 1.

[0058] The staff call button 812 is an image button that can accept user requests to call a staff member. In Figures 5-9, "staff member" refers to the attendant mentioned above, i.e., an operator located in the center or another room in the store. When the staff call button 812 is pressed by the user, the transmit / receive control unit 213 transmits a signal to the information processing device at the center via the communication unit 106 to call an operator.

[0059] The payment method selection screen 90 also includes a conversation history display field 910 where the entered voice is displayed as text. Since no voice has been entered yet when the payment method selection screen 90 is displayed, the conversation history display field 910 is empty.

[0060] Returning to Figure 4A, if the voice reception unit 205 receives voice input (S3 "Yes"), the image acquisition unit 203 acquires an image of the user's face from the camera 113. Then, the face direction detection unit 204 detects the orientation of the user's face from the image acquired by the image acquisition unit 203. If the orientation of the user's face detected by the face direction detection unit 204 is facing towards the self-POS device 1 (S4 "Yes"), the voice recognition unit 206 transcribes the input voice using voice recognition and converts it into text (S5).

[0061] Furthermore, if the orientation of the user's face detected by the face direction detection unit 204 is not facing the self-POS device 1 (S4 "No"), the voice recognition unit 206 does not recognize the input voice and returns to processing S3.

[0062] After processing in S5, the keyword detection unit 207 detects keywords from the text converted from speech by the speech recognition unit 206. If the keyword detection unit 207 detects keywords related to a payment method (S6 "Yes"), the identification unit 211 identifies the payment method to be used for accounting settlement based on the detected keywords (S7).

[0063] Furthermore, the display control unit 209 displays the text generated by the speech recognition unit 206 on the display unit 111 (S8). At this time, the display control unit 209 highlights the keywords contained in the text that have been detected by the keyword detection unit 207.

[0064] Figure 6 shows an example of a payment method selection screen 90 that displays payment method candidates based on the user's voice according to the embodiment. In the example shown in Figure 6, the display control unit 209 displays the text generated from the user's voice, "What should I pay with? Maybe a credit card or XX Pay." in the conversation history display field 910. In addition, since "credit card" and "XX Pay" have been detected as keywords by the keyword detection unit 207, the display control unit 209 highlights these words. The highlighting method can be, for example, displayed in a different color from the background text, made bold, highlighted or underlined, enclosed in a box, etc., but is not particularly limited.

[0065] Furthermore, the display control unit 209 displays the selection buttons for the payment method candidates identified by the identification unit 211 on the display unit 111 (S9). In the example shown in Figure 6, the display control unit 209 displays the "Credit Card Payment" button 801 and the "○○Pay" button 802 on the payment method selection screen 90.

[0066] Furthermore, the response generation unit 208 generates a response to the text converted from speech by the speech recognition unit 206 (S10).

[0067] The display control unit 209 displays the response generated by the response generation unit 208 in the conversation history display area 910. In the example shown in Figure 6, the display control unit 209 displays the response text "Payment method candidates have been displayed on the screen. To proceed with payment, please speak your payment method again or touch the screen." in the conversation history display area 910. The voice output control unit 210 also outputs the response generated by the response generation unit 208 as a voice message from the voice output unit 105 (S11).

[0068] If the payment method is determined by either voice input by the user or by operating the selection button for the payment method candidates (S12 "Yes"), the accounting processing unit 212 executes the payment process using the determined payment method (S13). If the payment method is not determined (S12 "No"), the reception unit 201 and the voice reception unit 205 wait for user operation or voice input.

[0069] Figure 7 shows an example of the payment method selection screen 90 after a user has selected a payment method candidate in the payment method selection screen 90 of Figure 6. In the example shown in Figure 7, the voice reception unit 205 receives the voice input from the user, who has selected a payment method candidate, "I will pay by credit card," and confirms the payment method, thereby confirming "credit card payment." Voice recognition by the voice recognition unit 206 and keyword detection by the keyword detection unit 207 are also performed on this voice. In this case, the response generation unit 208 may also generate a response that repeats the user's confirmed content, such as "Credit card, understood. I will proceed with payment."

[0070] The display control unit 209 may change the display mode of the button selected by the user from among the "Credit Card Payment" button 801 and the "○○Pay" button 802, which indicate the payment method options. The change in display mode may be, for example, a change in the color of the image button, blinking, etc., but is not limited to these. Furthermore, after the payment method is confirmed, the display control unit 209 may transition the display unit 111 to the screen for the confirmed payment method.

[0071] Furthermore, the payment method can also be confirmed by the user pressing either the "Credit Card Payment" button 801 or the "○○Pay" button 802 shown in Figure 6. For example, if the reception unit 201 receives notification that the user has pressed the "Credit Card Payment" button 801, the accounting processing unit 212 will execute the payment process using credit card payment.

[0072] Furthermore, if only one payment method is identified and explicitly stated by the user, the user's confirmation step may be omitted.

[0073] Figure 8 shows an example of the selection of payment method candidates based on member registration information or shopping history according to the embodiment. In the example shown in Figure 8, the user has voice-inputted "I'll pay with the usual method," and the card reader 109 has obtained the user's member registration information from the member card, etc. It is also assumed that the user's member registration information already has one payment method registered that the user uses by default (for example, "credit card payment").

[0074] In this case, the identification unit 211 may specify "credit card payment" as the payment method. Alternatively, the display control unit 209 may display a response indicating that the payment method has been confirmed, such as "Understood. You will pay by credit card. We will proceed to payment."

[0075] Returning to Figure 4A, if the keyword detection unit 207 does not detect any keywords related to payment methods (S6 "No"), but detects keywords related to the user's questions or requests for assistance (S14 "Yes"), the display control unit 209 highlights the keywords and displays the text converted from the user's voice on the payment method selection screen 90 (S15).

[0076] The transmission / reception control unit 213 then transmits a signal to the information processing device at the center via the communication unit 106 to call an attendant. At the same time, the transmission / reception control unit 213 transmits the conversation history with the user to the information processing device at the center via the speech recognition unit 206 (S16). The conversation history includes text converted from the user's voice and the response generated by the response generation unit 208.

[0077] Subsequently, the transmission / reception control unit 213 initiates a video call with the attendant (S17).

[0078] Figure 9 shows an example of a video call screen with an attendant according to the embodiment. The display control unit 209, for example, provides a video call display field 911 on the payment method selection screen 90 and displays the attendant's video screen. The display control unit 209 may also display a video screen of the user next to the attendant's video screen. The voice recognition unit 206 transcribes the conversation between the attendant and the user during the video call using voice recognition and converts it into text. The display control unit 209 displays this text in the video call display field 911. After the payment method is determined by the video call, the process proceeds to S13, and the accounting processing unit 212 executes the payment process using the determined payment method. In this case, the attendant may remotely operate the self-POS device 1 to determine the payment method.

[0079] Next, as shown in Figure 4B, if the keyword detection unit 207 does not detect any keywords related to payment methods, or keywords related to user questions or requests for assistance (S14 "No"), the display control unit 209 displays the text generated by the speech recognition unit 206 on the display unit 111 (S18). In this case, for example, the user may still be considering payment methods or may have paused speaking in the middle of voice input. In this case, the process returns to S3, and the voice reception unit 205 waits for the user to continue voice input. If the state of not detecting keywords continues for a specified time or longer, the transmission / reception control unit 213 may call an attendant to assist the user. At this point, the processing of this flowchart ends.

[0080] As described above, the self-service POS device 1 of this embodiment recognizes the user's voice and converts it into text, detects keywords from the text, and displays the text on the display unit 111. Furthermore, it performs accounting-related processing based on the detected keywords. In other words, with the self-service POS device 1 of this embodiment, even if the customer user utters keywords without being aware of them, the self-service POS device 1 automatically detects them, allowing the customer to use it with intuitive voice operation.

[0081] Furthermore, if the system can be operated by customer voice input, customers may feel uneasy or fail to notice misrecognition by the self-POS device 1 if they are unaware of what kind of input operation their voice has resulted in. In contrast, the self-POS device 1 of this embodiment displays the text recognized from the user's voice on the display unit 111, so that customers can proceed with processing after understanding what kind of voice input their voice has resulted in for the self-POS device 1. Therefore, the self-POS device 1 of this embodiment can be used by customers through intuitive voice operation, improving customer convenience.

[0082] Furthermore, while selecting a payment method is a necessary process for any self-service POS system, the screen layout and operating procedures may differ depending on the specifications of the self-service POS system. Therefore, in systems where payment method selection is based on touch panel operation, users may have difficulty selecting their desired payment method. In contrast, the self-service POS system 1 of this embodiment identifies the payment method to be used for settlement based on keywords detected from the user's voice, and executes the payment using the identified payment method. This allows the user to execute their desired payment method through intuitive voice input, regardless of the screen layout or other factors.

[0083] Furthermore, the self-service POS device 1 of this embodiment highlights keywords contained in the text when displaying the text recognized from the user's voice on the display unit 111. This allows the user to understand which words from their voice were recognized as information for processing by the self-service POS device 1. For example, in the case of an ambiguous utterance such as, "I have a credit card, but... hmm, cash please," the user may be confused when "credit card payment" is presented as a payment method option even though they did not intend to select it. In such cases, by understanding the text of their voice, the keywords within it, and the recognized words, the user can notice and correct unintended input.

[0084] Furthermore, the self-POS device 1 of this embodiment detects the orientation of the user's face from an image of the user's face, and converts the input voice into text if the user's face is facing the self-POS device 1. However, the self-POS device 1 does not convert the input voice into text if the user's face is not facing the self-POS device 1. Therefore, the self-POS device 1 of this embodiment can reduce the error of misrecognizing voices that the user did not intend to input into the self-POS device 1, or voices spoken by people other than the user.

[0085] Furthermore, in this embodiment, if the keywords detected from the text recognized from the user's voice are related to the user's questions or requests for assistance, the self-service POS device 1 summons an attendant and transmits the text recognized from the user's voice to the information processing device used by the attendant. Therefore, with the self-service POS device 1 of this embodiment, assistance can be provided quickly when the user is experiencing difficulties, and it becomes easier for the attendant to understand the user's situation from the user's past voice input history.

[0086] (Variation 1) In the above embodiment, the payment method selection process was given as an example of accounting-related processing that the self-POS device 1 supports voice input for, but other processes may also support voice input. For example, the user may be able to voice input the selection of whether or not a shopping bag is needed at the beginning of the product registration process, the presence or absence of a parking ticket or coupon at the beginning of the payment process, and whether or not a receipt is needed after the payment process. In this case, for example, keywords corresponding to each processing screen may be stored in the storage unit 104.

[0087] (Modification 2) Furthermore, in the above-described embodiment, the transmission / reception control unit 213 summoned an attendant located remotely, but the method of summoning an attendant is not limited to this method. For example, the transmission / reception control unit 213 may send a summoning notification to a terminal held by a store employee located within the store where the self-POS device 1 is installed. In this case, instead of a video call, the store employee may actually go to the self-POS device 1.

[0088] Furthermore, the self-service POS device 1 may employ a method of calling nearby store staff using voice or alert sounds. In this case, the voice output control unit 210 may be an example of a calling unit.

[0089] (Variation 3) Furthermore, in the above-described embodiment, the self-service POS device 1 restricted voice recognition based on the user's facial orientation, but the method of restriction is not limited to this.

[0090] For example, the speech recognition unit 206 may perform speech recognition only while the display control unit 209 is displaying a processing screen corresponding to the voice input on the display unit 111.

[0091] Furthermore, the self-service POS device 1 may be equipped with a motion sensor that detects a person located within a specified range in the front direction. In this case, the voice recognition unit 206 may perform voice recognition only when a person is detected by the motion sensor.

[0092] In addition, if the speech recognition unit 206 does not perform speech recognition, it may be a state where speech collection by the speech input unit 114 is enabled but text conversion is not performed, or the speech collection by the speech input unit 114 itself may be turned off.

[0093] Furthermore, the voice recognition unit 206 may recognize the voice input after a predetermined wake word (a verbal cue) only if that wake word is input.

[0094] The various speech recognition limitation techniques described above may be used individually, or in combination of some or all of them.

[0095] (Modification 4) Furthermore, although the above-described embodiment obtained customer identification information from a membership card or the like, the method of obtaining customer identification information is not limited to this. For example, the self-POS device 1 may identify customers by facial recognition. Specifically, a photograph of the customer's face may be registered in the customer's membership registration information in advance. In this case, the identification unit 211 may identify the customer from the image of the customer taken by the camera 113 and identify a payment method registered in the customer's membership registration information or a payment method from past purchase history as a candidate payment method.

[0096] Furthermore, when identifying a customer by facial recognition, the display control unit 209 may display a message including the customer's name, such as "Welcome, Mr. / Ms. ○○," on the display unit 111.

[0097] (Variation 5) Furthermore, while the self-service POS device 1 was used as an example of a payment device in the above-described embodiment, the payment device is not limited to this. For example, the same functions as the self-service POS device 1 in the above-described embodiment can be applied to a POS system that can be attached to a shopping cart, or a smartphone equipped with POS functionality. In this case, a POS system that can be attached to a shopping cart, or a smartphone equipped with POS functionality, may be an example of a payment device.

[0098] (Experimental variation 6) Furthermore, although the above-described embodiment shows a configuration in which the self-service POS device 1 is equipped with a camera 113 as an example, the camera 113 may also be an external peripheral device attached to the self-service POS device 1. In this case, the self-service POS device 1 is connected to the camera 113 via wired or wireless means so as to be able to communicate with it. The code reading unit 108, the card reading unit 109, and the printing unit 110 may also be external peripheral devices attached to the self-service POS device 1.

[0099] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations are possible without departing from the spirit of the invention. These embodiments and their variations are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as described in the claims. [Explanation of Symbols]

[0100] 1 POS device 10. First cabinet 11. Second cabinet 12 Basket stand 14 card slots 15 Display operation section 24 Receipt issuing slots 90 Payment Method Selection Screen 100 Control Unit 101 CPU 102 ROM 103 RAM 104 Storage section 105 Audio output section 106 Communications Department 107 Deposit and Withdrawal Department 108 Code reading unit 109 Card reader 110 Printing section 111 Display section 112 Operation section 113 Camera 114 Voice Input Section 201 Reception Department 202 Product Registration Department 203 Image acquisition unit 204 Face direction detection unit 205 Voice Reception Department 206 Voice Recognition Unit 207 Keyword Detection Unit 208 Answer generation part 209 Display Control Unit 210 Audio output control unit 211 Specific section 212 Accounting Processing Section 213 Transmit / Receive Control Unit 801 "Credit Card Payment" button 802 "○○Pay" button 811 Payment Method Display Button 812 Call a staff member button 910 Conversation history display area 911 Video Call Display Field [Prior art documents] [Patent Documents]

[0101] [Patent Document 1] Japanese Patent Publication No. 2023-104103 [Patent Document 2] Japanese Patent Publication No. 2023-69784

Claims

1. A voice reception unit that accepts voice input from the user, A speech recognition unit that recognizes the input speech and converts it into text, A keyword detection unit that detects keywords from the aforementioned text, A display control unit that displays the aforementioned text on the display unit, Based on the detected keywords, an accounting processing unit performs accounting-related processing, An accounting device equipped with the following features.

2. The system further comprises a specification unit that identifies a settlement method to be used for settling the accounting based on the detected keywords, The accounting processing unit executes the settlement using the specified settlement method. The accounting device according to claim 1.

3. The display control unit, when displaying the text on the display unit, highlights the keywords contained in the text. The accounting device according to claim 1.

4. An image acquisition unit that acquires an image of the user's face, The system further includes a face direction detection unit that detects the orientation of the user's face from the acquired image, The voice recognition unit converts the input voice into text if the user's face is facing the accounting device, and does not convert the input voice into text if the user's face is not facing the accounting device. The accounting device according to claim 1.

5. The system further includes a calling unit that, if the detected keyword is related to the user's questions or requests for assistance, calls an attendant and transmits the text to the information processing device used by the attendant. The accounting device according to any one of claims 1 to 4.

6. A voice reception step that accepts voice input from the user, A speech recognition step that recognizes the input speech and converts it into text, A keyword detection step for detecting keywords in the aforementioned text, A display control step that causes the aforementioned text to be displayed on the display unit, Based on the detected keywords, an accounting processing step is performed to carry out accounting-related processing, A program that causes a computer to execute something.

Images