Video playback system

Abstract

To facilitate an after-the-fact of check a face-to-face interaction between a plurality of people with a conversation.SOLUTION: A video playback system comprises video acquisition means, video storage means, text acquisition means, text storage means, video display means, and text display means. The video storage means stores video data that represents a video obtained by imaging an imaging range and is acquired by the video acquisition means. The text storage means stores text data that is generated by recognizing voice collected around the imaging range and is acquired by the text acquisition means. The video display means causes the video represented by the video data to be displayed. The text display means selects, out of the stored text data, text data generated by recognizing voice collected around the imaging range at timing of imaging the video displayed by the video display means and causes a text represented by the text data to be displayed.SELECTED DRAWING: Figure 16

Description

Technical Field

[0001] Embodiments of the present invention relate to a video playback system.

Background Art

[0002] In cases where multiple people interact face-to-face, such as in customer service by a store clerk accompanied by conversation at a store counter, etc., it has already been done to record the interaction as a video file with audio. However, when trying to check a location where a certain statement is made from a video file with audio, the checker has no choice but to search while checking the image and audio, and the burden on the checker is excessive. Under such circumstances, it has been desired to facilitate the subsequent confirmation work for the face-to-face interaction between multiple people accompanied by conversation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide a video playback system that can facilitate the subsequent confirmation work for the face-to-face interaction between multiple people accompanied by conversation.

Means for Solving the Problems

[0005] The video playback system of the embodiment includes a video acquisition means, a video storage means, a text acquisition means, a text storage means, Corresponding data storage means, a video display means and a text display means. The video acquisition means The area in which customers and staff are visible shoots within a As for the shooting range, predetermined shooting range ta Image data is acquired. The video storage means stores the video data acquired by the video acquisition means. Multiple video files containing video data extracted from and including time information representing the start and end dates and times of the said video data. It memorizes the following. The means of obtaining the text is, Between the start and stop instructions regarding customer service by store staff Shooting range Aiming for Text generated by recognizing collected audio file The text storage means obtains the text obtained by the text acquisition means. file of Associated with recognition period data representing the period during which the text file was acquired. Remember. The correspondence data storage means stores correspondence data for each of multiple customer interactions, which is included in the barcode data attached to the prescription submitted by the customer in relation to the customer interaction, and associates an identifier that identifies the customer with the date and time of the interaction. The means of displaying the video is, When a customer response is specified for any of the multiple response data stored in the response data storage means, the video storage means selects a video file from among the video files stored in the video file storage means that contains time information representing the period including the response date and time included in the response data for the specified customer response, and then processes the video file accordingly. The video is displayed. The text display means displays the text stored by the text storage means. file From among them, the video displayed by the video display device Associated with recognition period data that represents a period including the start date and time represented by the time information contained in the file. text file Select the text in question. file Display the text that represents it. [Brief explanation of the drawing]

[0006] [Figure 1] A block diagram showing the schematic configuration of a recording system according to one embodiment, and the main circuit configuration of a recording / playback device and a counter terminal included in the recording system. [Figure 2] A schematic diagram showing the data structure of the correspondence management data in Figure 1. [Figure 3] Flowchart for the registration process. [Figure 4] A diagram showing an example of a registration screen. [Figure 5] Flowchart of the recording and playback process. [Figure 6] A diagram showing an example of a registration screen. [Figure 7] A diagram showing an example of a registration screen. [Figure 8] A diagram showing an example of a registration screen. [Figure 9] A flowchart for speech recognition processing. [Figure 10] A diagram showing an example of the start instruction screen. [Figure 11] A diagram showing an example of a stop instruction screen. [Figure 12] A flowchart of the search process. [Figure 13] Flowchart of the search process. [Figure 14] Flowchart of the search process. [Figure 15] Diagram showing an example of a search screen. [Figure 16] Diagram showing an example of a playback screen. [Figure 17] Diagram showing an example of a screen with an editing window popped up.

Mode for Carrying Out the Invention

[0007] Hereinafter, an example of an embodiment will be described with reference to the drawings. In this embodiment, an example in which a video playback system is applied to a recording system that records the response of a clerk at a counter to a patient receiving a prescription at a pharmacy (hereinafter referred to as patient response) will be described. Note that a patient receiving a prescription at a pharmacy is a customer of the pharmacy.

[0008] FIG. 1 is a block diagram showing the schematic configuration of a recording system 100 according to this embodiment and the main circuit configurations of a recording / playback device 1 and a counter terminal 2 included in the recording system 100. The recording system 100 includes a recording / playback device 1, a counter terminal 2, a camera unit 3, a scanner unit 4, and a microphone unit 5. The recording / playback device 1, the counter terminal 2, and the camera unit 3 are communicable via a communication network 200. The scanner unit 4 and the microphone unit 5 are connected to the counter terminal 2.

[0009] As the communication network 200, the Internet, a VPN (virtual private network), a LAN, a public communication network, a mobile communication network, or various other networks for computer communication can be used alone or in appropriate combination. As the communication network 200, for example, the Internet is used.

[0010] The recording and playback device 1 is an information processing device that stores video files and text files recording patient interactions by store employees at the counter where the counter terminal 2 is installed, and performs information processing for playback. Window terminal 2 is a terminal device installed in a pharmacy where a counter is provided for patient interactions that need to be recorded. Window terminal 2 performs various information processing related to patient interactions by staff, as described below.

[0011] Camera unit 3 captures a designated area of ​​the counter where staff interact with patients, and simultaneously inputs audio from the surrounding area to obtain video data with sound. Camera unit 3 transmits the video data to the recording / playback device 1 via the communication network 200. For example, an existing unit known as a network camera can be used as camera unit 3. In other words, camera unit 3 has the function of a means of capturing images. The scanner unit 4 is equipped with a barcode scanner and outputs barcode data obtained by reading barcodes with this barcode scanner.

[0012] The microphone unit 5 includes a patient-side microphone and a staff-side microphone. The patient-side microphone is positioned with its directional direction facing the side where the patient is located at the counter, and outputs audio data (hereinafter referred to as patient audio data) that receives ambient sound mainly consisting of the patient's voice. The staff-side microphone is positioned with its directional direction facing the side where the staff member is located at the counter, and outputs audio data (hereinafter referred to as staff member audio data) that receives ambient sound mainly consisting of the staff member's voice. In other words, the microphone unit 5 is an example of a sound collection means that collects sound around the counter, which is within the shooting range of the camera unit 3.

[0013] Thus, the counter terminal 2, camera unit 3, scanner unit 4, and microphone unit 5 constitute a device set for recording customer interactions at one counter. Although Figure 1 shows only one device set, multiple similar device sets may be provided, each associated with a different counter.

[0014] The recording and playback device 1 comprises a processor 11, a main storage unit 12, an auxiliary storage unit 13, a communication unit 14, and a transmission line 15. By connecting the processor 11, the main storage unit 12, and the auxiliary storage unit 13 via a transmission line 15, a computer is configured to perform information processing to realize the functions of a recording and playback device 1.

[0015] The processor 11 corresponds to the central part of the computer described above. The processor 11 performs information processing to control each part in order to realize various functions of the recording and playback device 1, in accordance with information processing programs such as the operating system and application programs.

[0016] The main memory unit 12 corresponds to the main memory portion of the computer described above. The main memory unit 12 includes a read-only memory area and a rewritable memory area. The main memory unit 12 stores a portion of the information processing program described above in the read-only memory area. The main memory unit 12 may also store data necessary for the processor 11 to perform processing to control each part in the read-only memory area or the rewritable memory area. The main memory unit 12 uses the rewritable memory area as a work area for the processor 11.

[0017] The auxiliary storage unit 13 corresponds to the auxiliary storage portion of the computer described above. The auxiliary storage unit 13 can utilize, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), an SSD (solid state drive), or various other well-known storage devices. The auxiliary storage unit 13 stores data used by the processor 11 in performing various processes and data generated by the processing performed by the processor 11. The auxiliary storage unit 13 may also store the above-mentioned information processing program. In this embodiment, the auxiliary storage unit 13 stores the recording and playback program PRA, which is one of the information processing programs. The recording and playback program PRA is an application program that describes the procedures for the information processing described later (hereinafter referred to as recording and playback processing) for recording video and text and playing back recorded video and text. A portion of the storage area of ​​the auxiliary storage unit 13 is used as an area for storing the video file group FIA, the text file group FIB, and the corresponding management data DAA. The video file group FIA is a collection of video files that have been converted from video data transmitted from camera unit 3 into data files in a predetermined file format. The text file group FIB is a collection of text files that have been transmitted from counter terminal 2 as described later. The response management data DAA is data used to manage patient interactions at the counter. Details of the response management data will be described later.

[0018] The communication unit 14 performs communication processing for data communication via the communication network 200. The communication unit 14 can use, for example, an existing communication device for the Internet. The transmission line 15 includes an address bus, a data bus, and control signal lines, and transmits data and control signals exchanged between the connected parts.

[0019] As the hardware for the recording and playback device 1, for example, a general-purpose computer device for server equipment can be used. In this case, the transfer of the recording and playback device 1 is generally carried out with the recording and playback program PRA stored in the auxiliary storage unit 13. However, the hardware of the recording and playback device 1 and the recording and playback program PRA may be transferred separately, without the recording and playback program PRA stored in the auxiliary storage unit 13. The recording and playback program PRA may then be written to the auxiliary storage unit 13 in response to the operation of any operator. Alternatively, the hardware of the recording and playback device 1 and the recording and playback program PRA may be transferred separately, with a different version of the same type of information processing program stored in the auxiliary storage unit 13. The recording and playback program PRA may then be written to replace the information processing program already stored in the auxiliary storage unit 13. The transfer of the recording and playback program PRA can be carried out by recording it on a removable recording medium such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory, or by communication over a network. The recording and playback program PRA may be stored in the main storage unit 12.

[0020] The customer service terminal 2 comprises a processor 21, a main storage unit 22, an auxiliary storage unit 23, an input / output unit 24, an interface unit 25, a communication unit 26, and a transmission line 27. The general functions of the processor 21, main memory unit 22, auxiliary memory unit 23, communication unit 26, and transmission line 27 are the same as those of the processor 11, main memory unit 12, auxiliary memory unit 13, communication unit 14, and transmission line 15. However, the auxiliary memory unit 23 does not store the recording / playback program PRA, but stores the correspondence registration program PRB, the video search program PRC, and the speech recognition program PRD. The correspondence registration program PRB is an application program that describes the procedure for the information processing described later (hereinafter referred to as the correspondence registration process) for registering patient care for the purpose of managing the implementation status of patient care. The video search program PRC is an application program that describes the procedure for the information processing described later (hereinafter referred to as the search process) for the user interface when searching for videos recorded by the recording / playback device 1. The speech recognition program PRD is an application program that describes the procedure for the information processing described later (hereinafter referred to as the speech recognition process) for controlling speech recognition processing targeting speech data output from the microphone unit 5.

[0021] The input / output unit 24 includes at least one of various user interface devices. The input / output unit 24 may include, for example, a touch panel and a sound device as user interface devices. The input / output unit 24 may also include various other devices as user interface devices, such as a mouse and a keyboard.

[0022] The interface unit 25 has a port to which external devices can be attached and detached, and the scanner unit 4 and microphone unit 5 are connected to this port. The interface unit 25 interfaces with the devices attached to these ports, such as the scanner unit 4 and microphone unit 5, for data exchange. As the interface unit 25, off-the-shelf devices conforming to existing standards such as the USB (universal serial bus) standard can be used.

[0023] The hardware for the counter terminal 2 can be various types of information processing terminals, such as desktop computers, laptop computers, tablet computers, or smartphones. Generally, the counter terminal 2 is transferred with the hardware in a state where the corresponding registration program PRB, video search program PRC, and voice recognition program PRD are not stored in the auxiliary storage unit 23, and each program is written to the auxiliary storage unit 23 according to the operation of any operator. However, the counter terminal 2 may also be transferred with the corresponding registration program PRB, video search program PRC, and voice recognition program PRD already stored in the auxiliary storage unit 23. Alternatively, the hardware of the counter terminal 2 in a state where a different version of the same type of information processing program as the corresponding registration program PRB, video search program PRC, and voice recognition program PRD is stored in the auxiliary storage unit 23 may be transferred with the corresponding registration program PRB, video search program PRC, and voice recognition program PRD being transferred separately. The corresponding registration program PRB, video search program PRC, and voice recognition program PRD may then be written to replace the information processing program already stored in the auxiliary storage unit 23. The transfer of the corresponding registration program PRB, video search program PRC, and voice recognition program PRD can be done by recording them on a removable recording medium such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory, or by communication over a network. The corresponding registration program PRB, video search program PRC, and voice recognition program PRD may also be stored in the main storage unit 22.

[0024] Figure 2 schematically represents the data structure of the correspondence management data DAA. If the recording and playback device 1 records patient interactions at multiple service counters, then multiple DAA (Digital Access Arrangement) data, each associated with one of those service counters, will be stored in the auxiliary storage unit 13. The correspondence management data DAA includes fields FAA, FAB, FAC, ... Field FAA contains the contact code, which serves as the identifier for the associated contact point. Each field from FAB onward may contain corresponding data. The number of corresponding data entries in the correspondence management data DAA varies depending on the operating status of the recording / playback device 1. The corresponding data is associated with a single patient interaction and is a collection of various data related to that patient interaction.

[0025] As shown in Figure 2, one corresponding data entry includes fields FBA, FBB, FBC, and FBD. Field FBA contains the patient code, which serves as an identifier for the patient subject to the associated patient management. Field FBB contains the start date and time of the associated patient management. Field FBC contains the file path (hereinafter referred to as the edit file path) used to identify the edit text file related to the associated patient management, as described below. Field FBD contains the file path (hereinafter referred to as the original file path) used to identify the original text file related to the associated patient management, as described below.

[0026] Next, the operation of the recording system 100 configured as described above will be explained. Note that the contents of the various processes described below are examples, and it is possible to change the order of some processes, omit some processes, or add other processes as appropriate. When the processor 21 receives an instruction to execute the correspondence registration process via a predetermined operation on the input / output unit 24 at the counter terminal 2, it starts the correspondence registration process based on the correspondence registration program PRB.

[0027] Figure 3 is a flowchart of the correspondence registration process. As ACT1, the processor 21 displays a registration screen on the touch panel provided in the input / output unit 24. The registration screen is a GUI (graphical user interface) screen that allows store staff to register the start of patient care.

[0028] Figure 4 shows an example of a registration screen. Note that in Figure 4 and other figures, some display objects may be omitted from the illustrations of the various screens shown. As shown in Figure 4, the registration screen includes a text CSA, an image IMA, a button BUA, and an area ARA.

[0029] The string CSA represents a text message instructing the store clerk to scan the barcode displayed on the prescription brought by the patient being attended to. The image IMA is an image that helps the store clerk visualize the task they need to perform. The button BUA is a soft key that the operator uses to signal the end of the attendance registration process. The area ARA displays a list of patient attendances that have been registered so far. When processor 21 executes ACT1, no patient attendances have been registered yet, so the list displayed in area ARA does not normally show specific information. However, Figure 4 shows the list in area ARA with specific information displayed in order to clarify what the list in area ARA looks like. For each patient attendance, the list in area ARA displays the patient code, counter code, and date and time of attendance as identifiers for the patient being attended to.

[0030] Furthermore, when generating the registration screen, the processor 21 may retrieve the information to be displayed in the patient correspondence list from the correspondence history data stored in the auxiliary storage unit 23, or it may retrieve it from the correspondence history data managed by another information processing device. The other information processing device could be a POS (point-of-sale) server that manages drug sales in a pharmacy.

[0031] Now, when a store employee in charge of patient care begins assisting a new patient, they have the barcode displayed on the prescription brought in by that patient read by the barcode scanner installed in scanner unit 4. Once scanner unit 4 has scanned the barcode with the barcode scanner and obtained barcode data, it outputs this barcode data.

[0032] After the processor 21 displays the registration screen as ACT1 in Figure 3, it proceeds to ACT2. As ACT2, processor 21 waits for a code to be entered. Then, if the barcode data output from scanner unit 4 as described above is received by interface unit 25, processor 21 determines that a code has been entered and YES, and proceeds to ACT3.

[0033] As ACT3, processor 21 attempts to retrieve the patient code contained in the barcode data. Specifically, processor 21 extracts the data at a predetermined location within the barcode data represented by the barcode displayed on the prescription, as the patient code of the patient to whom the prescription is issued.

[0034] In ACT4, processor 21 checks whether a valid patient code has been obtained. If a valid patient code has been obtained, processor 21 determines it is YES and proceeds to ACT5. As ACT5, the processor 21 requests the recording / playback device 1 to register a new patient interaction. For example, the processor 21 sends predetermined request data for this request to the recording / playback device 1 via the communication unit 26 to the communication network 200. The request data includes, for example, the patient code, the counter code as an identifier for the counter where the counter terminal 2 is being used, and the date and time of interaction. The date and time to be used as the date and time of interaction may be determined as appropriate by, for example, the creator of the interaction registration program PRB, but it is expected to be the date and time related to the timing when YES was determined in ACT2, or the execution timing of one of ACT3 to ACT5. Furthermore, the barcode scanning of the prescription for registering the transaction with the POS server that manages drug sales may be the same as the barcode scanning performed by the store clerk as described above. In this case, the processor 21, as ACT5, also requests transaction registration from the POS server.

[0035] When the above-mentioned request data is transmitted to the recording / playback device 1 via the communication network 200, the communication unit 14 receives the request data. In the recording and playback device 1, the processor 11 basically continuously performs recording and playback processing based on the recording and playback program PRA, except during special periods such as maintenance work. However, the processor 11 may also perform recording and playback processing only during predetermined service hours.

[0036] Furthermore, if the recording system 100 is equipped with multiple device sets, each consisting of a counter terminal 2, a camera unit 3, a scanner unit 4, and a microphone unit 5, and these sets are associated with multiple counters, the recording and playback processing for each of these multiple counters will be executed in parallel. However, in the following description, only the recording and playback processing for one counter equipped with the single device set shown in Figure 1 will be explained.

[0037] Figure 5 is a flowchart of the recording and playback process. As ACT11, processor 11 starts recording. That is, processor 11 converts the video data with audio transmitted from camera unit 3, transmitted via communication network 200, and received by communication unit 14 into a video file in a predetermined multimedia format, and adds the video file to the video file group FIA. For example, processor 11 cuts the video data into 5-minute segments and converts each segment into a single mp4 video file. Processor 11 also includes time information in each video file, indicating the start and end dates and times of the video represented by that video file.

[0038] Thus, the processor 11 acquires the video data transmitted from the camera unit 3 and stores the video file containing this video data in the auxiliary storage unit 13. In other words, by having the processor 11 perform information processing based on the recording and playback program PRA, the computer with the processor 11 as its central component functions as a video acquisition means. The auxiliary storage unit 13, which stores the video file group FIA containing the aforementioned video file, is an example of a video storage means.

[0039] Then, while continuing the recording that was started as described above, the processor 11 transitions to the waiting state of ACT12, ACT13, ACT14, ACT15, ACT16, and ACT17. In this waiting state of ACT12 to ACT17, the processor 11 waits for one of the following to be made: registration request, save request, search request, file transmission request, original file request, or text update request.

[0040] Then, if the request data sent from the counter terminal 2 to request registration, as described above, is received by the communication unit 14, the processor 11 determines that registration has been requested and YES in ACT 12, and proceeds to ACT 18. As ACT18, processor 11 updates the correspondence management data DAA. That is, processor 11 sets the patient code and correspondence date and time included in the request data into fields FBA and FBB. Processor 11 also sets fields FBC and FBD of the new correspondence data generated here to blank or to a predetermined null value. Processor 11 may also generate correspondence data consisting only of fields FBA and FBB.

[0041] As ACT19, processor 11 sends a response to counter terminal 2 indicating that registration in response to the registration request has been completed. For example, processor 11 sends predetermined response data for completion to counter terminal 2 from communication unit 14 to communication network 200. When this response data is transmitted to counter terminal 2 via communication network 200, communication unit 26 receives the response data. After this, processor 11 continues ACT12~ Return to ACT17's standby state.

[0042] At the counter terminal 2, the processor 21 requests registration as ACT5 in Figure 3, and then proceeds to ACT6. As ACT6, the processor 21 checks whether a communication error occurred during the registration request described above. If the response data for the completion response is received by the communication unit 26 as described above, the processor 21 determines that there was no communication error and determines NO, and proceeds to ACT7. As ACT7, processor 21 updates the registration screen to reflect the registration results of this operation. After this, processor 21 returns to the standby state of ACT2.

[0043] Figure 6 shows an example of a registration screen. Figure 6 shows an example where, starting from the registration screen shown in Figure 4, processing begins for a patient with the patient code "01234567890123456789". The registration screen shown in Figure 6 has an added area ARB compared to the registration screen shown in Figure 4, and the patient code obtained in ACT3 is displayed in this area ARB. In addition, the first row of the list displayed in the area ARA of the registration screen in Figure 6 shows the result of this registration.

[0044] On the other hand, if processor 21 confirms that some kind of communication error has occurred, it determines YES in ACT6 and proceeds to ACT8. As ACT8, processor 21 updates the registration screen to indicate that a communication error has occurred.

[0045] Figure 7 shows an example of a registration screen. Figure 7 shows an example where a communication error occurs when requesting registration for a patient with the patient code "01234567899999999999", starting from the state where the registration screen shown in Figure 4 is displayed. The registration screen shown in Figure 7 has an added area ARB compared to the registration screen shown in Figure 4, and this area ARB displays the patient code obtained in ACT3. Furthermore, the first line of the list displayed in the area ARA in Figure 7 shows the result of this registration, and includes the string "NG" to indicate an error. Additionally, the registration screen shown in Figure 7 has an added string CSB compared to the registration screen shown in Figure 4. The string CSB represents a text message indicating that a communication error occurred.

[0046] After updating the registration screen in ACT8 in Figure 3, processor 21 proceeds to ACT9. As ACT9, processor 21 performs communication error processing. For example, processor 21 may operate the sound device included in input / output unit 24 to output an audio message such as "Communication error." Alternatively, processor 21 may perform other appropriate processing as communication error processing, such as flashing the alarm lamp included in input / output unit 24. After this, processor 21 returns to the waiting state of ACT2.

[0047] If an inappropriate barcode is scanned, such as a barcode displayed on a prescription that is not subject to processing at the counter where counter terminal 2 is used, the code obtained in ACT3 is not a valid patient code. In this case, processor 21 determines NO in ACT4 in Figure 3 and proceeds to ACT10. As ACT10, processor 21 updates the registration screen to indicate that a read error has occurred.

[0048] Figure 8 shows an example of a registration screen. Figure 8 shows an example where, starting from the registration screen shown in Figure 4, a code with a different system from the regular patient code, "9999578e84636d0000xx," is obtained. The registration screen shown in Figure 8 has an added area ARB compared to the registration screen shown in Figure 4, and this area ARB displays the patient code obtained in ACT3. Furthermore, the first line of the list displayed in the area ARA in Figure 8 shows the result of this registration, and includes the string "NG" to indicate an error. Additionally, the registration screen shown in Figure 8 has the string CSC added compared to the registration screen shown in Figure 4. The string CSC represents a text message indicating that registration failed.

[0049] After updating the registration screen in ACT10 in Figure 3, processor 21 proceeds to ACT11. As ACT11, processor 21 performs read error processing. For example, as read error processing, processor 21 may operate the sound device included in input / output unit 24 to output an audio message such as "Registration failed." Alternatively, as read error processing, processor 21 may perform other appropriate processing, such as flashing the alarm lamp included in input / output unit 24. After this, processor 21 returns to the waiting state of ACT2.

[0050] If the registration of the patient is successful, the store clerk will then provide patient care based on the prescription. This patient care may follow a procedure like the following: First, the clerk requests the dispensing staff to prepare the medication based on the prescription. Next, once the dispensing is complete, the clerk hands the prepared medication to the patient while explaining it. Finally, the clerk processes the payment for the medication.

[0051] At the counter terminal 2, when the processor 21 is instructed to perform speech recognition processing by a predetermined operation at the input / output unit 24, it starts speech recognition processing based on the speech recognition program PRD. The store clerk may keep the speech recognition processing running continuously for the duration of any patient interaction, or may start it as needed.

[0052] Figure 9 is a flowchart of the speech recognition process. As ACT31, the processor 21 displays a start command screen on the touch panel provided on the input / output unit 24. The start command screen is a GUI screen for receiving start commands for voice recognition.

[0053] Figure 10 shows an example of a start instruction screen. The start command screen includes several display objects, as shown in Figure 10, one of which is a button called BUB. The button BUB is a soft key used by the operator to initiate voice recognition. If a store employee wants to start voice recognition for patient-related conversations, they can initiate the recognition process through a predetermined operation, such as tapping button BUB.

[0054] After the processor 21 displays the start instruction screen as ACT31 in Figure 9, it proceeds to ACT32. As ACT32, processor 21 waits for a start command. Then, if the processor 21 is instructed to start speech recognition as described above, it determines it is YES and proceeds to ACT33.

[0055] As ACT33, the processor 21 initiates speech recognition. For example, the processor 21 requests the cloud server (not shown in Figure 1) to initiate speech recognition. After this, the processor 21 sends the patient voice data and store clerk voice data output from the microphone unit 5 to the cloud server. Alternatively, the processor 21 may perform information processing according to the speech recognition program PRD or another information processing program to generate text data representing the content of the utterances contained in the voice data of each patient voice and store clerk. As ACT34, the processor 21 displays a stop command screen on the touch panel provided in the input / output unit 24. The stop command screen is a GUI screen for receiving stop commands via voice recognition.

[0056] Figure 11 shows an example of a stop instruction screen. The stop command screen includes multiple display objects, as shown in Figure 11, one of which is the BUC button. The BUC button is a soft key used by the operator to signal the stop of voice recognition. If a store employee wants to stop voice recognition related to patient interaction, they can do so by performing a predetermined action, such as tapping button BUC.

[0057] After the processor 21 displays the stop instruction screen as ACT34 in Figure 9, it proceeds to ACT35. As ACT35, the processor 21 waits for a stop command. For example, while waiting for a stop command, the processor 21 continues to send patient voice data and store clerk voice data output from the microphone unit 5 to the cloud server. Then, if the processor 21 receives a command to stop voice recognition as described above, it determines it is YES and proceeds to ACT36.

[0058] As ACT36, processor 21 stops speech recognition. In other words, processor 21 requests the aforementioned cloud server to stop speech recognition after terminating the transmission of patient voice data and store clerk voice data. As ACT37, processor 21 obtains the text file generated by speech recognition from the aforementioned cloud server. Alternatively, if the cloud server transmits the text data generated by speech recognition in real time, processor 21 could collect this text data while in the waiting state as ACT35 and then convert this collected text data into a text file as ACT37.

[0059] The text file, for example, contains content such as: "2022 / 01 / 01 16:10:10,1,Welcome,2022 / 01 / 01 16:10:13,2,Thank you,2022 / 01 / 01 16:10:20,1,I will take your prescription,2022 / 01 / 01 16:10:31,2,Oh, your medication record book,2022 / 01 / 01 16:10:37,1,Thank you,2022 / 01 / 01 16:10:40,1,I will take it,2022 / 01 / 01 16:10:45,1,Please have a seat and wait." For example, in "2022 / 01 / 01 16:10:10,1,Welcome", "2022 / 01 / 01 16:10:10" represents the date and time of the audio being recognized, "1" indicates that it is the recognition result from the employee's voice data, and "Welcome" represents the text data that is the recognition result. In addition, the "2" in "2,Thank you" indicates that it is the recognition result from the patient's voice data.

[0060] As ACT38, processor 21 requests recording / playback device 1 to save the text file. For example, the request data for this request includes the text file acquired in ACT37 and data representing the period during which the audio data subject to speech recognition was acquired (hereinafter referred to as recognition period data). After this, processor 21 repeats the processing from ACT31 onward in the same manner as described above.

[0061] Now, in the recording and playback device 1, when the request data for saving a text file is received by the communication unit 14, the processor 11 determines that a save request has been made, and in ACT 13 in Figure 5, it determines YES and proceeds to ACT 20. As ACT20, processor 11 saves a text file in response to a request from the above-mentioned terminal 2. That is, for example, processor 11 adds the text file included in the request data to the text file group FIB as an edited text file, while associating it with the recognition period data also included in the request data. The edited text file is a text file that can be updated by the editing process described later. Then processor 11 sets the file path of the saved edited text file as the edited file path to the field FBC of the corresponding data set in the last field of the corresponding management data DAA. After this, processor 11 returns to the waiting state of ACT12~ACT17.

[0062] Thus, the processor 11 acquires a text file, which is an example of text data, and stores this text file in the auxiliary storage unit 13. In other words, by having the processor 11 perform information processing based on the recording and playback program PRA, the computer with the processor 11 as its central component functions as a text acquisition means. Furthermore, the auxiliary storage unit 13, which stores a group of text files FIB containing the text file, which is an example of text data, is an example of a text storage means. In this way, the interaction with patients at the counter is recorded as a video file with audio and a text file representing the speech recognition results, and the status of the patient interaction is recorded as DAA (Data Acquisition Account) management data.

[0063] At the counter terminal 2, when a verifying person, such as a store clerk or manager, instructs the processor 21 to execute a search process through a predetermined operation at the input / output unit 24, the processor 21 starts the search process based on the video search program PRC. Figures 12, 13, and 14 are flowcharts of the search process. In Figure 12, the processor 21, designated as ACT51, displays a search screen on the touch panel provided in the input / output unit 24. The search screen is a GUI screen for patient-related searches.

[0064] Figure 15 shows an example of a search screen. The search screen shown in Figure 15 includes areas ARC, ARD, ARE, ARF, ARG, ARH, and buttons BUD and BUE. Areas ARC, ARD, ARE, and ARF represent the entered search criteria. Area ARG represents a list of patient responses that match the search criteria. Area ARH represents information about a single selected patient response. Button BUD is a soft key used by the operator to initiate a search. Button BUE is a soft key used by the operator to initiate video playback.

[0065] When processor 21 executes ACT51, no search conditions have been specified yet, and no search has been performed. Therefore, there is no information to be displayed in areas ARC, ARD, ARE, ARF, ARG, and ARH, and they are all left blank. However, Figure 15 shows areas ARC, ARD, ARE, ARF, ARG, and ARH with specific information displayed in order to clarify how various types of information are represented.

[0066] After the processor 21 displays the search screen as ACT51 in Figure 12, it proceeds to the waiting state of ACT52 and ACT53. As ACT52 and ACT53, the processor 21 waits for an instruction to execute a search or for an operation to input search conditions. The store clerk performs a predetermined operation, for example using the input / output unit 24, to input search criteria for narrowing down several patient interactions from those recorded by the recording / playback device 1. In response, the processor 21 determines YES in ACT 53 and proceeds to ACT 54.

[0067] As ACT54, processor 21 modifies the search conditions according to the operation performed. Processor 21 also updates the information displayed in areas ARC, ARD, ARE, ARF, ARG, and ARH as appropriate to reflect the modified search conditions. After this, processor 21 returns to the waiting state for ACT52 and ACT53.

[0068] Once the verification operator has finished setting the desired search conditions, they instruct the search to be executed by performing a predetermined operation, such as tapping the BUD button on the search screen. In response, the processor 21 determines YES in ACT52 and proceeds to ACT55. As ACT55, the processor 21 requests a search from the recording / playback device 1. For example, the request data for this request from the processor 21 includes data representing the set search conditions.

[0069] Now, in the recording and playback device 1, when the request data for a search request is received by the communication unit 14, the processor 11 determines that a search request has been made, and in ACT 14 in Figure 5, it determines YES, and proceeds to ACT 21. As ACT21, processor 11 extracts all corresponding data from correspondence management data DAA that match the search conditions represented by the data included in the request data.

[0070] As ACT22, processor 11 notifies the customer service terminal 2 of the extracted corresponding data. In other words, processor 11 sends notification data containing the extracted corresponding data from communication unit 14 to communication network 200 addressed to customer service terminal 2. After this, processor 11 returns to the waiting state for ACT12 to ACT17.

[0071] When the above notification data is transmitted to the counter terminal 2 via the communication network 200, the communication unit 26 receives the notification data. At the counter terminal 2, the processor 21 requests a search as ACT55 in Figure 12, and then proceeds to ACT56. As ACT56, processor 21 updates the search screen to display a list of patient responses for each response data included in the notification data in area ARG. Note that at this stage, area ARH does not display specific information regarding patient responses on the search screen.

[0072] With the updated search screen displayed, processor 21 enters a waiting state for ACT57, ACT58, and ACT59. In this waiting state for ACT57-ACT59, processor 21 waits for an instruction to execute a search, an instruction to play back content, or an input operation for search conditions.

[0073] If the verification operator changes the search criteria and restarts the search, they perform a predetermined operation to input the search criteria. In response, processor 21 determines YES in ACT59 and proceeds to ACT60. As ACT60, processor 21 modifies the search conditions according to the operation performed. Processor 21 also updates the information represented in areas ARC, ARD, ARE, ARF, ARG, and ARH as needed to reflect the modified search conditions. After this, processor 21 returns to the waiting state of ACT57-ACT59.

[0074] Once the verification operator has finished changing the desired search criteria, they instruct the system to execute the search by performing a predetermined operation, such as tapping the BUD button on the search screen. In response, the processor 21 determines YES in ACT57 and repeats ACT55 onwards as described above.

[0075] Once the verification operator finds a patient interaction that needs verification, they specify that interaction using a predetermined procedure and then instruct playback using a predetermined procedure, such as tapping the BUE button on the search screen. In response to this instruction, the processor 21 determines YES in ACT58 and proceeds to ACT61.

[0076] As ACT61, processor 21 starts playback processing to play a video related to the specified patient care. Specifically, processor 21 obtains a video file from recording / playback device 1 for a period including the date and time of the specified patient care. Processor 21 also obtains an edited text file from recording / playback device 1 that is associated with recognition period data representing the recognition period including the start date and time of the obtained video file. Specifically, processor 21 requests file transmission from recording / playback device 1, for example, along with notification of the date and time of the specified patient care. Processor 21 then displays a playback screen on the touch panel of input / output unit 24, showing the video and text represented by the video file and edited text file sent from recording / playback device 1, respectively, with the frames and text related to the same date and time synchronized. Processor 21 also outputs audio based on the audio data contained in the video file from the sound device of input / output unit 24.

[0077] When the processor 11 in the recording / playback device 1 receives a file transmission request as described above, it determines YES at ACT15 in Figure 5 and proceeds to ACT23. As ACT23, processor 11 retrieves the video file and edited text file corresponding to the request from the video file group FIA and the text file group FIB, and sends these video file and edited text file to the customer service terminal 2. For example, processor 11 retrieves video files from the video file group FIA to be sent, where the start and end dates and times represented by the time information contained in the video file are before and after the response date and time notified at the time of the request. Processor 11 may also retrieve other predetermined video files related to the timing before and after the above video file to be sent. That is, for example, it may retrieve a video file whose start and end dates and times represented by the time information contained in the video file are before and after the response date and time notified at the time of the request, and one video file each related to the timing immediately before and after that video file, as targets for transmission. Processor 11 also searches for the response data contained in the response management data DAA, in which the customer service code associated with customer service terminal 2 is set in field FAA, in which the response date and time notified at the time of the request is set in field FBB. The processor 11 then retrieves the edited text file identified by the edited file path set in the corresponding data field FBC from the text file group FIB as the target for transmission. The processor 11 then sends the retrieved video file and edited text file from the communication unit 14 to the communication network 200, destined for the terminal 2. After this, the processor 11 returns to the waiting state for ACT12 to ACT17.

[0078] Figure 16 shows an example of a playback screen. The playback screen shown in Figure 16 is an example of a screen that appears when the button BUE is tapped on the search screen shown in Figure 15. The playback screen includes areas ARI, ARJ, and buttons BUF, BUG. The playback screen also includes display objects equivalent to the areas ARC, ARD, ARE, ARF, ARG, ARH, and button BUD in the search screen shown in Figure 15; however, these display objects are not interactable, for example, by being grayed out.

[0079] Area ARI displays the currently playing video within a hatched area and also represents multiple GUI objects for controlling video playback. Area ARJ represents the text of the editing text file. Button BUF is a soft key used by the user to display the editing window. Button BUG is a soft key used by the user to display the search screen.

[0080] Thus, the processor 21 displays the video data represented by the video file by displaying a playback screen. Thus, by the processor 21 executing information processing based on the video search program PRC, the computer with the processor 21 as its central component functions as a video display means. The processor 21 also displays text representing the audio collected at the time the displayed video was filmed by displaying a playback screen. Thus, by the processor 21 executing information processing based on the video search program PRC, the computer with the processor 21 as its central component functions as a text display means. The verification worker can confirm the content of what the store clerk and the patient said during patient interaction based on the audio output from the sound device and the text displayed in area ARJ.

[0081] However, the text displayed in area ARJ may differ from the actual spoken content due to errors in speech recognition. If the verification worker notices such an error and believes it should be corrected, they will instruct the display of the editing window by performing a predetermined operation, such as tapping button BUF. If the verification worker wishes to re-examine the search screen, they will instruct the display of the search screen by performing a predetermined operation, such as tapping button BUG.

[0082] After starting the playback process as ACT61 in Figure 12, processor 21 continues the playback process while waiting for ACT62 and ACT63. In this waiting state for ACT62 and ACT63, processor 21 waits for instructions to display the editing window or the search screen. Then, if an instruction to display the search screen is received as described above, processor 21 determines YES in ACT63 and proceeds to ACT64. As ACT64, processor 21 displays the search screen on the touch panel again, which was displayed before the playback screen was displayed in ACT61. After this, processor 21 returns to the standby state of ACT57-ACT59.

[0083] If the processor 21 is instructed to display the editing window as described above, it determines YES in ACT62 and proceeds to ACT71 in Figure 13. As ACT71, processor 21 causes an editing window to pop up on the touch panel provided on input / output unit 24. The editing window is a GUI screen for editing the text represented in the editing text file.

[0084] Figure 17 shows an example of a screen displaying a pop-up editing window (WIA). The editing window WIA includes the area ARK and buttons BUH, BUI, BUJ, BUK, and BUL. Area ARK represents the entire text that was displayed in area ARJ of the playback screen when the editing window was displayed. Button BUH is a soft key used by the operator to start text editing. Button BUI is a soft key used by the operator to save the edited text. Button BUJ is a soft key used by the operator to end text editing. Button BUK is a soft key used by the operator to display the original text before editing when the edited text is displayed in area ARK. Button BUL is a soft key used by the operator to send a text file representing the text displayed in area ARK to the outside of the recording system 100.

[0085] After displaying the editing window as ACT71 in Figure 13, processor 21 enters a waiting state for ACT72, ACT73, ACT74, and ACT75. In this waiting state for ACT72 to ACT75, processor 21 waits for one of the following commands to be received: termination command, original display command, transmission command, or editing start command.

[0086] The verification worker checks the text displayed in area ARK, and if they determine that no editing is necessary, they signal termination by performing a predetermined operation, such as tapping button BUJ. In response, processor 21 determines YES in ACT72 and proceeds to ACT76. As ACT76, processor 21 closes the editing window WIA. Then processor 21 returns to the waiting state of ACT62 and ACT63 in Figure 12.

[0087] The text displayed in area ARK of the editing window WIA may have been edited in the past. If the reviewer wishes to view the original text before such editing, they can instruct the reviewer to display the original text by performing a predetermined operation, such as tapping the button BUK. In response, processor 21 determines YES in ACT73 and proceeds to ACT77.

[0088] As ACT77, processor 21 executes the original text display process to display the original text. Specifically, processor 21 requests the original text file from recording / playback device 1, and if the original text file is sent from recording / playback device 1 in response, it pops up a new window displaying the text represented by the original text file, overlaid on the editing window WIN. After this, processor 21 waits for an instruction to finish displaying the original text, closes the above window, and returns to the waiting state of ACT72~ACT75.

[0089] Furthermore, when the processor 11 in the recording / playback device 1 receives a request for the original text file mentioned above, it determines YES at ACT16 in Figure 5 and proceeds to ACT24. As ACT24, processor 11 retrieves the original text file that matches the request from the text file group FIB and sends the original text file to the customer terminal 2. Specifically, processor 11 retrieves the original text file identified by the original file path set in the field FBD of the corresponding data found in ACT23 from the text file group FIB. Then processor 11 sends the original text file to customer terminal 2 from the communication unit 14 to the communication network 200. Note that if the edited text file sent in ACT23 has never been edited, the original file path is not set in the field FBD of the corresponding data found in ACT23. In this case, since the edited text file sent in ACT23 is equivalent to the original text file, processor 21 sends the edited text file sent in ACT23 as the original text file. Alternatively, if the original file path is not set, processor 11 may notify that the original text file does not exist instead of sending the original text file. Processor 11 then returns to the waiting state for ACT12 to ACT17.

[0090] Furthermore, if the processor 11 notifies that the original text file does not exist, as described above, the processor 21 at the counter terminal 2 will instruct the input / output unit 24 to display information to the operator confirming that the text already shown in area ARK in ACT77 in Figure 13 is the original file.

[0091] If the verification operator decides to send a text file representing the text displayed in area ARK to the outside of the recording system 100, they will instruct the transmission by performing a predetermined operation, such as tapping the BUL button in the editing window. Upon receiving this instruction, the processor 21 determines YES at ACT74 in Figure 13 and proceeds to ACT78.

[0092] As ACT78, processor 21 sends a text file representing the text displayed in area ARK to a predetermined destination. Alternatively, processor 21 may accept the destination specification and then send the text file to that specified destination. After this, processor 21 returns to the waiting state for ACT82~ACT86.

[0093] The verifier checks the text displayed in area ARK, and if they decide to edit it, they instruct the editing to begin by performing a predetermined operation, such as tapping button BUH in the editing window WIA. Upon receiving this instruction, processor 21 determines YES in ACT75 and proceeds to ACT81 in Figure 14.

[0094] As ACT81, processor 21 copies a text file representing the text displayed in area ARK to generate a working file. After this, processor 21 enters a waiting state for ACT82, ACT83, ACT84, ACT85, and ACT86. In this waiting state for ACT82-ACT86, processor 21 waits for one of the following commands to be received: termination command, original display command, transmission command, editing operation, or save command.

[0095] If an instruction to display the original document is given, processor 21 determines YES in ACT83 and proceeds to ACT87, where it executes the original document display process in the same way as ACT77 in Figure 13. After this, processor 21 returns to the waiting state of ACT82 to ACT86. If a transmission command is received, processor 21 determines YES at ACT84 and proceeds to ACT88, where it transmits the text file in the same manner as ACT78 in Figure 13. However, in this case, processor 21 uses a working file as the text file to be transmitted. After this, processor 21 returns to the waiting state of ACT82 to ACT86.

[0096] If the verifier edits the text displayed in area ARK, they perform a predetermined operation for that purpose. This operation may be similar to the operations performed for text editing in various existing information processing devices. For example, if the verifier notices that "A Oyaku Teho" is incorrectly displayed as "Aoyaku Teho," they perform an operation to replace "Ao" with "Ao." In response to such an editing operation, processor 21 determines YES in ACT85 and proceeds to ACT89. At this point, processor 21 has input an instruction to change the content of the text as an editing operation. In other words, by having processor 21 execute information processing based on the video search program PRC, the computer with processor 21 as its central component functions as an input means.

[0097] As ACT89, processor 21 updates the working files in response to the operation. As ACT90, processor 21 updates the editing window WIA so that the text represented by the updated working file is displayed in area ARK. After this, processor 21 returns to the waiting state of ACT82~ACT86.

[0098] Once the reviewer has finished editing and decided to save the edited result, they instruct the system to save by performing a predetermined operation, such as tapping a button BUI in the editing window WIA. When this save instruction is received, the processor 21 determines YES in ACT86 and proceeds to ACT91. As ACT91, processor 21 requests the recording / playback device 1 to update the editing text file, accompanied by a notification of the working file. After this, processor 21 returns to the waiting state of ACT82~ACT86.

[0099] When the processor 11 in the recording / playback device 1 receives the above update request, it determines YES in ACT17 in Figure 5 and proceeds to ACT25. As ACT25, processor 11 checks whether the original text file corresponding to the edited text data to be updated is already saved. For example, processor 11 checks whether the original file path is set in the FBD field of the corresponding data found in ACT23. If the original file path is not set, processor 11 determines NO, stating that the original text file is not saved, and proceeds to ACT26.

[0100] As ACT26, processor 11 copies and saves the edited text file sent in ACT23 within the text file group FIB, and uses this as the original text file. Furthermore, processor 11 sets the file path of this original text file as the original file path in the corresponding data field FBD found in ACT23.

[0101] After completing ACT26, processor 11 proceeds to ACT27. Also, if processor 11 confirms in ACT25 that the relevant original file path is set, it determines that the original text file is saved and YES, skips ACT26, and proceeds to ACT27.

[0102] As ACT27, processor 11 overwrites the edit text file identified by the edit file path set in the corresponding data field FBC found in ACT23 with the notified working file, thereby rewriting the edit text file. After this, processor 11 returns to the waiting state for ACT12 to ACT17. Thus, by having the processor 11 perform information processing based on the recording and playback program PRA, the computer with the processor 11 as its central component functions as an update means.

[0103] If the verifier wishes to finish editing, they will indicate termination by performing a predetermined operation, such as tapping button BUJ displayed in the editing window WIA. In response, processor 21 determines YES at ACT82 in Figure 14 and proceeds to ACT92. As ACT92, processor 21 closes the editing window WIA. Then processor 21 returns to the waiting state of ACT62 and ACT63 in Figure 12. Note that if the working file has been updated but not saved at this time, the edits made by the reviewer up to this point will be discarded. Therefore, it is also possible to notify the reviewer that any unsaved edits will be invalid. Furthermore, it is also possible to have the reviewer specify whether or not to terminate, and if they specify not to terminate, return to the waiting state of ACT82~ACT86 in Figure 14.

[0104] As described above, the recording system 100 displays video footage of patient interactions and text transcripts of events related to those interactions, synchronized in time. In this way, it facilitates post-interaction review of face-to-face interactions between multiple people that involve conversation.

[0105] The recording system 100 also records the status of patient care, and when one of the past patient care interactions is selected, it displays a video and text related to that interaction. This makes it easy to review the spoken content related to patient care interactions that need to be checked, along with the video.

[0106] Furthermore, the recording system 100 updates the text in response to instructions from the verification worker. Therefore, even if an error occurs in speech recognition, the verification worker can easily correct the resulting deficiencies in the text.

[0107] This embodiment can be modified in various ways as follows: The processor 11 may generate the playback screen in the recording and playback device 1. In this case, the functions of video display and text display will be realized by a computer with the processor 11 as the central component.

[0108] The video file group FIA and the text file group FIB may be stored in the auxiliary storage unit 23 at the counter terminal 2, and the recording and playback processing may be performed by the processor 21 at the counter terminal 2. In this case, the functions of video acquisition and text acquisition will be implemented by a computer centered around the processor 21. In this case, the updating of the text file to reflect the editing results should also be performed by the processor 21. When such an update of the taste file is performed, the function of the update means will be implemented by a computer centered around the processor 21.

[0109] At least one of the search process and the speech recognition process may be performed by an information processing device other than the counter terminal 2. In other words, the search process may be performed by an information processing device other than the counter terminal 2. Similarly, the speech recognition process may be performed by an information processing device other than the counter terminal 2. In this case, the information processing device that performs the search process and the information processing device that performs the speech recognition process may be the same or different.

[0110] In ACT20, processor 11 may simply add only the original text file to the text file group FIB. Alternatively, if editing is performed, processor 11 may generate an edited text file reflecting the edits and add it to the text file group FIB.

[0111] Each of the functions realized by processors 11 and 21 through information processing can also be partially or entirely realized by hardware that performs non-program-based information processing, such as logic circuits. Furthermore, each of the above functions can also be realized by combining the above-mentioned hardware, such as logic circuits, with software control.

[0112] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. The invention described in the original claims of this application is listed below. [Note 1] A video acquisition means that acquires video data representing a video shot within a predetermined shooting range, A video storage means for storing video data acquired by the aforementioned video acquisition means, A text acquisition means that acquires text data generated by recognizing sound collected around the aforementioned shooting range, A text storage means for storing text data acquired by the text acquisition means, A video display means for displaying a video represented by video data stored in the video storage means, A text display means selects text data from among the text data stored by the text storage means, which recognizes and generates text data generated by the video display means by recognizing sound collected around the shooting range at the timing of shooting the video to be displayed, and displays the text represented by said text data. A video playback system equipped with the following features. [Note 2] The shooting means further comprises shooting the aforementioned shooting range and generating video data, The video acquisition means acquires video data generated by the shooting means. The video playback system described in Appendix 1. [Note 3] The system further comprises a sound collection means for collecting sound around the shooting range, The text acquisition means acquires text data obtained by recognizing the sound collected by the sound collection means. The video playback system described in Appendix 1. [Note 4] An input means for inputting instructions to change the content of the text displayed by the text display means, An update means updates text data stored in the text storage means to change the content of the text displayed by the text display means in accordance with a change instruction entered by the input means, A video playback system as described in Appendix 1, further comprising the features described in Appendix 1. [Note 5] A server device equipped with the video acquisition means, video storage means, text acquisition means and the text storage means, A terminal device equipped with the aforementioned video display means and the aforementioned text display means, A video playback system as described in Appendix 1, equipped with the following features. [Explanation of symbols]

[0113] 1...Recording and playback device, 2...Counter terminal, 3...Camera unit, 4...Scanner unit, 5...Microphone unit, 11,21...Processor, 12,22...Main memory unit, 13,23...Auxiliary memory unit, 14,26...Communication unit, 15,27...Transmission line, 24...Input / output unit, 25...Interface unit, 100...Recording system, 200...Communication network.

Claims

1. A video acquisition means that acquires video data of a shooting range in which customers and store employees are captured, with the shooting range being a predetermined shooting range. A video storage means that stores multiple video files, each containing video data extracted from video data acquired by the aforementioned video acquisition means, and including time information representing the start date and end date and time of the said video data. A text acquisition means that acquires a text file generated by recognizing audio collected while directed towards the shooting range during the period from when a start instruction is given to the store employee to when a stop instruction is given regarding customer service to the customer, A text storage means that stores the text file acquired by the text acquisition means in association with recognition period data representing the period during which the text file was acquired, A correspondence data storage means stores correspondence data for each of the multiple customer interactions, which is included in the barcode data attached to the prescription submitted by the customer in relation to the customer interaction, and associates an identifier that identifies the customer with the date and time of the interaction. When a customer response relating to any of the multiple corresponding data stored in the corresponding data storage means is specified, the video storage means selects a video file from among the video files stored in the video storage means that contains the time information representing the period including the date and time of the response included in the corresponding data relating to the specified customer response, and displays a video corresponding to that video file. A text display means selects from among the text files stored by the text storage means that a text file is associated with recognition period data representing a period including the start date and time represented by the time information contained in the video file displayed by the video display means, and displays the text represented by that text file. A video playback system equipped with the following features.

2. The system further comprises a shooting means that captures the aforementioned shooting range as a video and generates video data, The video acquisition means acquires video data generated by the shooting means. The video playback system according to claim 1.

3. The system further comprises a sound collection means for collecting sound around the aforementioned shooting range, The text acquisition means acquires a text file obtained by recognizing the sound collected by the sound collection means. The video playback system according to claim 1.

4. An input means for inputting instructions to change the content of the text displayed by the text display means, An update means updates a text file stored in the text storage means to change the content of the text displayed by the text display means in accordance with a change instruction entered by the input means, The video playback system according to claim 1, further comprising:

5. A server device comprising the video acquisition means, the video storage means, the text acquisition means, the text storage means, and the corresponding data storage means, A terminal device equipped with the aforementioned video display means and the aforementioned text display means, A video playback system according to claim 1, comprising:

Images