Record support system and record support method

Abstract

Traditional record-keeping methods presented challenges such as a heavy burden on record-keepers and difficulties in ensuring reliability and auditability. [Solution] The present invention relates to a record support system and method that extract contextual information from multimodal inputs such as voice, handwriting, and text, and make it possible to save it as a structured record. The input data is decomposed and classified into semantic units through natural language processing, and signatures and hash values ​​are added to ensure the reliability and resistance to tampering of the record. Furthermore, a chain structure is formed between the recorded data to enhance verifiability and auditability. The present invention can automatically generate structurally verifiable records while minimizing the operational burden on the recorder, and is applicable to various fields where the legitimacy of records is questioned, such as nursing care, medical care, education, and administration.

Description

TECHNICAL FIELD

[0001] The present invention relates to a record support technology that enables acquisition of information from natural business activities such as voice and handwriting, and enables generation, storage, and auditing of structured recorded documents. In particular, the present invention relates to a configuration that ensures accuracy, reliability, and admissibility as evidence in fields requiring accurate records such as medical care, nursing care, legal affairs, finance, and welfare, while eliminating the need for users to consciously perform recording acts.

BACKGROUND ART

[0002] Background Art 0002 Conventionally, business records have been created by workers writing and inputting each time, but problems such as forgetting to record, transcription errors, subjective expressions, and tampering have become apparent. 0003 In recent years, support technologies such as voice input and OCR have been increasingly introduced, but they are fragmented and unstructured, leaving issues in the coverage, reliability, and auditability of records.

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0003] Problems to be Solved by the Invention 0004 An object of the present invention is to realize an environment in which work can be performed without making users conscious of recording, and to ensure that the records have accuracy and audit evidential admissibility that are legally and institutionally valid.

MEANS FOR SOLVING THE PROBLEMS

[0004] Means for Solving the Problems 0005 The present invention is a record support system combining a voice input unit, a handwritten character acquisition unit, a semantic analysis unit, a record generation unit, a reliability evaluation unit, a signature adding unit, and a structure storage unit, and is a record support method using the same. Brief Description of the Drawings 0006 Figure 1 is an overall diagram of the recording support AI agent of the present invention, illustrating the process by which records are automatically generated and saved from the user's natural behavior. 0007 Figure 2 shows the recording generation process flow using voice and handwritten information as input. 0008 Figure 3 shows the configuration of ensemble processing and confidence scoring using multiple generative AIs. Figure 4 shows the configuration in which hash signatures and distributed recording areas are stored during the record preservation process. 0010 Figure 5 shows the physical configuration of the servers, terminals, recording nodes, etc. that make up this system. Embodiment of the Invention 0011 The record support AI agent shown in Figure 1 consists of a voice acquisition means, a handwriting OCR means, a semantic analysis means, a record generation engine, an evaluation means, a record signing means, and a storage means. 0012 When a user speaks or takes handwritten notes during their normal work, audio and image data are acquired and serve as the starting point for processing. 0013 Audio data is converted to text using a speech-to-text conversion tool (e.g., Whisper), and image data is converted to text using OCR processing (e.g., Tesseract, Google OCR, etc.). 0014 This information is analyzed using semantic analysis tools to extract the business context, and then formatted into a record format by a document generation tool. Support for SOAP format, LIFE format, and facility-specific formats is possible. 0015 Multiple generative models (LLMs) generate output candidates, which are then scored based on factors such as confidence level, consistency with past history, etc. Candidates below a certain threshold are automatically discarded, and outputs with a confidence level above a certain level are adopted as records. 0016 Record documents are assigned hash values ​​and stored in a tamper-proof manner by the record storage method. The storage location can be local storage, an on-site NAS, or a blockchain-type distributed ledger; the storage method is selectable. 0017 As shown in Figure 5, this configuration consists of user terminals (smartphones, voice devices), edge processing nodes (e.g., Android devices with Termux), local servers (Ubuntu), and recording nodes (distributed storage). 0018 Each node has a processor, storage, and communication interface, and is responsible for one of the following processes: preprocessing, recording, signing, or storage of audio data. 0019 The audio conversion process is performed on the terminal or on a local server, and the data is automatically passed to the next step. In particular, in the Termux environment, automatic uploading of recorded files is possible via script. 0020 Record generation and evaluation are performed using lightweight models for edge processing, or heavy processing is distributed and performed on servers equipped with local GPUs. 0021 The hash signature method calculates and assigns a hash value, including time, terminal, and identifier, to the entire structure, which includes the recorded content and evaluation results. 0022 The storage means stores this structure in an encrypted or tamper-proof format on the recording node. Searching is possible by tag, score, time series, and user. 0023 As a result, the entire workflow of input → analysis → recording → saving can be completed without requiring explicit action from the employee. [Effects of the Invention]

[0005] The entire workflow—input → analysis → recording → saving—is completed without requiring any explicit action from the employee. [Brief explanation of the drawing]

[0006] [Figure 1] Conceptual Diagram [Figure 2]Ensemble Configuration Diagram [Figure 3] Speaker Identification Button Image Diagram [Figure 4] Speech-to-Text Conversion Diagram [Figure 5] Ensemble-RAG Cooperation Diagram [Mode for Carrying Out the Invention]

[0007] Embodiment of the Invention 0011 The record support AI agent shown in Figure 1 is composed of a voice acquisition unit, a handwriting OCR unit, a semantic analysis unit, a record generation engine, an evaluation unit, a record signature unit, and a storage unit. 0012 When a user makes utterances or writes handwritten memos during routine work, voice data and image data are acquired, which serves as the starting point of processing. 0013 The voice data is converted into a character string by a voice conversion unit (e.g., Whisper), and the image data is converted into text through OCR processing (e.g., Tesseract, Google OCR, etc.). 0014 For these pieces of information, the work context is extracted by the semantic analysis unit, and the information is formatted into a record format by the record document generation unit. Here, compatibility with the SOAP format, LIFE format, facility-specific proprietary formats, and the like can be achieved. 0015 Output candidates are generated by a plurality of generation models (a group of LLMs), and are scored based on reliability, consistency, consistency with past history, and the like. Candidates below the threshold are automatically discarded, and outputs with reliability above a certain level are adopted as records. 0016 A hash value is assigned to the record document, and the record document is stored in a tamper-proof manner by the record storage unit. The storage destination is local storage, an in-facility NAS, or a blockchain-type distributed ledger, and the storage method is selectable. 0017 As shown in Figure 5, this configuration consists of user terminals (smartphones, voice devices), edge processing nodes (e.g., Android devices with Termux), local servers (Ubuntu), and recording nodes (distributed storage). 0018 Each node has a processor, storage, and communication interface, and is responsible for one of the following processes: preprocessing, recording, signing, or storage of audio data. 0019 The audio conversion process is performed on the terminal or on a local server, and the data is automatically passed to the next step. In particular, in the Termux environment, automatic uploading of recorded files is possible via script. 0020 Record generation and evaluation are performed using lightweight models for edge processing, or heavy processing is distributed and performed on servers equipped with local GPUs. 0021 The hash signature method calculates and assigns a hash value, including time, terminal, and identifier, to the entire structure, which includes the recorded content and evaluation results. 0022 The storage method involves storing this structure in an encrypted or tamper-proof format on the recording node. Searching is possible by tag, score, time series, and user. [Examples]

[0008] Examples 0024 Example 1: An example of a vital sign recording support system in a nursing care setting will be described in detail below. 0025 (Occurrence and input of the situation) A caregiver takes vital signs of a facility resident (e.g., takes their temperature) and says, "36.8 degrees, no fever." This speech is captured by a voice input device on a smartphone. The smartphone is equipped with a Termux environment with a script that periodically uploads the recorded files. 0026 (Transferring audio files) The acquired audio files are temporarily stored on the device and then automatically transferred via HTTP to a local server within the facility. The transfer is performed via crontab processing on Termux or by an API. 0027 (Text conversion from audio) The audio data is converted into text by a speech-to-text conversion tool (such as Whisper) on the local server. As a result, the sentence "36.8 degrees, no fever" is obtained. 0028 (Semantic analysis) The string is passed to a semantic analysis tool, and internal natural language processing extracts structured items such as "body temperature = 36.8" and "symptoms = no fever." In addition, a comparison with past records is performed, and additional elements such as "no change" may be added. 0029 (Generating record documents) The extracted information is formatted into a SOAP-style report by a record generation device. The report items include subject ID, measurement time, measurement value, and condition evaluation, and are output in a format that conforms to the facility's format and LIFE report form. 0030 (Ensemble evaluation) For identical content, multiple generative models (LLMs) generate candidate records. Evaluation methods are applied to these candidates, and scoring is performed using metrics such as matching rate, lexical consistency, and historical log consistency. Outputs with a score of 85 or higher are automatically selected. 0031 (Sign and save) Selected record documents are assigned a hash value that integrates the record content, evaluation score, processing time, and processing terminal information using a hash signature method. This enables tamper detection. 0032 (Location where records are stored) The signed structures are stored via a storage mechanism in a local recording node and optionally in a blockchain-type distributed ledger. Tag information (e.g., vital signs, date and time, target ID) is added during storage to facilitate future searches. 0033 (Reference and output of records) The records can be accessed via a dedicated interface (UI or API), and can be searched by criteria such as user or date. Furthermore, the data can be exported as CSV or PDF files for LIFE reporting, supporting secondary use (insurance claims and reports). 0034 Example 2: The following details an implementation of OCR recording support starting from handwritten notes. 0035 (Occurrence and input of the situation) A caregiver notices a user saying, "I'm feeling well today," and writes it down on a piece of paper. This memo is then photographed using the camera function on the caregiver's smartphone and saved as an image. 0036 (Transferring image files) Image files are periodically uploaded to a local server within the facility via a Termux script on a smartphone. The image data is then transmitted to the recording and processing device using HTTP communication or SFTP. 0037 (Recognition of handwritten characters) The image is input into an OCR system, where handwritten characters are converted to text. Tesseract or a cloud-based OCR API is used for the OCR process, and the result after character recognition is obtained as "I'm feeling good today." 0038 (Semantic analysis and structuring) The acquired text is sent to a semantic analysis system, where it undergoes labeling for emotion, state, and behavior classification. In this example, it is tagged as "Mental state = Good" and converted into a structured item. 0039 (Generating record documents) The record generation method generates report data based on the analysis results. In SOAP format, the data is formatted so that the "Subjective Findings" column includes the statement, "The person said they were feeling well." 0040 (Ensemble evaluation) The LLM group generates candidate texts with identical content, and the evaluation method scores them using consistency, content validity, and vocabulary appropriateness as evaluation indicators. Candidates that score above a certain threshold (e.g., 80 points or higher) are selected. 0041 (Sign and save) The output of the record generation means is passed to the hash signing means and converted into a signed structure that includes the recording time, terminal identification, and score information. 0042 (Preservation process) The data is stored in a local database or in a distributed recording area using tamper-proof ledger technology. Meta tags such as "subjective evaluation" and "mental state" are used during searches. 0043 (Reference and reporting processing) Records can be viewed immediately from the management interface and are linked to functions for exporting them as part of report output or facility reports. In particular, it enables the automatic generation of records for subjective items such as LIFE. 0044 Example 3: The following shows an embodiment of voice input-based record-keeping support in home care. 0045 (Occurrence of the situation and acquisition of audio) A family member providing home care might say, "They slept well today," when checking on the patient's condition. This utterance is captured by the family member's Android device and recorded using a voice input device. The device is equipped with a Termux environment and has an automated recording script running in the background. 0046 (Local saving and transfer of audio files) Spoken audio is saved as a recording file on the device and automatically transferred to the facility server at regular intervals. A secure route via local Wi-Fi or VPN is used for communication. 0047 (Speech analysis processing) The speech conversion device on the local server analyzes the recorded data and extracts the string "I slept well today." 0048 (Contextual understanding and documentation) The semantic analysis tool determines from this string that "sleep state = good," classifies it into a sleep evaluation category, and generates structured items. It compares this with the previous day's record as needed. 0049 (Form generation) The record generation method outputs data in an XML structure based on the "LIFE submission format," and completes the report by including the relevant user ID and date in the record. 0050 (Evaluation and scoring) The output is input to multiple generative models, and a confidence score is calculated using evaluation metrics such as validity, consistency, and contextual coherence of each model's response. Candidate sentences that exceed a certain threshold are selected. 0051 (Signature process) The signature method adds a hash signature to the record structure, and the user terminal information and processing ID are included in the record. 0052 (Preservation process) The storage method involves saving records to a home-use recording node or a facility-side backup node, and synchronizing them with a distributed storage system as needed. 0053 (Submission and reference of documents) The completed forms are either linked to the local government's submission system via API or stored as USB output. Records can be shared among family members, visiting nurses, and facilities. 0054 Example 4: Family members keep records during home care. Speech is acquired via smartphone, recorded and converted in a local Termux environment, and then transferred to a recording server. The recording is submitted to the local government in LIFE format. [Industrial applicability]

[0009] Industrial applicability 0061 This invention is useful in all industries where accuracy and evidentiary value of records are required, and can be introduced into businesses that require institutionally based forms, such as healthcare, nursing care, legal affairs, finance, and education. 0062 This invention makes it possible to generate highly accurate record documents from natural actions such as speech and handwriting, without requiring conscious record-keeping by on-site staff, thereby achieving both a reduction in workload and ensuring the accuracy of records. 0063 Furthermore, the recorded data is stored in a structured format and is tamper-resistant through hash signatures and distributed storage, making it suitable for audits, legal evidence, and provision to third parties. 0064 Furthermore, it can automatically generate forms based on the recorded content and can be linked with documents submitted to local governments (e.g., LIFE reports, remuneration invoices, education evaluation reports), making it usable as a foundation for institutional digital transformation. 0065 In the future, we envision developing this as an international business record standard through PCT (Patent Cooperation Treaty) applications, and it will be easy to expand the output format and signature structure to suit the systems of each country. 0066 Based on the above, the present invention has broad applicability, extending beyond supporting individual tasks to encompass systems, audits, and international cooperation, and possesses extremely high industrial value as a core technology for next-generation record-keeping infrastructure. 0067 Furthermore, the nonce used in the hash signature may be dynamically determined by a program periodically constructed by a generating AI at each node. This ensures both randomness and structural consistency in the signature, improving the traceability of the recorded signature, preventing duplication, and enhancing the accuracy of tamper detection. The generating AI autonomously changes the signature configuration while referring to the state of each node and past nonce patterns. 0068 This configuration allows the signature pattern to change dynamically over time while maintaining consistency with the entire record structure, thus providing greater resistance to unauthorized access and tampering from external sources.

Claims

1. Claim 1 A record support system that acquires audio and / or text as input information, analyzes the information, and generates a structured record document, characterized in that it comprises one or more of the following: input means, semantic analysis means, record generation means, evaluation means, selection means, signing means, and storage means.

2. Claim 2 The record support system according to claim 1, characterized in that the storage means assigns hash signatures and / or identification information to the recorded documents, enabling tamper detection and integrity verification.

3. Claim 3 The recording support system according to claim 1 or 2, characterized in that the evaluation means calculates a reliability score of the recorded document and / or a degree of agreement with past records, and adds the score to the recording structure.

4. Claim 4 The record support system according to any one of claims 1 to 3, characterized in that the structure includes one or more of the following: tag information, history reference information, time-series classification information, and user identification information.

5. Claim 5 The record support system according to any one of claims 1 to 4, characterized in that the record document is automatically formatted into one or more output formats: SOAP format, local government designated format, or LIFE submission format.

6. Claim 6 The recording support system according to any one of claims 1 to 5, characterized in that the input information is audio information, or text information, and / or a handwritten memo image, and the handwritten memo image is converted to text by image-to-text processing such as OCR and then processed by the semantic analysis means.

7. Claim 7 The record support system according to any one of claims 1 to 6, characterized in that the storage means has a storage configuration using blockchain and / or distributed ledger nodes, and the record structure is maintained in a tamper-proof manner.

8. Claim 8 The record support system according to any one of claims 1 to 7, characterized in that the evaluation means uses a group of natural language processing models to score the record document using lexical consistency, contextual relevance, and consistency with similar records as evaluation indicators.

9. Claim 9 A record-keeping support system according to any one of paragraphs 1 to 8, characterized in that the tasks to be recorded are in fields where accuracy and evidentiary value are institutionally required, such as medical care, nursing care, legal affairs, finance, and education.

10. Claim 10 A recording support system according to any one of claims 1 to 9, characterized in that the input means, semantic analysis means, record generation means, evaluation means, selection means, signature means, and storage means functions automatically cooperate to complete the process without requiring explicit operation from the person being recorded.

11. Claim 11 A record support system according to any one of claims 1 to 10, characterized in that each node periodically constructs a program for generating hash values ​​used for signing in the record storage means, thereby ensuring randomness and structural consistency in the signature, and improving the traceability and tamper detection accuracy of the record signature.

12. Claim 12 A recording support method characterized by acquiring audio and / or text and / or memo images, analyzing them to generate a structured record document, and processing and / or saving the record document.

13. Claim 13 A record support method according to claim 12, characterized in that it includes a step of outputting and / or submitting the generated record document for audit or reporting.

Images