Multi-channel medical bill recognition method, device and equipment combined with large language model, storage medium and product

Abstract

The application discloses a multi-channel medical bill recognition method, device and equipment combined with a large language model, a storage medium and a product, relates to the technical field of bills, and comprises the following steps: in response to receiving a medical bill recognition request, obtaining medical bill picture data to be recognized; according to the medical bill picture data, extracting bill metadata of the medical bill; according to the bill type and / or the hospital name, dynamically matching and routing selecting the corresponding target recognition prompt word from the preset prompt word library; obtaining the target channel configuration corresponding to the medical bill recognition request, calling the corresponding large language model interface; inputting the medical bill picture data and the target recognition prompt word into the large language model for reasoning to obtain an original output result; and performing engineering specification processing on the original output result to generate structured bill recognition data and output. The application significantly improves the efficiency, generalization ability and stability of medical bill recognition, and greatly reduces the artificial input cost and maintenance complexity.

Description

Technical Field

[0001] This application relates to the field of invoice technology, and in particular to a method, apparatus, device, storage medium and product for recognizing multi-channel medical invoices that combines a large language model. Background Technology

[0002] In corporate medical reimbursement processes, each reimbursement involves numerous detailed items, which currently rely heavily on manual completion and review. Finance personnel must manually verify a large number of paper or electronic invoices. Meanwhile, medical invoices from hospitals at all levels have complex formats and extremely low standardization. Significant differences exist in layout, field naming, and printing styles between different hospitals and even within the same hospital for different services (outpatient, inpatient, and medical insurance settlement). This highly heterogeneous environment poses a significant challenge to automating invoice recognition.

[0003] While traditional OCR recognition solutions have been applied in some standardized invoice scenarios, they fundamentally rely on fixed templates or training with large amounts of labeled data, resulting in severely insufficient generalization ability. Once a new format or minor format adjustment is encountered, the original template becomes ineffective, requiring the collection of new samples for model optimization. Furthermore, OCR output consists only of discrete text blocks, necessitating the writing of numerous manually crafted post-processing rules to assemble them into structured fields. These rules expand linearly with the number of formats, leading to extremely high maintenance costs and a high likelihood of conflicts. Directly using a general-purpose large model for end-to-end recognition faces practical bottlenecks such as high input / output token overhead, low efficiency in processing large images, and the difficulty of adapting a single model channel to the personalized access needs of different enterprises. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and product for recognizing multi-channel medical invoices by combining a large language model, aiming to systematically solve the problems of poor generalization, high maintenance cost and rigid channel adaptation in the automated recognition of medical invoices.

[0005] To achieve the above objectives, this application proposes a method for recognizing multi-channel medical invoices by combining a large language model, the method comprising: In response to receiving a medical invoice recognition request, the system obtains the image data of the medical invoice to be recognized. Based on the medical invoice image data, extract the invoice metadata; the invoice metadata includes at least the invoice type and hospital name; Based on the ticket type and / or the hospital name, dynamically match and route the corresponding target recognition prompt words from the preset prompt word library; Obtain the target channel configuration corresponding to the medical invoice recognition request, and call the corresponding large language model interface according to the target channel configuration. The target channel configuration is determined by routing from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request. The medical invoice image data and the target recognition prompts are input into the large language model for inference to obtain the original output results; The original output results are processed in an engineering manner to generate structured ticket recognition data and output it.

[0006] In one embodiment, before the step of obtaining the image data of the medical invoice to be identified in response to receiving a medical invoice recognition request, the method further includes: Construct a prompt word library, and classify and store the pre-collected sample prompt words according to the recognition stage and applicable objects into the prompt word library. Each prompt word contains at least one of the following: task description, field definition, field processing rules, output format constraints, and output examples. Configure a unified processor interface for different model suppliers, and register the channel type and corresponding enterprise identifier of each processor in the channel configuration table. The channel configuration table includes multiple pre-registered model service channels and their corresponding enterprise identifiers.

[0007] In one embodiment, extracting the medical invoice metadata based on the medical invoice image data includes: The hospital / patient identification prompts are used to perform first-level identification on the medical bill image data, extracting the hospital name and patient name to obtain the first-level identification result; Based on the first-level identification result, the second-level identification is performed by calling the invoice type judgment prompt words to determine the specific invoice type code of the medical invoice and obtain the invoice metadata.

[0008] In one embodiment, the step of dynamically matching and routing the corresponding target recognition prompt word from a preset prompt word library based on the ticket type and / or the hospital name includes: If the invoice type code is an invoice detail, then the corresponding central unified invoice prompt word is queried according to the invoice type code and used as the target identification prompt word; If the invoice type code is a list detail, then the hospital name is used as the routing key to query the hospital's exclusive list prompt words, which are used as the target identification prompt words; If no matching specific prompt word is found, the system will automatically downgrade and use the default fallback prompt word as the target identification prompt word.

[0009] In one embodiment, the step of obtaining the image data of the medical invoice to be identified in response to receiving a medical invoice recognition request includes: In response to receiving a medical invoice recognition request, the original medical invoice image to be recognized is obtained; Determine whether the file size or resolution of the original medical invoice image exceeds a preset threshold; If so, then perform proportional resolution compression based on the target width while maintaining the image aspect ratio. Read the EXIF ​​orientation parameters of the original medical bill image, and rotate and align the image according to the orientation parameters; The processed image is converted into Base64 encoded data to serve as standardized medical invoice image data.

[0010] In one embodiment, the design of the prompt words further includes: At the beginning of each prompt word, a professional role description is set for the model to activate the model's knowledge in the field of medical invoices; the professional role description is used to instruct the large language model to perform the recognition task as a medical invoice reviewer. Each prompt word is structurally assembled and configured to contain at least one of the following independent adjustable modules: task description module, field definition module, field processing rules module, output format constraint module, and output example module; The assembled prompt text is bound to the model inference parameters for joint fine-tuning; the model inference parameters include at least a temperature parameter, a maximum number of output tokens dynamically calculated based on the estimated number of output records, and a stop sequence for truncating redundant outputs. The complete medical bill recognition task is broken down into multiple cascaded lightweight quantum tasks according to the execution order, and each subtask is configured with an independent prompt word. The subtask includes at least a first-level prompt word for extracting the hospital name and the patient's name, a second-level prompt word for determining the bill type code, and a third-level prompt word for extracting the detailed expense list. The output of the previous subtask serves as the basis for prompt word routing or dynamic assembly in the next subtask.

[0011] In one embodiment, the output format of the original output result is a custom delimiter format; the custom delimiter format concatenates each target field into a single line of text according to a predetermined delimiter order.

[0012] In one embodiment, obtaining the target channel configuration corresponding to the medical invoice recognition request and calling the corresponding large language model interface according to the target channel configuration includes: The system queries the channel configuration table based on the enterprise identifier carried in the medical invoice recognition request to obtain the active channel configuration; if no channel is found or the current channel is disabled, it will automatically downgrade to the preset default channel. Traverse the set of registered processors, match the corresponding processor according to the channel type, and the matched processor converts the unified recognition request containing the medical invoice image data, the target recognition prompt words and extended configuration into the specific interface protocol of the corresponding model supplier through the internal adapter for invocation; After the call ends, the adapter parses the original response returned by the model supplier into a unified recognition result structure, which is then used as the original output result.

[0013] In one embodiment, the step of performing engineering standardization processing on the original output result to generate structured ticket recognition data and outputting it includes: Remove extra spaces and newline characters from the original output and extract the valid data rows; The system performs data type compatibility processing on each field of the extracted valid data rows, converting numeric fields to numeric types; maps preset enumerated fields to system standard dictionary items, filtering out non-standard content; fills empty strings and records exception logs for fields that fail to be parsed; and generates and outputs structured ticket recognition data.

[0014] Furthermore, to achieve the above objectives, this application also proposes a recognition device for multi-channel medical invoices that combines a large language model, the device comprising: The acquisition module is used to acquire the image data of the medical invoice to be recognized in response to a received medical invoice recognition request. The extraction module is used to extract the invoice metadata of the medical invoice based on the image data of the medical invoice; the invoice metadata includes at least the invoice type and the hospital name; The prompt word matching module is used to dynamically match and route the corresponding target recognition prompt words from a preset prompt word library based on the ticket type and / or the hospital name; The model invocation module is used to obtain the target channel configuration corresponding to the medical invoice recognition request, and invoke the corresponding large language model interface according to the target channel configuration. The target channel configuration is determined by routing from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request. The reasoning module is used to input the medical invoice image data and the target recognition prompts into the large language model for reasoning and to obtain the original output results; The result processing module is used to perform engineering standardization processing on the original output results, generate structured ticket recognition data, and output it.

[0015] Furthermore, to achieve the above objectives, this application also proposes a multi-channel medical invoice recognition device that incorporates a large language model. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the multi-channel medical invoice recognition method that incorporates a large language model as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the multi-channel medical invoice recognition method combining a large language model as described above.

[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multi-channel medical invoice recognition method combining a large language model as described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: Upon receiving a medical invoice recognition request, the system acquires image data of the medical invoice to be recognized; extracts invoice metadata from the medical invoice image data; the invoice metadata includes at least the invoice type and hospital name; dynamically matches and routes corresponding target recognition prompts from a preset prompt word library based on the invoice type and / or the hospital name; acquires the target channel configuration corresponding to the medical invoice recognition request, and calls the corresponding large language model interface based on the target channel configuration, wherein the target channel configuration is determined from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request; inputs the medical invoice image data and the target recognition prompts into the large language model for inference, and obtains the original output result; performs engineering standardization processing on the original output result to generate structured invoice recognition data and output it. This application achieves fully automated, zero-sample recognition from raw invoice images to structured data. It ensures accurate matching of prompt words through chained pre-recognition and dynamic routing, supports flexible enterprise access through multi-channel configuration, and ensures the reliability of output results through engineering specifications. Ultimately, it significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual data entry costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the method for recognizing multi-channel medical invoices that incorporates a large language model, as described in this application. Figure 2 This is a schematic diagram of the system architecture involved in the embodiment of the multi-channel medical invoice recognition method combining a large language model in this application; Figure 3 This is a schematic diagram of the overall recognition process of the multi-channel medical invoice recognition method embodiment of this application, which combines a large language model. Figure 4 This is a schematic diagram of the module structure of the multi-channel medical invoice recognition device that incorporates a large language model according to an embodiment of this application. Figure 5 This is a schematic diagram of the hardware operating environment involved in the multi-channel medical invoice recognition method combining a large language model in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of this application embodiment is as follows: Upon receiving a medical invoice recognition request, obtain image data of the medical invoice to be recognized; extract invoice metadata based on the medical invoice image data; the invoice metadata includes at least the invoice type and hospital name; dynamically match and route corresponding target recognition prompts from a preset prompt word library based on the invoice type and / or the hospital name; obtain the target channel configuration corresponding to the medical invoice recognition request, and call the corresponding large language model interface based on the target channel configuration, wherein the target channel configuration is determined from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request; input the medical invoice image data and the target recognition prompts into the large language model for inference, and obtain the original output result; perform engineering standardization processing on the original output result to generate structured invoice recognition data and output it. This application achieves fully automated, zero-sample recognition from raw invoice images to structured data. It ensures accurate matching of prompt words through chained pre-recognition and dynamic routing, supports flexible enterprise access through multi-channel configuration, and ensures the reliability of output results through engineering specifications. Ultimately, it significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual data entry costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition.

[0026] This application's embodiments take into account that in enterprises' medical reimbursement business, each reimbursement involves numerous detailed items, which currently generally rely on manual completion and review. Financial personnel need to manually verify a large number of paper or electronic invoices. At the same time, the medical invoice formats of hospitals at all levels are complex and have a very low degree of standardization. There are significant differences in layout, field naming, and printing style between different hospitals and even between different services (outpatient, inpatient, and medical insurance settlement) within the same hospital. This highly heterogeneous environment poses a huge challenge to the automation of invoice recognition.

[0027] While traditional OCR recognition solutions have been applied in some standardized invoice scenarios, they fundamentally rely on fixed templates or training with large amounts of labeled data, resulting in severely limited generalization capabilities. Once a new format or minor format adjustment is encountered, the original template becomes ineffective, requiring the collection of new samples and model optimization. Furthermore, OCR output consists only of discrete text blocks, necessitating the writing of numerous manually crafted post-processing rules to assemble them into structured fields. These rules expand linearly with the number of formats, leading to extremely high maintenance costs and a high likelihood of conflicts. Directly using a general-purpose large model for end-to-end recognition faces practical bottlenecks such as high input / output token overhead, low efficiency in processing large images, and the difficulty of adapting a single model channel to the personalized access needs of different enterprises.

[0028] To address the aforementioned issues, this application proposes a multi-channel medical invoice recognition scheme that combines a large language model. This scheme leverages the semantic understanding capabilities of an LLM (Large Language Model) to achieve zero-sample, multi-format invoice recognition through prompt word engineering, eliminating the need for pre-set templates or specialized training. Simultaneously, it constructs a unified multi-channel management mechanism, supporting enterprises to expand customized recognition services as needed. Furthermore, it introduces intelligent image preprocessing and output standardization engineering, reducing token overhead and improving recognition efficiency and stability across the entire chain from input compression and precise routing to structured constraints. This systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition.

[0029] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone; or an electronic device capable of performing the above functions, a multi-channel medical invoice recognition device or system combining a large language model, etc. The following description uses a multi-channel medical invoice recognition system combining a large language model as an example to illustrate this embodiment and the subsequent embodiments.

[0030] Based on this, embodiments of this application provide a method for recognizing multi-channel medical invoices by combining a large language model, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multi-channel medical invoice recognition method combining a large language model according to this application.

[0031] In this embodiment, the method for recognizing multi-channel medical invoices using a large language model includes steps S10 to S60. The following provides a detailed explanation of each step.

[0032] like Figure 1 As shown, the first embodiment of this application proposes a method for recognizing multi-channel medical invoices by combining a large language model, the method comprising: Step S10: In response to receiving a medical invoice recognition request, obtain the image data of the medical invoice to be recognized; Combination Figure 2 As shown, Figure 2 This is a schematic diagram of the overall system architecture involved in the embodiments of this application. The system architecture includes: a client, a business layer, a channel layer, and a large model. The client can convert invoices into lists, and users can also manually upload invoice lists. The business layer includes image preprocessing, a recognition module, and prompt word management, etc. Figure 2 The logical relationships between the multi-channel access layer, image preprocessing module, prompt word management module, recognition routing layer, large model calling layer, and result normalization module are demonstrated.

[0033] This solution uses Large Language Model (LLM) and prompt word engineering as the core recognition engine to build a full-chain medical list recognition capability with multi-channel access, image preprocessing, intelligent recognition, and structured output.

[0034] The core idea is to leverage the powerful semantic understanding and format generalization capabilities of LLM to design differentiated recognition prompts for different hospital invoices, covering multiple invoice formats without the need for preset templates or special training; reduce token overhead and improve response speed by compressing images on the input side and simplifying the format with delimiters on the output side; and encapsulate the access logic with an independent channel layer to support standardized expansion of branch self-built channels.

[0035] First, in response to a received medical invoice recognition request, the image data of the medical invoice to be recognized is acquired. This image data can undergo adaptive preprocessing. Adaptive compression before inputting the image into the large model reduces input token overhead, and adaptive rotation is achieved using the image's EXIF ​​orientation parameters. The preprocessing flow may include: (1) Size judgment: Read the uploaded image and determine whether the file size / resolution exceeds the preset threshold; (2) Proportional compression: Compress images exceeding the threshold proportionally according to the target width, keeping the aspect ratio unchanged, and avoiding text distortion that affects recognition; (3) Error handling: When image reading fails (e.g., due to format error), the original data is returned directly without interrupting the recognition process; (4) Rotation processing: The image is aligned using the EXIF ​​orientation parameters; (5) Output after compression: The compressed image is converted to Base64 encoding and sent to the subsequent recognition stage.

[0036] Specifically, when a company's reimbursement system initiates a medical invoice recognition request, the request carries the original medical invoice image file to be recognized (usually a scanned copy or a photo taken with a mobile phone, in formats including JPEG, PNG, PDF, etc.). The system first reads the image file and determines whether its file size or resolution exceeds a preset threshold. The preset threshold can be configured according to actual business needs, for example, setting the file size threshold to 2MB or the resolution width threshold to 2000 pixels. If it exceeds the threshold, the resolution is compressed proportionally to the target width, for example, compressing the image width to a uniform 1024 pixels while maintaining the image height proportionally to the original aspect ratio to avoid text distortion that could affect subsequent recognition accuracy. If it does not exceed the threshold, the original image remains unchanged.

[0037] Next, the system reads the EXIF ​​orientation parameters of the original image. EXIF ​​(Exchangeable Image File Format) is metadata embedded in the image file, which contains rotation information at the time the image was captured (such as 0°, 90°, 180°, 270°, etc.). The system automatically rotates and aligns the image according to these orientation parameters to ensure that the text is oriented correctly, avoiding errors in large model recognition due to image tilt.

[0038] After compression and rotation, the system converts the processed image data into Base64 encoded format. Base64 is a common method for encoding binary data into ASCII characters, facilitating transmission in HTTP requests. Simultaneously, the system incorporates a fault-tolerance mechanism: if image reading fails (e.g., due to file corruption or unsupported format), the original data is returned directly, without interrupting subsequent recognition processes, ensuring high service availability.

[0039] Through the above preprocessing steps, the original image is preprocessed before entering the large model, which reduces the input token consumption of the subsequent visual encoder and improves the recognition accuracy, laying a high-quality input foundation for the entire recognition process.

[0040] Step S20: Extract the medical invoice metadata based on the medical invoice image data; the invoice metadata includes at least the invoice type and hospital name; Then, based on the medical invoice image data, the invoice metadata is extracted; the invoice metadata includes at least the invoice type and the hospital name.

[0041] As one implementation method, extracting the medical invoice metadata based on the medical invoice image data includes: The hospital / patient identification prompts are used to perform first-level identification on the medical bill image data, extracting the hospital name and patient name to obtain the first-level identification result; Based on the first-level identification result, the second-level identification is performed by calling the invoice type judgment prompt words to determine the specific invoice type code of the medical invoice and obtain the invoice metadata.

[0042] In this embodiment, a chain-based pre-recognition strategy is adopted to decompose the complex recognition task into two lightweight sub-tasks. First, the system calls the hospital / patient recognition prompt to perform the first-level recognition on the preprocessed Base64 image data. The prompt is a pre-designed text instruction, such as: "Please analyze this medical receipt, extract the hospital name and patient name, and output them in the format 'hospital name|patient name'". The large model extracts relevant information from the image based on the prompt and returns the first-level recognition result. The system splits this result into hospital name and patient name by the separator "|" and temporarily stores it.

[0043] Then, based on the first-level recognition results (especially the hospital name), the system calls the ticket type judgment prompts for second-level recognition. The large model outputs the specific ticket type code based on the image content. Thus, the system obtains ticket metadata, namely the hospital name and ticket type code, providing a basis for subsequent routing.

[0044] The advantage of chain-based pre-identification is that each subtask has a single responsibility, the prompt words are focused, the recognition accuracy is higher than that of the single prompt word method which completes all tasks at once, and the output of the preceding stage can dynamically affect the subsequent stages, achieving fine-grained control.

[0045] Step S30: Based on the ticket type and / or the hospital name, dynamically match and route the corresponding target recognition prompt words from the preset prompt word library; Then, based on the ticket type and / or the hospital name, the corresponding target recognition prompt word is dynamically matched and routed from a preset prompt word library.

[0046] In this embodiment, the system maintains an internal prompt word library, which categorizes and stores prompt words according to different dimensions. Upon obtaining the ticket metadata, the system distributes the data based on the ticket type code: If the invoice type code is Invoice Details (e.g., Central Unified Invoice), the system will query the "Central Unified Invoice Prompt Word" in the prompt word library according to the invoice type code. This prompt word is specifically designed for invoice layout and includes the corresponding field definitions and output format.

[0047] If the invoice type code is "detailed list," the system uses the extracted hospital name as the routing key to search the prompt word library for the hospital's unique prompt word. If found, this unique prompt word is used as the target identification prompt word.

[0048] If no matching exclusive prompt words are found, the system will automatically downgrade to using the preset default list of prompt words. These prompt words are universally designed and can cover most common layouts, ensuring that hospitals that are not pre-configured can still be recognized normally.

[0049] This embodiment employs a dynamic routing mechanism to ensure a high degree of matching between prompt words and ticket features, thereby improving recognition accuracy. At the same time, a degradation strategy ensures the system's fault tolerance and universality.

[0050] Step S40: Obtain the target channel configuration corresponding to the medical invoice recognition request, and call the corresponding large language model interface according to the target channel configuration. The target channel configuration is determined by routing from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request. Then, the target channel configuration corresponding to the medical invoice recognition request is obtained, and the corresponding large language model interface is called according to the target channel configuration. The target channel configuration is determined from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request.

[0051] Specifically, the system pre-configures available model service channels for different enterprises (identified by their enterprise ID, ComId). For example, one branch might use a cloud service provider's OCR large model interface, while another branch might use a self-built LLM service. The channel configuration table records extended parameters such as channel type, model endpoint, authentication information, and timeout for each enterprise ID.

[0052] When an identification request arrives, the system extracts the enterprise identifier from the request and queries the channel configuration table to obtain the currently active channels enabled by that enterprise. If the query is successful and the channel status is "ON", then the channel is determined to be the target channel; if no channel is found or the current channel is disabled (status is "OFF"), the system automatically downgrades to the preset default channel (such as the public Extreme Computing OCR channel) to ensure uninterrupted service.

[0053] The system then iterates through the registered set of processors, each corresponding to a model provider (such as a visual cloud processor, a high-performance computing processor, or a self-built model processor). Each processor implements a unified interface and declares the channel type it is responsible for. The system matches the corresponding processor according to the target channel type; once a match is successful, that processor is responsible for executing subsequent model calls.

[0054] The hot update and dynamic start / stop functions for channel configurations allow administrators to modify channel status online without restarting the service, and automatically downgrade when disabled, ensuring high availability and flexibility of the system.

[0055] Step S50: Input the medical bill image data and the target recognition prompt words into the large language model for reasoning and obtain the original output result; Then, the medical invoice image data and the target recognition prompts are input into the large language model for inference to obtain the original output result; the output format of the original output result is a custom delimiter format. In this step, the matched processor encapsulates the preprocessed Base64 image data and the selected target recognition prompts, along with the model inference parameters (such as Temperature set to 0.1, Top_p set to 0.9, and Max Tokens dynamically calculated based on the estimated number of records), and sends a request according to the interface protocol of the model supplier. After receiving the instruction, the large language model performs visual encoding and semantic understanding of the image, and generates the result according to the output format constraints in the prompts.

[0056] In this embodiment, a custom delimiter format is used instead of the traditional JSON format. For example, the prompt word requires the output format to be "Bill Name|Expense Name|Expense Category|Specification|Unit Price|Quantity|Amount", with one detailed record per line. The model output is: ``` Laboratory Fee | Factor B Determination | ① | Test | 35.00 | 1.0 | 35.00 Examination Fee | CT Scan | ② | Session | 200.00 | 1.0 | 200.00 ``` This format eliminates redundant characters such as key names, curly braces, and quotation marks in JSON, retaining only field values ​​and vertical bar separators, which greatly reduces the number of output tokens, improves response speed, and reduces inference costs.

[0057] Step S60: Perform engineering standardization processing on the original output results to generate structured ticket recognition data and output it.

[0058] Finally, the original output is processed for engineering standardization to generate structured invoice recognition data and output it. This embodiment takes into account the inherent instability of large model outputs, which may include extra spaces, line breaks, unexpected explanatory text, or mismatched field types. Therefore, it is necessary to standardize the output of large models, including but not limited to: field type compatibility handling (e.g., numbers may be output as strings), dictionary-based processing of project categories (to prevent content in category fields from not meeting the scenario requirements), and overall formatting (e.g., spaces, line breaks, etc.).

[0059] Specifically, this may include: (1) Format cleaning: Remove extra spaces, newlines and any non-delimiter formatted descriptive text from the original output results, and extract valid data rows connected by delimiters.

[0060] (2) Data type compatibility processing: Convert numeric fields (such as unit price, quantity, amount) from strings to numeric types. For example, convert “35.00” to the floating-point number 35.00 to avoid errors in subsequent business calculations.

[0061] (3) Dictionary mapping: For predefined enumeration fields (such as cost categories), map them to standard dictionary items in the system. For example, the model may output "①" or "Class A", and the system will uniformly map them to the predefined standard codes "A" or "B", and filter out non-standard content that is not within the dictionary range to ensure data consistency.

[0062] (4) Exception handling: For fields that fail to be parsed (such as missing or seriously formatted errors), fill in empty strings and record exception logs, while ensuring that the remaining fields of the record are output normally. A single data exception does not affect the overall execution of the batch process.

[0063] After the above standardization process, the system outputs structured invoice recognition data (such as JSON objects or database records) for direct use in subsequent reimbursement review processes.

[0064] Through the above steps, this embodiment achieves fully automated, zero-shot recognition from raw invoice images to structured data. Image preprocessing reduces the input token, chained pre-recognition and dynamic routing ensure accurate matching of prompt words, delimiter output compresses the output token, multi-channel configuration supports flexible enterprise access, and engineering standards ensure the reliability of output results. Ultimately, this significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, and greatly reduces manual data entry costs and maintenance complexity.

[0065] Based on the first embodiment described above, a second embodiment of this application is proposed. In this embodiment, before step S10, a system pre-configuration step is further included, namely, building a prompt word library and registering a multi-channel processor to provide metadata and access capability support for subsequent recognition.

[0066] As one implementation, before the step of obtaining the image data of the medical invoice to be identified in response to receiving a medical invoice recognition request, the method further includes: A prompt word library is constructed, and pre-collected sample prompt words are classified and stored in the prompt word library according to the recognition stage and applicable objects. The prompt words of the entire system are uniformly stored, organized and managed for lifecycle.

[0067] Specifically, system developers or business experts design multiple sets of prompt word templates for different stages of medical invoice recognition. These recognition stages include hospital / patient identification, invoice type determination, and list content extraction; the applicable targets include specific hospitals, specific invoice types, and default general scenarios. The specific prompt word classification and organization are shown in Table 1 below: Table 1

[0068] As one implementation method, each prompt is stored as structured data, containing at least the following five parts (but not all of them are required; they can be combined as needed): Task Description: Used to clarify the target of the current prompt, such as "Please analyze and identify the expense details on medical bills"; Field definition: List the names of the target fields to be extracted, such as "Bill Name, Expense Name, Expense Category, Specification, Unit Price, Quantity, Amount"; Field processing rules include value format constraints (such as "the expense category must be a numeric code"), exclusion rules (such as "remove rows containing the words 'subtotal' or 'total'"), and missing value handling rules (such as "output an empty string if it cannot be recognized"). Output format constraints: Specify the delimiter format, such as "Bill Name|Expense Name|Expense Category|Specification|Unit Price|Quantity|Amount", and constrain "No other content may be added"; Output example: Provides typical output examples to guide the model in generating results in the expected format, such as "Lab fee | Factor B determination | ① | times | 35.00 | 1.0 | 35.00".

[0069] Meanwhile, the prompt word library supports dynamic addition, deletion, modification, and querying. Administrators can add new exclusive prompt words at any time for newly emerging hospital layouts without modifying the code.

[0070] Furthermore, in the pre-configuration scheme, this embodiment also configures a unified processor interface for different model suppliers, and registers the channel type and corresponding enterprise identifier of each processor in the channel configuration table. The channel configuration table includes multiple pre-registered model service channels and their corresponding enterprise identifiers.

[0071] In this embodiment, the system employs the strategy pattern and adapter pattern to design the multi-channel management layer. First, a unified processor interface (e.g., MedicalOCRHandler) is defined, which declares the recognition method, channel type acquisition method, etc. Each model vendor (e.g., Alibaba Cloud Visual Intelligence Platform, Tencent Cloud OCR, and a branch's self-built LLM service) implements this interface, and encapsulates the calling logic for that vendor in its implementation class.

[0072] The channel configuration table is stored in a database or configuration center. Each record contains the following fields: Channel ID, Enterprise ID (ComId), Channel Name, OCR Channel Type (enumerated value), Enabled Status (ON / OFF), and Extended Configuration (JSON format, storing parameters such as model endpoints, authentication keys, and timeout). During registration, each channel type is bound to its corresponding enterprise identifier. One enterprise ID can be associated with one or more channels (but only one channel can be active at a time).

[0073] After pre-configuration, the system has the ability to route requests from different enterprises through different channels. It also provides plug-and-play extensibility for new channels. When a new branch builds its own model, it is only necessary to add a processor implementation class and a record in the channel configuration table. The core identification logic does not need to be modified.

[0074] Through the above pre-configuration steps, the system has achieved a standardized framework for centralized, structured management of prompt words and multi-channel access, laying a solid foundation for subsequent dynamic routing and flexible expansion, and greatly reducing maintenance costs and expansion difficulties.

[0075] Based on any of the above embodiments, a third embodiment of this application is proposed. In this embodiment, the design and optimization strategy of prompt words is described in detail. This strategy runs through the entire process of prompt word library construction, parameter fine-tuning and task decomposition, so as to improve the accuracy and efficiency of the large model in medical invoice recognition tasks.

[0076] First, role setting is performed. At the beginning of each prompt, a professional role description is set for the model to activate the model's knowledge in the field of medical bills.

[0077] Specifically, when writing the prompt text, a role definition statement is first added, such as: "You are a professional medical invoice auditor, familiar with the format and field meanings of various hospital expense lists." This role description aims to guide the large language model to utilize its medical finance knowledge learned during the pre-training phase, thereby more accurately understanding the professional terminology in the invoices (such as "treatment fee," "material fee," "out-of-pocket ratio," etc.) and employing professional auditing thinking for field extraction. Compared to prompts without a role setting, role setting significantly improves the accuracy of field semantic understanding, especially for ambiguous or abbreviated field names.

[0078] In addition, each prompt word is configured to contain the following five independent adjustable modules. The function and content of each module are as follows: Task Description Module: Clearly defines the objective of the current subtask. For example, for the prompt words for list content recognition, the task description is "Please extract each expense detail from this medical expense list, including item name, specifications, unit price, quantity, amount, etc." Field definition module: Lists the set of field names to be output, such as "Bill Name, Expense Name, Expense Category, Specification, Unit Price, Quantity, Amount"; Field processing rules module: Set processing constraints for each field, such as "the expense category should output a numeric code (1-9)", "if the amount field contains the word 'approximately', remove the word and take the value", "the entire row containing 'subtotal' or 'total' should be ignored", etc., and also specify that the missing value should be handled by "outputting an empty string". Output format constraint module: Specifies the use of a custom delimiter (such as "|") as the separator between fields, and requires the model to concatenate each detail into a single line of text in the order of the fields, and "no content other than field values ​​and delimiters (such as explanations, notes, punctuation marks, etc.) shall be added"; Output Example Module: Provides one or more typical output examples that conform to the format constraints, such as "Lab Fee | Factor B Measurement | ① | Times | 35.00 | 1.0 | 35.00", allowing the model to understand the desired form of the output format through few-shot learning.

[0079] The five modules mentioned above are independent of each other, and the content of any module can be adjusted according to different ticket types without modifying other modules, which greatly improves the flexibility and iteration efficiency of prompt word maintenance.

[0080] In addition, this embodiment also performs joint parameter fine-tuning for the prompt words and the model, that is, binding and configuring the assembled prompt word text with the model inference parameters.

[0081] This embodiment considers that, in addition to the prompt text itself, the parameter settings during model inference are also crucial to output stability. In this embodiment, for the medical invoice recognition task, the temperature parameter is set to a low value between 0 and 0.3 (e.g., 0.1), while Top_p is set to around 0.9 to reduce output randomness and encourage the model to generate the most probable answer. Furthermore, the system dynamically calculates the maximum number of output tokens (MaxTokens) based on the estimated number of output records to avoid unnecessary resource waste. Stop sequences, such as "\n\n" or "end," are also set; when the model encounters these sequences, generation stops to prevent redundant output.

[0082] By binding the prompt text to these parameters (i.e., different parameter combinations can be configured for different prompts), fine-grained control over the model's output behavior is achieved, ensuring determinism while improving inference efficiency.

[0083] In addition, a chain-like task decomposition is configured for the prompt words. In specific implementation, the complete medical invoice recognition task is split into multiple serial light quantum tasks according to the execution order, and an independent prompt word is configured for each sub-task.

[0084] This embodiment breaks down the complete recognition task into three sub-tasks, each using an independent prompt word: Level 1 prompt words (hospital / patient identification): responsible for extracting the hospital name and patient name from the ticket, with the output format being "hospital name|patient name"; Second-level prompt (invoice type determination): Based on the first-level recognition results (especially the hospital name, which can be supplemented by context), determine whether the invoice belongs to "invoice details", "list details" or "referral form", and output the type code; The third-level prompt (list content recognition): Based on the hospital name and invoice type output from the first two levels, dynamically select the corresponding list content recognition prompt (such as a dedicated prompt or a default prompt) to perform the core cost details extraction.

[0085] In each subtask, the output of the previous level (such as the hospital name) serves as the basis for routing or dynamically assembling prompts for the next level. For example, if the second level determines the prompt as "list details," the third level will query the specific prompt based on the hospital name; if it is "invoice details," the invoice prompt will be used directly.

[0086] The advantages of this chain-like decomposition are: each prompt word has a single responsibility and focused instructions, resulting in a much higher accuracy rate than a large, comprehensive prompt word; each stage can be independently fine-tuned and tested without interfering with each other; and the output of the preceding stages can be used to dynamically determine the selection of subsequent prompt words, achieving fine-grained routing control.

[0087] Through the comprehensive optimization strategy of prompt word design, parameter fine-tuning and chain decomposition, this embodiment significantly improves the recognition accuracy of large models in complex medical invoice scenarios, reduces token overhead, and achieves maintainability and scalability of prompt words, thereby greatly reducing manual intervention while ensuring recognition quality.

[0088] Based on any of the above embodiments, a fourth embodiment of this application is proposed. In this embodiment, the specific implementation process of obtaining the target channel configuration and calling the corresponding large language model interface is described in detail, focusing on channel routing, processor matching and adapter protocol conversion mechanism.

[0089] As one implementation method, obtaining the target channel configuration corresponding to the medical invoice recognition request and calling the corresponding large language model interface according to the target channel configuration includes: The system queries the channel configuration table based on the enterprise identifier carried in the medical invoice recognition request to obtain the active channel configuration; if no channel is found or the current channel is disabled, it will automatically downgrade to the preset default channel. Traverse the set of registered processors, match the corresponding processor according to the channel type, and the matched processor converts the unified recognition request containing the medical invoice image data, the target recognition prompt words and extended configuration into the specific interface protocol of the corresponding model supplier through the internal adapter for invocation; After the call ends, the adapter parses the original response returned by the model supplier into a unified recognition result structure, which is then used as the original output result.

[0090] Specifically, firstly, the channel configuration table is queried based on the enterprise identifier carried in the medical invoice recognition request to obtain the active channel configuration; if no channel is found or the current channel is disabled, it is automatically downgraded to the preset default channel.

[0091] When a request enters the system, the request header or body contains an Enterprise ID (ComId), such as "ComId_1001". The system first queries the channel configuration table to find the corresponding channel record based on the ComId. Each record in the channel configuration table contains an enable / disabled field; if the status is "ON", the channel is considered active. If the query is successful and the status is enabled, the system retrieves detailed information about the channel (including channel type, model endpoints, authentication information, timeout, and other extended parameters). If no configuration for the Enterprise ID is found, or if the found channel status is "OFF" (disabled), the system automatically downgrades to a default channel (such as the public "Extreme Computing OCR" channel) to ensure that all requests can be processed. This downgrade mechanism ensures that service is not interrupted when a specific enterprise channel is abnormal or disabled.

[0092] In addition, channel configuration supports hot updates and dynamic start / stop. Administrators can modify channel status or configuration parameters through the management interface at runtime, and the changes take effect immediately without requiring a service restart.

[0093] Then, the registered processor set is traversed, and the corresponding processor is matched according to the channel type. The matched processor then uses an internal adapter to convert the unified recognition request containing the medical invoice image data, the target recognition prompt words, and the extended configuration into the specific interface protocol of the corresponding model supplier for invocation.

[0094] When the system starts, it scans all processor classes that implement the MedicalOCRHandler interface and registers them in a processor mapping table (Map<channel type, processor instance>). Each processor declares the channel types it supports through its getType() method.

[0095] After determining the target channel type, the system iterates through the mapping table to find a processor that matches the type. Then, the processor executes its internal logic: Unified Request Object: The system pre-encapsulates a general recognition request object, which includes image Base64 data, target prompt text, and channel extension configurations (such as timeout, model version, etc.).

[0096] Adapter Conversion: The adapter module inside the processor converts the unified request into the specific input parameter format required by the model vendor. For example, if the Visual Cloud interface requires images to be passed in URL format, the adapter uploads the Base64 data to temporary storage and generates the URL; if the Extreme Computing interface requires images to be passed in binary stream format, the adapter decodes the Base64 into a byte array; self-built models may require additional business parameters (such as department type), which the adapter extracts and assembles from the extended configuration.

[0097] Sending a request: The adapter uses the vendor's SDK or HTTP client to send a request according to its interface protocol, carrying authentication information.

[0098] After the call ends, the adapter parses the original response returned by the model supplier into a unified recognition result structure, which is then used as the original output result.

[0099] This embodiment takes into account that different model vendors have different interface protocols, input parameter formats (image encoding methods, prompt word transmission fields), and return structures. That is, the response formats returned by different model vendors vary greatly; for example, some return JSON objects, some return plain text, and some include additional status codes. In this embodiment, the adapter logic implemented internally by each processor is as follows: the input parameter side converts the unified recognition request (image Base64 + prompt words + channel extension configuration) into the interface format of that vendor; the output parameter side parses the vendor's original return into a unified recognition result structure. Upper-layer business is completely unaware of the differences between underlying vendors, achieving "one set of business logic, multiple model services."

[0100] In one implementation, the adapter in the processor parses the supplier's raw response into a standard result structure defined internally by the system. This standard result includes at least the following fields: recognition status (success / failure), raw text output (i.e., delimited text generated by the model), and error message (if any). Ultimately, all processors output a unified recognition result object for subsequent engineering specification processing.

[0101] Furthermore, this embodiment also supports a processor extension mechanism: employing the strategy pattern, each model supplier implements a unified processor interface and declares the channel types it is responsible for. When a branch connects to a self-built model, only the processor implementation class and channel registration configuration need to be added; the core code requires zero modification, making it plug-and-play. In other words, when a new model supplier connects, developers only need to add a processor implementation class, complete the adapter conversion logic, and then add a new channel record to the channel configuration table. The core system recognition process requires no modification, achieving a plug-and-play effect.

[0102] Through the aforementioned channel routing, processor matching, and adapter conversion mechanisms, this embodiment achieves complete decoupling between business logic and the underlying model interface, enabling upper-layer applications to transparently call model services from different vendors. It also supports hot updates, degradation, and expansion, greatly enhancing the system's flexibility, stability, and maintainability, and meeting the complex access needs of large enterprises in multi-branch and multi-cloud environments.

[0103] Reference Figure 3 As shown, Figure 3 This is a schematic diagram of the overall identification process of this application, combined with Figure 2 and Figure 3 As shown, the overall recognition process of this application specifically includes the execution order of image preprocessing, chained pre-recognition (hospital / patient recognition → invoice type judgment), prompt word routing, channel routing, model calling, and result post-processing.

[0104] This solution uses Large Language Model (LLM) and prompt word engineering as the core recognition engine to build a full-chain medical list recognition capability of "multi-channel access → image preprocessing → intelligent recognition → structured output".

[0105] The core idea is to leverage the powerful semantic understanding and format generalization capabilities of LLM to design differentiated recognition prompts for different hospital invoices, covering multiple invoice formats without the need for preset templates or special training; reduce token overhead and improve response speed by compressing images on the input side and simplifying the format with delimiters on the output side; and encapsulate the access logic with an independent channel layer to support standardized expansion of branch self-built channels.

[0106] Its core technical solution includes: adaptive preprocessing of medical invoice images (proportional compression + EXIF ​​rotation) to reduce the input token; extraction of invoice metadata through chained pre-recognition (hospital identification → type judgment), and dynamic routing to the most suitable prompt words accordingly; routing from multiple registration channels to the target model service based on enterprise identifiers, and using a custom delimiter format to constrain model output to save output tokens; and finally, outputting structured results through engineering standardization processing (format cleaning, type conversion, dictionary mapping).

[0107] This embodiment manages channels within the recognition process: model channel configurations for each branch are managed using the enterprise ID as the dimension, decoupling the access logic from the recognition logic. Each channel record includes: channel ID, enterprise ID (ComId), channel name, OCR channel type (enumerated values ​​such as Visual Cloud / Extreme Computing / branch custom), activation status (ON / OFF), and extended configurations (JSON parameters such as model endpoints, timeout, and authentication information).

[0108] The channel routing process includes: requesting the enterprise ID and querying the active channel configuration by ComId; if a channel is found, its OCR channel type is retrieved; if no channel is found, the default channel type (such as Extreme Computing OCR) is used as a fallback; traversing the set of registered processors and matching processors by `getType()==ocrType`; after a successful match, calling the processor to perform recognition and exiting immediately to ensure single-processor handling.

[0109] In addition, a processor extension mechanism can be adopted, using the strategy pattern, where each model vendor implements a unified processor interface and declares the channel types they are responsible for. When branches integrate self-built models, only the processor implementation class and channel registration configuration need to be added, with zero modification to the core code, achieving a plug-and-play effect.

[0110] In addition, channel configuration supports hot updates and dynamic start / stop; when a channel is disabled, requests are automatically downgraded to the default channel to ensure uninterrupted service, thereby achieving dynamic channel management.

[0111] Furthermore, channel adapters and protocols are standardized because different model vendors have different interface protocols, input parameter formats (image encoding methods, prompt word transmission fields), and return structures. In this embodiment, the adapter logic implemented within each processor is as follows: the input parameter side converts the unified recognition request (image Base64 + prompt words + channel extension configuration) into the vendor's interface format, and the output parameter side parses the vendor's original return into a unified recognition result structure. Upper-layer business processes are completely unaware of the differences between underlying vendors, achieving the effect of one set of business logic serving multiple model services.

[0112] In this embodiment, the most suitable prompt word is dynamically matched for each ticket during the recognition process. The overall routing process is as follows: (1) Pre-identification: First, use hospital identification prompts to extract the hospital name and patient name (separated by "|" and then split after returning), and then use invoice type prompts to determine the invoice type (invoiceType); (2) Divide by type: Invoice type: Invoice Details → Search Central Invoice Details by Invoice Type Code (prompt keywords); The invoice type is a detailed list → search for the hospital's specific list prompts by hospital name; (3) Fallback: When no exclusive prompt words are found, the default list of prompt words will be used to ensure the availability of hospitals not covered.

[0113] Collaboration with channel routing: Channel routing determines which model to use, while prompt word routing determines which prompt words to use; the two operate independently. The result of prompt word routing is passed as input parameters to the processor matched by channel routing for recognition.

[0114] The rules are extensible: the routing key is currently the hospital name, and a rule preprocessing extension point is reserved. In the future, it will be possible to support obtaining rules based on other information on the invoice (such as invoice number prefix, department type).

[0115] This application achieves the following overall technical effects through the technical solutions described in the above embodiments: 1. Strong generalization ability: Utilizing the semantic understanding capabilities of large language models, it can recognize medical invoices of all levels and various formats from hospitals with zero samples without the need to train special models or preset templates for each invoice format, completely eliminating the dependence of traditional OCR on templates.

[0116] 2. High recognition accuracy: Through prompt word engineering strategies such as role setting, chained task decomposition, and dynamic prompt word routing, the model focuses on the current sub-task. Combined with low temperature parameters, the accuracy and determinism of the output are significantly improved.

[0117] 3. Fast inference speed: Image preprocessing compresses the input token, and a custom delimiter format compresses the output token. Combined with dynamic estimation of Max Tokens, the computation time and cost of large model inference are greatly reduced, meeting the high concurrency needs of enterprises.

[0118] 4. Excellent scalability: The multi-channel management framework supports plug-and-play functionality for different model vendors. Adding a new channel only requires implementing the processor interface and registering the configuration, with zero changes to the core code. The prompt dictionary supports hot updates, and there is no need to redeploy the system to cope with new invoice formats.

[0119] 5. High stability: Multiple protection mechanisms, including automatic degradation in case of channel failure, fault tolerance for image reading, and handling of output parsing anomalies, ensure high service availability and the integrity of batch processing.

[0120] 6. Low maintenance cost: The structured prompt word module is independently adjustable, and the post-processing rules are uniformly engineered cleaning, eliminating the cost of a large amount of manual maintenance of post-rules in traditional solutions, and fundamentally solving the problems of rule expansion and conflict.

[0121] In summary, this application provides a complete, efficient, intelligent, and easy-to-maintain medical invoice recognition solution. This method achieves zero-sample, high-generalization, and low-cost medical invoice recognition, and supports flexible expansion to multiple enterprises and channels, demonstrating promising prospects for industrial applications.

[0122] Furthermore, to achieve the above objectives, this application also proposes a recognition device for multi-channel medical invoices that combines a large language model, such as... Figure 4 As shown, the device includes: The acquisition module 10 is used to acquire the image data of the medical invoice to be recognized in response to receiving a medical invoice recognition request; Extraction module 20 is used to extract the invoice metadata of the medical invoice based on the medical invoice image data; the invoice metadata includes at least the invoice type and hospital name; The prompt word matching module 30 is used to dynamically match and route the corresponding target recognition prompt word from a preset prompt word library according to the ticket type and / or the hospital name; The model invocation module 40 is used to obtain the target channel configuration corresponding to the medical invoice recognition request, and invoke the corresponding large language model interface according to the target channel configuration. The target channel configuration is determined by routing from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request. The reasoning module 50 is used to input the medical bill image data and the target recognition prompt words into the large language model for reasoning and to obtain the original output results; The result processing module 60 is used to perform engineering standardization processing on the original output results, generate structured ticket recognition data, and output it.

[0123] The multi-channel medical invoice recognition device combining a large language model provided in this application adopts the multi-channel medical invoice recognition method combining a large language model in the above embodiments, which significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual input costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition. Compared with the prior art, the beneficial effects of the multi-channel medical invoice recognition device combining a large language model provided in this application are the same as the beneficial effects of the multi-channel medical invoice recognition method combining a large language model provided in the above embodiments, and other technical features in the multi-channel medical invoice recognition device combining a large language model are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0124] Furthermore, to achieve the above objectives, this application also proposes a multi-channel medical invoice recognition device that incorporates a large language model. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the multi-channel medical invoice recognition method that incorporates a large language model as described above.

[0125] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a multi-channel medical invoice recognition device that incorporates a large language model, suitable for implementing embodiments of this application. The multi-channel medical invoice recognition device in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The illustrated multi-channel medical invoice recognition device combining a large language model is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0126] like Figure 5As shown, the multi-channel medical invoice recognition device incorporating a large language model may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the multi-channel medical invoice recognition device incorporating a large language model. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the multi-channel medical invoice recognition device incorporating a large language model to wirelessly or wiredly communicate with other devices to exchange data. While the figure shows a multi-channel medical invoice recognition device incorporating a large language model with various systems, it should be understood that implementing or possessing all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0127] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0128] The multi-channel medical invoice recognition device combining a large language model provided in this application adopts the multi-channel medical invoice recognition method combining a large language model in the above embodiments, which significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual input costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition. Compared with the prior art, the beneficial effects of the multi-channel medical invoice recognition device combining a large language model provided in this application are the same as the beneficial effects of the multi-channel medical invoice recognition method combining a large language model provided in the above embodiments, and other technical features in the multi-channel medical invoice recognition device combining a large language model are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0129] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0130] Furthermore, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the multi-channel medical invoice recognition method combining a large language model as described above.

[0131] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0132] The aforementioned computer-readable storage medium may be included in a multi-channel medical invoice recognition device incorporating a large language model; or it may exist independently and not be assembled into a multi-channel medical invoice recognition device incorporating a large language model.

[0133] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a multi-channel medical invoice recognition device incorporating a large language model, the device performs the following actions: upon receiving a medical invoice recognition request, it acquires image data of the medical invoice to be recognized; extracts invoice metadata based on the image data, wherein the invoice metadata includes at least the invoice type and hospital name; dynamically matches and routes corresponding target recognition prompts from a preset prompt word library based on the invoice type and / or the hospital name; acquires the target channel configuration corresponding to the medical invoice recognition request, and calls the corresponding large language model interface based on the target channel configuration, wherein the target channel configuration is determined from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request; inputs the medical invoice image data and the target recognition prompts into the large language model for inference, and obtains the original output result; performs engineering standardization processing on the original output result to generate structured invoice recognition data and outputs it.

[0134] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0137] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described multi-channel medical invoice recognition method combined with a large language model. This significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual data entry costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multi-channel medical invoice recognition method combined with a large language model provided in the above embodiments, and will not be repeated here.

[0138] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multi-channel medical invoice recognition method combining a large language model as described above.

[0139] The computer program product provided in this application significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual data entry costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the multi-channel medical invoice recognition method combined with a large language model provided in the above embodiments, and will not be repeated here.

[0140] One or more technical solutions proposed in this application have at least the following technical effects: Upon receiving a medical invoice recognition request, the system acquires image data of the medical invoice to be recognized; extracts invoice metadata from the medical invoice image data; the invoice metadata includes at least the invoice type and hospital name; dynamically matches and routes corresponding target recognition prompts from a preset prompt word library based on the invoice type and / or the hospital name; acquires the target channel configuration corresponding to the medical invoice recognition request, and calls the corresponding large language model interface based on the target channel configuration, wherein the target channel configuration is determined from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request; inputs the medical invoice image data and the target recognition prompts into the large language model for inference, and obtains the original output result; performs engineering standardization processing on the original output result to generate structured invoice recognition data and output it. This application achieves fully automated, zero-sample recognition from raw invoice images to structured data. It ensures accurate matching of prompt words through chained pre-recognition and dynamic routing, supports flexible enterprise access through multi-channel configuration, and ensures the reliability of output results through engineering specifications. Ultimately, it significantly improves the efficiency, generalization ability, and stability of medical invoice recognition, greatly reduces manual data entry costs and maintenance complexity, and systematically solves the problems of poor generalization, high maintenance costs, and rigid channel adaptation in automated medical invoice recognition.

[0141] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for recognizing multi-channel medical invoices by combining a large language model, characterized in that, The method includes: In response to receiving a medical invoice recognition request, the system obtains the image data of the medical invoice to be recognized. Based on the medical invoice image data, extract the invoice metadata; the invoice metadata includes at least the invoice type and hospital name; Based on the ticket type and / or the hospital name, dynamically match and route the corresponding target recognition prompt words from the preset prompt word library; Obtain the target channel configuration corresponding to the medical invoice recognition request, and call the corresponding large language model interface according to the target channel configuration. The target channel configuration is determined by routing from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request. The medical invoice image data and the target recognition prompts are input into the large language model for inference to obtain the original output results; The original output results are processed in an engineering manner to generate structured ticket recognition data and output it.

2. The method as described in claim 1, characterized in that, Before the step of obtaining the image data of the medical invoice to be identified in response to receiving a medical invoice recognition request, the method further includes: Construct a prompt word library, and classify and store the pre-collected sample prompt words according to the recognition stage and applicable objects into the prompt word library. Each prompt word contains at least one of the following: task description, field definition, field processing rules, output format constraints, and output examples. Configure a unified processor interface for different model suppliers, and register the channel type and corresponding enterprise identifier of each processor in the channel configuration table. The channel configuration table includes multiple pre-registered model service channels and their corresponding enterprise identifiers.

3. The method as described in claim 2, characterized in that, The step of extracting the medical invoice metadata based on the medical invoice image data includes: The hospital / patient identification prompts are used to perform first-level identification on the medical bill image data, extracting the hospital name and patient name to obtain the first-level identification result; Based on the first-level identification result, the second-level identification is performed by calling the invoice type judgment prompt words to determine the specific invoice type code of the medical invoice and obtain the invoice metadata.

4. The method as described in claim 3, characterized in that, The step of dynamically matching and routing the corresponding target recognition prompt words from a preset prompt word library based on the invoice type and / or the hospital name includes: If the invoice type code is an invoice detail, then the corresponding central unified invoice prompt word is queried according to the invoice type code and used as the target identification prompt word; If the invoice type code is a list detail, then the hospital name is used as the routing key to query the hospital's exclusive list prompt words, which are used as the target identification prompt words; If no matching specific prompt word is found, the system will automatically downgrade and use the default fallback prompt word as the target identification prompt word.

5. The method as described in claim 1, characterized in that, The step of receiving a medical invoice recognition request and obtaining the image data of the medical invoice to be recognized includes: In response to receiving a medical invoice recognition request, the original medical invoice image to be recognized is obtained; Determine whether the file size or resolution of the original medical invoice image exceeds a preset threshold; If so, then perform proportional resolution compression based on the target width while maintaining the image aspect ratio. Read the EXIF ​​orientation parameters of the original medical invoice image, and rotate and align the image according to the orientation parameters; The processed image is converted into Base64 encoded data to serve as standardized medical invoice image data.

6. The method as described in claim 1, characterized in that, The design of the prompt words also includes: At the beginning of each prompt word, a professional role description is set for the model to activate the model's knowledge in the field of medical invoices; the professional role description is used to instruct the large language model to perform the recognition task as a medical invoice reviewer. Each prompt word is structurally assembled and configured to contain at least one of the following independent adjustable modules: task description module, field definition module, field processing rules module, output format constraint module, and output example module; The assembled prompt text is bound to the model inference parameters for joint fine-tuning; the model inference parameters include at least a temperature parameter, a maximum number of output tokens dynamically calculated based on the estimated number of output records, and a stop sequence for truncating redundant outputs. The complete medical bill recognition task is broken down into multiple cascaded lightweight quantum tasks according to the execution order, and each subtask is configured with an independent prompt word. The subtask includes at least a first-level prompt word for extracting the hospital name and the patient's name, a second-level prompt word for determining the bill type code, and a third-level prompt word for extracting the detailed expense list. The output of the previous subtask serves as the basis for prompt word routing or dynamic assembly in the next subtask.

7. The method as described in claim 1, characterized in that, The original output result is in a custom delimiter format; the custom delimiter format concatenates each target field into a single line of text according to a predetermined delimiter order.

8. The method as described in claim 2, characterized in that, The step of obtaining the target channel configuration corresponding to the medical invoice recognition request and calling the corresponding large language model interface according to the target channel configuration includes: The system queries the channel configuration table based on the enterprise identifier carried in the medical invoice recognition request to obtain the active channel configuration; if no channel is found or the current channel is disabled, it will automatically downgrade to the preset default channel. Traverse the set of registered processors, match the corresponding processor according to the channel type, and the matched processor converts the unified recognition request containing the medical invoice image data, the target recognition prompt words and extended configuration into the specific interface protocol of the corresponding model supplier through the internal adapter for invocation; After the call ends, the adapter parses the original response returned by the model supplier into a unified recognition result structure, which is then used as the original output result.

9. The method as described in claim 1, characterized in that, The process of performing engineering standardization on the original output results to generate structured invoice recognition data and outputting it includes: Remove extra spaces and newline characters from the original output and extract the valid data rows; The system performs data type compatibility processing on each field of the extracted valid data rows, converting numeric fields to numeric types; maps preset enumerated fields to system standard dictionary items, filtering out non-standard content; fills empty strings and records exception logs for fields that fail to be parsed; and generates and outputs structured ticket recognition data.

10. A recognition device for multi-channel medical invoices combining a large language model, characterized in that, The device includes: The acquisition module is used to acquire the image data of the medical invoice to be recognized in response to a received medical invoice recognition request. The extraction module is used to extract the invoice metadata of the medical invoice based on the image data of the medical invoice; the invoice metadata includes at least the invoice type and the hospital name; The prompt word matching module is used to dynamically match and route the corresponding target recognition prompt words from a preset prompt word library based on the ticket type and / or the hospital name; The model invocation module is used to obtain the target channel configuration corresponding to the medical invoice recognition request, and invoke the corresponding large language model interface according to the target channel configuration. The target channel configuration is determined by routing from multiple pre-registered model service channels based on the enterprise identifier carried in the medical invoice recognition request. The reasoning module is used to input the medical invoice image data and the target recognition prompts into the large language model for reasoning and to obtain the original output results; The result processing module is used to perform engineering standardization processing on the original output results, generate structured ticket recognition data, and output it.

11. A recognition device for multi-channel medical invoices combining a large language model, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for recognizing multi-channel medical invoices incorporating a large language model as described in any one of claims 1 to 9.

12. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for recognizing multi-channel medical invoices combined with a large language model as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method for recognizing multi-channel medical invoices incorporating a large language model as described in any one of claims 1 to 9.

Images