Methods and apparatus for controlling desktop robots to collect medical documents

Abstract

This application discloses a method and apparatus for controlling a desktop robot to acquire medical documents. The method includes: controlling the desktop robot to acquire medical documents; recognizing the acquired image; calculating the current recognition confidence and current recognition completeness rate based on the recognition results, and constructing a current state; calculating the current quality score of the acquired image based on the current state; generating a set of candidate actions based on multiple current shooting parameters; inputting the current state and each candidate action into a trained action quality prediction model to obtain a predicted quality value corresponding to each candidate action; calculating the predicted quality improvement of each candidate action based on the current quality score; selecting the candidate action with the largest predicted quality improvement as the current optimal action, controlling the desktop robot to execute the current optimal action, and performing the next acquisition. This invention can improve the effectiveness of acquisition, reduce the number of repeated acquisitions, and reduce the cost of manual intervention.

Description

Technical Field

[0001] This application relates to the field of medical document processing, specifically to methods, devices, media, electronic devices, and computer program products for controlling desktop robots to collect medical documents. Background Technology

[0002] With the rapid development of medical informatization, the electronic collection and recognition of medical documents has become a crucial aspect of medical data management. Medical documents, especially laboratory reports and prescriptions, contain a large amount of structured information that needs to be accurately and efficiently collected and converted into electronic data for subsequent storage, analysis, and utilization. Currently, medical document collection is primarily conducted using scanners, mobile phone cameras, or dedicated collection devices, followed by the conversion of images into editable text information using Optical Character Recognition (OCR) technology.

[0003] However, existing technologies have the following shortcomings in medical document acquisition and recognition:

[0004] First, most existing medical document acquisition systems use a one-time shooting method, lacking a feedback adjustment mechanism based on the recognition results. This one-way acquisition mode cannot dynamically adjust the acquisition parameters according to the recognition results, resulting in a lack of closed-loop integration between the acquisition and recognition processes. When the quality of the initially acquired image is poor or the recognition result is unsatisfactory, manual intervention is often required to re-acquire the data, reducing work efficiency.

[0005] Secondly, existing technologies primarily use conventional image quality indicators such as image sharpness and contrast as evaluation standards, failing to directly optimize for the structured requirements of medical data entry. This means that even with good image quality, incomplete identification of specific test items or errors in extracting key values ​​may prevent the fulfillment of structured data entry requirements.

[0006] Third, existing technologies lack intelligent decision-making mechanisms based on recognition quality, making it impossible to adjust acquisition parameters in a targeted manner to address current recognition problems. For example, when certain inspection items fail to be recognized due to insufficient light, the system cannot automatically adjust exposure parameters; when certain areas experience reflections or shadows due to improper shooting angles, the system cannot automatically adjust the shooting angle.

[0007] Finally, due to the lack of an intelligent parameter optimization mechanism, existing technologies often require multiple repeated shots to obtain satisfactory recognition results when processing complex medical documents, resulting in high manual intervention costs and low overall collection efficiency. Summary of the Invention

[0008] This application provides a method, apparatus, medium, electronic device, and computer program product for controlling a desktop robot to collect medical documents.

[0009] In a first aspect, embodiments of this application provide a method for controlling a desktop robot to collect medical documents, for use in an electronic device, the method comprising:

[0010] The acquisition step involves controlling the acquisition unit of the desktop robot to acquire medical documents and obtain the acquired image. The acquisition unit has multiple current capture parameters.

[0011] The identification step involves identifying the currently acquired image. If the identification result meets a preset termination condition, the subsequent steps are terminated and the identification result is output; otherwise, the construction step is initiated.

[0012] The construction step involves calculating the current recognition confidence and current recognition completeness rate based on the recognition results, and constructing the current state based on the currently acquired image, the current recognition confidence, and the current recognition completeness rate.

[0013] The scoring calculation step involves using a trained quality assessment model based on the current state to calculate the current quality score of the currently acquired image.

[0014] The generation step involves generating a set of candidate actions based on the multiple current shooting parameters, which includes at least one candidate action. The candidate action includes adjusting at least one of the shooting angle, shooting distance, and exposure parameters of the acquisition unit.

[0015] The prediction step involves inputting the current state and each candidate action in the candidate action set into the trained action quality prediction model to obtain the predicted quality value corresponding to each candidate action, which is used to predict the quality score that the acquisition unit can achieve in the next acquisition after executing each candidate action.

[0016] The improvement calculation step involves calculating the predicted quality improvement for each candidate action based on the current quality score, where the predicted quality improvement is the difference between the predicted quality value and the current quality score.

[0017] In the selection step, the candidate action with the largest predicted quality improvement is selected as the current optimal action, and the desktop robot is controlled to execute the current optimal action and perform the next data collection.

[0018] Secondly, embodiments of this application provide a device for controlling a desktop robot to collect medical documents, the device comprising:

[0019] The acquisition unit controls the desktop robot to acquire medical documents and obtain the acquired image. The acquisition unit has multiple current capture parameters.

[0020] The recognition unit recognizes the currently acquired image. When the recognition result meets the preset termination condition, it terminates the subsequent steps and outputs the recognition result; otherwise, it proceeds to the construction step.

[0021] The construction unit calculates the current recognition confidence and current recognition completeness rate based on the recognition results, and constructs the current state based on the currently acquired image, the current recognition confidence and the current recognition completeness rate;

[0022] The scoring calculation unit calculates the current quality score of the currently acquired image based on the current state and using a trained quality assessment model.

[0023] The generation unit generates a set of candidate actions based on the plurality of current shooting parameters, which includes at least one candidate action. The candidate action includes adjusting at least one of the shooting angle, shooting distance, and exposure parameters of the acquisition unit.

[0024] The prediction unit inputs the current state and each candidate action in the candidate action set into the trained action quality prediction model to obtain the predicted quality value corresponding to each candidate action, which is used to predict the quality score that the acquisition unit can achieve in the next acquisition after executing each candidate action.

[0025] The improvement calculation unit calculates the predicted quality improvement for each candidate action based on the current quality score, wherein the predicted quality improvement is the difference between the predicted quality value and the current quality score.

[0026] The selection unit selects the candidate action with the largest improvement in predicted quality as the current optimal action, controls the desktop robot to execute the current optimal action, and performs the next data collection.

[0027] Thirdly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method for controlling a desktop robot to collect medical documents as described in the first aspect.

[0028] Fourthly, embodiments of this application provide an electronic device, including: one or more processors; one or more memories; the one or more memories storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the method for controlling a desktop robot to collect medical documents as described in the first aspect.

[0029] Fifthly, embodiments of this application provide a computer program product including computer-executable instructions, which are executed by a processor to implement the method of controlling a desktop robot to collect medical documents as described in the first aspect.

[0030] The present invention can achieve the following beneficial effects.

[0031] (1) Achieve closed-loop integration of the data collection and identification process to improve the intelligence level of the system.

[0032] This invention introduces a state-based approach. With action The decision-making model uses the recognition results to provide feedback control for the next collection action after each collection, transforming the document collection process from a traditional one-time shooting to a closed-loop process of "collection-recognition-evaluation-adjustment", thereby significantly improving the automation and intelligence level of the system.

[0033] (2) Improve the availability of medical data by optimizing the quality of structured data.

[0034] Unlike existing technologies that primarily evaluate image sharpness, this invention uses a quality function. The optimization goal is to "completely identify test items, correctly extract key values, and achieve the required confidence level". This optimizes the data collection process to meet the needs of medical data storage, thereby significantly improving the completeness and accuracy of structured data.

[0035] (3) Improve the effectiveness of data collection decisions by using a quality prediction-based action selection mechanism.

[0036] By constructing a quality assessment model Predicting the impact of different candidate actions on subsequent recognition results and selecting the optimal action to execute enables the robot to make targeted adjustments for current recognition problems (such as missing fields, low confidence, etc.), avoids blindly repeating data collection, and improves the effectiveness of each data collection.

[0037] (4) Reduce the number of repeated data collections and lower the cost of manual intervention.

[0038] Because it can automatically adjust the shooting angle, distance and exposure parameters based on the recognition results, and gradually optimize the acquisition conditions, key fields are gradually identified, thereby significantly reducing the need for repeated shooting due to recognition failures, reducing reliance on manual intervention and improving overall acquisition efficiency. Attached Figure Description

[0039] Figure 1 According to an embodiment of this application, a flowchart of a method for controlling a desktop robot to collect medical documents is shown;

[0040] Figure 2 According to an embodiment of this application, a structural diagram of a device for controlling a desktop robot to collect medical documents is shown;

[0041] Figure 3According to an embodiment of this application, a block diagram of an electronic device is shown. Detailed Implementation

[0042] The illustrative embodiments of this application include, but are not limited to, methods, apparatuses, media, electronic devices, and computer program products for controlling a desktop robot to collect medical documents.

[0043] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0044] Figure 1 A flowchart illustrating a method for controlling a desktop robot to collect medical documents according to an embodiment of this application is shown, for use in... Figure 3 The electronic device shown.

[0045] Before implementing the method of this invention, initial setup is required. For example, in scenarios such as a hospital laboratory window, outpatient clinic, or health checkup center, the medical document to be collected is placed in a designated area on the desktop. In this embodiment, the medical document is, for example, a laboratory report, which includes multiple test items and their corresponding values.

[0046] During initialization, upon receiving a "test report collection task," the corresponding test report field template is automatically loaded. The template must include at least the test items, corresponding values, units, and reference ranges. Preferably, the priority of key fields can be set simultaneously, with the test items and corresponding values ​​being core fields that must be fully identified.

[0047] During initialization, the following initialization settings are made for the shooting parameters (shooting angle, shooting distance, exposure parameters) of the desktop robot's acquisition unit (e.g., camera).

[0048] (1) Initialize the shooting angle.

[0049] The camera's optical axis is initially set to be substantially perpendicular to the table surface, that is, .

[0050] (2) Initialize shooting distance.

[0051] The initial shooting distance between the camera and the document is set to a distance that can fully cover the entire report and ensure that the central area is clearly identifiable.

[0052] (3) Initialize exposure parameters.

[0053] The initial exposure parameters are set to standard indoor lighting mode. Preferably, when expressed in exposure levels, the initial exposure parameters are:

[0054] The robot controls the robotic arm to move above the report form, performs the initial image capture, and obtains the first image. In real-world scenarios, report forms may exhibit the following characteristics: the document is tilted to one side of the desktop; the document edges are obscured by other papers; overhead lighting causes localized glare; or the document is partially warped, resulting in perspective distortion. Therefore, after acquiring the captured image, the first step is to detect whether a complete document outline exists within the image. If the document is detected as not fully within the frame, an automatic re-capture is immediately triggered.

[0055] After initialization, control the desktop robot's data acquisition unit to collect data. The specific process is as follows.

[0056] In acquisition step S11, the acquisition unit of the desktop robot is controlled to acquire the medical document and obtain the acquired image. The acquisition unit has multiple current shooting parameters.

[0057] The acquisition unit is, for example, a camera, and multiple current shooting parameters include the current shooting angle, current shooting distance, and current exposure parameters. The current acquisition is, for example, the t-th acquisition; the following will describe the current acquisition t in detail.

[0058] In the recognition step S12, the currently acquired image is recognized. When the recognition result meets the preset termination condition, the subsequent steps are terminated and the recognition result is output; otherwise, the process proceeds to the construction step S13.

[0059] Specifically, for the currently acquired image Perform the following processing:

[0060] 1) Detect the boundaries of the report and crop the effective area; 2) Perform geometric correction and brightness equalization on the image; 3) Perform OCR recognition to extract text content; 4) Locate the inspection item area and identify the name of each inspection item and its corresponding value; 5) Map the recognition results to the structured field, which includes at least one inspection item identified under the current acquisition t and its corresponding value.

[0061] During this process, the following situations may occur: some test item names are not recognized; some test values ​​are empty or incorrectly identified; some fields are recognized, but with low confidence; key fields are unstable due to reflection, blurring, or occlusion. Therefore, it is necessary to determine whether the preset termination conditions are met based on at least one test item and its corresponding value identified under the current acquisition t in the structured fields of the test report.

[0062] The preset termination condition is that all of the following conditions are met simultaneously: all target inspection items for the currently acquired image are identified; and each inspection item has a valid corresponding value.

[0063] For example, if the number of target inspection items in the preset template is 5, and the number of identified inspection items is 3, then it means that not all target inspection items have been identified.

[0064] Understandably, if the preset termination condition is not met, the subsequent steps will be terminated and the recognition result will be output, that is, the structured fields will be output; otherwise, the construction step S13 will be entered.

[0065] In step S13, based on the recognition results, the current recognition confidence and the current recognition completeness rate are calculated, and the current state is constructed based on the currently acquired image, the current recognition confidence and the current recognition completeness rate.

[0066] Specifically, firstly, the current recognition confidence is calculated based on the following formula 1), and the current recognition completeness rate is calculated based on formula 2).

[0067] Formula 1).

[0068] Formula 2).

[0069] Where N represents the image currently being acquired. The target number of test items, t represents the current number of data collected. Indicates the current collection The first one identified The identification confidence level of each test item. Indicates the current collection The current identification confidence level, Indicates the current collection The number of inspection items identified below. Indicates the current collection The current recognition completeness rate.

[0070] Understandably, if a certain test item fails to be identified, then Current identification confidence level This reflects the overall reliability of the recognition of the currently acquired image. When the image is clear, the angle is appropriate, and the exposure is normal, the recognition confidence of each test item is high, indicating a high current recognition confidence level. The confidence level will be increased accordingly; however, if the image is blurry, tilted, reflective, or occluded, the confidence level of some test items will decrease or be recorded as 0, and the current confidence level will be increased accordingly. The corresponding reduction.

[0071] Current recognition completeness rate This indicates the coverage of the target inspection item by the currently acquired image.

[0072] Use the currently acquired image Current identification confidence level and current recognition completeness rate Build the current state .

[0073] In the score calculation step S14, based on the current state, the trained quality assessment model is used to calculate the current quality score of the currently acquired image.

[0074] Current status Including currently acquired images Current identification confidence level and current recognition completeness rate The currently acquired image The input image encoding network yields image feature vectors. An example image encoding network is ResNet18, which processes the currently acquired image. Inputting ResNet18 yields image feature vectors. .

[0075] Then, the image feature vector Current identification confidence level Current recognition completeness rate By concatenating the vectors, we obtain the state representation vector. .

[0076] Formula 3).

[0077] Quality assessment models, such as multilayer perceptrons (MLPs), consist of three fully connected layers and ultimately output a quality score between 0 and 1.

[0078] In this embodiment, the state representation vector Input MLP to get the current quality score . It is used to characterize the overall level of currently acquired images in terms of image quality, recognition reliability, and recognition integrity.

[0079] The quality assessment model is a sub-model specifically designed to comprehensively score the currently acquired image. Its function is to take the image features extracted by the image coding network, along with the current recognition confidence and current recognition completeness rate, as input, and output the current quality score. The quality assessment model is a pre-trained model whose parameters remain unchanged after deployment and are not updated during online runtime; it only performs quality assessments on the currently acquired image.

[0080] In generation step S15, based on multiple current shooting parameters, a set of candidate actions is generated, which includes at least one candidate action. The candidate action includes adjusting the shooting angle of the acquisition unit, the shooting distance, and the exposure parameters.

[0081] The candidate action set includes at least one of the following candidate actions:

[0082] The shooting angle adjustment action is used to fine-tune the current shooting angle within a preset angle range in fixed angle increments;

[0083] The shooting distance adjustment action is used to fine-tune the current shooting distance within a preset distance range in fixed distance steps;

[0084] The exposure parameter adjustment action is used to fine-tune the current exposure parameter within a preset exposure range using a fixed exposure step.

[0085] Candidate actions in the candidate action set are generated in the manner of "adding one step to the current parameter" or "subtracting one step from the current parameter". If the adjusted parameter exceeds the corresponding preset allowable range, the candidate action is discarded.

[0086] The candidate actions are described in detail below.

[0087] 1) Adjusting the shooting angle.

[0088] The shooting angle is used to adjust the camera's shooting posture relative to the desktop document, in order to improve recognition problems caused by document tilt, local reflections, and perspective distortion.

[0089] As mentioned above, during initial setup, the shooting angle is set to... ,in, This indicates that the camera's optical axis is basically perpendicular to the desktop plane.

[0090] Preferably, the allowable adjustment range of the shooting angle (preset angle range) is: Each time an angle adjustment is performed, the current shooting angle is adjusted in fixed steps. Fine-tuning is performed. Preferably, the fixed angular step size for a single adjustment is: In the first Next data collection (current data collection) After that, the angle adjustment action can be represented as: And satisfy: .

[0091] When a document is detected to be tilted, have distorted perspective at the edges, or have critical field areas affected by reflections, the camera angle adjustment should be performed first.

[0092] 2) Shooting distance adjustment action.

[0093] Shooting distance is used to adjust the spatial relationship between the camera and the document, in order to improve problems such as the document occupying an unsuitable proportion of the frame, blurry key fields, or unclear local details.

[0094] During initial setup, the shooting distance is set to... Preferably, the preset distance range (i.e., the allowable adjustment range of the shooting distance) is: .

[0095] Each time a distance adjustment is performed, the current shooting distance is fine-tuned according to a fixed distance step. Preferably, the fixed distance step for a single adjustment is: In the first After the first data collection, the distance adjustment action can be represented as: And satisfy: .

[0096] When a document is detected to occupy too small a portion of the screen, or when key field areas are blurry or text edges are unclear, distance adjustment actions should be performed first.

[0097] 3) Exposure parameter adjustment action.

[0098] Exposure parameters are used to adjust the brightness of the camera image to improve problems such as images that are too dark, too bright, have excessive local reflections, or have uneven grayscale distribution in key areas.

[0099] During initial setup, the exposure parameters are set as follows: .in, This indicates the default exposure level.

[0100] Preferably, the preset exposure range (the allowable adjustment range of the exposure parameters) is: Each time an exposure adjustment is performed, the current exposure parameters (levels) are fine-tuned according to a fixed exposure step size. Preferably, the single adjustment step size (fixed exposure step size) is: In the first After the first data acquisition, the exposure adjustment action can be represented as: And satisfy: .

[0101] When an image is detected to be generally too dark, generally too bright, or with overexposed or low-contrast key areas, exposure adjustment actions should be performed first.

[0102] In the After the first acquisition, candidate actions for the next round are generated based on the current shooting parameters. Preferably, candidate actions are generated in the manner of "adding one step to the current parameters" or "subtracting one step from the current parameters".

[0103] Correspondingly, the candidate action can be represented as:

[0104]

[0105] ,

[0106] .

[0107] If the adjusted parameters exceed the corresponding allowable range, the candidate action is discarded. The candidate actions that remain within the valid range are ultimately combined to form the currently collected candidate action set. .

[0108] In prediction step S16, the current state and each candidate action in the candidate action set are input into the trained action quality prediction model to obtain the predicted quality value corresponding to each candidate action, which is used to predict the quality score that the acquisition unit can achieve in the next acquisition after executing each candidate action.

[0109] The motion quality prediction model is processed as follows:

[0110] The input candidate action 'a' is encoded to obtain the action vector. ;

[0111] Action vectors and state representation vector Perform fusion to obtain a fusion vector;

[0112] The fused vector is input into a multilayer perceptron (MLP) to obtain the prediction quality value for each candidate action.

[0113]

[0114] This represents the encoded action vector of candidate action a; This represents a multilayer perceptron; Indicates the current state Lower camera to perform actions Then, the quality score that can be achieved in the next collection is the prediction result of the quality of the next collection.

[0115] The motion quality prediction model can be understood as a multi-input neural network structure, where the core prediction part can be implemented using an MLP. The motion quality prediction model employs a multi-input structure, and its prediction output can be implemented using an MLP.

[0116] In step S17, based on the current quality score, the predicted quality improvement for each candidate action is calculated. The predicted quality improvement is the difference between the predicted quality value and the current quality score, as shown in Formula 4).

[0117] Formula 4).

[0118] in, This represents the improvement in prediction quality for candidate action a. The larger the value, the more likely the next round of data collection will improve the completeness and reliability of the identification, given that the action is performed in the current state.

[0119] In step S18, the candidate action with the largest predicted quality improvement is selected as the current optimal action. The camera of the desktop robot is controlled to execute the current optimal action and the next data acquisition is performed.

[0120] The current optimal action includes, for example:

[0121] ,

[0122] ,

[0123] .

[0124] In other words, return to step S11, use the shooting parameters contained in the current optimal action to perform the next (i.e., the (t+1)th) acquisition, and repeat the above process iteratively.

[0125] It is understandable that both the motion quality prediction model and the quality assessment model are pre-trained through supervised learning or reinforcement learning, and the parameters of each model remain unchanged during the execution of the method.

[0126] Understandably, if the selected action is an angle adjustment action, the robot's robotic arm or camera mount is controlled to adjust the shooting posture; if the selected action is a distance adjustment action, the camera is controlled to change the shooting distance; and if the selected action is an exposure adjustment action, the camera's exposure parameters are adjusted. After executing the current optimal action, a new acquired image is obtained. .

[0127] Understandably, when acquiring new images Then, repeat Figure 1 The process involves multiple rounds of closed-loop iteration to gradually improve: the document's position and angle within the image; the clarity of key fields; the impact of reflective areas on recognition; and the stability and completeness of field recognition.

[0128] Therefore, this invention is not a one-time image recognition, but an active closed-loop method based on recognition feedback that continuously drives the robot to optimize its acquisition actions.

[0129] Optionally, the data collection will be automatically terminated when all of the following conditions are met: all target test items have been identified; each test item has a valid value; the current identification confidence level reaches a preset threshold; and the current quality score reaches the data entry threshold.

[0130] The model training process is described below.

[0131] The model training process consists of two parts: training the quality assessment model and training the motion quality prediction model. The quality assessment model outputs the current quality score for the currently acquired image. The motion quality prediction model is used to predict motion quality in the current state. The predicted quality value (quality change) of the next acquisition result after executing different candidate actions.

[0132] The training process for the quality assessment model is as follows.

[0133] First, medical document images under different acquisition conditions are acquired and recognized to obtain corresponding image features, recognition confidence, and recognition completeness. For the first... Secondary image acquisition Image feature vectors are extracted using an image coding network: .

[0134] Then, the image feature vector Current identification confidence level and the current recognition completeness rate The vectors are then concatenated to form the input vector for the quality assessment model. .

[0135] Finally, the input vector is fed into the quality assessment model to obtain the current quality score: .in, Represents an image coding network; Indicates the first Feature vector of the acquired image; This represents the input vector of the quality assessment model; This represents a quality assessment model; These represent the parameters of the quality assessment model; Indicates the first The current quality score of the current collection results.

[0136] During training, a corresponding true quality label is assigned to each sample. The true quality label can be manually labeled or generated according to business rules, and is used to represent the overall quality level of the current collection results.

[0137] Based on training samples By minimizing the error between the model output and the true quality label, the parameters of the quality assessment model are optimized. Optimize. The loss function is defined as:

[0138] .

[0139] in, This represents the training loss of the quality assessment model; This indicates the number of samples in a training batch. Indicates the first The input vector of each sample; Indicates the first The true quality label corresponding to each sample.

[0140] Through the above training, the quality assessment model can learn the mapping relationship between image features, recognition confidence, recognition completeness rate and the quality of the current acquisition results.

[0141] The training process of the motion quality prediction model is as follows:

[0142] The motion quality prediction model is used to predict motion quality in the current state. Execute a candidate action Then, the quality score of the next data collection result. Its expression is: .in, Indicates the current state Next action Then, the predicted value for the quality of the next round of data collection; These represent the parameters of the motion quality prediction model; Represents the feature vector of the current state; Indicates action The encoded vector; This represents a multilayer perceptron.

[0143] Wherein, the current state feature vector It consists of image feature vectors, recognition confidence, and recognition completeness rate, namely: For each state in the training samples, i.e., the action pair After performing this action, the next round of image acquisition can be obtained. And further obtain the recognition confidence level for the next round of data collection. and recognition completeness rate The quality assessment model will then calculate the quality score for the next round. , as the supervision target of the motion quality prediction model.

[0144] Based on training samples By minimizing the error between the model's predicted values ​​and the actual quality scores for the next round, the parameters are adjusted. Optimize accordingly. The supervised training loss function is defined as:

[0145] .

[0146] in, This represents the supervised training loss of the motion quality prediction model; Indicates the first The current state of each sample; Indicates the first The execution action of each sample; Indicates the first The actual next-round quality score after each sample action is performed.

[0147] Through the above training, the motion quality prediction model can learn the mapping relationship between "current state + current action" and "next round of acquisition quality".

[0148] To enable the model to prioritize actions that improve recognition quality, reinforcement learning is further employed to fine-tune the action quality prediction model based on supervised training.

[0149] Through a joint optimization process, this invention can simultaneously possess the ability to assess the quality of current data collection and the ability to predict the quality after an action.

[0150] The present invention has the following beneficial effects:

[0151] (1) Achieve closed-loop integration of the data collection and identification process to improve the intelligence level of the system.

[0152] This invention introduces a state-based approach. With action The decision-making model uses the recognition results to provide feedback control for the next collection action after each collection, transforming the document collection process from a traditional one-time shooting to a closed-loop process of "collection-recognition-evaluation-adjustment", thereby significantly improving the automation and intelligence level of the system.

[0153] (2) Improve the availability of medical data by optimizing the quality of structured data.

[0154] Unlike existing technologies that primarily evaluate image sharpness, this invention uses a quality function. The optimization goal is to "completely identify test items, correctly extract key values, and achieve the required confidence level". This optimizes the data collection process to meet the needs of medical data storage, thereby significantly improving the completeness and accuracy of structured data.

[0155] (3) Improve the effectiveness of data collection decisions by using a quality prediction-based action selection mechanism.

[0156] By constructing a quality assessment model Predicting the impact of different candidate actions on subsequent recognition results and selecting the optimal action to execute enables the robot to make targeted adjustments for current recognition problems (such as missing fields, low confidence, etc.), avoids blindly repeating data collection, and improves the effectiveness of each data collection.

[0157] (4) Reduce the number of repeated data collections and lower the cost of manual intervention.

[0158] Because it can automatically adjust the shooting angle, distance and exposure parameters based on the recognition results, and gradually optimize the acquisition conditions, key fields are gradually identified, thereby significantly reducing the need for repeated shooting due to recognition failures, reducing reliance on manual intervention and improving overall acquisition efficiency.

[0159] This invention provides a device for controlling a desktop robot to collect medical documents. For example... Figure 2 As shown, device 20 includes:

[0160] Acquisition unit 201 controls the acquisition unit of the desktop robot to perform current acquisition of medical documents and obtain the current acquisition image, wherein the acquisition unit has multiple current shooting parameters;

[0161] The recognition unit 202 recognizes the currently acquired image. When the recognition result meets the preset termination condition, it terminates the subsequent steps and outputs the recognition result; otherwise, it proceeds to the construction step.

[0162] Construction unit 203 calculates the current recognition confidence and current recognition completeness based on the recognition results, and constructs the current state based on the currently acquired image, the current recognition confidence, and the current recognition completeness.

[0163] The scoring calculation unit 204 calculates the current quality score of the currently acquired image based on the current state and using a trained quality assessment model.

[0164] The generation unit 205 generates a candidate action set containing at least one candidate action based on the plurality of current shooting parameters. The candidate action includes adjusting at least one of the shooting angle, shooting distance, and exposure parameters of the acquisition unit.

[0165] Prediction unit 206 inputs the current state and each candidate action in the candidate action set into the trained action quality prediction model to obtain the predicted quality value corresponding to each candidate action, which is used to predict the quality score that the acquisition unit can achieve in the next acquisition after executing each candidate action.

[0166] The improvement calculation unit 207 calculates the predicted quality improvement for each candidate action based on the current quality score, wherein the predicted quality improvement is the difference between the predicted quality value and the current quality score.

[0167] The selection unit 208 selects the candidate action with the largest predicted quality improvement as the current optimal action, controls the desktop robot to execute the current optimal action, and performs the next data collection.

[0168] It is understandable that the acquisition unit 201, identification unit 202, construction unit 203, scoring calculation unit 204, generation unit 205, prediction unit 206, improvement calculation unit 207, and selection unit 208 can be... Figure 3 The processor 1404 in the electronic device 1400 has the functions of these modules or units to implement them.

[0169] The present invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform [operations]. Figure 1 The method shown.

[0170] The present invention also provides a computer program product, including computer-executable instructions, which are executed by processor 1404 to carry out [the program]. Figure 1 The method shown.

[0171] Now for reference Figure 3 , Figure 3 An example electronic device 1400 according to an embodiment of the present invention is illustrated schematically. In one embodiment, the electronic device 1400 may include one or more processors 1404, a system control logic unit 1408 connected to at least one of the processors 1404, a system memory 1412 connected to the system control logic unit 1408, a non-volatile memory (NVM) 1416 connected to the system control logic unit 1408, and a network interface 1420 connected to the system control logic unit 1408.

[0172] In some embodiments, processor 1404 may include one or more single-core or multi-core processors. In some embodiments, processor 1404 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where electronic device 1400 employs an eNB (Evolved Node B) or RAN (Radio Access Network) controller, processor 1404 may be configured to perform various conforming embodiments, such as... Figure 1 The example shown.

[0173] In some embodiments, the system control logic unit 1408 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1404 and / or any suitable device or component communicating with the system control logic unit 1408.

[0174] In some embodiments, the system control logic unit 1408 may include one or more memory controllers to provide an interface to the system memory 1412. The system memory 1412 may be used to load and store data and / or instructions. In some embodiments, the system memory 1412 of the electronic device 1400 may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0175] The non-volatile memory 1416 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 1416 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of HDD (Hard Disk Drive), CD (Compact Disc) drive, and DVD (Digital Versatile Disc) drive.

[0176] The non-volatile memory 1416 may include a portion of the storage resources on the device on which the electronic device 1400 is installed, or it may be accessible by the electronic device, but is not necessarily part of the electronic device. For example, the non-volatile memory 1416 may be accessed over a network via network interface 1420.

[0177] Specifically, system memory 1412 and non-volatile memory 1416 may each include a temporary copy and a permanent copy of instruction 1424. Instruction 1424 may include, when executed by at least one of processors 1404, causing electronic device 1400 to perform, as Figure 1 The instructions for the method shown. In some embodiments, the instructions 1424, hardware, firmware and / or software components thereof may additionally / alternatively be located in the system control logic unit 1408, the network interface 1420 and / or the processor 1404.

[0178] Network interface 1420 may include a transceiver for providing a radio interface to electronic device 1400, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 1420 may be integrated into other components of electronic device 1400. For example, network interface 1420 may be integrated into at least one of processor 1404, system memory 1412, non-volatile memory 1416, and firmware device (not shown) with instructions, which, when at least one of processor 1404 executes the instructions, enable electronic device 1400 to perform as follows: Figure 1 The method shown.

[0179] The network interface 1420 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1420 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0180] In one embodiment, at least one of the processors 1404 may be packaged together with the logic of one or more controllers for the system control logic unit 1408 to form a system package (SiP). In another embodiment, at least one of the processors 1404 may be integrated on the same die with the logic of one or more controllers for the system control logic unit 1408 to form a system on chip (SoC).

[0181] The electronic device 1400 may further include an input / output (I / O) device 1432. The I / O device 1432 may include a user interface enabling a user to interact with the electronic device 1400; the peripheral component interface is designed to allow peripheral components to also interact with the electronic device 1400. In some embodiments, the electronic device 1400 may also include sensors for determining at least one of environmental conditions and location information related to the electronic device 1400.

[0182] In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash), and a keyboard.

[0183] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0184] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0185] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0186] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (CD-ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other propagation signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0187] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0188] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0189] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0190] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A method for controlling a desktop robot to collect medical documents, used in an electronic device, characterized in that, The method includes: The acquisition step involves controlling the acquisition unit of the desktop robot to acquire medical documents and obtain the acquired image. The acquisition unit has multiple current capture parameters. The identification step involves identifying the currently acquired image. If the identification result meets a preset termination condition, the subsequent steps are terminated and the identification result is output; otherwise, the construction step is initiated. The construction step involves calculating the current recognition confidence and current recognition completeness rate based on the recognition results, and constructing the current state based on the currently acquired image, the current recognition confidence, and the current recognition completeness rate. The scoring calculation step involves using a trained quality assessment model based on the current state to calculate the current quality score of the currently acquired image. The generation step involves generating a set of candidate actions based on the multiple current shooting parameters, which includes at least one candidate action. The candidate action includes adjusting at least one of the shooting angle, shooting distance, and exposure parameters of the acquisition unit. The prediction step involves inputting the current state and each candidate action in the candidate action set into the trained action quality prediction model to obtain the predicted quality value corresponding to each candidate action, which is used to predict the quality score that the acquisition unit can achieve in the next acquisition after executing each candidate action. The improvement calculation step involves calculating the predicted quality improvement for each candidate action based on the current quality score, where the predicted quality improvement is the difference between the predicted quality value and the current quality score. In the selection step, the candidate action with the largest predicted quality improvement is selected as the current optimal action, and the desktop robot is controlled to execute the current optimal action and perform the next data collection.

2. The method for controlling a desktop robot to collect medical documents according to claim 1, characterized in that, The recognition result is mapped to a structured field, which includes at least one detection item identified in the current acquisition and its corresponding value. The construction step further includes: The current identification confidence level is calculated based on the following formula 1), and the current identification completeness rate is calculated based on formula 2). Official 1), Official 2), Where N represents the total number of target inspection items for the currently acquired image, and t represents the current acquisition... Indicates the current collection The first one identified The identification confidence level of each test item. Indicates the current collection The current identification confidence level, Indicates the current collection The number of inspection items identified below. Indicates the current collection The current recognition completeness rate is as follows.

3. The method for controlling a desktop robot to collect medical documents according to claim 2, characterized in that, The scoring calculation step further includes: The currently acquired image is input into an image encoding network to obtain an image feature vector; The image feature vector, the current recognition confidence, and the current recognition completeness are concatenated to obtain a state representation vector; The state representation vector is input into the quality assessment model to obtain the current quality score.

4. The method for controlling a desktop robot to collect medical documents according to claim 2, characterized in that, The current recognition confidence level represents the overall reliability of the recognition of the currently acquired image, and the current recognition completeness rate represents the coverage of the target inspection items by the currently acquired image.

5. The method for controlling a desktop robot to collect medical documents according to claim 1, characterized in that, The multiple current shooting parameters include the current shooting angle, current shooting distance, and current exposure parameters. In the generation step, the candidate action set includes at least one of the following candidate actions: The shooting angle adjustment action is used to fine-tune the current shooting angle within a preset angle range in fixed angle increments; The shooting distance adjustment action is used to fine-tune the current shooting distance within a preset distance range in fixed distance steps; The exposure parameter adjustment action is used to fine-tune the current exposure parameter within a preset exposure range using a fixed exposure step.

6. The method for controlling a desktop robot to collect medical documents according to claim 5, characterized in that, The candidate actions in the candidate action set are generated in the manner of "adding one step to the current parameter" or "subtracting one step from the current parameter". If the adjusted parameter exceeds the corresponding preset allowable range, the candidate action is discarded.

7. The method for controlling a desktop robot to collect medical documents according to claim 3, characterized in that, The motion quality prediction model performs the following processing: The input candidate actions are encoded to obtain an action vector; The action vector and the state representation vector are fused to obtain a fused vector; The fusion vector is input into a multilayer perceptron to obtain the prediction quality value for each candidate action.

8. The method for controlling a desktop robot to collect medical documents according to claim 1, characterized in that, The preset termination condition is that all of the following conditions are met simultaneously: All target inspection items in the currently acquired image were identified; Each test item has a valid corresponding value.

9. The method for controlling a desktop robot to collect medical documents according to claim 1, characterized in that, Both the motion quality prediction model and the quality assessment model are pre-trained through supervised learning or reinforcement learning, and the parameters of each model remain unchanged during the execution of the method.

10. A device for controlling a desktop robot to collect medical documents, characterized in that, The device includes: The acquisition unit controls the desktop robot to acquire medical documents and obtain the acquired image. The acquisition unit has multiple current shooting parameters. The recognition unit recognizes the currently acquired image. When the recognition result meets the preset termination condition, it terminates the subsequent steps and outputs the recognition result; otherwise, it proceeds to the construction step. The construction unit calculates the current recognition confidence and current recognition completeness rate based on the recognition results, and constructs the current state based on the currently acquired image, the current recognition confidence, and the current recognition completeness rate; The scoring calculation unit calculates the current quality score of the currently acquired image based on the current state and using a trained quality assessment model. The generation unit generates a set of candidate actions based on the plurality of current shooting parameters, which includes at least one candidate action. The candidate action includes adjusting at least one of the shooting angle, shooting distance, and exposure parameters of the acquisition unit. The prediction unit inputs the current state and each candidate action in the candidate action set into the trained action quality prediction model to obtain the predicted quality value corresponding to each candidate action, which is used to predict the quality score that the acquisition unit can achieve in the next acquisition after executing each candidate action. The improvement calculation unit calculates the predicted quality improvement for each candidate action based on the current quality score, wherein the predicted quality improvement is the difference between the predicted quality value and the current quality score. The selection unit selects the candidate action with the largest improvement in predicted quality as the current optimal action, controls the desktop robot to execute the current optimal action, and performs the next data collection.

11. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer, cause the computer to perform the method for controlling a desktop robot to collect medical documents as described in any one of claims 1-9.

12. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the method of controlling a desktop robot to acquire medical documents as described in any one of claims 1-9.

13. A computer program product comprising computer-executable instructions, characterized in that, The instructions are executed by the processor to implement the method of controlling a desktop robot to collect medical documents as described in any one of claims 1-9.

Images