Intelligent order recording method, device and equipment and storage medium

Abstract

The invention discloses an intelligent order recording method and device, equipment and a storage medium, relates to the technical field of large models, and discloses an intelligent order recording method which comprises the steps that when a to-be-recognized picture input by a user is received, a vector database is retrieved to obtain a picture recognition result of the to-be-recognized picture; extracting key information of the to-be-recognized picture according to the picture recognition result and a target large model; performing field mapping on a system form field according to the key information to obtain a mapping result; and performing form rendering according to the mapping result to generate and display a pre-filled form page. Through the mode, the input of the form not only depends on character recognition, introduction of a large model and analysis of a picture vector, and more accurate field mapping is carried out at last, so that automatic form input and form page display are realized, and the efficiency and accuracy of intelligent form input are improved.

Description

Technical Field

[0001] The present application relates to the field of large model technology, and in particular to intelligent order recording methods, devices, equipment and storage media. Background Art

[0002] At present, applications such as electronic medical record systems and order management systems require a large amount of manual data entry. Due to factors such as handwriting marks, paper color, and handwriting size, the data entry efficiency is low and the accuracy is not high.

[0003] In response to the above problems, the industry currently has intelligent order recording tool solutions based on OCR technology, which has achieved the transformation from traditional manual data entry to automated and intelligent data entry. The current intelligent order recording tools based on OCR technology require a lot of data labeling: Traditional intelligent order recording tools are based on OCR (optical character recognition) technology to extract image information entered by users. Poor business adaptability: The formats of different documents vary greatly, and traditional intelligent order recording tools cannot adapt to the recognition of diverse documents. Poor recognition accuracy: The current system has poor recognition accuracy, lacks error correction capabilities, and requires user intervention. High user learning cost: The functional logic design of similar tools on the market is complex, and users need to spend a lot of time learning to use them.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide an intelligent recording method, device, equipment and storage medium, aiming to solve the technical problem of low accuracy of intelligent recording of text recognition.

[0006] To achieve the above objectives, the present application proposes an intelligent order recording method, the method comprising: When receiving a picture to be identified input by a user, searching a vector database to obtain a picture identification result of the picture to be identified; Extracting key information of the image to be identified according to the image recognition result and the target large model; Perform field mapping on the system form fields according to the key information to obtain a mapping result; A form rendering is performed based on the mapping result to generate and display a pre-filled form page.

[0007] Optionally, when receiving the picture to be identified input by the user, before the step of searching the vector database to obtain the picture identification result of the picture to be identified, the step further includes: Carry out data modeling and data definition for the preset system form to obtain the modeling results; Perform storage preprocessing on the preset entities to obtain a basic data set; The modeling results and the basic data set are stored in a vector database.

[0008] Optionally, when receiving the picture to be identified input by the user, the step of searching the vector database to obtain the picture recognition result of the picture to be identified includes: When receiving a picture to be identified input by a user, searching a vector database through a target neural network to obtain similarity information between the picture to be identified and the pictures in the vector database; The image recognition result of the image to be recognized is selected according to the similarity information.

[0009] Optionally, when receiving the picture to be identified input by the user, before the step of searching the vector database through the target neural network to obtain the similarity information between the picture to be identified and the picture in the vector database, the step further includes: Obtain candidate images from the image library, and convert the candidate images into image vectors through a preset convolutional neural network and store them in the vector database; The candidate image similarity information is calculated based on the image vector.

[0010] Optionally, the step of extracting key information of the image to be identified according to the image recognition result and the target large model includes: Inputting the image recognition result into the target large model for training; Inputting the image to be identified into the trained target large model to obtain image extraction information; Performing text recognition on the image to be recognized by using a text recognition model to obtain a text recognition result; The key information of the image to be recognized is determined according to the image extraction information and the text recognition result.

[0011] Optionally, the step of performing field mapping on the system form fields according to the key information to obtain a mapping result includes: Obtain a system form in a preset database, and perform field mapping between the key information and the fields in the system form to obtain a preliminary mapping result; The preliminary mapping result is subjected to field value validity verification and data correction to obtain a mapping result.

[0012] Optionally, the step of performing field value validity verification and data correction on the preliminary mapping result to obtain the mapping result includes: Identifying a field value of the preliminary mapping result; Verifying the validity of the field value and performing data correction on the verification result to obtain a corrected mapping result; Parameter assembly is performed according to the modified mapping result to obtain a mapping result.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes an intelligent order recording device, which includes: The image recognition module is used to retrieve the vector database to obtain the image recognition result of the image to be recognized when receiving the image to be recognized input by the user; An information extraction module, used to extract key information of the image to be identified based on the image recognition result and the target large model; A field mapping module is used to perform field mapping on the system form fields according to the key information to obtain a mapping result; The form rendering module is used to generate and display a pre-filled form page by performing form rendering according to the mapping result.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes an intelligent order recording device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the intelligent order recording method described above.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the intelligent recording method described above are implemented.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the intelligent recording method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: When receiving the image to be identified input by the user, the application retrieves the vector database to obtain the image recognition result of the image to be identified; extracts the key information of the image to be identified based on the image recognition result and the target large model; performs field mapping on the system form field based on the key information to obtain the mapping result; performs form rendering and generates and displays the pre-filled form page based on the mapping result. In this way, the entry of the form is not only based on text recognition, but also introduces large models and image vectors for analysis, and finally performs more accurate field mapping to achieve automatic order recording and form page display, thereby improving the efficiency and accuracy of intelligent order recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A flowchart of the first embodiment of the intelligent order recording method of this application is provided; Figure 2 A schematic diagram of the system architecture provided for an embodiment of the intelligent order recording method of this application; Figure 3 A schematic diagram of form modeling provided for an embodiment of the intelligent order recording method of this application; Figure 4 A schematic diagram of entity preprocessing provided for an embodiment of the intelligent order recording method of the present application; Figure 5 A schematic diagram of form rendering provided for an embodiment of the intelligent order recording method of this application; Figure 6 A flow chart of an embodiment of the intelligent order recording method of this application; Figure 7 A flowchart of the second embodiment of the intelligent order recording method of this application is provided; Figure 8 A field mapping diagram provided for an embodiment of the intelligent order recording method of this application; Fig. 9 This is a schematic diagram of the module structure of the intelligent order recording device according to an embodiment of the present application; Fig.10 This is a schematic diagram of the device structure of the hardware operating environment involved in the intelligent order recording method in the embodiment of the present application.

[0021] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0023] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of the embodiment of the present application is: when receiving the picture to be identified input by the user, searching the vector database to obtain the picture recognition result of the picture to be identified; extracting the key information of the picture to be identified based on the picture recognition result and the target large model; mapping the system form fields according to the key information to obtain the mapping result; performing form rendering and displaying the pre-filled form page according to the mapping result.

[0025] In this embodiment, for the convenience of description, the following description is made by taking the identification computer as the execution subject.

[0026] Due to the current existing technology, applications such as electronic medical record systems and order management systems require a lot of manual data entry. Due to factors such as handwriting marks, paper color, and handwriting size, the data entry efficiency is low and the accuracy is not high.

[0027] In response to the above problems, the industry currently has intelligent order recording tool solutions based on OCR technology, which has achieved the transformation from traditional manual data entry to automated and intelligent data entry. The current intelligent order recording tools based on OCR technology require a lot of data labeling: Traditional intelligent order recording tools are based on OCR (optical character recognition) technology to extract image information entered by users. Poor business adaptability: The formats of different documents vary greatly, and traditional intelligent order recording tools cannot adapt to the recognition of diverse documents. Poor recognition accuracy: The current system has poor recognition accuracy, lacks error correction capabilities, and requires user intervention. High user learning cost: The functional logic design of similar tools on the market is complex, and users need to spend a lot of time learning to use them.

[0028] The present application provides a solution, through this method, it is realized that the entry of the form does not only rely on text recognition, but also introduces large models and image vectors for analysis, and finally performs more accurate field mapping to achieve automatic recording and form page display, thereby improving the efficiency and accuracy of intelligent recording.

[0029] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a server, etc. The following takes a computer as an example to illustrate this embodiment and the following embodiments.

[0030] Based on this, the present application embodiment provides an intelligent order recording method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the intelligent order recording method of this application.

[0031] In this embodiment, the intelligent order recording method includes steps S10 to S40: Step S10, when receiving the picture to be identified input by the user, searching the vector database to obtain the picture identification result of the picture to be identified; It should be noted that if Figure 2 The system architecture diagram of the embodiment is shown as follows. Figure 2 The system shown in the embodiment includes form modeling: form modeling describes the data structure of the system form through structured data, including fields, valid value sets and their relationships. Visual language model: visual language model is an artificial intelligence model that can simultaneously process and understand both visual (image) and language (text) modal information. This model combines the technologies of computer vision and natural language processing, enabling them to perform well in complex tasks such as visual question answering, image description generation, and text-to-image search. AI application development platform: AI application development platform is an open source platform for building artificial intelligence applications. It can combine backend as a service and LLMops to simplify the development of generative AI solutions. By using AI application development platform, developers can skip complexity and focus on the most important thing - creating innovative AI applications that solve real-world problems. AI application development platform generally supports the following features: (1) Support for convenient calling of mainstream LLM; (2) High-quality RAG (Retrieval-augmented Generation) engine; (3) Graphical low-code workflow orchestration; (4) Easy-to-use interface and API.

[0032] It should be understood that the entire system follows the AI ​​agent construction paradigm of perception, decision-making, and execution. That is, (1) Perception: extract key document information from user input (images, text) through visual language model recognition and output in a standard format. (2) Decision-making: match the recognized document information with the form modeling information to complete data verification and error correction. (3) Memory: store and retrieve historical recognition records as part of the large model prompt to provide model output accuracy. (4) Execution: call the form creation API provided by the system to pre-fill key information, and the user generates the document after supplementing the remaining information.

[0033] In a specific implementation, firstly, a picture to be identified input by a user is received, and the user input can be a picture of an uploaded file or a specified address. After receiving the picture to be identified, the picture to be identified is first identified.

[0034] In a feasible implementation, in order to perform data modeling and definition in advance and build a vector database, before step S10, it also includes: performing data modeling and data definition on a preset system form to obtain a modeling result; performing storage preprocessing on a preset entity to obtain a basic data set; and storing the modeling result and the basic data set in a vector database.

[0035] It should be noted that data modeling and data definition are first performed as follows Figure 3 As shown, the specific data modeling is for the system form: define the document name, field name, data type (text / number / date / time / address / telephone number, etc.), and available value set of the system form.

[0036] It should be understood that if Figure 4 The following is a page for entity preprocessing. Customers and commodities commonly used in documents are taken as basic data sets and stored in a vector database for approximate matching in the value correction stage.

[0037] In a feasible implementation, in order to accurately obtain image recognition results, step S10 includes: when receiving the image to be recognized input by the user, searching the vector database through the target neural network to obtain similarity information between the image to be recognized and the image in the vector database; selecting the image recognition result of the image to be recognized based on the similarity information.

[0038] In the specific implementation, for the pictures input by the user, a similarity algorithm is used to retrieve the vector database to determine the similarity information of each picture, so as to select similar pictures and the recognition results of the corresponding pictures based on the similarity information, and submit them to the large model for reference.

[0039] It should be noted that for a picture uploaded by a user, its similarity with the pictures in the picture library will be calculated (using cosine similarity), and a list of pictures ranked from high to low in similarity and the corresponding recognition results of the images will be returned to the user as the result of image retrieval.

[0040] In a feasible implementation manner, in order to pre-train the target large model and construct a vector database, when receiving the picture to be identified input by the user, the vector database is retrieved through the target neural network, and before the step of obtaining the similarity information between the picture to be identified and the picture in the vector database, it also includes: obtaining the alternative pictures in the picture library, and converting the alternative pictures into picture vectors through a preset convolutional neural network and storing them in the vector database; calculating the similarity information of the alternative pictures according to the picture vectors; and obtaining the target large model according to the similarity information of the alternative pictures and the training of the alternative pictures.

[0041] It should be understood that image vectorization: first, the candidate images are converted into vector representations in high-dimensional space using a convolutional neural network (the solution of this embodiment is based on the ResNet50 model), and then the similarity between the vector representations in high-dimensional space of the two images is calculated (using cosine similarity). In the model training stage, the training goal is to make the similarity of images of the same category as high as possible and the similarity of images of different categories as low as possible.

[0042] In the specific implementation, vectorized storage: the vectorized results of the image are stored in a dedicated vector database. This project uses Milvus as the vector database. Milvus is a high-performance and highly scalable vector database.

[0043] Step S20, extracting key information of the image to be identified according to the image recognition result and the target large model; It should be understood that after obtaining the image recognition results, the target large model is trained and then the key information is extracted.

[0044] In a feasible implementation, in order to obtain key information, step S20 includes: inputting the image recognition result into the target large model for training; inputting the image to be recognized into the trained target large model to obtain image extraction information; performing text recognition on the image to be recognized through a text recognition model to obtain a text recognition result; and determining the key information of the image to be recognized based on the image extraction information and the text recognition result.

[0045] It should be noted that after obtaining the similarity information and candidate images, the target large model is trained and optimized again.

[0046] It should be understood that visual language model recognition: uses the visual language model to extract image information, and requires the output of standard structured data in the form of key-value, including field name, data type (text / number / date time / address / telephone number, etc.), to obtain image extraction information.

[0047] In the specific implementation, OCR recognition: use the OCR model to extract image information and obtain text recognition results. Finally, compare and combine the recognition results of the above two methods to obtain key information and improve the recognition accuracy.

[0048] Step S30, performing field mapping on the system form fields according to the key information to obtain a mapping result; It should be noted that after the key information is obtained, the fields in the key information are mapped with the fields in the pre-stored or pre-set system form, so that subsequent intelligent form entry can be achieved.

[0049] Step S40: Render and generate a pre-filled form page according to the mapping result.

[0050] It should be understood that if Figure 5 As shown, finally, the system pre-filled form is rendered based on the mapping result and then displayed to the user through a web page.

[0051] In the specific implementation, Figure 6The timing diagram of the embodiment is shown as follows. Figure 6 Finally, the user can supplement the missing information in the form and review it: the document image provided by the user may lack some key information in the system form, which needs to be supplemented by the user. And the review and confirmation of the corresponding final document.

[0052] This embodiment provides an intelligent order recording method. When receiving a picture to be identified input by a user, the vector database is retrieved to obtain the picture recognition result of the picture to be identified; the key information of the picture to be identified is extracted according to the picture recognition result and the target large model; the system form field is mapped according to the key information to obtain the mapping result; the form rendering is generated and the pre-filled form page is displayed according to the mapping result. In this way, the entry of the form is not only based on text recognition, but also introduces large models and picture vectors for analysis, and finally more accurate field mapping is performed to achieve automatic order recording and form page display, which improves the efficiency and accuracy of intelligent order recording.

[0053] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 6 , step S30 includes steps S301~S302: Step S301, obtaining a system form in a preset database, and performing field mapping between the key information and the fields in the system form to obtain a preliminary mapping result; It should be noted that if Figure 8 The diagram shown is a schematic diagram of field mapping. First, a system form in a preset database is obtained. The preset database is a pre-set database for storing forms.

[0054] It should be understood that after the system form is determined, field mapping is performed between the key information and the fields in the system form. Specific mapping methods include type mapping, name mapping and semantic mapping.

[0055] In the specific implementation, type mapping (mandatory rules): field mapping is done by data type first, that is, the data type of the identified field and the type of the mapped form field must be the same. For example, if the value corresponding to the "order quantity" field is identified as a numeric type, then the form field that can be mapped must also be a numeric type field such as "product quantity" or "inventory quantity", and cannot be a text or phone number type field such as "order number" or "contact number". According to the corresponding data type of the field, the identified fields are grouped, and then the subsequent steps are performed in each group.

[0056] It should be understood that name mapping: matching the name of the identified field with the field name of the form. Semantic mapping: matching fields with similar semantics through word2vec (word vector) representation in the field of NLP (natural language processing).

[0057] Step S302, verifying the validity of field values ​​and correcting data on the preliminary mapping result to obtain a mapping result.

[0058] In the specific implementation, after obtaining the preliminary mapping results, it is also necessary to verify the validity of each field in the preliminary mapping results, remove invalid fields and erroneous fields, and realize self-correction of intelligent entry.

[0059] It should be noted that before the validity verification, user correction is also performed: the results of the above steps are displayed to the user, and the user is required to make corrections and confirmation. The system saves the field mapping results after the user's correction for subsequent processing of the same type of documents to reduce the degree of user intervention. Data filling: Check the validity of the field value and correct it, and assemble the request parameters according to the format of the document creation API declaration.

[0060] In a feasible implementation, in order to accurately perform validity verification and data correction, step S302 includes: identifying the field value of the preliminary mapping result; performing validity verification on the field value, and performing data correction on the verification result to obtain a corrected mapping result; and performing parameter assembly according to the corrected mapping result to obtain a mapping result.

[0061] It should be noted that system forms often reference public basic data such as customers and products. When creating system forms, it is necessary to verify the validity of the identified field values ​​to ensure that customer and product-related data have been dimensioned in the public basic data.

[0062] It should be understood that data correction: In actual applications, there may be the following two situations that lead to invalid field values. 1) Model text recognition error: Some optical character recognition errors lead to invalid field values. For example, the letter O is mistakenly recognized as the number 0. 2) Because the basic data is not pulled together, there is an inconsistency between the document value and the system value. For example, the name of a product in the user document is "A Mineral Water 550ml", and the system maintenance value is "A Mineral Water 550ml". Based on the above two scenarios, the system is required to have a certain fault tolerance. Technical solutions such as text similarity algorithms and semantic understanding models can be used to complete data correction.

[0063] In the specific implementation, the parameter assembly is finally performed: according to the JSON Schema parameter format declared in the form, the parameter assembly is completed and the result is saved.

[0064] This embodiment obtains the system form in the preset database, and maps the key information with the fields in the system form to obtain a preliminary mapping result; the preliminary mapping result is validated and the data is corrected to obtain the mapping result. In this way, the validity verification and error correction of the table fields of the intelligent input after the field mapping is completed are realized, thereby improving the accuracy of the intelligent input form.

[0065] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the intelligent order recording method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0066] This application also provides an intelligent recording device, please refer to Fig. 9 , the intelligent order recording device comprises: The picture recognition module 10 is used to retrieve the vector database to obtain the picture recognition result of the picture to be recognized when receiving the picture to be recognized input by the user.

[0067] The information extraction module 20 is used to extract key information of the image to be identified based on the image recognition result and the target large model.

[0068] The field mapping module 30 is used to perform field mapping on the system form fields according to the key information to obtain a mapping result.

[0069] The form rendering module 40 is used to perform form rendering and generate and display a pre-filled form page according to the mapping result.

[0070] This embodiment retrieves the vector database to obtain the image recognition result of the image to be recognized when receiving the image to be recognized input by the user; extracts the key information of the image to be recognized based on the image recognition result and the target large model; performs field mapping on the system form field based on the key information to obtain the mapping result; performs form rendering and generates and displays the pre-filled form page based on the mapping result. In this way, the entry of the form is not only based on text recognition, but also introduces the large model and image vector for analysis, and finally performs more accurate field mapping to achieve automatic order recording and form page display, thereby improving the efficiency and accuracy of intelligent order recording.

[0071] In one embodiment, the image recognition module 10 is also used to perform data modeling and data definition on a preset system form to obtain a modeling result; perform storage preprocessing on a preset entity to obtain a basic data set; and store the modeling result and the basic data set in a vector database.

[0072] In one embodiment, the image recognition module 10 is also used to retrieve the vector database through the target neural network when receiving the image to be recognized input by the user, to obtain the similarity information between the image to be recognized and the image in the vector database; and select the image recognition result of the image to be recognized based on the similarity information.

[0073] In one embodiment, the image recognition module 10 is also used to obtain candidate images in the image library, and convert the candidate images into image vectors through a preset convolutional neural network and store them in the vector database; and calculate the similarity information of the candidate images based on the image vectors.

[0074] In one embodiment, the information extraction module 20 is also used to input the image recognition result into the target large model for training; input the image to be recognized into the trained target large model to obtain image extraction information; perform text recognition on the image to be recognized through a text recognition model to obtain a text recognition result; and determine the key information of the image to be recognized based on the image extraction information and the text recognition result.

[0075] In one embodiment, the field mapping module 30 is also used to obtain a system form in a preset database, and perform field mapping between the key information and the fields in the system form to obtain a preliminary mapping result; and perform field value validity verification and data correction on the preliminary mapping result to obtain a mapping result.

[0076] In one embodiment, the field mapping module 30 is further used to identify the field value of the preliminary mapping result; verify the validity of the field value and perform data correction on the verification result to obtain a corrected mapping result; and assemble parameters according to the corrected mapping result to obtain a mapping result.

[0077] The intelligent order recording device provided by the present application adopts the intelligent order recording method in the above embodiment, which can solve the technical problem of low accuracy of intelligent order recording by text recognition. Compared with the prior art, the beneficial effects of the intelligent order recording device provided by the present application are the same as the beneficial effects of the intelligent order recording method provided by the above embodiment, and the other technical features in the intelligent order recording device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0078] The present application provides an intelligent order recording device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intelligent order recording method in the above-mentioned embodiment one.

[0079] Reference below Fig.10, which shows a schematic diagram of the structure of an intelligent order recording device suitable for implementing the embodiment of the present application. The intelligent order recording device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The smart order recording device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0080] like Fig.10 As shown, the intelligent order recording device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the intelligent order recording device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the intelligent recording device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an intelligent recording device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0081] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0082] The intelligent order recording device provided by the present application adopts the intelligent order recording method in the above embodiment, which can solve the technical problem of low accuracy of intelligent order recording by text recognition. Compared with the prior art, the beneficial effects of the intelligent order recording device provided by the present application are the same as the beneficial effects of the intelligent order recording method provided by the above embodiment, and the other technical features in the intelligent order recording device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0083] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0084] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0085] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the intelligent order recording method in the above-mentioned embodiment.

[0086] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0087] The above-mentioned computer-readable storage medium may be included in the intelligent order recording device; or it may exist independently without being assembled into the intelligent order recording device.

[0088] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the intelligent order recording device, the intelligent order recording device: when receiving the picture to be identified input by the user, retrieves the vector database to obtain the image recognition result of the picture to be identified; extracts the key information of the picture to be identified based on the image recognition result and the target large model; maps the system form fields according to the key information to obtain the mapping result; and performs form rendering and generates and displays the pre-filled form page according to the mapping result.

[0089] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0090] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0092] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned intelligent order recording method, and can solve the technical problem of low accuracy of text recognition intelligent order recording. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the intelligent order recording method provided in the above-mentioned embodiment, and will not be repeated here.

[0093] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned intelligent recording method when executed by a processor.

[0094] The computer program product provided by this application can solve the technical problem of low accuracy of text recognition intelligent recording. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the intelligent recording method provided by the above embodiment, which will not be repeated here.

[0095] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An intelligent order recording method, characterized in that: The method includes: When receiving a picture to be identified input by a user, searching a vector database to obtain a picture identification result of the picture to be identified; Extracting key information of the image to be identified according to the image recognition result and the target large model; Perform field mapping on the system form fields according to the key information to obtain a mapping result; A form rendering is performed based on the mapping result to generate and display a pre-filled form page.

2. The method according to claim 1, characterized in that Before the step of retrieving the vector database to obtain the picture recognition result of the picture to be recognized when receiving the picture to be recognized input by the user, the step further includes: Carry out data modeling and data definition for the preset system form to obtain the modeling results; Perform storage preprocessing on the preset entities to obtain a basic data set; The modeling results and the basic data set are stored in a vector database.

3. The method according to claim 1, characterized in that The step of retrieving the vector database to obtain the picture recognition result of the picture to be recognized when receiving the picture to be recognized input by the user comprises: When receiving a picture to be identified input by a user, searching a vector database through a target neural network to obtain similarity information between the picture to be identified and the pictures in the vector database; The image recognition result of the image to be recognized is selected according to the similarity information.

4. The method according to claim 3, characterized in that Before the step of searching the vector database through the target neural network to obtain the similarity information between the image to be identified and the image in the vector database when receiving the image to be identified input by the user, the step further includes: Obtain candidate images from the image library, and convert the candidate images into image vectors through a preset convolutional neural network and store them in the vector database; The candidate image similarity information is calculated based on the image vector.

5. The method according to claim 1, characterized in that The step of extracting key information of the image to be identified based on the image recognition result and the target large model includes: Inputting the image recognition result into the target large model for training; Inputting the image to be identified into the trained target large model to obtain image extraction information; Performing text recognition on the image to be recognized by using a text recognition model to obtain a text recognition result; The key information of the image to be recognized is determined according to the image extraction information and the text recognition result.

6. The method according to claim 1, characterized in that The step of mapping the system form fields according to the key information to obtain the mapping result comprises: Obtain a system form in a preset database, and perform field mapping between the key information and the fields in the system form to obtain a preliminary mapping result; The preliminary mapping result is subjected to field value validity verification and data correction to obtain a mapping result.

7. The method according to claim 6, characterized in that The step of performing field value validity verification and data correction on the preliminary mapping result to obtain the mapping result comprises: Identifying a field value of the preliminary mapping result; Verifying the validity of the field value and performing data correction on the verification result to obtain a corrected mapping result; Parameter assembly is performed according to the modified mapping result to obtain a mapping result.

8. An intelligent order recording device, characterized in that: The device comprises: The image recognition module is used to retrieve the vector database to obtain the image recognition result of the image to be recognized when receiving the image to be recognized input by the user; An information extraction module, used to extract key information of the image to be identified based on the image recognition result and the target large model; A field mapping module is used to perform field mapping on the system form fields according to the key information to obtain a mapping result; The form rendering module is used to generate and display a pre-filled form page by performing form rendering according to the mapping result.

9. An intelligent order recording device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the intelligent order recording method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the intelligent order recording method as described in any one of claims 1 to 7 are implemented.

Images