Information input method based on large model, storage medium and computer equipment

Abstract

The invention discloses an information input method and device based on a large model, a storage medium and computer device.The method comprises the steps that a picture input page for a target function is displayed, and the target function is a function applied by providing specific type information; the specific type information comprises picture information of at least one demand picture type and text information of at least one demand text type; obtaining at least one picture input based on the picture input page, and performing picture type identification and text information analysis on the picture through the large model; based on the picture, the target picture type recognized by the large model and the analyzed target text information, an application information page of the target function is filled, and the application information page is a page which contains specific type information and can be modified; and displaying the application information page, and submitting input information of the target function based on the confirmed application information page under the condition that the application information page is confirmed.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a method, device, storage medium and computer equipment for information entry based on a large model. Background Art

[0002] In the field of information entry, traditional information entry methods often rely on users to manually input or upload files, and undergo manual review to ensure the accuracy and completeness of the information. However, with the rapid development of information technology and the growing demand of users for efficient and convenient services, traditional information entry methods have gradually exposed many shortcomings.

[0003] There are many scenarios for filling in qualification information in the industry. For example, merchants need to fill in form information when entering the online platform, which includes many certification materials, and need to fill in the corresponding text information synchronously according to the content of the materials. It is very easy to make mistakes or omissions, or to transmit the wrong location of the corresponding materials. For example, the picture frame of the food license needs to be uploaded as a business license, and the store name is entered incorrectly, or there are multiple missing characters; there are too many form fields, and it takes a long time for merchants to complete the form filling. Therefore, the pass rate of filling in the information once is not high, and the information needs to be modified and submitted repeatedly. In turn, it may also cause users to feel poor about filling in large forms and be unwilling to fill them in, or unwilling to submit them again after the form is rejected, resulting in the loss of platform merchants.

[0004] To sum up, the traditional information entry method has obvious defects in information entry efficiency, accuracy and user experience. There is an urgent need for an efficient, accurate and convenient information entry method to solve these problems. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a large-model-based information entry method, apparatus, storage medium and computer equipment, which help to improve the efficiency and accuracy of information entry, shorten the user's filling time, reduce the risk of errors, omissions and incorrect material location transmission, and at the same time improve the user experience and reduce user resistance and platform merchant loss caused by cumbersome processes.

[0006] According to one aspect of the present application, a method for inputting information based on a large model is provided, comprising:

[0007] Displaying a picture entry page for a target function, wherein the target function is a function applied for by providing specific type information, and the specific type information includes picture information of at least one required picture type and text information of at least one required text type;

[0008] Acquire at least one picture entered based on the picture entry page, and perform picture type recognition and text information analysis on the picture through a large model;

[0009] Based on the picture, the target picture type identified by the large model and the analyzed target text information, fill in the application information page of the target function, wherein the application information page is a modifiable page containing the specific type of information;

[0010] The application information page is displayed, and when the application information page is confirmed, the input information of the target function is submitted based on the confirmed application information page.

[0011] Optionally, before performing picture type recognition on the picture by using the large model, the method further includes:

[0012] Performing text recognition on each picture respectively to obtain picture text information of each picture;

[0013] For any picture, identify whether the picture text information of the picture hits any keyword corresponding to the required picture type, and determine the target picture type of the picture based on the required picture type corresponding to the hit keyword;

[0014] Accordingly, the image type is identified by the large model, including:

[0015] For the pictures corresponding to the picture text information that does not hit the keyword, the picture type is identified by the large model to determine the target picture type of the picture.

[0016] Optionally, identifying whether the picture text information of the picture matches any keyword corresponding to the required picture type includes:

[0017] Identify whether the picture text information of the picture matches a keyword corresponding to a first picture type, wherein the first picture type includes a standard picture type in the required picture type;

[0018] Accordingly, the image type is identified by the large model to determine the target image type of the image, including:

[0019] Determining a second picture type to be identified according to the required picture type and the identified target picture type;

[0020] For each picture of undetermined picture type, picture type identification prompt information is constructed according to the picture and the second picture type, and the target picture type of the picture is identified at least in the second picture type based on the picture type identification prompt information through a large model.

[0021] Optionally, constructing picture type identification prompt information according to the picture and the second picture type includes:

[0022] Obtaining feature description information corresponding to the second image type;

[0023] A picture type identification prompt template is filled according to the picture, the second picture type and the feature description information to obtain the picture type identification prompt information.

[0024] Optionally, before performing text information analysis on the image using the large model, the method further includes:

[0025] Performing an image integrity check on the target image type based on the required image type;

[0026] If the image integrity check passes, a first image confirmation page is displayed based on each image and a target image type of each image, and in response to a confirmation operation on the first image confirmation page, the image type of each image to be confirmed is used as the target image type of each image;

[0027] If the image integrity check fails, a second image confirmation page is displayed based on each image, the target image type of each image, and the missing image type, wherein the second image confirmation page includes an image upload control corresponding to the missing image type; when an image of the missing image type is uploaded based on the image upload control, in response to a confirmation operation on the second image confirmation page, the image type of each image to be confirmed is used as the target image type of each image.

[0028] Optionally, performing text information analysis on the image using a large model includes:

[0029] For each picture, text information analysis prompt information is constructed according to the preset text type corresponding to the picture and the target picture type of the picture, and the text information matching the preset text type is analyzed based on the text information analysis prompt information through a large model.

[0030] Optionally, before performing text information analysis on the image using the large model, the method further includes:

[0031] For each picture, according to the preset text type corresponding to the picture, text extraction is performed on the picture text information corresponding to the picture;

[0032] Accordingly, according to the picture and the preset text type corresponding to the target picture type of the picture, text information analysis prompt information is constructed, including:

[0033] Text information analysis prompt information is constructed according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture.

[0034] Optionally, constructing text information analysis prompt information according to the image, the text extraction result of the image, and the preset text type corresponding to the image includes:

[0035] For each picture from which text information has been extracted, a text information confirmation prompt template is filled according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture to obtain the text information analysis prompt information, wherein the text information confirmation prompt template includes content for guiding the large model to verify the text extraction result of the picture based on the picture and the preset text type corresponding to the picture;

[0036] For each picture from which text information has not been extracted, the text information generation prompt template is filled in according to the picture and the preset text type corresponding to the picture to obtain the text information analysis prompt information, wherein the text information generation prompt template includes content for guiding the large model to generate text information based on the picture and the preset text type corresponding to the picture.

[0037] Optionally, before filling in the application information page of the target function based on the picture, the target picture type identified by the large model and the analyzed target text information, the method further includes:

[0038] Performing an image integrity check on the target image type based on the required image type, and performing a text integrity check on the target text information based on the required text type;

[0039] If the image integrity check and the text integrity check are passed, the step of filling in the application information page of the target function based on the image, the target image type of the image and the target text information is performed;

[0040] If at least one of the image integrity check and the text integrity check fails, the missing item that fails the integrity check is determined; when the missing item belongs to a predicted item, the missing item is predicted based on the image and the target text information through a large model, and the application information page of the target function is filled based on the predicted missing item information, the image, the target image type of the image and the target text information; and when the missing item does not belong to a predicted item, the application information page of the target function is filled directly based on the image, the target image type of the image and the target text information.

[0041] Optionally, the predicted item includes a required text type; when the missing item belongs to the predicted item, predicting the missing item based on the image and the target text information by using a large model includes:

[0042] In the case that the missing item belongs to a predicted item, the associated item information corresponding to the missing item is obtained based on the target text information and the picture text information corresponding to the picture, and missing item prediction prompt information is constructed based on the associated item information and the missing item, and the missing item is predicted based on the missing item prediction prompt information through a large model.

[0043] Optionally, the target function includes opening a store online, the required image types include store front images, store environment images, business license images, and business license images, the required text types include store name, store address, business category, and business hours; the predicted items include business category and business hours;

[0044] In the case where the missing item is a predicted item, obtaining the associated item information corresponding to the missing item based on the target text information and the picture text information corresponding to the picture includes:

[0045] In the case where the missing item includes a business category, the associated item information corresponding to the business category is determined based on at least one of the store name, the image and text information corresponding to the store front image, the image and text information corresponding to the business license image, the image and text information corresponding to the business permit image, the dish image information, and the business category information of the relevant stores corresponding to the store name; wherein the relevant stores include chain stores, and the dish image information includes the image and text information identified in the image whose target image type is a menu and / or the image whose target image type is a dish;

[0046] In the case where the missing item includes business hours, determine the range of surrounding stores corresponding to the store address, obtain information of each surrounding store within the range, the surrounding store information includes business categories of surrounding stores and business hours of surrounding stores, and determine related item information corresponding to the business hours based on the surrounding store information and the business categories.

[0047] Optionally, the method further comprises:

[0048] Performing watermark recognition on the image and / or similar image recognition based on a preset image library to determine whether the image is a forged image, wherein the forged image includes a watermarked image and an image that meets a similarity condition;

[0049] If the picture is the forged picture, a reminder page is displayed based on the forged picture to remind the user to correct the forged picture.

[0050] Optionally, performing similar image recognition on the image based on a preset image library includes:

[0051] For any picture, based on the target picture type of the picture, at least one comparison picture is determined from each preset picture in the preset picture library, and the similarity between the picture and each comparison picture is calculated according to the comparison conditions corresponding to the target picture type of the picture, wherein the comparison conditions for text-type pictures include comparing text similarity, and the comparison conditions for image-type pictures include comparing pixel grayscale value similarity.

[0052] Optionally, after displaying the application information page, the method further includes:

[0053] Acquire information modification data for the application information page, and modify specific type of information on the application information page based on the information modification data;

[0054] The information modification data is recorded, and a large model training sample is constructed based on the information modification data to perform optimization training of the large model.

[0055] According to another aspect of the present application, there is provided an information entry device based on a large model, comprising:

[0056] A display module, configured to display a picture entry page for a target function, wherein the target function is a function applied for by providing specific type information, wherein the specific type information includes picture information of at least one required picture type and text information of at least one required text type;

[0057] An analysis module, used to obtain at least one picture entered based on the picture entry page, and perform picture type recognition and text information analysis on the picture through a large model;

[0058] A filling module, used to fill in the application information page of the target function based on the picture, the target picture type identified by the large model and the analyzed target text information, wherein the application information page is a modifiable page containing the specific type of information;

[0059] The display module is further used to display the application information page, and when the application information page is confirmed, submit the input information of the target function based on the confirmed application information page.

[0060] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned large model-based information entry method is implemented.

[0061] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned large model-based information entry method when executing the program.

[0062] By means of the above technical scheme, the information entry method, device, storage medium and computer equipment based on the large model provided by the embodiment of the present application can directly enter multiple pictures at one time on the picture entry page when entering information for the target function, and use the large model to identify the picture type and analyze the text information of the entered pictures, so as to display the application information page according to the entered pictures, the identified target picture type and the analyzed target text information, so that the user can modify and confirm the information on the page, and complete the information entry of the target function based on the information on the confirmed application information page. The embodiment of the present application introduces an information entry method based on a large model, which helps to improve the efficiency and accuracy of information entry, shorten the user filling time, reduce the risk of miswriting, missing and mistransmitting the material location, and improve the user experience, reduce the user resistance and platform merchant loss caused by cumbersome processes. In addition, the method also promotes the standardization and normalization of information entry, lays a solid foundation for the efficient processing and utilization of information, thereby optimizing the information entry process as a whole and meeting the user's demand for efficient and convenient services.

[0063] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0065] Figure 1 A schematic diagram of a flow chart of a method for inputting information based on a large model provided in an embodiment of the present application is shown;

[0066] Figure 2 A schematic diagram of a process of another large model-based information entry method provided in an embodiment of the present application is shown;

[0067] Figure 3 A schematic structural diagram of a large model-based information entry device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0069] In this embodiment, a method for inputting information based on a large model is provided, such as Figure 1 As shown, the method includes:

[0070] Step 101: displaying a picture entry page for a target function, wherein the target function is a function applied for by providing specific type information, and the specific type information includes picture information of at least one required picture type and text information of at least one required text type.

[0071] The embodiment of the present application provides an information entry method based on a large model, which is intended to simplify and optimize the process of users applying for specific target functions by providing specific types of information, such as applying for the function of opening a store online by filling in various posture information. First, under the application process of the target function, a picture entry page designed specifically for the target function is displayed. This page is the entrance for users to submit the information required for the application. The target function refers to the function that the user hopes to apply for by providing certain specific types of information. These specific types of information include picture information of at least one required picture type and text information of at least one required text type. Taking the target function as the online store opening function as an example, the required picture types can specifically include store front pictures, store environment pictures, business license pictures, business license pictures, ID card front pictures (portrait side), ID card back pictures (back of the portrait side), and the required text types can include store name, store address, business license number, business category, business hours, etc. Specifically, the picture entry page can include a picture upload box, and the user can drag and drop pictures to the picture upload box to upload multiple pictures at one time, or upload multiple pictures one by one. In addition, prompt information can also be displayed in the picture upload box to prompt the user of the type of picture to be uploaded.

[0072] Step 102: Obtain at least one picture entered based on the picture entry page, and perform picture type recognition and text information analysis on the picture through a large model.

[0073] In an embodiment of the present application, after a user submits at least one picture through the picture entry page, the large model can be used to perform picture type recognition and text information analysis on the submitted picture, so as to identify the type of the picture and extract the required information from the picture. The picture type recognition of the large model at least includes the recognition of the required picture type mentioned above, and the text information analysis at least includes the analysis of the required text type mentioned above. By implementing picture type recognition and text information analysis through the large model, the application information page is automatically filled in, which reduces the possibility of human error, improves the accuracy of information, helps to reduce the time of manual input by users and the workload of manual review, thereby improving the efficiency of information entry.

[0074] Step 103: based on the image, the target image type identified by the large model and the analyzed target text information, fill in the application information page of the target function, wherein the application information page is a modifiable page containing the specific type of information.

[0075] Step 104: display the application information page, and when the application information page is confirmed, submit the input information of the target function based on the confirmed application information page.

[0076] In the embodiment of the present application, based on the pictures submitted by the user and the picture types and text information identified by the large model, the application information page of the target function can be automatically filled and displayed. The application information page is an editable page containing specific types of information, allowing users to manually modify or supplement. If the user believes that the picture type and text information automatically identified by the large model are deviated from what they think, there are missing, then the user can modify and supplement the information on the application information page. If the user has no objection to the information on the page (or the modified information), after confirmation, the system will submit the input information of the target function based on the confirmed application information page to complete the entire application process. By simplifying the information entry process, reducing the user's operation steps, and providing clear reminder pages and guidance, users can complete information entry more easily and quickly. In addition, the automated processing process reduces the user's waiting time, improves the user's satisfaction and trust, and reduces the user's difficulty and resistance to filling in, and improves the user's willingness and satisfaction to fill in, thereby helping to reduce the loss of platform merchants.

[0077] By applying the technical solution of this embodiment, when entering information for the target function, multiple pictures can be directly entered at one time on the picture entry page, and the large model can be used to identify the picture type and analyze the text information of the entered pictures, so that the application information page is displayed according to the entered pictures, the identified target picture type and the analyzed target text information, so that the user can modify and confirm the various information on the page, and complete the information entry of the target function based on the various information on the confirmed application information page. The embodiment of the present application introduces an information entry method based on a large model, which helps to improve the efficiency and accuracy of information entry, shorten the user's filling time, reduce the risk of miswriting, missing and mis-transmitting materials, and improve the user experience, reduce user resistance and platform merchant loss caused by cumbersome processes. In addition, the method also promotes the standardization and normalization of information entry, lays a solid foundation for the efficient processing and utilization of information, thereby optimizing the information entry process as a whole and meeting the user's demand for efficient and convenient services.

[0078] In an embodiment of the present application, optionally, it also includes: performing watermark recognition on the image and / or similar image recognition based on a preset image library to determine whether the image is a forged image, wherein the forged image includes a watermarked image and an image that meets a similarity condition; if the image is a forged image, a reminder page is displayed based on the forged image to remind the user to correct the forged image.

[0079] In the above embodiments, in addition to improving the efficiency and accuracy of information entry, the embodiments of the present application further enhance the security and authenticity of information by identifying forged images. Among them, forged image recognition includes watermark recognition of images and similar image recognition based on a preset image library. This mechanism effectively identifies and intercepts forged images, including those with watermarks or images that are highly similar to other images, thereby avoiding information entry errors or fraud caused by the use of false materials. When the system detects a forged image, it can display a reminder page to remind the user of the forged image, and guide the user to correct or re-upload the image. This not only improves the accuracy of the information, but also enhances the user's trust in the system, further ensures the security and authenticity of the information, and provides users with a more reliable and efficient information entry experience.

[0080] In an embodiment of the present application, optionally, similar image recognition is performed on the image based on a preset image library, including: for any image, based on the target image type of the image, at least one comparison image is determined from each preset image in the preset image library, and the similarity between the image and each comparison image is calculated according to the comparison conditions corresponding to the target image type of the image, wherein the comparison conditions for text-type images include comparing text similarity, and the comparison conditions for image-type images include comparing pixel grayscale value similarity.

[0081] In the above embodiment, different similar image recognition rules can be adopted for different types of images. Specifically, for any image to be identified, at least one relevant comparison image can be screened out in the preset image library according to its target image type (such as business license, ID card, store environment map, etc.). The preset image library can specifically be a database containing multiple types of images, for example, containing multiple business license images, multiple ID card images (used with user permission), etc. Then, according to the target image type of the image, the corresponding comparison condition is selected to calculate the similarity between the image and the comparison image. For text-type images, such as business licenses, licenses, etc., the comparison condition mainly focuses on the similarity of text content, and the text content can be extracted by text recognition technology and natural language processing technology, and it is determined whether the text content is consistent, so as to determine whether the uploaded image is theft of other people's information. For image-type images, such as photos, screenshots, etc., the comparison condition focuses more on the similarity of pixel grayscale values, and the image is converted into a grayscale image by image processing technology, and the similarity score between the grayscale values ​​is calculated. Through this mechanism, it is possible to accurately identify images to be identified that are highly similar to images in the preset image library, thereby effectively intercepting forged or tampered images and ensuring the authenticity and accuracy of information entry. When images with a high degree of similarity are detected, a reminder page is triggered to guide the user to make corrections or re-upload to avoid using forged images for information entry. The similar image recognition function in the embodiment of the present application provides additional security for the information entry process through precise comparison conditions and efficient calculation methods, further improving the reliability of the system and user experience.

[0082] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another information entry method based on a large model is provided, such as Figure 2 As shown, the method includes:

[0083] Step 201: display a picture entry page for a target function, wherein the target function is a function applied for by providing specific type information, and the specific type information includes picture information of at least one required picture type and text information of at least one required text type.

[0084] In this embodiment, the picture entry page for a specific target function is first displayed. The target function is applied for by providing specific types of information, including picture information of at least one required picture type and text information of at least one required text type. This step provides a clear and intuitive entry interface for the user, guiding the user to upload relevant pictures and fill in necessary text information as required.

[0085] Step 202, obtain at least one picture entered based on the picture entry page; perform text recognition on each picture respectively to obtain the picture text information of each picture; for any picture, identify whether the picture text information of the picture hits any keyword corresponding to the required picture type, and determine the target picture type of the picture based on the required picture type corresponding to the hit keyword; for the picture corresponding to the picture text information that does not hit the keyword, perform picture type recognition on the picture through a large model to determine the target picture type of the picture.

[0086] In this embodiment, at least one picture entered by the user based on the picture entry page is obtained. Subsequently, these pictures are subjected to text recognition processing to extract the text information in each picture, that is, the picture text information. This step uses advanced OCR (optical character recognition) technology to realize the automatic recognition and extraction of text in the picture, which provides a basis for subsequent picture classification and information entry. Next, for any picture, the following operations are performed: First, identify whether the picture text information of the picture contains any keyword corresponding to the required picture type. If the keyword is hit, the target picture type of the picture is determined based on the required picture type corresponding to the keyword. This step realizes the rapid and accurate judgment of the picture type through keyword matching technology. For example, pictures containing keywords such as "resident identity card" and "validity period" can be judged as the back of the identity card; pictures containing keywords such as "food business license" can be judged as business license pictures. For pictures that do not realize category discrimination in this way or the text is not extracted from the picture, that is, pictures corresponding to the picture text information that do not hit the keyword, the picture type is identified through the large model. The large model is used to analyze and understand the content in the image, thereby determining the target image type of the image. This step further improves the accuracy and adaptability of image classification, especially for those image types that have less text information or are difficult to determine through keyword matching. Through this embodiment, not only is the rapid and accurate entry of image information achieved, but also the efficiency and accuracy of information entry are improved through intelligent image classification technology. At the same time, this process also provides a reliable data foundation for subsequent information processing and review, which helps to improve the performance and user experience of the entire information entry system.

[0087] In an embodiment of the present application, optionally, in step 202, identifying whether the picture text information of the picture hits the keyword corresponding to any required picture type includes: identifying whether the picture text information of the picture hits the keyword corresponding to the first picture type, wherein the first picture type includes the standard picture type in the required picture type; accordingly, performing picture type recognition on the picture through the large model to determine the target picture type of the picture, including: determining the second picture type to be recognized according to the required picture type and the recognized target picture type; for each picture of an undetermined picture type, constructing picture type recognition prompt information according to the picture and the second picture type, and using the large model to recognize the target picture type of the picture at least in the second picture type based on the picture type recognition prompt information. Wherein, the large model is pre-trained with pictures of different required picture types.

[0088] In the above embodiment, when identifying whether the image text information of the image hits the keyword, the first image type, i.e., the standard image type, is mainly concerned. Standard image types generally refer to standard images with fixed formats, contents, and requirements, such as business licenses, ID cards, etc. Through a predefined keyword library, the text information of these images is accurately matched to quickly determine their types. This keyword matching method is particularly effective for standard image types, and can quickly and accurately complete the identification. However, for non-standard image types or images with insufficient text information for keyword matching, a more flexible and intelligent identification method is adopted. Specifically, the remaining, unidentified second image type is first determined based on the required image type and the identified target image type, which may include non-standard, more complex or diverse images, such as product photos, scene screenshots, etc. Next, for those images whose types have not yet been determined, the image type identification prompt information corresponding to each image can be constructed to help the large model understand the image content more accurately and identify the target type of the image in the second image type. By utilizing the deep learning and image understanding capabilities of the large model, it is helpful to accurately locate the type that matches the image content, thereby realizing intelligent identification of the image type. The information entry process in the embodiment of the present application combines the two methods of keyword matching and large model image type recognition, which not only improves the accuracy and efficiency of image type recognition, but also enhances the flexibility and adaptability of the system.

[0089] In an embodiment of the present application, optionally, picture type identification prompt information is constructed based on the picture and the second picture type, including: obtaining feature description information corresponding to the second picture type; filling in a picture type identification prompt template based on the picture, the second picture type and the feature description information to obtain the picture type identification prompt information.

[0090] In the above embodiment, when constructing the picture type recognition prompt information, the feature description information corresponding to the second picture type is first obtained. The feature description information may include visual features such as key elements, colors, textures, shapes, etc. that should be included in the picture, as well as text features such as text, symbols or patterns that may appear in the picture. These feature description information is obtained in advance based on an in-depth understanding and analysis of the second picture type, and is intended to help the large model identify the picture type more accurately. Next, the picture type recognition prompt template is filled in according to the picture, the second picture type and the feature description information. The picture type recognition prompt template is a predefined framework for organizing and presenting information related to picture type recognition. By filling in this template, the system can generate a comprehensive prompt information containing the picture itself, its possible types and the feature description information corresponding to the type. This comprehensive prompt information not only provides direct visual information of the picture, but also provides additional clues and context for picture type recognition through feature description information. This helps the large model to understand the picture content more comprehensively and identify the target type of the picture more accurately in the second picture type. The embodiment of the present application can generate more accurate and useful picture type identification prompt information by obtaining feature description information of the second picture type and filling in the picture type identification prompt template according to this information, thereby further improving the accuracy and efficiency of the large model in picture type identification, thereby optimizing the performance and user experience of the entire information entry process.

[0091] Further, after determining the image type of each picture, in an embodiment of the present application, optionally, it also includes: performing an image integrity check on the target image type based on the required image type; if the image integrity check passes, then based on each picture and the target image type of each picture, a first image confirmation page is displayed, and in response to a confirmation operation on the first image confirmation page, the image type of each picture to be confirmed is used as the target image type of each picture; if the image integrity check fails, then based on each picture, the target image type of each picture, and the missing image type, a second image confirmation page is displayed, wherein the second image confirmation page includes an image upload control corresponding to the missing image type; in a case where a picture of the missing image type uploaded based on the image upload control is obtained, in response to a confirmation operation on the second image confirmation page, the image type of each picture to be confirmed is used as the target image type of each picture.

[0092] In the above embodiment, after the image type of each image is determined, in order to ensure the comprehensiveness of the image type, the target image type can also be checked for image integrity based on the required image type. This step is intended to check whether the user has uploaded all necessary image types as required and whether these images have been correctly classified. If the image integrity check passes, it means that after automatic type recognition, it is considered that the user has uploaded all necessary images. At this time, the system will display a first image confirmation page based on each image and the target image type of each image. This page will list all uploaded images and their corresponding image types for the user to make the final confirmation. If the user confirms that it is correct, these confirmed image types can be used as the final target image types of each image, and the subsequent information processing flow will continue. However, if the image integrity check fails, it means that the user may have missed some necessary types of images. At this time, a second image confirmation page can be displayed based on each image, the target image type of each image, and the missing image type. This page not only lists the uploaded images and their corresponding image types, but also specifically prompts which image types are missing, and provides image upload controls for these missing image types. Users can upload missing images through these upload controls and make necessary classification adjustments. After obtaining the picture of the missing picture type uploaded based on the picture upload control, the user can perform a confirmation operation on the second picture confirmation page. Once confirmed, these confirmed picture types will also be used as the final target picture types of each picture, and the subsequent information processing flow will continue. Through such picture integrity verification and confirmation links, the information entry process of the embodiment of the present application can ensure that all necessary pictures have been correctly uploaded and classified, thereby improving the accuracy and reliability of information entry. At the same time, this process also provides a good user interaction experience, allowing users to clearly understand which pictures have been uploaded, which pictures are missing, and how to make necessary adjustments and supplements.

[0093] Step 203: for each picture, construct text information analysis prompt information according to the picture and the preset text type corresponding to the target picture type of the picture, and analyze the text information matching the preset text type based on the text information analysis prompt information through a large model.

[0094] In the embodiment of the present application, in order to deeply analyze and extract the text information contained in the picture, a text information analysis prompt information can be constructed according to the preset text type corresponding to the picture and its target picture type, and then the prompt information is parsed using the large model to extract the text information matching the preset text type. Specifically, one or more preset text types are determined according to each picture and its target picture type. These preset text types are defined according to the characteristics of the target picture type and the requirements of information entry. For example, for ID card pictures, the preset text types may include name, gender, date of birth, etc.; for business license pictures, the preset text types may include business license scope, etc. Next, a text information analysis prompt information is constructed, which may specifically include some basic features of the picture (such as color, texture, shape, etc.), recognized picture text information (such as text extracted by OCR technology), and related descriptions or keywords of the preset text type. The combination of this information is intended to provide a comprehensive and specific context for the large model to more accurately analyze and extract text information matching the preset text type. Then, the text information analysis prompt information is input into the big model for analysis. The big model will use its powerful natural language processing capabilities and deep learning algorithms to deeply analyze the prompt information and try to extract text information that matches the preset text type. Among them, the big model is pre-trained with different types of pictures and text information of preset text types corresponding to the pictures. Finally, based on the analysis results of the big model, the text information that matches the preset text type is extracted and further processed as part of the information of the picture. This information can be used for subsequent information entry, review, analysis and other links, providing strong support for the entire information entry process. The information entry process of the embodiment of the present application can realize the accurate extraction and analysis of the text information contained in the picture, thereby improving the accuracy and efficiency of information entry. At the same time, this step also provides a more comprehensive and accurate data basis for subsequent information processing and review.

[0095] In an embodiment of the present application, optionally, before performing text information analysis on the image through a large model, the method also includes: for each image, according to a preset text type corresponding to the image, performing text extraction on the image text information corresponding to the image; accordingly, according to the image and the preset text type corresponding to the target image type of the image, constructing text information analysis prompt information, including: constructing text information analysis prompt information according to the image, the text extraction result of the image, and the preset text type corresponding to the image.

[0096] In this embodiment, in order to further improve the accuracy and efficiency of text information analysis, before using the large model to analyze the text information of the picture, text extraction can also be performed first. This step is intended to extract the text information in the picture in a targeted manner according to the preset text type corresponding to the picture, so as to provide more accurate and useful data for the subsequent text information analysis. Specifically, for each picture, the system will extract the text information in the picture according to its corresponding preset text type. This extraction process may involve steps such as recognition of the text in the picture (such as through OCR technology), filtering and screening of the text content (such as removing irrelevant text information, retaining text information related to the preset text type). Through this step, text information closely related to the preset text type can be extracted from the picture, providing strong support for subsequent analysis. For example, the preset text types corresponding to the front image of the ID card include ID number, name, etc. After the text extraction is completed, a text information analysis prompt information is constructed based on the extraction result, the picture itself, and the preset text type corresponding to the target picture type of the picture. This prompt information not only contains the basic information of the image and the extracted text information, but also integrates the relevant descriptions or keywords of the preset text type, aiming to provide a comprehensive and specific context for the big model. Through such prompt information, the big model can more accurately understand the relationship between the image content and the preset text type, so as to more effectively extract and analyze the text information that matches the preset text type.

[0097] In an embodiment of the present application, optionally, text information analysis prompt information is constructed according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture, including: for each picture from which text information has been extracted, a text information confirmation prompt template is filled in according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture, to obtain the text information analysis prompt information, wherein the text information confirmation prompt template includes content for guiding the large model to verify the text extraction result of the picture based on the picture and the preset text type corresponding to the picture; for each picture from which text information has not been extracted, a text information generation prompt template is filled in according to the picture and the preset text type corresponding to the picture, to obtain the text information analysis prompt information, wherein the text information generation prompt template includes content for guiding the large model to generate text information based on the picture and the preset text type corresponding to the picture.

[0098] In this embodiment, in order to more accurately process the text information in the image, two different strategies can be used to construct text information analysis prompt information, respectively for images from which text information has been extracted and images from which text information has not been extracted. For images from which text information has been extracted, the text information confirmation prompt template is filled in according to the image itself, the text extraction result of the image, and the preset text type corresponding to the image. The design of this template is intended to guide the large model to verify the extracted text information to ensure that it matches the image content and the preset text type. Specifically, the template may contain some guiding sentences or questions, such as "Please confirm whether the following text information matches the [preset text type] in the image", and attach the extracted text information for comparison and verification by the large model. In this way, the system can further improve the accuracy and reliability of the text information. For images from which text information has not been extracted, the text information generation prompt template will be filled in according to the image itself and the preset text type corresponding to the image. The design of this template focuses on guiding the large model to generate text information based on the image content and the preset text type. Specifically, the template may contain some inspiring statements or prompts, such as "Please generate corresponding text information based on [image features] and [preset text type] in the image", to stimulate the creativity and understanding ability of the large model, so as to generate text information that matches the image content and the preset text type. Through these two different strategies, the system can flexibly process different types of images and provide targeted text information analysis prompts according to actual conditions. This not only improves the accuracy and efficiency of text information processing, but also enhances the adaptability and robustness of the system.

[0099] Step 204: Perform an image integrity check on the target image type based on the required image type, and perform a text integrity check on the target text information based on the required text type.

[0100] In the information entry process of the embodiment of the present application, in order to ensure that all necessary information has been correctly entered, so as to support the accurate filling of the application information page of the target function. The target image type can also be checked for image integrity according to the required image type to check whether all required images have been uploaded and classified correctly. At the same time, the system will also perform a text integrity check on the target text information according to the required text type to ensure that all necessary text information has been correctly extracted or generated.

[0101] Step 205: If the image integrity check and the text integrity check are passed, then based on the image, the target image type of the image and the target text information, fill in the application information page of the target function.

[0102] In the embodiment of the present application, if both the image integrity check and the text integrity check are passed, it means that all necessary information has been completely and accurately entered. At this time, based on the image, the target image type of the image, and the target text information, the application information page of the target function is automatically filled in. This step realizes the automatic entry of information, greatly improving the efficiency and accuracy of information entry.

[0103] Step 206: If at least one of the image integrity check and the text integrity check fails, the missing items that fail the integrity check are determined; when the missing items are predicted items, the missing items are predicted based on the image and the target text information through a large model, and based on the predicted missing item information, the image, the target image type of the image and the target text information, the application information page of the target function is filled in; and when the missing items are not predicted items, the application information page of the target function is filled in directly based on the image, the target image type of the image and the target text information.

[0104] In the embodiment of the present application, if at least one of the image integrity check and the text integrity check fails, the missing items that fail the integrity check can be determined first. These missing items may include images that have not been uploaded, images that are misclassified, missing text information, etc. Next, the system will adopt different processing strategies according to the nature of the missing items: if the missing items belong to the predicted items, that is, the missing items can be predicted by other information that has been entered (such as pictures and target text information), then continue to use the large model to predict the missing items, and fill in the application information page of the target function based on the predicted missing item information, pictures, target picture types of pictures, and target text information. This step makes full use of the prediction ability of the large model and can make up for the inconvenience caused by the lack of information to a certain extent. If the missing items do not belong to the predicted items, that is, the missing items cannot be predicted by other information that has been entered, then the application information page of the target function is directly filled based on the information that has been entered (pictures, target picture types of pictures, and target text information), and the user is prompted to pay attention to the missing items on the page. This step ensures the continuity and integrity of information entry, and reminds users to supplement the missing information in time. Through such a processing flow, the information entry process of the embodiment of the present application can ensure that all necessary information has been correctly entered, and provide a reasonable processing strategy when information is missing, thereby supporting the accurate filling of the application information page of the target function. This process not only improves the efficiency and accuracy of information entry, but also enhances the adaptability and user-friendliness of the system.

[0105] In an embodiment of the present application, optionally, the predicted item includes a required text type; in step 206, when the missing item belongs to the predicted item, the missing item is predicted based on the image and the target text information through the large model, including: when the missing item belongs to the predicted item, based on the target text information and the picture text information corresponding to the picture, obtaining the associated item information corresponding to the missing item, and constructing the missing item prediction prompt information based on the associated item information and the missing item, and predicting the missing item based on the missing item prediction prompt information through the large model.

[0106] In this embodiment, when it is detected that the missing item belongs to the predicted item (a certain type of demand text), first, based on the target text information and the picture text information corresponding to the picture, the associated item information related to the missing item is obtained. These associated item information may include other text information associated with the target text information, specific areas or features in the picture, etc., which have some logical or semantic connection with the missing item. After obtaining the associated item information, a missing item prediction prompt information is further constructed based on this information and the missing item itself. This prompt information is intended to provide a comprehensive and specific contextual environment for the large model to more accurately understand and predict the content of the missing item. The prompt information may include a detailed description of the associated item information, specific requirements of the missing item or expected prediction results, etc. Finally, this missing item prediction prompt information is input into the large model for prediction. The large model will use its powerful natural language processing capabilities and deep learning algorithms to deeply analyze the prompt information and try to infer the content of the missing item from it. The prediction results will be used as a supplement to the missing item information, together with other entered information, to fill in the application information page of the target function. Through such a processing flow, the information entry process of the embodiment of the present application can more accurately predict and process missing text information, especially when the required text type is missing. This strategy not only improves the completeness and accuracy of information entry, but also enhances the intelligence of the system and user experience.

[0107] In an embodiment of the present application, optionally, the target function includes opening an online store, the required image types include store facade images, store environment images, business license images, and business license images, and the required text types include store name, store address, business category, and business hours; the predicted items include business category and business hours; in the case where the missing item belongs to a predicted item, based on the target text information and the image text information corresponding to the image, obtaining the associated item information corresponding to the missing item, including: in the case where the missing item includes the business category, based on the store name, the image text information corresponding to the store facade image, the image text information corresponding to the business license image, and the business license image Corresponding picture and text information, dish image information, at least one of the business category information of related stores corresponding to the store name, determine the associated item information corresponding to the business category; wherein the related stores include chain stores, and the dish image information includes picture and text information recognized in a picture whose target image type is a menu and / or a picture whose target image type is a dish; in the case where the missing item includes business hours, determine the range of surrounding stores corresponding to the store address, obtain information of each surrounding store within the range of surrounding stores, the surrounding store information includes business categories of surrounding stores and business hours of surrounding stores, and determine the associated item information corresponding to the business hours based on the surrounding store information and the business categories.

[0108] In this embodiment, for the target function of opening a store online, the required image types include store front images, store environment images, business license images, and business license images. The required text types include store names, store addresses, business categories, and business hours. The predicted items include business categories and business hours. Of course, the predicted items can also include other types of items, which are not limited here. For the prediction item of business category, when it is detected that it is missing, the associated item information can be determined based on multiple information sources, and these information sources include but are not limited to: store name (which may imply the business direction or characteristics), image text information corresponding to the store front image (such as the text on the doorplate), image text information corresponding to the business license image (such as the description of the business scope), image text information corresponding to the business license image (such as the permitted business items), dish image information (including the image text information identified in the image of the target image type being the menu and / or the dishes identified in the image of the target image type being the dish, which information can directly reflect the type of dishes provided by the store), and the relevant store business category information corresponding to the store name (such as the business categories of chain stores or stores of the same brand, which information may be obtained through technical means such as big data analysis). By integrating this information, the big model can more accurately infer the missing business categories. It should be noted that the big model can be trained in advance for its ability to predict business categories. In terms of sample construction, due to the uneven distribution of merchant categories, the strategy of up and down sampling is used to balance the sample size of different categories to avoid deviations in model training. For the prediction item of business hours, when it is detected that it is missing, the surrounding store range corresponding to the store address can be determined first. This range may be determined based on geographic location information (such as longitude and latitude, street name, etc.), such as the surrounding store range with a preset distance as the radius based on the geographic location information of the store address, or the surrounding store range with the business district where the store address is located as the range; then, the information of each surrounding store within this range is obtained, including the business categories and business hours of the surrounding stores; after obtaining this information, the relevant surrounding store information is screened based on the business categories of the missing store, and the big model is used to try to infer the missing business hours from the business hours of these related stores. This strategy uses the business hours of surrounding stores as a reference, which improves the accuracy and practicality of the prediction. Through such a processing flow, the information entry process of the embodiment of the present application can more accurately predict and process the missing business categories and business hours information in the online store opening application. For the situation where certain items are not included in the pictures provided by the user, item prediction can be realized. This strategy not only improves the completeness and accuracy of information entry, but also enhances the intelligence of the system and user experience.

[0109] Step 207: display the application information page, and when the application information page is confirmed, submit the input information of the target function based on the confirmed application information page.

[0110] In this embodiment, the processed and filled application information page is presented to the user. This page will contain all necessary information, such as store name, store address, business category, business hours, and store front map, store environment map, business license map, business license map and other picture information. The user can check and confirm whether all information is accurate on this page. If any error or omission is found, the user has the right to modify or supplement. After the user confirms that all information on the application information page is accurate, the submission operation can be performed, the user's submission request is received, and based on the confirmed application information page, the input information of the target function is submitted to the corresponding back-end processing system or database. Through this step, not only the accuracy and completeness of all necessary information are ensured, but also an intuitive and convenient interface is provided for users to view and confirm their application information. In addition, the system also greatly improves the efficiency and accuracy of information entry through the automated submission process, providing users with a better experience.

[0111] In an embodiment of the present application, optionally, after the application information page is displayed in step 207, it also includes: obtaining information modification data for the application information page, and modifying specific types of information on the application information page based on the information modification data; recording the information modification data, and constructing a large model training sample based on the information modification data to perform optimization training of the large model.

[0112] In this embodiment, after the user modifies the information on the application information page, not only can the page content be updated accordingly, but also they can be used to optimize the training of the large model. Specifically, after the application information page is displayed, the user is allowed to view and confirm all information. If the user finds any errors or needs to supplement the information, they can modify it through the interface provided by the system. These modifications may involve specific types of information, such as store names, business categories, business hours, etc. If the user submits the modification data, the corresponding information on the application information page can be updated based on these modification data to ensure that the submitted information is the most accurate and complete. In addition, in addition to updating the application information page, the information modification data submitted by the user can be further recorded to optimize the training of the large model. For example, if the user modifies the type of the picture, then the picture classification sample can be constructed based on the modified picture and the modified picture type to train the picture classification ability of the large model. For another example, if the user modifies the text information of a certain demand text type (not belonging to the prediction item), then the text analysis sample can be constructed based on the modified demand text type, the modified text information, and the picture corresponding to the demand text type to train the text analysis ability of the large model. For another example, if a user modifies the text information of a certain demand text type (belonging to the prediction item), then a text prediction sample can be constructed based on the modified demand text type, the modified text information, and the associated item information corresponding to the demand text type to train the text prediction ability of the large model. This optimization training method based on actual user feedback can not only improve the prediction ability of the large model, but also make it more intelligent and more humane. Through continuous iteration and optimization, a more accurate and personalized service experience can be provided to users. The embodiment of the present application further improves the flexibility and intelligence level of the system by introducing information modification and page update functions, as well as recording modification data and optimizing the strategy of training large models. These functions not only enhance the user experience, but also provide strong support for the continuous optimization and upgrading of the system.

[0113] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides an information entry device based on a large model, such as Figure 3 As shown, the device comprises:

[0114] A display module, configured to display a picture entry page for a target function, wherein the target function is a function applied for by providing specific type information, wherein the specific type information includes picture information of at least one required picture type and text information of at least one required text type;

[0115] An analysis module, used to obtain at least one picture entered based on the picture entry page, and perform picture type recognition and text information analysis on the picture through a large model;

[0116] A filling module, used to fill in the application information page of the target function based on the picture, the target picture type identified by the large model and the analyzed target text information, wherein the application information page is a modifiable page containing the specific type of information;

[0117] The display module is further used to display the application information page, and when the application information page is confirmed, submit the input information of the target function based on the confirmed application information page.

[0118] Optionally, the analysis module is further used to:

[0119] Performing text recognition on each picture respectively to obtain picture text information of each picture;

[0120] For any picture, identify whether the picture text information of the picture hits any keyword corresponding to the required picture type, and determine the target picture type of the picture based on the required picture type corresponding to the hit keyword;

[0121] For the pictures corresponding to the picture text information that does not hit the keyword, the picture type is identified by the large model to determine the target picture type of the picture.

[0122] Optionally, the analysis module is further used to:

[0123] Identify whether the picture text information of the picture matches a keyword corresponding to a first picture type, wherein the first picture type includes a standard picture type in the required picture type;

[0124] Determining a second picture type to be identified according to the required picture type and the identified target picture type;

[0125] For each picture of undetermined picture type, picture type identification prompt information is constructed according to the picture and the second picture type, and the target picture type of the picture is identified at least in the second picture type based on the picture type identification prompt information through a large model.

[0126] Optionally, obtaining feature description information corresponding to the second image type;

[0127] A picture type identification prompt template is filled according to the picture, the second picture type and the feature description information to obtain the picture type identification prompt information.

[0128] Optionally, the analysis module is further used to: perform image integrity check on the target image type based on the required image type;

[0129] The display module is further configured to: if the image integrity check is passed, display a first image confirmation page based on each image and the target image type of each image, and in response to a confirmation operation on the first image confirmation page, use the image type of each image to be confirmed as the target image type of each image;

[0130] If the image integrity check fails, a second image confirmation page is displayed based on each image, the target image type of each image, and the missing image type, wherein the second image confirmation page includes an image upload control corresponding to the missing image type; when an image of the missing image type is uploaded based on the image upload control, in response to a confirmation operation on the second image confirmation page, the image type of each image to be confirmed is used as the target image type of each image.

[0131] Optionally, the analysis module is further used to:

[0132] For each picture, text information analysis prompt information is constructed according to the preset text type corresponding to the picture and the target picture type of the picture, and the text information matching the preset text type is analyzed based on the text information analysis prompt information through a large model.

[0133] Optionally, the analysis module is further used to:

[0134] For each picture, according to the preset text type corresponding to the picture, text extraction is performed on the picture text information corresponding to the picture;

[0135] Text information analysis prompt information is constructed according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture.

[0136] Optionally, the analysis module is further used to:

[0137] For each picture from which text information has been extracted, a text information confirmation prompt template is filled according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture to obtain the text information analysis prompt information, wherein the text information confirmation prompt template includes content for guiding the large model to verify the text extraction result of the picture based on the picture and the preset text type corresponding to the picture;

[0138] For each picture from which text information has not been extracted, the text information generation prompt template is filled in according to the picture and the preset text type corresponding to the picture to obtain the text information analysis prompt information, wherein the text information generation prompt template includes content for guiding the large model to generate text information based on the picture and the preset text type corresponding to the picture.

[0139] Optionally, the analysis module is further used to: perform an image integrity check on the target image type based on the required image type, and perform a text integrity check on the target text information based on the required text type;

[0140] The filling module is further used for: if the image integrity check and the text integrity check are passed, then executing the step of filling the application information page of the target function based on the image, the target image type of the image and the target text information;

[0141] If at least one of the image integrity check and the text integrity check fails, the missing item that fails the integrity check is determined; when the missing item belongs to a predicted item, the missing item is predicted based on the image and the target text information through a large model, and the application information page of the target function is filled based on the predicted missing item information, the image, the target image type of the image and the target text information; and when the missing item does not belong to a predicted item, the application information page of the target function is filled directly based on the image, the target image type of the image and the target text information.

[0142] Optionally, the predicted item includes a required text type; the analysis module is further used to: when the missing item belongs to the predicted item, based on the target text information and the picture text information corresponding to the picture, obtain the related item information corresponding to the missing item, and construct missing item prediction prompt information based on the related item information and the missing item, and predict the missing item based on the missing item prediction prompt information through a large model.

[0143] Optionally, the target function includes opening a store online, the required image types include store front images, store environment images, business license images, and business license images, the required text types include store name, store address, business category, and business hours; the predicted items include business category and business hours;

[0144] In the case where the missing item is a predicted item, obtaining the associated item information corresponding to the missing item based on the target text information and the picture text information corresponding to the picture includes:

[0145] In the case where the missing item includes a business category, the associated item information corresponding to the business category is determined based on at least one of the store name, the image and text information corresponding to the store front image, the image and text information corresponding to the business license image, the image and text information corresponding to the business permit image, the dish image information, and the business category information of the relevant stores corresponding to the store name; wherein the relevant stores include chain stores, and the dish image information includes the image and text information identified in the image whose target image type is a menu and / or the image whose target image type is a dish;

[0146] In the case where the missing item includes business hours, determine the range of surrounding stores corresponding to the store address, obtain information of each surrounding store within the range, the surrounding store information includes business categories of surrounding stores and business hours of surrounding stores, and determine related item information corresponding to the business hours based on the surrounding store information and the business categories.

[0147] Optionally, the analysis module is further used to:

[0148] Performing watermark recognition on the image and / or similar image recognition based on a preset image library to determine whether the image is a forged image, wherein the forged image includes a watermarked image and an image that meets a similarity condition;

[0149] If the picture is the forged picture, a reminder page is displayed based on the forged picture to remind the user to correct the forged picture.

[0150] Optionally, the analysis module is further used to:

[0151] For any picture, based on the target picture type of the picture, at least one comparison picture is determined from each preset picture in the preset picture library, and the similarity between the picture and each comparison picture is calculated according to the comparison conditions corresponding to the target picture type of the picture, wherein the comparison conditions for text-type pictures include comparing text similarity, and the comparison conditions for image-type pictures include comparing pixel grayscale value similarity.

[0152] Optionally, the analysis module is further used to:

[0153] Acquire information modification data for the application information page, and modify specific type of information on the application information page based on the information modification data;

[0154] The information modification data is recorded, and a large model training sample is constructed based on the information modification data to perform optimization training of the large model.

[0155] It should be noted that for other corresponding descriptions of the functional units involved in the information entry device based on a large model provided in the embodiment of the present application, reference can be made to Figure 1 to Figure 2 The corresponding description in the method will not be repeated here.

[0156] The embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a bus, a processor, a memory and a communication interface, and can also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0157] Those skilled in the art will appreciate that the structure of the above-mentioned computer device is only a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0158] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0159] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for information entry based on a large model, characterized in that: The method comprises: Displaying a picture entry page for a target function, wherein the target function is a function applied for by providing specific type information, and the specific type information includes picture information of at least one required picture type and text information of at least one required text type; Acquire at least one picture entered based on the picture entry page, and perform picture type recognition and text information analysis on the picture through a large model; Based on the picture, the target picture type identified by the large model and the analyzed target text information, filling in the application information page of the target function, wherein the application information page is a modifiable page containing the specific type of information; The application information page is displayed, and when the application information page is confirmed, the input information of the target function is submitted based on the confirmed application information page.

2. The method according to claim 1, characterized in that Before performing picture type recognition on the picture by using the large model, the method further includes: Performing text recognition on each picture respectively to obtain picture text information of each picture; For any picture, identifying whether the picture text information of the picture hits the keyword corresponding to the first picture type, and determining the target picture type of the picture based on the required picture type corresponding to the hit keyword, wherein the first picture type includes the standard picture type in the required picture type; Accordingly, the image type is identified by the large model, including: Determining a second picture type to be identified according to the required picture type and the identified target picture type; For each picture of undetermined picture type, obtain feature description information corresponding to the second picture type, fill in a picture type identification prompt template according to the picture, the second picture type and the feature description information to obtain the picture type identification prompt information, and identify the target picture type of the picture at least in the second picture type based on the picture type identification prompt information through a large model.

3. The method according to claim 1, characterized in that Before performing text information analysis on the image using the large model, the method further includes: Performing an image integrity check on the target image type based on the required image type; If the image integrity check passes, then based on each image and the target image type of each image, a first image confirmation page is displayed, and in response to a confirmation operation on the first image confirmation page, the image type of each image to be confirmed is used as the target image type of each image; If the image integrity check fails, a second image confirmation page is displayed based on each image, the target image type of each image, and the missing image type, wherein the second image confirmation page includes an image upload control corresponding to the missing image type; when an image of the missing image type is uploaded based on the image upload control, in response to a confirmation operation on the second image confirmation page, the image type of each image to be confirmed is used as the target image type of each image.

4. The method according to claim 1, characterized in that: The image is analyzed for text information using a large model, including: For each picture, according to the preset text type corresponding to the picture, text extraction is performed on the picture text information corresponding to the picture; For each picture, construct text information analysis prompt information according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture, and analyze the text information matching the preset text type based on the text information analysis prompt information through the big model; According to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture, constructing text information analysis prompt information includes: For each picture from which text information has been extracted, a text information confirmation prompt template is filled according to the picture, the text extraction result of the picture, and the preset text type corresponding to the picture to obtain the text information analysis prompt information, wherein the text information confirmation prompt template includes content for guiding the large model to verify the text extraction result of the picture based on the picture and the preset text type corresponding to the picture; For each picture from which text information has not been extracted, the text information generation prompt template is filled in according to the picture and the preset text type corresponding to the picture to obtain the text information analysis prompt information, wherein the text information generation prompt template includes content for guiding the large model to generate text information based on the picture and the preset text type corresponding to the picture.

5. The method according to claim 1, characterized in that Before filling in the application information page of the target function based on the image, the target image type identified by the large model and the analyzed target text information, the method further includes: Performing an image integrity check on the target image type based on the required image type, and performing a text integrity check on the target text information based on the required text type; If the image integrity check and the text integrity check are passed, the step of filling in the application information page of the target function based on the image, the target image type of the image and the target text information is performed; If at least one of the image integrity check and the text integrity check fails, the missing item that fails the integrity check is determined; in a case where the missing item belongs to a predicted item, based on the target text information and the image text information corresponding to the image, the associated item information corresponding to the missing item is obtained, based on the associated item information and the missing item, missing item prediction prompt information is constructed, the missing item is predicted based on the missing item prediction prompt information through a large model, and based on the predicted missing item information, the image, the target image type of the image and the target text information, the application information page of the target function is filled; and in a case where the missing item does not belong to a predicted item, the application information page of the target function is filled directly based on the image, the target image type of the image and the target text information.

6. The method according to claim 5, characterized in that The target function includes opening a store online, the required image types include store facade images, store environment images, business license images, and business license images, the required text types include store name, store address, business category, and business hours; the predicted items include business category and business hours; In the case where the missing item is a predicted item, obtaining the associated item information corresponding to the missing item based on the target text information and the picture text information corresponding to the picture includes: In the case where the missing item includes a business category, the associated item information corresponding to the business category is determined based on at least one of the store name, the image and text information corresponding to the store front image, the image and text information corresponding to the business license image, the image and text information corresponding to the business permit image, the dish image information, and the business category information of the relevant stores corresponding to the store name; wherein the relevant stores include chain stores, and the dish image information includes the image and text information identified in the image whose target image type is a menu and / or the image whose target image type is a dish; In the case where the missing item includes business hours, determine the range of surrounding stores corresponding to the store address, obtain information of each surrounding store within the range, the surrounding store information includes business categories of surrounding stores and business hours of surrounding stores, and determine related item information corresponding to the business hours based on the surrounding store information and the business categories.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Performing watermark recognition on the image and / or similar image recognition based on a preset image library to determine whether the image is a forged image, wherein the forged image includes a watermarked image and an image that meets a similarity condition; If the picture is a forged picture, a reminder page is displayed based on the forged picture to remind the user to correct the forged picture; Among them, similar image recognition based on a preset image library is performed on the image, including: for any image, based on the target image type of the image, at least one comparison image is determined from each preset image in the preset image library, and the similarity between the image and each comparison image is calculated according to the comparison conditions corresponding to the target image type of the image, wherein the comparison conditions for text-type images include comparing text similarity, and the comparison conditions for image-type images include comparing pixel grayscale value similarity.

8. The method according to any one of claims 1 to 6, characterized in that After displaying the application information page, the method further includes: Acquire information modification data for the application information page, and modify specific type of information on the application information page based on the information modification data; The information modification data is recorded, and a large model training sample is constructed based on the information modification data to perform optimization training of the large model.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images