Information inspection method, device and equipment based on large model and storage medium

Abstract

The invention provides an information detection method and device based on a large model, equipment and a storage medium, and relates to the field of data processing and image processing, in particular to the technical fields of artificial intelligence, large models and the like. The specific implementation scheme is as follows: acquiring a to-be-detected image, and acquiring an initial recognition result of the to-be-detected image; wherein the initial recognition result is a result obtained after text recognition is carried out on the to-be-detected image, and at least the to-be-detected image is input into a target large model, so that target scene information of the to-be-detected image is obtained; the target scene information and the initial recognition result are input into the target large model to obtain an output target recognition result, and the target recognition result is at least a result which is obtained after the initial recognition result is corrected and matched with the target scene information. Comprising key information of the to-be-detected image.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of data processing and image processing, and in particular to technical fields such as artificial intelligence and large models. Background Art

[0002] The recognition of key information in documents often appears in various scenarios, and its importance cannot be ignored. For example, in the verification scenario of bidding / tendering documents, the accuracy of the key information in the documents is particularly important; however, in the existing key information recognition scenarios, the accuracy of the key information is often low. Therefore, how to effectively improve the accuracy of the recognition results has become a problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a large model-based information inspection method, device, equipment and storage medium.

[0004] According to one aspect of the present disclosure, there is provided a large model-based information detection method, comprising:

[0005] Acquire an image to be detected, and acquire an initial recognition result of the image to be detected; wherein the initial recognition result is a result obtained after performing text recognition on the image to be detected;

[0006] At least input the image to be detected into a target large model to obtain target scene information of the image to be detected;

[0007] The target scene information and the initial recognition result are input into the target large model to obtain an output target recognition result, wherein the target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

[0008] According to another aspect of the present disclosure, there is provided an information detection device based on a large model, comprising:

[0009] A preprocessing unit, used to obtain an image to be detected and obtain an initial recognition result of the image to be detected; wherein the initial recognition result is a result obtained after performing text recognition on the image to be detected;

[0010] A target processing unit is used to input at least the image to be detected into a target large model to obtain target scene information of the image to be detected; input the target scene information and the initial recognition result into the target large model to obtain an output target recognition result; wherein the target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0017] In this way, the disclosed scheme effectively utilizes the large model to correct the initial recognition results of the image to be detected. Compared with the existing recognition methods, the target recognition results obtained by the disclosed scheme are more accurate. Moreover, the above process does not require human intervention, and is intelligent, efficient and accurate, thereby providing strong support for improving user experience.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 The following is a flow chart of the information detection method based on a large model according to an embodiment of the present disclosure. Figure 1 ;

[0021] Figure 2 The following is a flow chart of the information detection method based on a large model according to an embodiment of the present disclosure. Figure 2 ;

[0022] Figure 3 It is a schematic diagram of a process for determining training data required for training a prompt word generator according to an embodiment of the present disclosure;

[0023] Figure 4It is a schematic diagram of the processing flow of the prompt word generator in the information detection method based on the large model according to an embodiment of the present disclosure;

[0024] Figure 5 It is a schematic diagram of the connection relationship between the modules in the preset recognition model according to the embodiment of the present disclosure;

[0025] Figure 6 is a flowchart of a specific example of a large model-based information detection method according to an embodiment of the present disclosure;

[0026] Figure 7 is a structural schematic diagram of an information detection device based on a large model according to an embodiment of the present disclosure;

[0027] Figure 8 It is a block diagram of an electronic device used to implement the large model-based information detection method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0029] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, it can mean including any one or more elements selected from the set consisting of A, B and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0030] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0031] The following is a description of the related technologies of the embodiments of the present disclosure. The following related technologies are optional solutions that can be arbitrarily combined with the technical solutions of the embodiments of the present disclosure, and they all belong to the protection scope of the embodiments of the present disclosure.

[0032] Large models usually refer to machine learning models with large parameters and capable of handling complex tasks, especially deep learning models.

[0033] Fine-tuning is an important concept in the field of machine learning, especially in the fields of natural language processing (NLP) and computer vision. Fine-tuning refers to further training a pre-trained model using labeled data for a specific task to optimize the model's performance on the task. Pre-trained models are usually trained on large-scale datasets to learn general feature representations, while fine-tuning is to adapt the model to specific application scenarios based on these general features.

[0034] Figure 1 This is a schematic flow chart of an information detection method based on a large model according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0035] Further, the method includes at least part of the following contents. Figure 1 As shown, including:

[0036] Step S101: obtaining an image to be detected, and obtaining an initial recognition result of the image to be detected.

[0037] Here, the initial recognition result is a result obtained after performing text recognition on the image to be detected. For example, in one example, the initial recognition result includes at least part of the key information in the image to be detected.

[0038] Step S102: at least input the image to be detected into a target large model to obtain target scene information of the image to be detected.

[0039] Here, it should be noted that the target scene information can indicate the scene corresponding to the image to be detected, thus laying a foundation for subsequently improving the accuracy of the recognition result.

[0040] Furthermore, in one example, in order to further improve the accuracy of the inference result so that the obtained target scene information accurately expresses the real scene of the image to be detected, the initial recognition result and the image to be detected can be input into the target macro model to obtain the target scene information. This lays the foundation for further improving the accuracy of the target recognition result.

[0041] Step S103: input the target scene information and the initial recognition result into the target large model to obtain an output target recognition result.

[0042] The target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

[0043] In this way, the disclosed scheme effectively utilizes the large model to correct the initial recognition results of the image to be detected. Compared with the existing recognition methods, the target recognition results obtained by the disclosed scheme are more accurate. Moreover, the above process does not require human intervention, and is intelligent, efficient and accurate, thereby providing strong support for improving user experience.

[0044] In addition, since the disclosed solution also fully considers the scene information (that is, the target scene information) of the image to be detected during the process of correcting the initial recognition results using the large model, the large model is more targeted during the recognition process, laying the foundation for improving the accuracy of the recognition results.

[0045] In a specific example of the disclosed scheme, the disclosed scheme can also be applied to verification scenarios. For example, after obtaining the key information of the image to be detected, the key information of the image to be detected is compared with the target key information (for example, with the target key information obtained from other systems) to obtain the target detection result of the image to be detected. At this time, the target detection result can represent the authenticity of the image to be detected, or can be called the authenticity of the key information of the image to be detected.

[0046] For example, in a bidding scenario, the image to be detected can be a contract agreement image; at this time, the key information of the contract in the contract agreement image can be obtained by using the disclosed solution. At this time, the key information of the contract in the obtained contract agreement image can be compared with the key information of the theoretical contract (that is, the target key information), so as to determine the authenticity of the contract agreement image. The above process can automatically and intelligently distinguish the authenticity without human intervention. In this way, compared with the existing manual authenticity identification, the disclosed solution can save labor costs and is more efficient.

[0047] In this way, the disclosed solution provides an intelligent authenticity detection solution, which does not require manual intervention in the detection process, and effectively saves labor costs compared to existing manual detection. Moreover, since the target recognition results obtained by the disclosed solution are highly accurate, the target detection results obtained by the disclosed solution are also more accurate than those obtained by manual detection, further effectively reducing the error rate. In other words, the authenticity detection solution provided by the disclosed solution is both efficient and intelligent, thus laying a foundation for further improving user experience.

[0048] In a specific example of the disclosed solution, the above-mentioned image to be detected can be obtained in the following manner. Specifically, before obtaining the image to be detected, or before obtaining the initial recognition result of the image to be detected, the method further includes:

[0049] Step S1001: Obtain the file to be detected.

[0050] Step S1002: Perform keyword recognition on the file to be detected to obtain a keyword recognition result.

[0051] For example, in one example, the keyword recognition result includes at least one target keyword. In this way, it is convenient to determine the image to be detected that includes the required key information based on the recognized target keyword, which provides support for the subsequent effective recognition of the key information.

[0052] For example, in one example, the layout analysis of the file to be detected can be performed, and based on the layout analysis results, the area in the text to be detected that requires keyword recognition can be determined, and then keyword recognition can be performed on the area in the file to be detected that requires keyword recognition to obtain at least one target keyword.

[0053] Step S1003: Based on the keyword recognition result, determine the image to be detected that contains key information corresponding to the target keyword.

[0054] It should be noted that, in an example, the keyword recognition result may include a target keyword. In this case, at least one image to be detected that includes key information corresponding to the target keyword may be determined based on the target keyword.

[0055] It is understandable that, in order to include all key information corresponding to the target keyword, the number of images to be detected corresponding to the target keyword may be one or more. In other words, there may not be a one-to-one correspondence between the images to be detected and the target keywords.

[0056] Or, in another example, the keyword recognition result may include multiple target keywords. In this case, the image to be detected that includes key information corresponding to each target keyword may be determined based on each target keyword.

[0057] It is understandable that when the keyword recognition result contains multiple target keywords, it is necessary to obtain an image to be detected that contains key information corresponding to each target keyword, that is, in this scenario, multiple images to be detected can be obtained. In this way, different needs of different scenarios can be met, which lays a foundation for enriching the application scenarios of the disclosed solution, improving the adaptability of the disclosed solution, and further improving the user experience.

[0058] In this way, the disclosed solution provides a specific solution for obtaining an image to be detected, and the solution can efficiently determine the image to be detected from the file to be detected that needs to be identified with key information, and the process does not require manual intervention, so it is both intelligent and efficient. Moreover, the disclosed solution does not impose any restrictions on the file to be detected. In other words, the disclosed solution can identify any file that needs to be identified with key information, so it is also universal.

[0059] Specifically, Figure 2 This is a schematic flow chart of an information detection method based on a large model according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figure 1 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0060] Further, the method includes at least part of the following contents. Figure 2 As shown, including:

[0061] Step S201: obtaining an image to be detected, and obtaining an initial recognition result of the image to be detected.

[0062] Here, the initial recognition result is a result obtained after performing text recognition on the image to be detected.

[0063] It should be noted that the relevant content about the initial recognition result can be found in the above description and will not be repeated here.

[0064] Step S202: at least input the image to be detected into a target large model to obtain target scene information of the image to be detected.

[0065] It should be noted that the relevant content about the target scene information can be found in the above description and will not be repeated here.

[0066] Here, in one example, in order to further improve the accuracy of the inference result so that the obtained target scene information accurately expresses the real scene of the image to be detected, the disclosed solution can also input the initial recognition result and the image to be detected into the target large model to obtain the target scene information. In this way, the foundation is laid for further improving the accuracy of the target recognition result.

[0067] Step S203: obtaining a target prompt word that matches the target scene information.

[0068] Here, the target prompt word is at least used to indicate the output of the target data structure matching the target scene information. For example, in one example, the target scene information can be input into the prompt word generator, or the target scene information and the initial recognition result can be input into the prompt word generator to output the target prompt word. In other words, the prompt word generator can be used to quickly determine the data format matching the target scene information.

[0069] It should be further explained that the target prompt word can indicate the data format required to be output for the target scene information, so that the data output by the subsequent model (that is, the target recognition result) meets the data format requirements, thereby laying the foundation for further improving the level of intelligence and improving user experience.

[0070] For example, in one example, the prompt generator can be trained in advance, for example, Figure 3 As shown, multiple scene information can be determined first, and then the core field (or core query field, which can be recorded as scene query) of each scene information in the multiple scene information can be determined. Finally, for the core field of each scene information, the data format required by each scene information can be determined to accurately customize the structured output data format required by each scene. Further, multiple scene information and the data format required by each scene information can be unified as training data, or the core field of each scene information can also be used as training data. In this way, the prompt word generator is trained to obtain a prompt word generator that can accurately output the data format required by the scene information.

[0071] Furthermore, for example, Figure 4As shown, in one example, in the prompt word generator obtained after the training is completed, the initial recognition result (for example, in one example, the initial recognition result can be specifically an optical character recognition (OCR) result (which may include recognized characters and / or tables)) can be vector matched with the target scene information (for example, with the core field in the target scene information) to obtain the text information associated with the core field in the initial recognition result, and then determine the data format required for the text information associated with the core field to construct the target prompt word. In other words, in one example, the target prompt word can specifically indicate the data format required to output the text information associated with the core field of the target scene information in the initial recognition result.

[0072] It is understandable that, for the disclosed solution, the target prompt word may include multiple prompt words, or may also include a prompt instance and / or a prompt word associated with the prompt instance, and the disclosed solution does not impose specific restrictions on the target prompt word. For example, for the invoice scenario, the target prompt word may be specifically: please output the invoice recognition result in JSON (JavaScript Object Notation) format, including the following keywords: invoice number (invoice_num), invoice type (invoice_type), invoice code (invoice_code), invoice amount (invoice_amount), etc.

[0073] Step S204: input the target prompt word and the initial recognition result into the target macro model to obtain an output target recognition result having the target data structure.

[0074] Here, the target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

[0075] That is to say, in this example, the target prompt word and the initial recognition result can be input into the target large model together, so as to use the reasoning ability of the target large model to correct the initial recognition result. For example, after obtaining the input target prompt word and the initial recognition result, the target large model uses few-shot learning and / or in-context learning to adjust the content in the initial recognition result, such as text or the data format of the text, or regenerate new text, etc., to obtain a target recognition result containing the key information of the image to be detected. At this time, the output format of the key information in the target recognition result meets the requirements of the required data format, thereby providing strong support for subsequent processing.

[0076] In this way, the disclosed scheme provides a refined scheme for obtaining target recognition results. The scheme first obtains a target prompt word that matches the scene information of the image to be detected (that is, the target scene information), and then uses the target prompt word to guide the reasoning of the large model, so that the large model can not only modify the initial recognition results and output target recognition results with higher accuracy, but also make the output target recognition results meet the data format requirements. In this way, on the basis of effectively ensuring accuracy, the degree of intelligence is further improved. At the same time, it also lays the foundation for further improving the user experience.

[0077] In a specific example, the above-mentioned initial recognition result can be obtained in the following manner, that is, before obtaining the initial recognition result of the image to be detected, the method further includes:

[0078] The acquired image to be detected is input into a preset recognition model to obtain the outputted initial recognition result. In this way, the model is used to quickly obtain the initial recognition result, which provides strong support for the subsequent rapid acquisition of high-precision target recognition results.

[0079] Here, in one example, an OCR model can be used for optical character recognition to obtain an optical character recognition result (i.e., the initial recognition result described above), for example, it may include recognized characters and / or tables. However, it should be noted that the accuracy of the initial recognition result obtained at this time is relatively low, for example, there may be missing characters, garbled characters, etc. Therefore, after obtaining the initial recognition result, the reasoning ability of the large model can be further used to correct the initial recognition result, thereby effectively improving the accuracy of the final recognition result.

[0080] Furthermore, in one example, Figure 5 As shown, the preset recognition model may mainly include a text detection module, a direction classifier (also called a text box correction module) and a text recognition module.

[0081] Furthermore, the text detection module is mainly used to detect the text area in the input image to be detected. Here, it should be noted that the image to be detected may contain multiple text areas. At this time, all text areas of the image to be detected can be detected by using the text detection module. For example, for the invoice scenario described above, the text detection module can be used to obtain text areas such as the invoice number (invoice_num), invoice type (invoice_type), invoice code (invoice_code), invoice amount (invoice_amount), etc. For example, a text box containing the above text can be specifically output.

[0082] Furthermore, the direction classifier is mainly used to determine whether the direction of the text in the text area satisfies a preset rule, thereby providing strong support for improving the accuracy of subsequent inspections.

[0083] Furthermore, the text recognition module is mainly used to perform text recognition on the text area where the direction of the text meets the preset rule to obtain the initial recognition result. For example, using the text recognition module, text recognition is performed on the text area where the direction of the text meets the preset rule, so that the text recognition results corresponding to each text area are obtained to obtain the initial recognition result.

[0084] In this way, the disclosed scheme provides a refined structure of a recognition model, which is simple and efficient and can quickly obtain initial recognition results. Moreover, since the disclosed scheme fully considers the influence of the direction of the text on the recognition results, the disclosed scheme can also maximize the accuracy of the initial recognition results, thereby providing strong support for the subsequent rapid and efficient correction of the initial recognition results, and at the same time, it also provides strong support for quickly obtaining target recognition results.

[0085] Furthermore, in one example, the direction classifier is also used to rotate the text area when it is determined that the direction of the text in the text area does not satisfy the preset rule, so that the direction of the text in the text area after the rotation operation satisfies the preset rule. For example, it can be determined whether the angle between the text area, such as the text in the text box, and the preset direction is a preset value (for example, whether it is zero degrees); for text at zero degrees, it can be considered that the text is in the horizontal direction. Otherwise, if it is not zero degrees, the line where the text is located can be rotated, for example, an angle rotation operation is performed to make the line where the text is located at zero degrees, and then the text area after the rotation operation is sent to the text recognition module after affine transformation to recognize the text content.

[0086] In this way, the disclosed scheme further refines the processing logic of the direction classifier, and the above processing logic is simple and efficient, thereby providing strong support for the subsequent rapid and efficient correction of the initial recognition results, and at the same time, also provides strong support for quickly obtaining target recognition results.

[0087] Furthermore, in a specific example, after obtaining the target recognition result described above, the target recognition result can also be used to fine-tune the preset recognition model, so as to further improve the recognition accuracy of the model. For example, after obtaining the output target recognition result, the method further includes:

[0088] The target recognition result is used to fine-tune at least some of the network parameters in the text detection module in the preset recognition model, and at least some of the network parameters in the text recognition module in the preset recognition model, so as to train the preset recognition model and obtain the preset recognition model after parameter fine-tuning. In this way, the similarity between the recognition result output by the fine-tuned preset recognition model and the target recognition result satisfies the preset rules, and in this way, the recognition accuracy of the preset recognition model is further improved by using the target recognition result, thereby further providing strong support for improving the accuracy of the target recognition result.

[0089] A specific application scenario of the disclosed solution is given below, namely, the verification scenario of bidding / tender documents:

[0090] During the project bidding process, the importance of verifying the key information in the bidding documents cannot be ignored. For example, to ensure the authenticity of various materials in the bidding documents, it is necessary to verify key information such as scanned copies of business licenses, contracts, and related invoices. However, the current verification process has the following problems: the labor cost is high during the verification process, and the professional quality of the personnel is high; the accuracy is difficult to guarantee, because the amount of information in the bidding documents is large, and it is difficult for staff to efficiently and accurately identify and verify various types of information within a limited time.

[0091] Based on this, this example proposes a file intelligent verification assistant based on a large model (hereinafter referred to as the intelligent verification assistant). The intelligent assistant has the following advantages:

[0092] High efficiency and accuracy: The intelligent verification assistant can quickly process a large number of bidding / tendering documents, and can automatically identify and verify the authenticity of invoices, contracts, etc., greatly improving the efficiency of verification work. Compared with traditional manual verification methods, it significantly reduces processing time and makes the verification process faster. At the same time, the intelligent verification assistant can accurately identify and verify various types of information, reducing the risk of misjudgment and missed judgment, thereby improving the accuracy of verification.

[0093] High degree of automation: The intelligent verification assistant can automatically execute the verification process without human intervention. It can automatically extract key information from the file and compare it with the information in the database to realize the automated verification process. This effectively reduces the cost and error rate of manual operations and improves work efficiency.

[0094] Strong scalability: The intelligent verification assistant based on the large model has strong scalability and can continuously adapt to new verification requirements and data changes. For example, new verification rules and algorithms can be added according to actual needs to meet more complex and diverse verification needs.

[0095] High security: Since the intelligent verification assistant uses a large model to perform the verification process, it can better protect privacy compared to manual operations. In addition, the large model also uses strict data encryption and privacy protection measures when processing sensitive information, further ensuring the security of information.

[0096] Specifically, the following combination Figure 6 The verification process of the intelligent verification assistant proposed in this example is described in detail:

[0097] Step S601, file upload: upload the bidding / tendering documents in the smart management portal.

[0098] Here, you can also authenticate the identity of the currently logged in account before uploading the bidding / tender documents, and upload the bidding / tender documents after the authentication is passed.

[0099] Step S602, obtaining the image to be detected: by keyword recognition and / or layout analysis, the area in the bidding / tendering document (e.g., scanned copy or image, etc.) that needs to be keyword recognized is recognized to automatically extract key scanned copy information, obtain multiple target keywords, and then obtain the image to be detected containing the key information corresponding to the target keywords. For example, the image of the business license scan, the screenshot of the invoice, and the contract agreement image are obtained to be detected.

[0100] Step S603, initial text recognition: using the preset recognition model described above (including the text detection module, the direction classifier (also called the text box correction module) and the text recognition module) to perform text detection and text recognition on the input image to be detected to obtain an initial recognition result.

[0101] Step S604, large model scene discrimination: the initial recognition result is sent to the target large model so that the target large model performs scene discrimination to obtain scene information corresponding to the image to be detected. For example, in one example, the scene information may be one of the following: invoice, contract, business license, etc.

[0102] Step S605, constructing a target prompt word: inputting the scene information of the image to be detected obtained above and the initial recognition result obtained above into a prompt word generator to obtain a target prompt word, which is used to indicate the data format required for the key information in the image to be detected to be output.

[0103] Step S606, the large model outputs the target recognition result: the obtained target prompt word and the initial recognition result are input into the target large model to obtain the target recognition result.

[0104] Step S607, authenticity comparison, compare and analyze the target recognition result obtained by the target large model with the real data one by one, and return the verification result.

[0105] It should be noted that, in actual applications, anomalies can also be marked based on the verification results to provide prompts.

[0106] The disclosed solution also provides an information detection device based on a large model, such as Figure 7 As shown, including:

[0107] The preprocessing unit 701 is used to obtain an image to be detected and obtain an initial recognition result of the image to be detected; wherein the initial recognition result is a result obtained after performing text recognition on the image to be detected;

[0108] The target processing unit 702 is used to input at least the image to be detected into the target large model to obtain the target scene information of the image to be detected; input the target scene information and the initial recognition result into the target large model to obtain an output target recognition result; wherein the target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

[0109] In a specific example of the disclosed solution,

[0110] The preprocessing unit is further used to obtain a target prompt word that matches the target scene information; the target prompt word is at least used to indicate outputting a target data structure that matches the target scene information;

[0111] The target processing unit is specifically used to input the target prompt word and the initial recognition result into the target large model to obtain the output target recognition result having the target data structure.

[0112] In a specific example of the disclosed solution, the preprocessing unit is further used to input the image to be detected into a preset recognition model to obtain the output initial recognition result.

[0113] In a specific example of the disclosed solution, the preset recognition model includes a text detection module, a direction classifier and a text recognition module; wherein,

[0114] The text detection module is used to detect the text area of ​​the input image to be detected;

[0115] The direction classifier is used to determine whether the direction of the text in the text area meets a preset rule;

[0116] The text recognition module is used to perform text recognition on a text area where the direction of the text meets the preset rule to obtain the initial recognition result.

[0117] In a specific example of the present disclosure, the direction classifier is also used to rotate the text area when it is determined that the direction of the text in the text area does not satisfy a preset rule, so that the direction of the text in the text area after the rotation operation satisfies the preset rule.

[0118] In a specific example of the disclosed solution, the preprocessing unit is further used for:

[0119] The target recognition result is used to fine-tune at least some of the network parameters in the text detection module and at least some of the network parameters in the text recognition module to train the preset recognition model and obtain the preset recognition model after parameter fine-tuning.

[0120] In a specific example of the disclosed solution, the preprocessing unit is further used for:

[0121] Get the file to be tested;

[0122] Performing keyword recognition on the file to be detected to obtain a keyword recognition result; the keyword recognition result includes at least one target keyword;

[0123] Based on the keyword recognition result, the image to be detected containing key information corresponding to the target keyword is determined.

[0124] In a specific example of the disclosed solution, it further includes: a comparison unit;

[0125] The comparison unit is used to compare the key information of the image to be detected with the target key information to obtain the target detection result of the image to be detected, and the target detection result is used to indicate the authenticity of the image to be detected.

[0126] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0127] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0128] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0129] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0130] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0131] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as information detection methods based on large models. For example, in some embodiments, the information detection method based on the large model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the information detection method based on the large model described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the information detection method based on the large model in any other appropriate manner (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0138] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0140] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A large model-based information detection method, comprising: Acquire an image to be detected, and acquire an initial recognition result of the image to be detected; wherein the initial recognition result is a result obtained after performing text recognition on the image to be detected; At least input the image to be detected into a target large model to obtain target scene information of the image to be detected; The target scene information and the initial recognition result are input into the target large model to obtain an output target recognition result, wherein the target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

2. The method according to claim 1, further comprising: Obtaining a target prompt word that matches the target scene information; The target prompt word is at least used to indicate outputting a target data structure matching the target scene information; The step of inputting the target scene information and the initial recognition result into the target large model to obtain an output target recognition result includes: The target prompt word and the initial recognition result are input into the target large model to obtain the output target recognition result having the target data structure.

3. The method according to claim 1 or 2, further comprising: The image to be detected is input into a preset recognition model to obtain the outputted initial recognition result.

4. The method according to claim 3, wherein: The preset recognition model includes a text detection module, a direction classifier and a text recognition module; wherein, The text detection module is used to detect the text area in the input image to be detected; The direction classifier is used to determine whether the direction of the text in the text area meets a preset rule; The text recognition module is used to perform text recognition on a text area where the direction of the text meets the preset rule to obtain the initial recognition result.

5. The method according to claim 4, wherein: The direction classifier is also used to rotate the text area if it is determined that the direction of the text in the text area does not meet the preset rule, so that the direction of the text in the text area after the rotation operation meets the preset rule.

6. The method according to claim 4, wherein: The method further comprises: The target recognition result is used to fine-tune at least some of the network parameters in the text detection module and at least some of the network parameters in the text recognition module to train the preset recognition model and obtain the preset recognition model after parameter fine-tuning.

7. The method according to claim 1 or 2, further comprising: Get the file to be tested; Performing keyword recognition on the file to be detected to obtain a keyword recognition result; The keyword recognition result includes at least one target keyword; Based on the keyword recognition result, the image to be detected containing key information corresponding to the target keyword is determined.

8. The method according to claim 1 or 2, further comprising: The key information of the image to be detected is compared with the target key information to obtain the target detection result of the image to be detected, and the target detection result is used to indicate the authenticity of the image to be detected.

9. An information detection device based on a large model, comprising: A preprocessing unit, used to obtain an image to be detected and obtain an initial recognition result of the image to be detected; wherein the initial recognition result is a result obtained after performing text recognition on the image to be detected; A target processing unit is used to input at least the image to be detected into a target large model to obtain target scene information of the image to be detected; input the target scene information and the initial recognition result into the target large model to obtain an output target recognition result; wherein the target recognition result is at least a result obtained by correcting the initial recognition result and matching the target scene information, including key information of the image to be detected.

10. The device according to claim 9, wherein: The preprocessing unit is further used to obtain a target prompt word that matches the target scene information; the target prompt word is at least used to indicate outputting a target data structure that matches the target scene information; The target processing unit is specifically used to input the target prompt word and the initial recognition result into the target large model to obtain the output target recognition result having the target data structure.

11. The device according to claim 9 or 10, wherein: The preprocessing unit is further used to input the image to be detected into a preset recognition model to obtain the output initial recognition result.

12. The device according to claim 11, wherein The preset recognition model includes a text detection module, a direction classifier and a text recognition module; wherein, The text detection module is used to detect the text area of ​​the input image to be detected; The direction classifier is used to determine whether the direction of the text in the text area meets a preset rule; The text recognition module is used to perform text recognition on a text area where the direction of the text meets the preset rule to obtain the initial recognition result.

13. The device according to claim 12, wherein: The direction classifier is also used to rotate the text area if it is determined that the direction of the text in the text area does not meet the preset rule, so that the direction of the text in the text area after the rotation operation meets the preset rule.

14. The device according to claim 13, wherein: The preprocessing unit is further used for: The target recognition result is used to fine-tune at least some of the network parameters in the text detection module and at least some of the network parameters in the text recognition module to train the preset recognition model and obtain the preset recognition model after parameter fine-tuning.

15. The device according to claim 9 or 10, wherein: The preprocessing unit is further used for: Get the file to be tested; Performing keyword recognition on the file to be detected to obtain a keyword recognition result; the keyword recognition result includes at least one target keyword; Based on the keyword recognition result, the image to be detected containing key information corresponding to the target keyword is determined.

16. The device according to claim 9 or 10, wherein: Also includes: The comparison unit is used to compare the key information of the image to be detected with the target key information to obtain the target detection result of the image to be detected, and the target detection result is used to indicate the authenticity of the image to be detected.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images