A power nameplate recognition method based on multi-model fusion and expert correction mechanism
The power nameplate recognition method, which combines multi-model fusion and expert correction mechanisms, integrates optical character recognition and multimodal large models, and incorporates multiple expert mechanisms for information correction. This solves the problem of unstable output in power nameplate recognition under different scenarios, and improves recognition accuracy and system reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LONGSHINE TECH
- Filing Date
- 2026-03-17
- Publication Date
- 2026-06-26
Smart Images
Figure CN122290128A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power, and in particular to a method for identifying power nameplates based on multi-model fusion and expert correction mechanisms. Background Technology
[0002] Nameplates in the power sector are usually installed outdoors. After on-site personnel take photos, professionals manually enter the information from the nameplates into the system, which involves a large amount of manual work.
[0003] Extracting nameplate information in the power sector has several unique characteristics compared to other sectors. First, because power sector nameplates are installed outdoors, the on-site conditions during photography are often variable. The degree to which photographed nameplates are affected by factors such as lighting and dust varies, impacting subsequent identification. Second, on-site personnel often do not use fixed angles or standardized mobile devices when photographing nameplates, resulting in images of varying sizes and tilt levels, increasing the difficulty of image processing. Third, the information on power sector nameplates is highly diverse, including a large number of stamped characters, making extraction challenging.
[0004] There are already some technical methods for extracting nameplate information.
[0005] This approach employs an Optical Character Recognition (OCR) model and simple post-processing. This type of method first uses an OCR model to extract text information. However, since the OCR model only returns text, it loses crucial structural and semantic information. For example, the OCR model cannot determine that "IP55" is a protection rating or "AC 220V" is a rated voltage. Therefore, post-processing techniques, such as regular expression matching, are needed to extract the target information. This method is very effective in scenarios where the shooting angle and content are relatively fixed. However, in power scenarios, nameplates are greatly affected by outdoor conditions. The information on the nameplates is usually fixed in layout but varied in style, with complex backgrounds and blurred text. Therefore, it is difficult for the OCR model to return accurate text information, and it is also difficult to parse it using post-processing techniques.
[0006] Image preprocessing and OCR model extraction. This approach is quite mainstream. For information extraction tasks, image enhancement is usually performed first, such as locating and cropping the target region, performing grayscale processing on stamped or blurred text, eliminating noise, and highlighting text, thereby improving the accuracy of OCR model extraction. This type of method is suitable for use in simple scenarios, where the background, brightness, and color difference of the nameplate are relatively uniform. Image details can be highlighted by adjusting thresholds or using image enhancement techniques. However, in real-world power scenarios, the environment, shooting equipment, and shooting conditions of each image are different. It is difficult to process all nameplate images using a uniform preprocessing method. This specific preprocessing method may not be applicable to all scenarios, thus limiting the method's effectiveness.
[0007] Another approach is information extraction based on multimodal large models. With the development of multimodal large models, they have begun to demonstrate their capabilities in the field of OCR. However, using only multimodal large models cannot guarantee the stability of the output due to the illusion problem of the model, and the results of multiple parsings may also be different, which can cause problems in practical applications. Summary of the Invention
[0008] This invention provides a power nameplate recognition method based on multi-model fusion and expert correction mechanism to solve the defect of unstable power nameplate recognition output in different scenarios in the prior art, so as to achieve stable nameplate recognition output under different shooting conditions and improve accuracy.
[0009] This invention provides a method for identifying power nameplates based on multi-model fusion and expert correction mechanisms, comprising the following steps.
[0010] Information is extracted from power nameplate images using optical character recognition models and multimodal large models. The extracted information is scored using a pre-set reward model to obtain a score. If the score exceeds the preset reliability threshold, the extracted information will be determined as the output result; If the score does not exceed the preset reliability threshold, the extracted information is corrected according to the modification opinions with different weights through a hybrid expert correction mechanism to obtain the output result. The extracted information is corrected using a hybrid expert correction mechanism based on modification suggestions with different weights, resulting in an output that includes: The extracted information is examined by visual inspection expert mechanism, rule and syntax inspection expert mechanism and semantic inspection expert, respectively, to obtain inspection suggestions, verification suggestions and correction information; The inspection suggestion, the verification suggestion, and the correction information are scored and normalized using a reward model and a softmax function, respectively, to obtain the inspection suggestion score, the verification suggestion score, and the correction information score. Based on the inspection suggestion score, the verification suggestion score, and the correction information score, the extracted information is corrected based on the power nameplate image to obtain the output result.
[0011] According to the present invention, a power nameplate recognition method based on multi-model fusion and expert correction mechanism is provided. This method extracts information from power nameplate images using an optical character recognition model and a multimodal large model, obtaining the extracted information, including: The optical character recognition model is used to extract the text from the power nameplate image to obtain the output text image. By using a multimodal large model, text information is extracted from the output text image to obtain the extracted information.
[0012] According to the present invention, a power nameplate identification method based on multi-model fusion and expert correction mechanism is provided. The extracted information is scored using a preset reward model to obtain a score, including: The output header of the multimodal model is modified and transformed to obtain the regression model; The extracted information is scored using a regression model to obtain a score.
[0013] According to the present invention, a power nameplate identification method based on multi-model fusion and expert correction mechanism is provided. The extracted information is scored using a preset reward model to obtain a score, including: A score is obtained by scoring based on the extracted information and the output text and images using a multimodal reward model.
[0014] According to the present invention, a power nameplate recognition method based on multi-model fusion and expert correction mechanism is provided. This method uses a visual inspection expert mechanism, a rule-based grammar inspection expert mechanism, and a semantic inspection expert mechanism to inspect the extracted information, obtaining inspection suggestions, verification suggestions, and correction information, including: Through a visual inspection expert mechanism, secondary identification is performed based on the extracted information and the image of the power nameplate to obtain inspection suggestions; among which, the inspection suggestions include: suspected error information and extraction suggestions; By using a rule-based syntax checking expert mechanism, the extracted information is validated against the rules and syntax, and validation suggestions are obtained. By having semantic inspection experts perform semantic correction on the extracted information, corrected information is obtained.
[0015] The present invention provides a method for identifying power nameplates based on multi-model fusion and expert correction mechanisms. This method utilizes a rule-based grammar check expert mechanism to perform rule-based grammar verification on the extracted information, obtaining verification suggestions, including: Through a rule-based syntax checking expert mechanism, the extracted information is subjected to format checks, value range checks, and character replacements to obtain the first verification suggestion. The extracted information is verified by the rule-based grammar checker mechanism based on empirical rules to obtain a second verification suggestion. The empirical rules are automatically generated by the rule-based grammar checker mechanism based on historical correction experience.
[0016] The present invention also provides a power nameplate identification device based on multi-model fusion and expert correction mechanism, comprising the following modules: The information extraction module is used to extract information from power nameplate images using optical character recognition models and multimodal large models to obtain the extracted information; The scoring module is used to score the extracted information using a preset reward model to obtain a score. The output module is used to determine the extracted information as the output result when the score exceeds a preset reliability threshold; The correction output module is used to correct the extracted information and obtain the output result by means of a hybrid expert correction mechanism, based on modification opinions with different weights, when the score does not exceed the preset reliability threshold. The correction output module is specifically used for: The extracted information is examined by visual inspection expert mechanism, rule and syntax inspection expert mechanism and semantic inspection expert, respectively, to obtain inspection suggestions, verification suggestions and correction information; The inspection suggestion, the verification suggestion, and the correction information are scored and normalized using a reward model and a softmax function, respectively, to obtain the inspection suggestion score, the verification suggestion score, and the correction information score. Based on the inspection suggestion score, the verification suggestion score, and the correction information score, the extracted information is corrected based on the power nameplate image to obtain the output result.
[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the power nameplate identification method based on multi-model fusion and expert correction mechanism as described above.
[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the power nameplate identification method based on multi-model fusion and expert correction mechanism as described above.
[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the power nameplate identification method based on multi-model fusion and expert correction mechanism as described above.
[0020] The present invention provides a power nameplate recognition method based on multi-model fusion and expert correction mechanisms. This method extracts information from power nameplate images using an optical character recognition model and a multimodal large model. The extracted information is then scored using a preset reward model. If the score exceeds a preset reliability threshold, the extracted information is determined as the output. If the score does not exceed the preset reliability threshold, a hybrid expert correction mechanism is used to correct the extracted information based on modification opinions with different weights, resulting in the final output. This hybrid expert correction mechanism, which corrects the extracted information based on modification opinions with different weights, includes: visual inspection expert mechanisms, rule-based syntax inspection expert mechanisms, and semantic inspection expert mechanisms. The extracted information is checked separately to obtain inspection suggestions, verification suggestions, and correction information. These suggestions are then scored and normalized using a reward model and a softmax function, resulting in inspection suggestion scores, verification suggestion scores, and correction information scores, respectively. Based on these scores, the extracted information is corrected using the power nameplate image, yielding the output result. Compared to existing technologies where power nameplate recognition output is unstable in different scenarios using OCR models or multimodal large models, this invention uses an optical character recognition model and a multimodal large model to extract information from power nameplate images, eliminating the influence of background on text extraction through multi-model fusion. Simultaneously, a hybrid expert correction mechanism corrects the extracted information based on modification opinions with different weights, significantly improving the accuracy of power nameplate recognition by addressing errors caused by uneven lighting, shadows, reflections, tilt, blur, and stamped markings. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0022] Figure 1This is a flowchart illustrating the power nameplate identification method based on multi-model fusion and expert correction mechanism provided by the present invention.
[0023] Figure 2 This is an example image of the output text obtained after extracting text from a page using an OCR model, as provided by this invention.
[0024] Figure 3 This is an image of an electrical nameplate provided by this invention.
[0025] Figure 4 The identification information is obtained by processing an image of an electric power nameplate provided by this invention using traditional identification methods.
[0026] Figure 5 This is an example image of the identification information obtained by processing an image of an electric power nameplate provided by this invention using the identification method of this invention. Figure 6 This is a flowchart illustrating the information extraction, processing, and hybrid expert correction mechanism provided by the present invention.
[0027] Figure 7 This is a schematic diagram of the structure of the power nameplate identification device based on multi-model fusion and expert correction mechanism provided by the present invention.
[0028] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0030] The following is combined Figures 1-8 This invention is described.
[0031] Figure 1 This is a flowchart illustrating the power nameplate identification method based on multi-model fusion and expert correction mechanism provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 101: Extract information from the power nameplate image using the optical character recognition model and the multimodal large model to obtain the extracted information.
[0032] Optionally, step 101 includes steps A1 to A2: Step A1: Extract the text from the power nameplate image using an optical character recognition model to obtain the output text image.
[0033] Step A2: Extract text information from the output text image using a multimodal large model to obtain the extracted information.
[0034] In steps A1 and A2 above, text is extracted from the layout using an OCR model, and the text information is restored according to the original image coordinates to maintain certain structural information. This step accepts the original image to be recognized and returns pure text information with the background removed. This step can eliminate the influence of the background on information extraction. Figure 2 As shown, Figure 2 This is to eliminate the background from the output text image obtained after OCR text extraction, thus removing the influence of the background on information extraction.
[0035] By using a multimodal large model to extract text information from the output text image and return structured information, the multimodal model can understand the field relationships in the text and thus return the correct extracted information. For example, product model: S11-M. The prompt controls the model's understanding and output of information.
[0036] Step 102: Use a preset reward model to score the extracted information and obtain a score.
[0037] Optionally, step 102 includes steps B1 to B2: Step B1: Modify and transform the output header of the multimodal model to obtain the regression model; Step B2: Use a regression model to score the extracted information and obtain a score.
[0038] Optionally, step 102 includes step C1: Step C1: Using a multimodal reward model, score the extracted information and output text and images to obtain a score.
[0039] In step 102 above, the preset reward model includes either a reward model or a scoring model. The extraction result, i.e., the extracted information, is scored using the reward model or scoring model. If the reward model determines that the extraction result exceeds the confidence threshold, the final result is returned directly. If it does not exceed the confidence threshold, a hybrid expert correction mechanism is initiated. Figure 6 This is a flowchart illustrating the information extraction, processing, and hybrid expert correction mechanism provided by the present invention. The information extraction and processing flow for steps 101 to 102 above is as follows: Figure 6 Before enabling hybrid expert correction, proceed with the following steps.
[0040] There are two approaches to reward models. One is to modify the output header of the multimodal model to transform it into a regression model and output a score. The other is to use an existing multimodal reward model, which can output a score based on the current extraction results and image information.
[0041] Step 103: If the score exceeds the preset reliability threshold, the extracted information is determined as the output result.
[0042] Step 104: If the score does not exceed the preset reliability threshold, the extracted information is corrected according to the modification opinions with different weights through a hybrid expert correction mechanism to obtain the output result.
[0043] In step 104 above, the hybrid expert correction mechanism analyzes potential errors from multiple different perspectives. Each expert model is responsible for only one type of problem, thereby enhancing the accuracy of correction. Furthermore, through a reward model + softmax, this invention adds weight information to the modification opinions of each expert. Finally, through a comprehensive decision model, all weighted modification opinions are integrated to produce the final information correction output, greatly improving the accuracy of recognition.
[0044] Optionally, step 104 includes steps D1 to D3: Step D1: The extracted information is checked by visual inspection expert mechanism, rule and syntax inspection expert mechanism and semantic inspection expert, respectively, and inspection suggestions, verification suggestions and correction information are obtained; Step D2: Using the reward model and softmax function, score and normalize the inspection suggestions, verification suggestions, and correction information respectively to obtain the inspection suggestion score, verification suggestion score, and correction information score. Step D3: Based on the inspection suggestion score, verification suggestion score, and correction information score, correct the extracted information based on the power nameplate image to obtain the output result.
[0045] In step D3 above, different experts have varying degrees of importance for corrective actions. To avoid misclassifying correct actions as errors, each expert's feedback needs to be scored to determine whether the correction is good or bad. In the final comprehensive decision, corrections with higher importance are given priority. This step is accomplished using a reward model. Subsequently, a softmax function is used to normalize the scores of each expert's correction, representing the importance of the correction. The formula for the softmax function is as follows: in, The reward model assigns scores to inspection recommendations, verification recommendations, and corrective information, including: inspection recommendation score, verification recommendation score, or corrective information score.
[0046] The experts' suggestions were ranked according to their scores. Taking into account all the suggestions and the power nameplate images, errors were corrected and correct information was retained.
[0047] The reward model is used to give a confidence score to the final extraction result. If the score exceeds a preset threshold, the final result is output. If the score does not exceed the threshold, it means that even after the correction mechanism, the correct information still cannot be extracted. This may be due to image corruption, extreme shooting conditions, etc. Such cases are reported to human judge.
[0048] Optionally, step D1 includes steps D11 to D13: Step D11: Through the visual inspection expert mechanism, secondary recognition is performed based on the extracted information and the power nameplate image to obtain inspection suggestions; among which, the inspection suggestions include: suspected error information and extraction suggestions; Step D12: Through the rule-based syntax checking expert mechanism, the extracted information is checked for rule-based syntax and validation suggestions are obtained; Step D13: Through semantic inspection experts, the extracted information is semantically corrected to obtain corrected information.
[0049] In steps D11 to D13 above, the visual inspection expert refocuses on the image itself, takes the extracted information and the power nameplate image as input, performs secondary recognition, outputs information on fields that may have extraction errors, and provides their own extraction suggestions.
[0050] Rule and grammar checkers are responsible for ensuring the accuracy of the pre-extracted text and performing rule and grammar checks on the extracted text.
[0051] Semantic inspection experts are responsible for correcting any semantic errors that may exist during pre-extraction. Due to the influence of nameplate layout, there may be a phenomenon of mixed extraction of information from multiple fields. For example, if the production date appears in the factory serial number, the semantic inspection expert is responsible for correction.
[0052] For example, “Production Number”: “10070808 August 2022” should be corrected to “Production Number”: “10070808”.
[0053] The expert models described above can be built using a large model and a prompt, or they can be implemented through rule programming.
[0054] Optionally, step D12 includes steps D121 to D122: Step D121: Through the rule-based syntax checking expert mechanism, the extracted information is checked for format, value range, and character replacement to obtain the first verification suggestion; Step D122: The extracted information is verified by the rule-based grammar check expert mechanism based on empirical rules to obtain a second verification suggestion; wherein, the empirical rules are empirical rules automatically generated by the rule-based grammar check expert mechanism based on historical correction experience.
[0055] In steps D121 to D122 above, the rule syntax of the rule syntax checker mainly includes static rules and dynamic rules.
[0056] Static rules primarily perform format checks, value range checks, and character replacements. For example, voltage values have a range that won't exceed 10000KV. Common typo replacement tables include replacing "ONAM" for cooling method with the correct "ONAN". There are also some general rules, such as "the model field usually contains letters and numbers," "current values should be followed by 'A'," and "IP protection rating format is 'IP' + two digits," etc.
[0057] Dynamic rules can be automatically mined by the system. For example, if the system finds that "0A" is corrected to "OA" multiple times, it will automatically generate a regular expression replacement rule 0A -> OA and apply it to subsequent recognition.
[0058] The specific processing flow of the hybrid expert correction mechanism in step 104 above is as follows: Figure 6 The steps after enabling hybrid expert correction in the middle.
[0059] This invention eliminates the influence of background on the extraction of text from power nameplates through multi-model fusion. Simultaneously, the proposed multi-expert correction mechanism allows each model to independently handle the correction of a specific type of information extraction, significantly improving the extraction accuracy against errors caused by uneven lighting, shadows, reflections, tilt, blurring, and stamped markings. This correction mechanism also exhibits good scalability. Furthermore, the multi-expert approach mitigates the failure risk and shortcomings of a single expert, resulting in higher overall system reliability. This invention creatively uses a reward model to output confidence levels for each stage of the extraction and modification process, enhancing the interpretability of information extraction and modification, and improving the accuracy of nameplate information extraction in power scenarios.
[0060] This invention elevates the traditional preprocessing + OCR + postprocessing paradigm for nameplate information extraction to a system engineering approach that relies on multi-model, multi-rule collaborative decision-making, resulting in a wider range of applications and stronger applicability.
[0061] Compared with existing technologies, this invention integrates multiple models to handle the extraction of nameplate information from power industry scenes under various shooting conditions and scenarios, achieving high accuracy. The proposed hybrid expert correction mechanism has excellent scalability; when new rules emerge, additional expert models can be added for expansion. Furthermore, due to the presence of a reward model, the output has a degree of interpretability. This multi-expert hybrid approach mitigates the failure risk and shortcomings of a single expert, resulting in higher overall system reliability.
[0062] like Figures 3 to 5 As shown, Figure 3 This is a photograph of an electricity company nameplate. Figure 4 The information identified by the OCR model in the existing technology is largely missing due to the influence of the tilt of the shooting angle. Figure 5 The output of the power nameplate recognition based on multi-model fusion and expert correction mechanism of the present invention can be seen to contain complete extracted information in the output of the present invention.
[0063] This invention provides a power nameplate recognition method based on multi-model fusion and expert correction mechanisms. The method extracts information from power nameplate images using an optical character recognition model and a multimodal large model. The extracted information is then scored using a preset reward model. If the score exceeds a preset reliability threshold, the extracted information is determined as the output. If the score does not exceed the preset reliability threshold, a hybrid expert correction mechanism is used to correct the extracted information based on modification opinions with different weights, resulting in the final output. This hybrid expert correction mechanism, which corrects the extracted information based on modification opinions with different weights, includes: visual inspection expert mechanisms, rule-based grammar inspection expert mechanisms, and semantic inspection expert mechanisms. The system examines the extracted information, obtaining inspection suggestions, verification suggestions, and correction information. Using a reward model and a softmax function, these suggestions are scored and normalized to obtain inspection suggestion scores, verification suggestion scores, and correction information scores, respectively. Based on these scores, the extracted information is corrected using the power nameplate image, resulting in the output. Compared to existing technologies where power nameplate recognition output is unstable in different scenarios using OCR models or multimodal large models, this invention uses an optical character recognition model and a multimodal large model to extract information from power nameplate images, eliminating the influence of background on text extraction through multi-model fusion. Simultaneously, a hybrid expert correction mechanism corrects the extracted information based on modification opinions with different weights, significantly improving the accuracy of power nameplate recognition by addressing errors caused by uneven lighting, shadows, reflections, tilt, blur, and stamped seals.
[0064] The following describes the power nameplate recognition device based on multi-model fusion and expert correction mechanism provided by the present invention. The power nameplate recognition device based on multi-model fusion and expert correction mechanism described below can be referred to in correspondence with the power nameplate recognition method based on multi-model fusion and expert correction mechanism described above.
[0065] Figure 7 This is a flowchart illustrating the power nameplate identification device based on multi-model fusion and expert correction mechanism provided by the present invention, as shown below. Figure 7 As shown, the device includes the following: The information extraction module 701 is used to extract information from power nameplate images using an optical character recognition model and a multimodal large model to obtain the extracted information.
[0066] The scoring module 702 is used to score the extracted information using a preset reward model to obtain a score.
[0067] The output module 703 is used to determine the extracted information as the output result when the score exceeds a preset reliability threshold.
[0068] The correction output module 704 is used to correct the extracted information and obtain the output result by means of a hybrid expert correction mechanism, based on modification opinions with different weights, when the score does not exceed the preset reliability threshold. The correction output module 704 is specifically used for: The extracted information is examined by visual inspection expert mechanism, rule and syntax inspection expert mechanism and semantic inspection expert, respectively, to obtain inspection suggestions, verification suggestions and correction information; The inspection suggestion, the verification suggestion, and the correction information are scored and normalized using a reward model and a softmax function, respectively, to obtain the inspection suggestion score, the verification suggestion score, and the correction information score. Based on the inspection suggestion score, the verification suggestion score, and the correction information score, the extracted information is corrected based on the power nameplate image to obtain the output result.
[0069] This invention provides a power nameplate recognition device based on multi-model fusion and expert correction mechanisms. It extracts information from power nameplate images using an optical character recognition model and a multimodal large model. The extracted information is then scored using a preset reward model. If the score exceeds a preset reliability threshold, the extracted information is determined as the output result. If the score does not exceed the preset reliability threshold, a hybrid expert correction mechanism is used to correct the extracted information based on modification opinions with different weights, resulting in the output result. This hybrid expert correction mechanism, which corrects the extracted information based on modification opinions with different weights, includes: visual inspection expert mechanisms, rule-based syntax inspection expert mechanisms, and semantic inspection expert mechanisms. The system examines the extracted information, obtaining inspection suggestions, verification suggestions, and correction information. Using a reward model and a softmax function, these suggestions are scored and normalized to obtain inspection suggestion scores, verification suggestion scores, and correction information scores, respectively. Based on these scores, the extracted information is corrected using the power nameplate image, resulting in the output. Compared to existing technologies where power nameplate recognition output is unstable in different scenarios using OCR models or multimodal large models, this invention uses an optical character recognition model and a multimodal large model to extract information from power nameplate images, eliminating the influence of background on text extraction through multi-model fusion. Simultaneously, a hybrid expert correction mechanism corrects the extracted information based on modification opinions with different weights, significantly improving the accuracy of power nameplate recognition by addressing errors caused by uneven lighting, shadows, reflections, tilt, blur, and stamped seals.
[0070] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logic instructions from the memory 830 to execute a power nameplate identification method based on multi-model fusion and expert correction mechanisms.
[0071] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0072] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the power nameplate identification method based on multi-model fusion and expert correction mechanism provided by the above methods.
[0073] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the power nameplate identification method based on multi-model fusion and expert correction mechanism provided by the above methods.
[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0075] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A power nameplate recognition method based on multi-model fusion and expert correction mechanism, characterized in that, include: Information is extracted from power nameplate images using optical character recognition models and multimodal large models. The extracted information is scored using a preset reward model to obtain a score; If the score exceeds a preset reliability threshold, the extracted information is determined as the output result; If the score does not exceed the preset reliability threshold, the extracted information is corrected according to the modification opinions with different weights through a hybrid expert correction mechanism to obtain the output result. The extracted information is corrected using a hybrid expert correction mechanism based on modification suggestions with different weights, resulting in an output that includes: The extracted information is examined by visual inspection expert mechanism, rule and syntax inspection expert mechanism and semantic inspection expert, respectively, to obtain inspection suggestions, verification suggestions and correction information; The inspection suggestion, the verification suggestion, and the correction information are scored and normalized using a reward model and a softmax function, respectively, to obtain the inspection suggestion score, the verification suggestion score, and the correction information score. Based on the inspection suggestion score, the verification suggestion score, and the correction information score, the extracted information is corrected based on the power nameplate image to obtain the output result.
2. The power nameplate recognition method based on multi-model fusion and expert correction mechanism according to claim 1, characterized in that, The process involves extracting information from power nameplate images using an optical character recognition model and a multimodal large model, yielding extracted information including: The optical character recognition model is used to extract the text from the power nameplate image to obtain the output text image. By using a multimodal large model, text information is extracted from the output text image to obtain the extracted information.
3. The power nameplate recognition method based on multi-model fusion and expert correction mechanism according to claim 1, characterized in that, The step of scoring the extracted information using a preset reward model to obtain a score includes: The output header of the multimodal model is modified and transformed to obtain the regression model; The extracted information is scored using the regression model to obtain a score.
4. The power nameplate recognition method based on multi-model fusion and expert correction mechanism according to claim 2, characterized in that, The step of scoring the extracted information using a preset reward model to obtain a score includes: A score is obtained by scoring based on the extracted information and the output text image using a multimodal reward model.
5. The power nameplate recognition method based on multi-model fusion and expert correction mechanism according to claim 1, characterized in that, The extracted information is examined through visual inspection expert mechanisms, rule-based grammar inspection expert mechanisms, and semantic inspection experts to obtain inspection suggestions, verification suggestions, and correction information, including: The visual inspection expert mechanism performs secondary identification based on the extracted information and the power nameplate image to obtain inspection suggestions; the inspection suggestions include: suspected error information and extraction suggestions. The extracted information is validated using a rule-based syntax checking expert mechanism to obtain validation suggestions. The extracted information is semantically corrected by a semantic inspection expert to obtain corrected information.
6. The power nameplate identification method based on multi-model fusion and expert correction mechanism according to claim 5, characterized in that, The step of using a rule-based syntax checking expert mechanism to perform rule-based syntax verification on the extracted information and obtain verification suggestions includes: The extracted information is subjected to format checks, value range checks, and character replacements through a rule-based syntax checking expert mechanism to obtain a first verification suggestion. The extracted information is verified by a rule-based syntax checking expert mechanism based on empirical rules to obtain a second verification suggestion; wherein the empirical rules are empirical rules automatically generated by the rule-based syntax checking expert mechanism based on historical correction experience.
7. A power nameplate identification device based on multi-model fusion and expert correction mechanism, characterized in that, include: The information extraction module is used to extract information from power nameplate images using optical character recognition models and multimodal large models to obtain the extracted information; The scoring module is used to score the extracted information using a preset reward model to obtain a score; The output module is used to determine the extracted information as the output result when the score exceeds a preset reliability threshold. The correction output module is used to correct the extracted information and obtain the output result by means of a hybrid expert correction mechanism, based on modification opinions with different weights, when the score does not exceed a preset reliability threshold. The correction output module is specifically used for: The extracted information is examined by visual inspection expert mechanism, rule and syntax inspection expert mechanism and semantic inspection expert, respectively, to obtain inspection suggestions, verification suggestions and correction information; The inspection suggestion, the verification suggestion, and the correction information are scored and normalized using a reward model and a softmax function, respectively, to obtain the inspection suggestion score, the verification suggestion score, and the correction information score. Based on the inspection suggestion score, the verification suggestion score, and the correction information score, the extracted information is corrected based on the power nameplate image to obtain the output result.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the power nameplate recognition method based on multi-model fusion and expert correction mechanism as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the power nameplate recognition method based on multi-model fusion and expert correction mechanism as described in any one of claims 1 to 6.