New energy station information collection method and system based on multi-modal data identification
By adopting multimodal data recognition technology in the collection work of new energy stations, the modal features of text and image data are extracted and fused, the problem of low heterogeneous data processing efficiency is solved, and efficient information fusion and rapid modeling of new energy stations are achieved.
Patent Information
- Application Number
- CN202411963306.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-18
AI Technical Summary
In the new energy station collection work, facing massive heterogeneous data, it is difficult to fully obtain the information required for modeling in a single data mode, resulting in cumbersome and complex, high repetition and waste of labor costs.
Using a method based on multimodal data recognition, the modal features of text and image data are extracted and deeply fusion is carried out to achieve cross-modal information fusion through natural language processing and image recognition technology.
It has greatly improved the efficiency of collecting funds in new energy stations, reduced labor and time costs, and helped the transformation of new power systems and the realization of the "dual carbon" goal.
Smart Images
Figure CN119851293A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a method and system for collecting funds at new energy stations based on multimodal data recognition. Background Art
[0002] As the scale of new energy access to the power grid continues to increase, the importance of transient modeling of new energy stations has become increasingly prominent. The collection of funds for new energy stations is a prerequisite for transient modeling of new energy stations. This process mainly involves collecting important information about the station, including transformer parameters, unit information, station topology, collector information, etc. However, there are a large number of heterogeneous data in these collection materials, such as pictures and videos taken by on-site personnel, test reports of various forms issued by various test units, various forms stored by operation and maintenance units, and station information recorded in other forms. These data modes are different but interrelated.
[0003] It is difficult to fully obtain the information required for new energy station modeling by extracting features from a single data modality. How to extract effective cross-modal features from a large amount of heterogeneous data in the collection materials and perform deep fusion is a key issue in improving the efficiency of new energy station collection. As a preparatory work for transient modeling of new energy stations, some of the work content of station collection is cumbersome, complex, highly repetitive, and wastes manpower costs. It is necessary to further enhance the digitalization level of process data processing.
[0004] At present, the collection of funds for new energy stations can only be done by manual identification, summarization and manual filling of modeling forms. In the context of comprehensively building a new power system with new energy as the main body, the number of stations to be modeled is growing. New energy modeling requires practitioners to have complex knowledge in the fields of electrical, computer and automatic control. It is very costly to train professional modelers, and manual collection of funds for new energy stations consumes a lot of valuable time of professional modelers. Not only does it seriously waste the human cost of new energy modeling test units, but the investment and return are not proportional.
[0005] Faced with a massive amount of original funding materials for new energy stations, manual processing is almost inevitable to make mistakes, and the only way to reduce the error rate and improve the efficiency of new energy station funding is to add manpower for review. However, the manpower and time costs generated by manual funding work have placed a huge burden on various new energy modeling and testing units. These costs will eventually be passed on to the modeling costs of new energy station owners, which will not only prolong the commissioning time of newly built stations, but also seriously hinder the construction of new power systems and the realization of "dual carbon" goals. Summary of the invention
[0006] The purpose of this application is to address the problem that when faced with massive amounts of original funding materials for new energy stations, the manpower and time costs generated by manual funding work impose a huge cost burden on various new energy modeling and testing units, and not only prolong the commissioning time of newly built stations, but also seriously hinder the construction of new power systems and the realization of "dual carbon" goals. This application provides a new energy station funding method and system based on multimodal data recognition.
[0007] In order to achieve the above objectives, this application adopts the following technical solutions:
[0008] On the one hand, the present application provides a new energy station collection method based on multimodal data recognition, comprising the following steps:
[0009] S1, obtaining multimodal data of a new energy station and confirming the type of the multimodal data;
[0010] S2, performing text preprocessing on the text data in the multimodal data to obtain text preprocessed data in a digital storage format with a uniform length; performing preprocessing on the image data in the multimodal data to obtain image preprocessed data with a uniform size;
[0011] S3, uses natural language processing (NLP) technology to train text preprocessing data, and extracts modal features from text preprocessing data based on recurrent neural networks; uses optical character recognition (OCR) technology to recognize image preprocessing data, and extracts modal features from image preprocessing data based on recurrent neural networks;
[0012] S4, in the new energy station database F D The data matching of the extracted multimodal features F is completed, and the new energy station database F is matched based on the matching results. D Update; wherein the multimodal feature F includes the modal features of the extracted text preprocessing data and / or the modal features of the extracted image preprocessing data.
[0013] Furthermore, in step S1, when confirming the type of the multimodal data, classification confirmation is performed according to the format of the collected multimodal data.
[0014] Furthermore, in step S2, the text preprocessing includes word segmentation processing, data length standardization processing and text denoising processing on the text data;
[0015] Among them, data length standardization is used to convert digital characters in text data into a digital storage format of uniform length; text denoising is used to remove noise in text data; and word segmentation processing of text data precedes data length standardization processing of text data.
[0016] Furthermore, in step S2, the following contents are included when extracting modal features from the text preprocessing data:
[0017] The Word2Vec word embedding model is used to convert the text preprocessing data into a vector of fixed dimension, and vectorized data for representing the semantic information of the text preprocessing data is obtained;
[0018] Build a recurrent neural network model for feature extraction, use the vectorized data as a training data set, and train the recurrent neural network model;
[0019] Based on the text preprocessing data, the modal features in the text preprocessing data are extracted through the trained recurrent neural network model.
[0020] Furthermore, in step S2, the image data is preprocessed including image segmentation, image sharpening and image denoising using a bilateral filtering method; wherein,
[0021] In the image segmentation processing, the Sobel operator is first used to detect the horizontal and vertical edges in the image data; then the maximum edge detected by the Sobel operator is cropped and segmented to extract the content in the image data into image sub-data of uniform size.
[0022] Furthermore, in step S3, when extracting the modal features of the image preprocessing data, the following contents are included:
[0023] Determine an image data set for model training, recognize the image data set by optical character recognition technology (OCR), and obtain a training set for model training;
[0024] Construct a recurrent neural network model for feature extraction, and train the recurrent neural network model using the training set;
[0025] Based on the image preprocessing data, the modal features in the image preprocessing data are extracted through the trained recurrent neural network model.
[0026] Furthermore, in step S4, a data matching algorithm is used to perform data matching, and the data matching result Score is determined by the following formula:
[0027] Score = Similarity(F,F D )
[0028] in, i is the component of the multidimensional vector along a certain direction, n is the number of dimensions, F i 、F Di The vectors F and F are D The component along dimension i.
[0029] Further, in step S4, when the matching result is greater than a preset threshold, the data matching is determined to be successful, and the multimodal feature F is completed in the new energy station database F D wherein the data fusion comprises:
[0030] In the new energy station database F D When some information is missing in , the missing information is fused and stored based on the extracted multimodal features F;
[0031] In the new energy station database F D When there is no extracted multimodal feature F in the database, the multimodal feature F is updated to the new energy station database F D In order to complete the new energy station database F D Updates.
[0032] Furthermore, when the data matching is unsuccessful, when it is determined that the extracted multimodal feature F is inconsistent with the corresponding original information through manual verification, the new energy station database F D Enter the corresponding original information.
[0033] On the other hand, the present application also provides a system for the above-mentioned new energy station collection method based on multimodal data recognition, including a multimodal data type classification confirmation module, a multimodal data preprocessing module, a multimodal data model feature extraction module, and a data matching and updating module; wherein,
[0034] The multimodal data type classification confirmation module is used to obtain the multimodal data of the new energy station and confirm the type of the multimodal data;
[0035] The multimodal data preprocessing module is used to perform text preprocessing on the text data in the multimodal data and obtain the text preprocessed data in a digital storage format of uniform length; and the multimodal data preprocessing module is used to perform preprocessing on the image data in the multimodal data and obtain the image preprocessed data of uniform size;
[0036] The multimodal data model feature extraction module is used to train the text preprocessing data using natural language processing (NLP) technology, and extract the modal features in the text preprocessing data based on a recurrent neural network; and the multimodal data model feature extraction module is used to recognize the image preprocessing data using optical character recognition (OCR) technology, and extract the modal features of the image preprocessing data based on a recurrent neural network;
[0037] The data matching and updating module is used to update the new energy station database F DThe data matching of the extracted multimodal features F is completed, and the new energy station database F is matched based on the matching results. D Update is performed; wherein the multimodal feature F includes the modal features of the extracted text preprocessing data and the modal features of the extracted image preprocessing data.
[0038] Beneficial effects of this application
[0039] The present application provides a new energy station funding collection method based on multimodal data recognition, which processes multimodal new energy station funding collection materials based on natural language processing and image recognition technology, can realize automatic extraction of key information and cross-modal information fusion, can greatly improve the efficiency of new energy station funding collection work, and assist in the transformation of a new power system with new energy as the main body. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0041] Figure 1 This is a schematic diagram of the process of Example 1.
[0042] Figure 2 This is a schematic diagram of the results of collecting wind turbine parameters using the method provided in Example 1. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application. At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0044] Example 1
[0045] In view of the problem that when faced with massive original funding materials for new energy stations, the manpower and time costs generated by manual funding work impose a huge cost burden on various new energy modeling and testing units, and not only prolong the commissioning time of newly built stations, but also seriously hinder the construction of new power systems and the realization of "dual carbon" goals, this embodiment 1 provides a new energy station funding collection method based on multimodal data recognition, which processes multimodal new energy station funding materials based on natural language processing and image recognition technology, realizes automatic extraction of key information and cross-modal information fusion, greatly improves the efficiency of new energy station funding collection work, and can help the transformation of new power systems with new energy as the main body.
[0046] like Figure 1 and Figure 2 As shown, this embodiment 1 provides a new energy station fee collection method based on multimodal data recognition, including the following steps:
[0047] S1, obtaining multimodal data of new energy stations and confirming the type of the multimodal data.
[0048] The collection of funds for new energy stations is the preparatory work for transient modeling of new energy stations, and mainly collects important information of new energy stations. Specifically, when obtaining multimodal data of new energy stations, the multimodal data obtained in step S1 may include unit parameters, transformer parameters, station information, equipment nameplate information, document scans, reports, etc., text or image data information, etc.; wherein, the multimodal data format collected in step S1 may involve bmp, jpg, png, tif, txt, doc, csv, xlsx, pdf and other formats.
[0049] When confirming the type of the multimodal data, step S1 can classify and confirm according to the format of the collected multimodal data. For example, data in text format such as txt, doc, csv, xlsx or pdf can be classified as text data, and data in image format such as bmp, jpg, png or tif can be classified as image data.
[0050] S2, performing text preprocessing on the text data in the multimodal data to obtain text preprocessed data in a digital storage format with a uniform length.
[0051] Specifically, in step S2, the text preprocessing may include performing data length standardization on the text data, and the data length standardization is used to convert the digital characters in the text data into a digital storage format of uniform length, and obtain the text preprocessing data, so as to facilitate the subsequent recognition processing of the text preprocessing data. In this embodiment 1, the data length standardization processing can adopt the existing length standardization processing method.
[0052] In order to ensure the effectiveness and reliability of subsequent processing of the text preprocessing data, the text preprocessing in step S2 can also include text denoising processing, through which irrelevant information in the text data is cleaned, thereby ensuring the effectiveness and reliability of subsequent data processing.
[0053] S3 uses natural language processing (NLP) technology to train text preprocessing data and extracts modal features from text preprocessing data based on a recurrent neural network.
[0054] In this embodiment 1, using natural language processing (NLP) technology to train text preprocessing data may include the following:
[0055] The Word2Vec word embedding model is used to convert the text preprocessing data into a vector of fixed dimension and obtain vectorized data. The vectorized data can represent the semantic information of the text preprocessing data.
[0056] Build a recurrent neural network model for feature extraction, use the vectorized data as a training dataset, and train the recurrent neural network model.
[0057] Based on the text preprocessing data, the modal features in the text preprocessing data are extracted through the trained recurrent neural network model.
[0058] Since the modal features in the text data are determined by means of the Word2Vec word embedding model, in step S2, the text preprocessing of the text data may also include word segmentation of the text data, so as to improve the accuracy of the Word2Vec word embedding model training; wherein, the Chinese word segmentation may be implemented using a variety of technologies including the Unigram word segmentation algorithm.
[0059] Further preferably, in step S2, the word segmentation process of the text data should be performed before the data length normalization process of the text data, so that the Word2Vec word embedding model can map the words with data length normalization process to a continuous vector space of fixed size, so that the words in the vector space can capture the semantic and grammatical relationship between words, so as to be able to predict the words related to the given word in its context based on the given word, which can extract the corresponding relationship of the required information from the preprocessed data. It should be understood that the given word is the vectorized data converted by the text preprocessing data using the Word2Vec word embedding model.
[0060] In step S1 of this embodiment 1, the multimodal data collected includes text data and image formats, in order to further ensure the accuracy of the materials for collecting funds from new energy stations and the efficiency of the collection work. In step S2 of this embodiment 1, the image data in the multimodal data is preprocessed to obtain image preprocessed data of uniform size; and in step S3, the image preprocessed data is recognized by using optical character recognition technology OCR, and the modal features of the image preprocessed data are extracted based on a recurrent neural network.
[0061] Specifically, when preprocessing image data, the Sobel operator can be used to detect horizontal and vertical edges in the image data, and the maximum result obtained is the row and column edges of the table; then the maximum edge detected by the Sobel operator is cropped and segmented, and the image is divided into rows, columns, and cells to extract the content in the image data into structured data of uniform size for further analysis.
[0062] More preferably, when preprocessing the image data, it also includes denoising the image data. In the denoising process, a bilateral filtering method can be used to remove irregular noise in the image while retaining the edge clarity of the characters. The bilateral filtering method is a nonlinear filtering method that not only considers the spatial distance of pixels, but also the similarity of pixel values, and can smooth the image while retaining the edge.
[0063] In order to increase the high-frequency part of the image data and highlight the details in the image data so that the image data becomes clearer, this embodiment 1 may use sharpening technology in the image data when pre-processing the image data, and through the sharpening processing of the image, the edges of the image characters can be enhanced to improve the accuracy of subsequent OCR recognition, which is especially helpful for blurred character images.
[0064] In step S3, the image preprocessing data is recognized by using optical character recognition technology OCR, and the modal features of the image preprocessing data are extracted based on a recurrent neural network, which may specifically include the following contents:
[0065] An image data set for model training is determined, the image data set is recognized by optical character recognition technology (OCR), and a training set for model training is obtained.
[0066] Build a recurrent neural network model for feature extraction, and use the training set to train and debug the recurrent neural network model to achieve text cleaning, vector representation, semantic analysis, etc.
[0067] Based on the image preprocessing data and through the trained recurrent neural network model, the corresponding modal features in the image preprocessing data are extracted.
[0068] Preferably, when determining the image dataset for model training, the image dataset used can be an open source Chinese image dataset, which can cover multiple scenes such as documents and handwriting, among which text recognition can include 11,000 images.
[0069] Specifically, the GPU inference time of the recurrent neural network model can be completed based on the NVIDIA Tesla T4 machine. The recurrent neural network RNN is used to process sequence data, and in the optical character recognition technology OCR, the character sequence sometimes shows a certain temporal sequence in the image. By using the recurrent neural network RNN, the dependency between characters can be effectively captured. Since the spatial arrangement of characters is not fixed, and the recurrent neural network RNN is used to perform sequence modeling loss function on the characters in the image data, the temporal information of the characters can be effectively processed to avoid the problem of mismatch between input and label lengths; among them, for the recurrent neural network model used for feature extraction, the loss function used is preferably CTC loss Connectionist Temporal Classification. It should be understood that the structure of the recurrent neural network model in this embodiment 1 can refer to the existing recurrent neural network model structure.
[0070] S4, in the new energy station database F D The data matching of the extracted multimodal features F is completed, and the new energy station database F is matched based on the matching results. D Update is performed; wherein the multimodal feature F includes the modal features of the extracted text preprocessing data and the modal features of the extracted image preprocessing data.
[0071] In this embodiment 1, step S4 can use a fast fund collection matching algorithm to perform data matching to ensure the accuracy of fund collection. The data matching result Score is determined by the following formula:
[0072] Score = Similarity(F,F D )
[0073] Among them, Similarity uses cosine similarity and is determined by the following formula:
[0074]
[0075] Among them, i is the component of the multidimensional vector along a certain direction, n is the number of dimensions, and F i 、F Di The vectors F and F are D The component along dimension i.
[0076] It should be understood that the new energy station database F DIt will store information in multiple dimensions, including station information, equipment nameplate information, unit parameters, transformer parameters, etc. D When performing data matching, matching should be performed in various dimensions.
[0077] Furthermore, in step S4, this embodiment 1 can perform a threshold determination of the matching result Score, and when the data matching is successful, complete the multimodal feature F in the new energy station database F D For example, the threshold can be set to 0.9, that is, when Score>0.9, the data is determined to be matched successfully.
[0078] Among them, the multimodal feature F is in the new energy station database F D Data fusion is performed in the following ways:
[0079] In the new energy station database F D When some information is missing in , the missing information is fused and stored based on the extracted multimodal features F.
[0080] In the new energy station database F D When there is no extracted multimodal feature F in the database, the multimodal feature F is updated to the new energy station database F D In order to complete the new energy station database F D Updates.
[0081] When the data matching fails, the data accuracy is confirmed by manual verification and then stored and updated. In the manual verification, the extracted multimodal features F are verified to be consistent with the original information. If they are inconsistent, the original information is manually entered.
[0082] In the new energy station database F D After the update is completed, it will be convenient for subsequent modeling and report generation.
[0083] This embodiment 1 provides a new energy station funding collection method based on multimodal data recognition, which processes multimodal new energy station funding collection materials based on natural language processing and image recognition technology, can realize automatic extraction of key information and cross-modal information fusion, can greatly improve the efficiency of new energy station funding collection, and assist in the transformation of a new power system with new energy as the main body.
[0084] Example 2
[0085] In view of the problem that when faced with massive amounts of original funding materials for new energy stations, the manpower and time costs generated by manual funding work will impose a huge cost burden on various new energy modeling and testing units, which will not only prolong the commissioning time of newly built stations, but also seriously hinder the construction of new power systems and the realization of "dual carbon" goals, this embodiment 2 provides a new energy station funding collection system based on multimodal data recognition, which processes multimodal new energy station funding materials based on natural language processing and image recognition technology, realizes automatic extraction of key information and cross-modal information fusion, greatly improves the efficiency of new energy station funding collection work, and can help the transformation of new power systems with new energy as the main body.
[0086] This embodiment 2 provides a system for the new energy station collection method based on multimodal data recognition in embodiment 1. Specifically, the system provided in this embodiment 2 includes a multimodal data type classification confirmation module, a multimodal data preprocessing module, a multimodal data model feature extraction module, and a data matching and updating module; wherein,
[0087] A multimodal data type classification confirmation module is used to obtain multimodal data of new energy stations and confirm the type of the multimodal data;
[0088] The multimodal data preprocessing module is used to perform text preprocessing on the text data in the multimodal data and obtain the text preprocessing data in a digital storage format of uniform length; and the multimodal data preprocessing module is used to perform preprocessing on the image data in the multimodal data and obtain the image preprocessing data of uniform size;
[0089] The multimodal data model feature extraction module is used to train the text preprocessing data using natural language processing (NLP) technology, and extract the modal features in the text preprocessing data based on a recurrent neural network; and the multimodal data model feature extraction module is used to recognize the image preprocessing data using optical character recognition (OCR) technology, and extract the modal features of the image preprocessing data based on a recurrent neural network;
[0090] The data matching and updating module is used to complete the data matching of the extracted multimodal features in the new energy station database, and update the new energy station database based on the matching results; wherein the multimodal features include the modal features of the extracted text preprocessing data and the modal features of the extracted image preprocessing data.
[0091] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between it and other embodiments.
[0092] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the specific order or sequential order shown in the process depicted in the drawings is not necessarily required to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0093] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A new energy station collection method based on multimodal data recognition, characterized by: The following steps are included: S1, obtaining multimodal data of a new energy station and confirming the type of the multimodal data; S2, performing text preprocessing on the text data in the multimodal data to obtain text preprocessed data in a digital storage format with a uniform length; performing preprocessing on the image data in the multimodal data to obtain image preprocessed data with a uniform size; S3, uses natural language processing (NLP) technology to train text preprocessing data, and extracts modal features from text preprocessing data based on recurrent neural networks; Use optical character recognition technology (OCR) to identify image preprocessing data, and extract modal features of image preprocessing data based on recurrent neural network; S4, in the new energy station database F D The data matching of the extracted multimodal features F is completed, and the new energy station database F is matched based on the matching results. D Update; wherein the multimodal feature F includes the modal features of the extracted text preprocessing data and / or the modal features of the extracted image preprocessing data.
2. The new energy station funding method based on multimodal data recognition according to claim 1 is characterized in that: In step S1 , when confirming the type of the multimodal data, classification confirmation is performed according to the format of the collected multimodal data.
3. The new energy station funding method based on multimodal data recognition according to claim 1 is characterized in that: In step S2, text preprocessing includes word segmentation processing, data length standardization processing and text denoising processing on text data; Among them, data length standardization is used to convert digital characters in text data into a digital storage format of uniform length; text denoising is used to remove noise in text data; and word segmentation processing of text data precedes data length standardization processing of text data.
4. The new energy station funding method based on multimodal data recognition according to claim 3 is characterized by: In step S2, the following contents are included when extracting modal features from the text preprocessing data: The Word2Vec word embedding model is used to convert the text preprocessing data into a vector of fixed dimension, and vectorized data for representing the semantic information of the text preprocessing data is obtained; Build a recurrent neural network model for feature extraction, use the vectorized data as a training data set, and train the recurrent neural network model; Based on the text preprocessing data, the modal features in the text preprocessing data are extracted through the trained recurrent neural network model.
5. The new energy station funding method based on multimodal data recognition according to claim 1 is characterized in that: In step S2, the image data is preprocessed including image segmentation, image sharpening and image denoising using a bilateral filtering method; wherein, In the image segmentation processing, the Sobel operator is first used to detect the horizontal and vertical edges in the image data; then the maximum edge detected by the Sobel operator is cropped and segmented to extract the content in the image data into image sub-data of uniform size.
6. The new energy station funding method based on multimodal data recognition according to claim 5 is characterized by: In step S3, when extracting the modal features of the image preprocessing data, the following contents are included: Determine an image data set for model training, recognize the image data set by optical character recognition technology (OCR), and obtain a training set for model training; Construct a recurrent neural network model for feature extraction, and train the recurrent neural network model using the training set; Based on the image preprocessing data, the modal features in the image preprocessing data are extracted through the trained recurrent neural network model.
7. The new energy station funding method based on multimodal data recognition according to any one of claims 1 to 6, characterized in that: In step S4, data matching is performed using a collection matching algorithm, and the data matching result Score is determined by the following formula: Score=Similarity(F,F D ) in, i is the component of the multidimensional vector along a certain direction, n is the number of dimensions, F i 、F Di The vectors F and F are D The component along dimension i.
8. The new energy station funding method based on multimodal data recognition according to claim 7 is characterized in that: In step S4, when the matching result is greater than a preset threshold, the data matching is determined to be successful, and the multimodal feature F is completed in the new energy station database F D wherein the data fusion comprises: In the new energy station database F D When some information is missing in , the missing information is fused and stored based on the extracted multimodal features F; In the new energy station database F D When there is no extracted multimodal feature F in the database, the multimodal feature F is updated to the new energy station database F D In order to complete the new energy station database F D Updates.
9. The new energy station funding method based on multimodal data recognition according to claim 8 is characterized in that: When data matching fails, when it is determined that the extracted multimodal feature F is inconsistent with the corresponding original information through manual verification, the new energy station database F D Enter the corresponding original information.
10. A system for a new energy station funding method based on multimodal data recognition according to any one of claims 1 to 9, characterized in that: It includes a multimodal data type classification confirmation module, a multimodal data preprocessing module, a multimodal data model feature extraction module, and a data matching and updating module; among them, The multimodal data type classification confirmation module is used to obtain the multimodal data of the new energy station and confirm the type of the multimodal data; The multimodal data preprocessing module is used to perform text preprocessing on the text data in the multimodal data and obtain the text preprocessed data in a digital storage format of uniform length; and the multimodal data preprocessing module is used to perform preprocessing on the image data in the multimodal data and obtain the image preprocessed data of uniform size; The multimodal data model feature extraction module is used to train the text preprocessing data using natural language processing (NLP) technology, and extract the modal features in the text preprocessing data based on a recurrent neural network; and the multimodal data model feature extraction module is used to recognize the image preprocessing data using optical character recognition (OCR) technology, and extract the modal features of the image preprocessing data based on a recurrent neural network; The data matching and updating module is used to update the new energy station database F D The data matching of the extracted multimodal features F is completed, and the new energy station database F is matched based on the matching results. D Update is performed; wherein the multimodal feature F includes the modal features of the extracted text preprocessing data and the modal features of the extracted image preprocessing data.