Multi-modal data service calling method and device, storage medium, electronic equipment and product

Abstract

The invention discloses a service calling method and device for multi-modal data, a storage medium, electronic equipment and a product. The method comprises the steps that to-be-processed data are obtained, and the to-be-processed data comprise data items of at least one data type; identifying the data type of each data item in the to-be-processed data; determining a processing strategy of the data item based on the data type of each data item, and performing data extraction on each data item based on the processing strategy of each data item to obtain data content corresponding to each data item; converting the data content corresponding to each data item into a preset data format, and integrating the data in the preset data format corresponding to each data item into target data; and in response to the service calling operation, calling the target service interface, and processing the target data based on the target service interface to obtain a data processing result. According to the invention, the service calling of the multi-modal data can be realized, and the flexibility and universality of the service are improved.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a service calling method, device, storage medium, electronic device and product for multimodal data. Background Art

[0002] With the development of Internet technology, data sources are becoming more and more diverse, including structured data, unstructured data, and semi-structured data. The different forms of these data bring challenges to data processing, especially in the process of service interface calls.

[0003] Traditional service interface calls often target single-modal data and lack support for multimodal data, thus limiting the flexibility and versatility of the service. Summary of the invention

[0004] The present invention provides a method, device, storage medium, electronic device and product for calling a service of multimodal data to realize calling a service of multimodal data.

[0005] According to one aspect of the present invention, a method for invoking a service of multimodal data is provided, comprising:

[0006] Acquire data to be processed, wherein the data to be processed includes data items of at least one data type;

[0007] Identify the data type of each data item in the data to be processed, wherein the data type includes one or more of structured data, unstructured data, and semi-structured data;

[0008] Determine a processing strategy for each data item based on the data type of each data item, and extract data from each data item based on the processing strategy for each data item to obtain data content corresponding to each data item;

[0009] Converting the data content corresponding to each of the data items into a preset data format, and integrating the data in the preset data format corresponding to each of the data items into target data;

[0010] In response to the service calling operation, the target service interface is called, and the target data is processed based on the target service interface to obtain a data processing result.

[0011] Optionally, the identifying the data type of each data item in the data to be processed includes:

[0012] The file extension of each of the data items in the data to be processed is obtained, and the data type of the data item is determined based on a preset correspondence between the extension and the data type and the file extension of the data item.

[0013] Optionally, identifying the data type of each data item in the data to be processed includes: obtaining source data corresponding to each of the data items, and determining the data type of each of the data items based on a file descriptor or header information in the source data.

[0014] Optionally, obtaining the data to be processed includes: displaying a data processing page, the data processing page including data collection areas corresponding to each data type; collecting data items through each of the data collection areas, and setting the data type identifier of the data item based on the data type corresponding to the data collection area.

[0015] Optionally, identifying the data type of each data item in the data to be processed includes: determining the data type of the data item according to a data type identifier carried by the data item.

[0016] Optionally, descriptive information of the data type corresponding to the data collection area is displayed in each of the data collection areas; or, the method further includes: in response to a trigger operation in any of the data collection areas, descriptive information of the data type corresponding to the data collection area is displayed.

[0017] Optionally, after collecting data items respectively through each of the data collection areas, the method further includes: verifying the data type of the data item collected by the data collection area; and generating a prompt message when the data type of the data item collected by the data collection area is inconsistent with the data type corresponding to the data collection area.

[0018] Optionally, at least one data item in the data to be processed comes from at least one data source; obtaining the data to be processed includes: sending data requests to each of the data sources respectively through a connection interface pre-established with at least one data source, and obtaining data items from each of the data sources.

[0019] Optionally, the processing strategy for the structured data includes: extracting data content from the structured data; the processing strategy for the unstructured data includes: extracting data from the unstructured data based on a pre-trained data processing model to obtain extracted data content; the processing strategy for the semi-structured data includes: parsing the semi-structured data, extracting keywords from the parsed data, and obtaining extracted data content.

[0020] Optionally, after obtaining the data content corresponding to each of the data items, the method further includes: performing standardization processing on the data content.

[0021] Optionally, after integrating the data in the preset data format corresponding to each of the data items into target data, it also includes: converting the target data into target vector data, and matching the target vector data in a preset cache library; wherein the preset cache library includes historical vector data and processing results corresponding to the historical vector data; when the matching degree between the target vector data and the historical vector data meets the matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data.

[0022] Optionally, the data to be processed is financial business data, and the target service interface is a financial business service interface.

[0023] According to one aspect of the present invention, there is provided a device for invoking a service of multimodal data, comprising:

[0024] A to-be-processed data acquisition module, used to acquire the to-be-processed data, wherein the to-be-processed data includes data items of at least one data type;

[0025] A data type identification module, used to identify the data type of each data item in the data to be processed, wherein the data type includes one or more of structured data, unstructured data and semi-structured data;

[0026] A data content extraction module, used to determine a processing strategy for each data item based on the data type of each data item, and to extract data from each data item based on the processing strategy of each data item to obtain data content corresponding to each data item;

[0027] A data integration module, used to convert the data content corresponding to each of the data items into a preset data format, and integrate the data in the preset data format corresponding to each of the data items into target data;

[0028] The service calling module is used to call the target service interface in response to the service calling operation, process the target data based on the target service interface, and obtain the data processing result.

[0029] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0030] at least one processor; and

[0031] a memory communicatively connected to the at least one processor; wherein,

[0032] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the service calling method for multimodal data described in any embodiment of the present invention.

[0033] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the service calling method for multimodal data described in any embodiment of the present invention when executed.

[0034] According to another aspect of the present invention, a computer program product is provided, characterized in that the computer program product includes a computer program, and when the computer program is executed by a processor, the service calling method for multimodal data described in any embodiment of the present invention is implemented.

[0035] The technical solution of the embodiment of the present invention is to obtain data to be processed, wherein the data to be processed includes data items of at least one data type; identify the data type of each data item in the data to be processed, and the data type includes one or more of structured data, unstructured data and semi-structured data; determine the processing strategy of the data item based on the data type of each data item, extract data from each data item based on the processing strategy of each data item, and obtain the data content corresponding to each data item; convert the data content corresponding to each data item into a preset data format, and integrate the data of the preset data format corresponding to each data item into target data; in response to a service call operation, call the target service interface, and process the target data based on the target service interface to obtain a data processing result. By identifying the data type, extracting the data content and converting the data format of the data to be processed in multiple modes, the multi-modal data to be processed is integrated into data in a unified format, so that the corresponding service interface is called to process the data in the unified format, and the service call of the multi-modal data is realized, thereby improving the flexibility and versatility of the service.

[0036] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 is a flow chart of a method for calling a service of multimodal data provided in the first embodiment of the present invention;

[0039] Figure 2is a flow chart of a method for calling a service of multimodal data provided by Embodiment 2 of the present invention;

[0040] Figure 3 This is a flow chart of a method for calling a service of multimodal data in a financial scenario provided by Embodiment 3 of the present invention;

[0041] Figure 4 This is a flowchart of a method for calling a service of multimodal data in a medical scenario provided by a fourth embodiment of the present invention;

[0042] Figure 5 It is a structural diagram of a multimodal data service invocation device provided in Embodiment 5 of the present invention;

[0043] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0045] It should be noted that the terms "first characteristic data", "second characteristic data", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0046] The acquisition, storage and / or processing of data in the technical solutions involved in this application shall comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solutions of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0047] Embodiment 1

[0048] Figure 1 This is a flow chart of a method for calling a service of multimodal data provided in the first embodiment of the present invention. This embodiment is applicable to the case of calling a service of multimodal data. The method can be executed by a device for calling a service of multimodal data. The device for calling a service of multimodal data can be implemented in the form of hardware and / or software. The device for calling a service of multimodal data can be configured in the electronic device provided in the embodiment of the present invention. Figure 1 As shown, the method includes:

[0049] S110. Acquire data to be processed, wherein the data to be processed includes data items of at least one data type.

[0050] Among them, the data to be processed refers to multimodal data to be processed using the target service interface. The data to be processed corresponds to the service interface. Specifically, if the data to be processed is financial business data, the target service interface called is the financial business service interface; illustratively, financial business data includes bank statements, income certificates, credit reports, transaction data, etc. If the data to be processed is medical business data, the target service interface called is the medical business service interface. Exemplarily, medical business data includes medical history records, examination results, diagnostic reports, medical images, etc. It can be understood that the data to be processed includes data items of different data types, and data items of different data types correspond to different modalities.

[0051] Based on the above embodiment, optionally, at least one data item in the data to be processed comes from at least one data source; obtaining the data to be processed includes: sending data requests to each of the data sources respectively through a connection interface pre-established with at least one data source, and obtaining data items from each of the data sources.

[0052] The data source is used to provide data to be processed, and specifically, the data source includes but is not limited to a database, a file, an API interface, a real-time data stream, etc. In this embodiment, for each data source, a connection interface is pre-established to communicate with the data source and obtain data, and a data request is sent to the data source through the pre-established connection interface, and a data item returned by the data source according to the data request is received.

[0053] S120: Identify the data type of each data item in the data to be processed, where the data type includes one or more of structured data, unstructured data, and semi-structured data.

[0054] Among them, structured data refers to data with a fixed format and pattern, such as table data in a database, which is easy to process through tools such as relational database query language (SQL). Unstructured data refers to data with unfixed format and content; for example: text files, images, videos, etc., which require specific parsing and processing technologies. Semi-structured data refers to data with a certain structure but not as strict as structured data, such as: JSON, XML, etc., which are usually used for Web services and data transmission.

[0055] Based on the above embodiment, optionally, the identifying the data type of each data item in the data to be processed includes: obtaining the file extension of each data item in the data to be processed, and determining the data type of the data item based on a preset correspondence between the extension and the data type and the file extension of the data item.

[0056] The file extension is usually located at the end of the file name and is used to indicate the type or format of the file. For example, in the file name "example.txt", the file extension is ".txt", indicating that the file is a text file. In this embodiment, the correspondence between the extension and the data type can be preset and stored in a configuration file, a database or an external service. For each data item in the data to be processed, the file extension of the data item is obtained, and based on the obtained file extension of the data item, a search and match is performed in the preset correspondence to determine the data type of the data item.

[0057] Based on the above embodiment, optionally, the identifying the data type of each data item in the data to be processed includes: obtaining source data corresponding to each of the data items, and determining the data type of each of the data items based on the file descriptor or header information in the source data.

[0058] The file descriptor is a non-negative integer used to identify an open file. The header information refers to the information located at the beginning of a file or a data block, which is used to describe the type, format, version, metadata, etc. of the file. In this embodiment, for each data item in the data to be processed, the source data corresponding to the data item is obtained, and the file descriptor or header information in the source data is read. The type of the data line can be determined based on the file descriptor and the mapping relationship between the file descriptor and the data type; the data type of the data item can also be determined based on the header information and the mapping relationship between the header information and the data type. The mapping relationship can be a predefined mapping relationship table or a predefined mapping logic.

[0059] S130. Determine a processing strategy for each of the data items based on the data type of each of the data items, and extract data from each of the data items based on the processing strategy for each of the data items to obtain data content corresponding to each of the data items.

[0060] In this embodiment, according to the data type of the identified data item, a corresponding processing strategy can be formulated for each type of data item; according to the processing strategy of each data item, data extraction is performed on each data item to obtain the specific content of the data item. Wherein, the processing strategy is essentially a method for extracting data content from a data item. Specifically, the processing strategy for structured data includes: extracting data content from the structured data; the processing strategy for unstructured data includes: extracting data from unstructured data based on a pre-trained data processing model to obtain the extracted data content; the processing strategy for semi-structured data includes: parsing the semi-structured data, extracting keywords from the parsed data, and obtaining the extracted data content. Exemplarily, for structured data, SQL query can be directly performed to extract data content; for unstructured data, a pre-trained data processing model can be used to extract data from unstructured data to obtain data content; wherein, the data processing model includes but is not limited to a natural language processing model (Natural Language Processing, NLP), an image recognition model, etc.; for semi-structured data, a parser can be used to parse the data, and the data content can be obtained by extracting keywords from the parsed data.

[0061] In some embodiments, optionally, after obtaining the data content corresponding to each of the data items, the method further includes: performing standardization processing on the data content.

[0062] Specifically, the data content is cleaned, the data format is unified, the data is normalized, and other operations are performed. Exemplarily, data cleaning includes: removing duplicate data, invalid data, and noise data, and correcting or filling missing values. Data format unification includes: converting date and time data into a unified date and time format, converting currency data into a unified currency symbol and format; for text data, converting the text to lowercase or uppercase, removing punctuation marks, stop words, etc., for subsequent text analysis. Data normalization includes: normalizing numerical data to more fairly compare different features in the algorithm.

[0063] S140: Convert the data content corresponding to each of the data items into a preset data format, and integrate the data in the preset data format corresponding to each of the data items into target data.

[0064] The preset data format refers to the input parameter format required by the service interface. Specifically, the preset data format is determined based on the service interface and is not limited here. In this embodiment, the data content corresponding to each extracted data item can be converted into a preset data format using a preset mapping rule. Further, all converted data items in the preset data format are integrated into a target data for subsequent unified processing. The target data is a collection of data items in the preset data format.

[0065] In some embodiments, optionally, after integrating the data in the preset data format corresponding to each of the data items into target data, it also includes: converting the target data into target vector data, and matching the target vector data in a preset cache library; wherein the preset cache library includes historical vector data and processing results corresponding to the historical vector data; when the matching degree between the target vector data and the historical vector data meets the matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data.

[0066] Wherein, the target vector data is target data in vector form. In this embodiment, the integrated target data is converted into target vector data in vector form, and the target vector data is matched in a preset cache library. When the matching degree between the target vector data and the historical vector data meets the matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data. Wherein, the matching condition can be that the similarity between the target vector data and the historical vector data reaches a preset similarity threshold. Specifically, the similarity between the target vector data and the historical vector data can be calculated using measurement methods such as cosine similarity and Euclidean distance. When the similarity between the target vector data and a certain historical vector data reaches a preset similarity threshold, the two vectors are considered to be matched.

[0067] This embodiment can quickly find data similar or identical to the target data by matching the target vector data with the historical vector data in the cache library, and directly use the processing results of these data, thereby avoiding repeated calls to the service interface for processing and improving processing efficiency.

[0068] S150 . In response to the service calling operation, calling the target service interface, processing the target data based on the target service interface, and obtaining a data processing result.

[0069] The service call operation is used to call the target service interface. Specifically, the service call operation can be a request initiated by the user or a scheduled task. In this embodiment, in response to the service call operation, the corresponding target service interface is called. The target service interface executes the preset processing logic on the incoming target data and generates a data processing result. The preset processing logic includes but is not limited to data analysis, machine learning prediction, etc., which is associated with the service interface and is not limited here.

[0070] The technical solution of this embodiment is to obtain the data to be processed, wherein the data to be processed includes data items of at least one data type; identify the data type of each data item in the data to be processed, and the data type includes one or more of structured data, unstructured data and semi-structured data; determine the processing strategy of the data item based on the data type of each data item, extract data from each data item based on the processing strategy of each data item, and obtain the data content corresponding to each data item; convert the data content corresponding to each data item into a preset data format, and integrate the data of the preset data format corresponding to each data item into target data; in response to the service call operation, call the target service interface, and process the target data based on the target service interface to obtain the data processing result. By identifying the data type, extracting the data content and converting the data format of the data to be processed in multiple modes, the multi-modal data to be processed is integrated into data in a unified format, so as to call the corresponding service interface to process the data in the unified format, realize the service call of multi-modal data, and improve the flexibility and versatility of the service.

[0071] Embodiment 2

[0072] Figure 2 It is a flowchart of a service calling method for multimodal data provided by the second embodiment of the present invention. In this embodiment, based on the above embodiments, the acquisition of the data to be processed includes: displaying a data processing page, the data processing page includes data collection areas corresponding to each data type; collecting data items through each of the data collection areas, and setting the data type identifier of the data item based on the data type corresponding to the data collection area. Correspondingly, the identification of the data type of each data item in the data to be processed includes: determining the data type of the data item according to the data type identifier carried by the data item. Among them, the explanation of the terms that are the same or corresponding to the above embodiments will not be repeated here.

[0073] like Figure 2 As shown, the method includes:

[0074] S210, displaying a data processing page, wherein the data processing page includes data collection areas corresponding to each data type; collecting data items through each of the data collection areas, and setting data type identifiers of the data items based on the data types corresponding to the data collection areas.

[0075] The data collection area is used for data items of data types corresponding to the data collection area, and each data collection area can call the connection interface of the data source. In this embodiment, a data processing page is displayed, and in the data collection page, each data collection area can collect data items of data types corresponding to the data collection area by calling the connection interface of the data source, and set the data type identifier of the data item.

[0076] Based on the above embodiment, optionally, description information of the data type corresponding to the data collection area is displayed in each of the data collection areas; or, the method further includes: in response to a trigger operation in any of the data collection areas, description information of the data type corresponding to the data collection area is displayed.

[0077] The description information of the data type is text data used to clarify and explain the meaning, format, scope, purpose or restriction of the data type. Specifically, the description information includes but is not limited to the name, definition, format, example, etc. of the data type, which is not limited here. In this embodiment, a text label can be added in the data collection area, and the description information of the data type corresponding to the data collection area is filled into the text label for display.

[0078] Optionally, an event processing function may be written, and logic may be written in the event processing function to display the description information. The event processing function is used to respond when the user performs a trigger operation on any data collection area. In this embodiment, on the data processing page, the user performs a trigger operation on any data collection area. In response to the trigger operation on any data collection area, the event processing function is triggered to display the description information of the data type corresponding to the data collection area. The trigger operation of the data collection area includes, but is not limited to, hovering the mouse over the data collection area, clicking the data collection area, and clicking the description information display control corresponding to the data collection area. Exemplarily, the description information may be displayed in the form of a pop-up window or a triggered display.

[0079] In some embodiments, optionally, after collecting data items respectively through each of the data collection areas, the method further includes: verifying the data type of the data item collected by the data collection area; and generating a prompt message when the data type of the data item collected by the data collection area is inconsistent with the data type corresponding to the data collection area.

[0080] In order to ensure the consistency of the data type of the collected data item and the data type corresponding to the data area, after each data collection area collects data items, the data type of the data line collected by the data collection area can be verified to determine whether the data type of the data item collected by the data collection area is consistent with the data type corresponding to the data collection area. If the data type of the data item collected by the data collection area is inconsistent with the data type corresponding to the data collection area, a prompt message is generated to avoid incorrect setting of the data type identifier of the data item.

[0081] S220. Determine the data type of the data item according to the data type identifier carried by the data item.

[0082] In this embodiment, the data type identifier carried by the data item may be matched with the data type corresponding to the data type identifier, and the data type may be used as the data type of the data item.

[0083] S230: Determine a processing strategy for each of the data items based on the data type of each of the data items, and extract data from each of the data items based on the processing strategy for each of the data items to obtain data content corresponding to each of the data items.

[0084] S240: Convert the data content corresponding to each of the data items into a preset data format, and integrate the data in the preset data format corresponding to each of the data items into target data.

[0085] S250 . In response to the service calling operation, calling the target service interface, and processing the target data based on the target service interface to obtain a data processing result.

[0086] The technical solution of this embodiment realizes the collection of data items of the data to be processed through the data collection area of ​​the data processing page, and sets a data type identifier for each collected data item to facilitate rapid identification of the type of the data item, thereby improving the processing rate of the data to be processed.

[0087] Embodiment 3

[0088] Figure 3 This is a flow chart of a method for calling a service of multimodal data in a financial scenario provided by Embodiment 3 of the present invention. This embodiment is a preferred embodiment provided on the basis of the above embodiments, and this embodiment is a method for calling a service of multimodal data in a financial scenario.

[0089] like Figure 3 As shown, the method includes:

[0090] S310, displaying a financial business data processing page, which includes a data collection area for financial business data corresponding to each data type; calling a data connection interface of a data source of the financial business data through the data collection area of ​​each financial business data, sending a data request to each data source respectively, and obtaining data items of each data source.

[0091] Among them, data sources include stock exchanges, banking systems, customer relationship management systems (CRM), etc. Financial business data includes bank statements, income certificates, credit reports, transaction data, etc.

[0092] S320: Verify the data type of the data item collected in the data collection area, and generate a prompt message when the data type of the data item collected in the data collection area for financial business data is inconsistent with the data type corresponding to the data collection area for financial business data.

[0093] S330. Obtain the file extension of each data item in the financial business data, and determine the data type of the data item based on the preset correspondence between the extensibility and the data type and the file extension of the data item; or, obtain the source data corresponding to each data item, and determine the data type of each data item based on the file descriptor or header information in the source data; or, set the data type identifier of the data item based on the data type corresponding to the data collection area, and determine the data type of the data item according to the data type identifier carried by the data item.

[0094] S340. Determine the processing strategy of the data item based on the data type of each data item. If the data item is structured data, perform data content extraction on the structured data; if the data item is unstructured data, perform data extraction on the unstructured data based on the pre-trained data processing mode to obtain the extracted data content; if the data item is semi-structured data, perform data analysis on the semi-structured data, perform keyword extraction on the analyzed data, and obtain the extracted data content.

[0095] S350: Standardize the data content, including data cleaning, format conversion, missing value processing and other operations.

[0096] S360: Convert the data content corresponding to each data item into a preset data format, and integrate the data in the preset data format corresponding to each data item into target data.

[0097] S370. Convert the target data into target vector data, and match the target vector data in a preset cache library; wherein the preset cache library includes historical vector data and processing results corresponding to the historical vector data; when the matching degree between the target vector data and the historical vector data meets the matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data.

[0098] S380. When the match in the preset cache library fails or real-time processing is required, the financial service personnel initiates a financial service call operation, responds to the financial service call operation, calls the financial service service interface to process the target data, and generates a processing result.

[0099] Among them, the financial business service interface includes risk assessment interface, investment strategy recommendation interface, etc. Correspondingly, the processing results include risk assessment results, recommended investment strategies, etc.

[0100] The technical solution of this embodiment integrates the multi-modal data to be processed into data in a unified format by performing data type identification, data content extraction, and data format conversion on the multi-modal financial business data, thereby calling the corresponding financial business service interface to process the financial business data in the unified format, realizing service calls for multi-modal financial business data, and improving the flexibility and versatility of financial services.

[0101] Embodiment 4

[0102] Figure 4 This is a flowchart of a method for calling a service of multimodal data in a medical scenario provided by Embodiment 4 of the present invention. This embodiment is a preferred embodiment provided on the basis of the above embodiments, and this embodiment is a method for calling a service of multimodal data in a medical scenario.

[0103] like Figure 4 As shown, the method includes:

[0104] S410, displaying a medical business data processing page, which includes a data collection area for medical business data corresponding to each data type; calling a data connection interface of a data source of the medical business data through the data collection area of ​​each medical business data, sending a data request to each data source respectively, and obtaining data items of each data source.

[0105] The data sources of medical business data include the hospital information management system (LIS), picture archiving and communication system (PACS) and other systems within the hospital, as well as external data sources (such as medical insurance system, third-party testing agencies). Medical business data includes medical history records, examination results, diagnosis reports, medical images, etc.

[0106] S420: Verify the data type of the data item collected in the data collection area, and generate prompt information when the data type of the data item collected in the data collection area for medical business data is inconsistent with the data type corresponding to the data collection area for medical business data.

[0107] S430. Obtain the file extension of each data item in the medical business data, and determine the data type of the data item based on the preset correspondence between the extensibility and the data type and the file extension of the data item; or, obtain the source data corresponding to each data item, and determine the data type of each data item based on the file descriptor or header information in the source data; or, set the data type identifier of the data item based on the data type corresponding to the data collection area, and determine the data type of the data item according to the data type identifier carried by the data item.

[0108] S440. Determine a processing strategy for the data item based on the data type of each data item. If the data item is structured data, perform data content extraction on the structured data. If the data item is unstructured data, perform data extraction on the unstructured data based on a pre-trained data processing model to obtain extracted data content. If the data item is semi-structured data, perform data analysis on the semi-structured data, perform keyword extraction on the analyzed data, and obtain extracted data content.

[0109] S450: Standardize the data content, including data cleaning, format conversion, missing value processing and other operations.

[0110] S460: Convert the data content corresponding to each data item into a preset data format, and integrate the data in the preset data format corresponding to each data item into target data.

[0111] S470. Convert the target data into target vector data, and match the target vector data in a preset cache library; wherein the preset cache library includes historical vector data and processing results corresponding to the historical vector data; when the matching degree between the target vector data and the historical vector data meets the matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data.

[0112] S480: When the preset buffer inventory match fails or real-time processing is required, the medical staff initiates a medical service call operation, responds to the medical service call operation, calls the medical business service interface to process the target data, and generates a processing result.

[0113] Among them, the medical business interfaces include hospital recommendation interface, disease diagnosis interface, treatment plan recommendation interface, etc. Correspondingly, the processing results include recommended hospitals, diagnosis results, recommended treatment plans, etc.

[0114] The technical solution of this embodiment integrates the multi-modal data to be processed into data in a unified format by identifying data types, extracting data content, and converting data formats for medical business data of multiple modalities, thereby calling the corresponding medical business service interface to process the medical business data in the unified format, realizing service calls for multi-modal medical business data, and improving the flexibility and versatility of medical services.

[0115] Embodiment 5

[0116] Figure 5 Schematic diagram of a multi-modal data service invocation device provided by Embodiment 5 of the present invention. Figure 5 As shown, the device comprises:

[0117] The to-be-processed data acquisition module 510 is used to acquire the to-be-processed data, wherein the to-be-processed data includes data items of at least one data type;

[0118] A data type identification module 520, used to identify the data type of each data item in the data to be processed, wherein the data type includes one or more of structured data, unstructured data and semi-structured data;

[0119] A data content extraction module 530 is used to determine a processing strategy for each data item based on the data type of each data item, and to extract data from each data item based on the processing strategy of each data item to obtain data content corresponding to each data item;

[0120] A data integration module 540, configured to convert the data content corresponding to each of the data items into a preset data format, and integrate the data in the preset data format corresponding to each of the data items into target data;

[0121] The service calling module 550 is used to call the target service interface in response to the service calling operation, and process the target data based on the target service interface to obtain a data processing result.

[0122] The technical solution of this embodiment is to obtain the data to be processed, wherein the data to be processed includes data items of at least one data type; identify the data type of each data item in the data to be processed, and the data type includes one or more of structured data, unstructured data and semi-structured data; determine the processing strategy of the data item based on the data type of each data item, extract data from each data item based on the processing strategy of each data item, and obtain the data content corresponding to each data item; convert the data content corresponding to each data item into a preset data format, and integrate the data of the preset data format corresponding to each data item into target data; in response to the service call operation, call the target service interface, and process the target data based on the target service interface to obtain the data processing result. By identifying the data type, extracting the data content and converting the data format of the data to be processed in multiple modes, the multi-modal data to be processed is integrated into data in a unified format, so as to call the corresponding service interface to process the data in the unified format, realize the service call of multi-modal data, and improve the flexibility and versatility of the service.

[0123] Based on the above embodiment, optionally, the data type identification module 520 is used to obtain the file extension of each data item in the data to be processed, and determine the data type of the data item based on a preset correspondence between the extension and the data type and the file extension of the data item.

[0124] Based on the above embodiment, optionally, the data type identification module 520 is used to obtain source data corresponding to each of the data items, and determine the data type of each of the data items based on the file descriptor or header information in the source data.

[0125] Based on the above embodiment, optionally, the module 510 for acquiring data to be processed is used to display a data processing page, which includes data collection areas corresponding to each data type; data items are collected respectively through each of the data collection areas, and the data type identifier of the data item is set based on the data type corresponding to the data collection area.

[0126] Based on the above embodiment, optionally, the data type identification module 520 is used to determine the data type of the data item according to the data type identifier carried by the data item.

[0127] Based on the above embodiment, optionally, description information of the data type corresponding to the data collection area is displayed in each of the data collection areas; or, the device also includes a data type description information trigger display module, which is used to display the description information of the data type corresponding to the data collection area in response to a trigger operation in any of the data collection areas.

[0128] Based on the above embodiment, optionally, the device also includes a data type verification module for verifying the data type of the data item collected in the data collection area; when the data type of the data item collected in the data collection area is inconsistent with the data type corresponding to the data collection area, a prompt message is generated.

[0129] Based on the above embodiment, optionally, at least one data item in the data to be processed comes from at least one data source; the module 510 for obtaining data to be processed is used to send data requests to each of the data sources through a connection interface pre-established with at least one data source, and obtain data items from each of the data sources.

[0130] On the basis of the above embodiments, optionally, the processing strategy for the structured data includes: extracting data content from the structured data; the processing strategy for the unstructured data includes: extracting data from the unstructured data based on a pre-trained data processing model to obtain extracted data content; the processing strategy for the semi-structured data includes: parsing the semi-structured data, extracting keywords from the parsed data, and obtaining extracted data content.

[0131] Based on the above embodiment, optionally, the device further includes a data content standardization module for performing standardization processing on the data content.

[0132] Based on the above embodiment, optionally, the device also includes a processing result matching module, which is used to convert the target data into target vector data, and match the target vector data in a preset cache library; wherein the preset cache library includes historical vector data and processing results corresponding to the historical vector data; when the matching degree between the target vector data and the historical vector data meets the matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data.

[0133] Based on the above embodiment, optionally, the data to be processed is financial business data, and the target service interface is a financial business service interface.

[0134] The service calling device for multimodal data provided in the embodiment of the present invention can execute the service calling method for multimodal data provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0135] Embodiment 6

[0136] Figure 61 is a schematic diagram of the structure of an electronic device provided in Embodiment 6 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0137] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0138] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0139] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a service call method for multimodal data.

[0140] In some embodiments, the service calling method for multimodal data may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the service calling method for multimodal data described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the service calling method for multimodal data in any other appropriate manner (e.g., by means of firmware).

[0141] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0143] Embodiment 7

[0144] Embodiment 7 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute a service calling method for multimodal data, the method comprising:

[0145] Acquire data to be processed, wherein the data to be processed includes data items of at least one data type;

[0146] Identify the data type of each data item in the data to be processed, where the data type includes one or more of structured data, unstructured data, and semi-structured data;

[0147] Determine a processing strategy for each data item based on the data type of each data item, extract data from each data item based on the processing strategy for each data item, and obtain data content corresponding to each data item;

[0148] Convert the data content corresponding to each data item into a preset data format, and integrate the data in the preset data format corresponding to each data item into target data;

[0149] In response to the service call operation, the target service interface is called, and the target data is processed based on the target service interface to obtain a data processing result.

[0150] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0152] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0153] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0154] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0155] Embodiment 8

[0156] Embodiment 8 of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the service calling method for multimodal data according to any embodiment of the present invention.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calling a service of multimodal data, characterized in that: include: Acquire data to be processed, wherein the data to be processed includes data items of at least one data type; Identify the data type of each data item in the data to be processed, wherein the data type includes one or more of structured data, unstructured data, and semi-structured data; Determine a processing strategy for each data item based on the data type of each data item, and extract data from each data item based on the processing strategy for each data item to obtain data content corresponding to each data item; Converting the data content corresponding to each of the data items into a preset data format, and integrating the data in the preset data format corresponding to each of the data items into target data; In response to the service calling operation, the target service interface is called, and the target data is processed based on the target service interface to obtain a data processing result.

2. The method according to claim 1, characterized in that The step of identifying the data type of each data item in the data to be processed includes: The file extension of each of the data items in the data to be processed is obtained, and the data type of the data item is determined based on a preset correspondence between the extension and the data type and the file extension of the data item.

3. The method according to claim 1, characterized in that The step of identifying the data type of each data item in the data to be processed includes: The source data corresponding to each of the data items is obtained, and the data type of each of the data items is determined based on the file descriptor or header information in the source data.

4. The method according to claim 1, characterized in that: The obtaining of the data to be processed comprises: Displaying a data processing page, wherein the data processing page includes a data collection area corresponding to each data type; Data items are collected respectively through each of the data collection areas, and data type identifiers of the data items are set based on the data types corresponding to the data collection areas.

5. The method according to claim 4, characterized in that The step of identifying the data type of each data item in the data to be processed includes: The data type of the data item is determined according to the data type identifier carried by the data item.

6. The method according to claim 4, characterized in that In each of the data collection areas, description information of the data type corresponding to the data collection area is displayed; or, The method further includes: in response to a trigger operation in any of the data collection areas, displaying description information of the data type corresponding to the data collection area.

7. The method according to claim 4, characterized in that After collecting data items respectively through each of the data collection areas, the following step further includes: Verifying the data type of the data item collected in the data collection area; When the data type of the data item collected in the data collection area is inconsistent with the data type corresponding to the data collection area, prompt information is generated.

8. The method according to claim 1, characterized in that: At least one data item in the data to be processed comes from at least one data source; The obtaining of the data to be processed comprises: Through a connection interface pre-established with at least one data source, data requests are sent to each of the data sources to obtain data items from each of the data sources.

9. The method according to claim 1, characterized in that: The processing strategy of the structured data includes: extracting data content from the structured data; The processing strategy of the unstructured data includes: extracting data from the unstructured data based on a pre-trained data processing model to obtain extracted data content; The processing strategy of the semi-structured data includes: performing data parsing on the semi-structured data, extracting keywords from the parsed data, and obtaining extracted data content.

10. The method according to claim 1, characterized in that After obtaining the data content corresponding to each of the data items, the method further includes: performing standardization processing on the data content.

11. The method according to claim 1, characterized in that After integrating the data in the preset data format corresponding to each of the data items into target data, the method further includes: Convert the target data into target vector data, and match the target vector data in a preset cache library; wherein the preset cache library includes historical vector data and processing results corresponding to the historical vector data; When the matching degree between the target vector data and the historical vector data satisfies a matching condition, the processing result corresponding to the historical vector data is used as the processing result corresponding to the target data.

12. The method according to claim 1, characterized in that The data to be processed is financial business data, and the target service interface is a financial business service interface.

13. A service invocation device for multimodal data, characterized in that: include: A to-be-processed data acquisition module, used to acquire the to-be-processed data, wherein the to-be-processed data includes data items of at least one data type; A data type identification module, used to identify the data type of each data item in the data to be processed, wherein the data type includes one or more of structured data, unstructured data and semi-structured data; A data content extraction module, used to determine a processing strategy for each data item based on the data type of each data item, and to extract data from each data item based on the processing strategy of each data item to obtain data content corresponding to each data item; A data integration module, used to convert the data content corresponding to each of the data items into a preset data format, and integrate the data in the preset data format corresponding to each of the data items into target data; The service calling module is used to call the target service interface in response to the service calling operation, process the target data based on the target service interface, and obtain the data processing result.

14. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the service calling method for multimodal data described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the service calling method for multimodal data according to any one of claims 1 to 12 when executed.

16. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the service invocation method for multimodal data according to any one of claims 1 to 12.

Images