Artificial Intelligence-Based Data Processing Method and Device, Medium, and Gateway Device

Abstract

The present invention discloses an artificial intelligence-based data processing method, device, medium, and gateway device, which relates to the field of artificial intelligence technology and mainly aims to solve the problem of low data processing efficiency of existing unstructured data. It includes: configuring a gateway data transmission interface and binding at least one data processing terminal, where the gateway data transmission interface is used to transmit data objects flowing through the gateway device; receiving a data processing request through the gateway data transmission interface, and classifying the data objects carried in the data processing request based on a data classification model that has completed training to determine the target data processing terminal of the data object. The training sample data of the data classification model is the classification label for classifying different data objects to the target data processing terminal according to the data access times; processing the data object in the target data processing terminal through the gateway data transmission interface.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a data processing method, apparatus, medium, and gateway device based on artificial intelligence. Background Art

[0002] With the rapid development of big data processing technology, the storage demand for unstructured data is increasing, especially in the field of medical and health, where unstructured data such as high-definition pictures, audio and video, and artificial intelligence models need to be stored.

[0003] Currently, when different existing business parties store unstructured data, they usually store it in a single object storage system based on each unstructured data type. However, the single object storage system cannot meet the security requirements and high-efficiency storage performance requirements in different business scenarios, and the storage and query costs are relatively high, which reduces the data processing efficiency of unstructured data, thereby affecting the efficiency of business processing based on stored and queried data in different business scenarios. Summary of the Invention

[0004] In view of this, the present invention provides a data processing method, apparatus, medium, and gateway device based on artificial intelligence, mainly aiming to solve the problem of low data processing efficiency of existing unstructured data.

[0005] According to one aspect of the present invention, there is provided a data processing method based on artificial intelligence, including:

[0006] Configuring a gateway data transmission interface and binding at least one data processing terminal, where the gateway data transmission interface is used to transmit data objects flowing through the gateway device;

[0007] Receiving a data processing request through the gateway data transmission interface, and classifying the data object carried in the data processing request based on a trained data classification model to determine the target data processing terminal of the data object, where the training sample data of the data classification model is classification labels for classifying different data objects to the target data processing terminal according to data access times;

[0008] Processing the data object in the target data processing terminal through the gateway data transmission interface.

[0009] Further, before classifying the data object carried in the data processing request based on the trained data classification model to determine the target data processing terminal of the data object, the method further includes:

[0010] Determine a training data processing terminal from the data processing terminals, and obtain training sample data in the training data processing terminal, where the classification labels in the training sample data are determined based on the data access times of each data processing terminal;

[0011] Establish a data classification model, and perform model training on the data classification model based on the classification labels and the training sample data to obtain a data classification model that has completed training.

[0012] Further, the determining a training data processing terminal from the data processing terminals includes:

[0013] Obtain service requirement information, where the service requirement information includes data classification types and data processing types corresponding to data objects in different service scenarios, the data classification types include image data types, audio data types, video data types, and model algorithm data types in unstructured data types, and the data processing types include storage processing types, query processing types, and update processing types;

[0014] Obtain data processing record information of the data processing terminals that have completed data processing;

[0015] If the data processing record information matches the service requirement information, determine the data processing terminal as the training data processing terminal, so as to obtain the classification labels of the marked data objects and the marked terminal identity information in the training data processing terminal as training sample data.

[0016] Further, the method further includes:

[0017] Send the service requirement information to the data processing terminals at a preset time interval, and instruct the data processing terminals to perform label marking processing based on the service requirement information;

[0018] Receive the marking indication information of the data processing terminals that have completed label marking processing, and execute the step of determining a training data processing terminal from the data processing terminals to update the training of the data classification model.

[0019] Further, after establishing the data classification model and performing model training on the data classification model based on the classification labels and the training sample data to obtain a data classification model that has completed training, the method further includes:

[0020] Obtain third-party application identifiers for data transmission through the gateway data transmission interface, and determine model verification metrics matching the third-party application identifiers based on a preset model verification relationship, where the preset model verification relationship includes the corresponding relationships between multiple third-party application identifiers and multiple model verification metrics;

[0021] If the model verification parameters of the trained data classification model match the model verification index, it is confirmed that the data classification model has passed the index verification, so that the data object can be classified through the data classification model.

[0022] Further, processing the data object in the target data processing terminal through the gateway data transmission interface includes:

[0023] If the data processing request is a data storage request, outputting the data object to the target data processing terminal through the gateway data storage interface, and instructing the target data processing terminal to store the data content of the data object;

[0024] If the data processing request is a data query request, sending the data query request of the data object to the target data processing terminal through the gateway data transmission interface, and instructing the target data processing terminal to query and feedback the data content of the data object;

[0025] If the data processing request is a data update request, the data object is output to the target data processing terminal through the gateway data transmission interface, and the target data processing terminal is instructed to replace and update the data content of the data object.

[0026] Furthermore, before outputting the data object to the target data processing terminal through the gateway data transmission interface, the method further includes:

[0027] Acquire data processing performance parameters of the target data processing terminal, wherein the data processing performance parameters include at least one of storage performance parameters, query performance parameters, terminal hardware operation parameters, and terminal operation line number;

[0028] If the data performance parameter does not match the gateway device transmission condition, then releasing the binding relationship with the target data processing terminal;

[0029] If the data performance parameter matches the gateway device transmission condition, then the step of outputting the data object to the target data processing terminal through the gateway data transmission interface is instructed to be executed.

[0030] According to another aspect of the present invention, there is provided a data processing device based on artificial intelligence, comprising:

[0031] A configuration module, used to configure a gateway data transmission interface and bind at least one data processing terminal, wherein the gateway data transmission interface is used to transmit data objects flowing through the gateway device;

[0032] A receiving module, configured to receive a data processing request through the gateway data transmission interface, and classify the data object carried in the data processing request based on a data classification model that has completed training, to determine the target data processing terminal of the data object. The training sample data of the data classification model is the classification label for classifying different data objects to the target data processing terminal according to the data access times;

[0033] A processing module, configured to process the data object in the target data processing terminal through the gateway data transmission interface.

[0034] Further, the device further includes:

[0035] A determining module, configured to determine a training data processing terminal from the data processing terminals, and obtain training sample data in the training data processing terminal, where the classification label in the training sample data is determined based on the data access times of each data processing terminal;

[0036] A training module, configured to establish a data classification model, and perform model training on the data classification model based on the classification label and the training sample data to obtain a data classification model that has completed training.

[0037] Further, the determining module includes:

[0038] A first obtaining unit, configured to obtain service requirement information, where the service requirement information includes the data classification types and data processing types corresponding to data objects in different service scenarios. The data classification types include image data type, audio data type, video data type, and model algorithm data type in the unstructured data type, and the data processing types include storage processing type, query processing type, and update processing type;

[0039] A second obtaining unit, configured to obtain data processing record information of the data that has been processed in the data processing terminal;

[0040] A determining unit, configured to, if the data processing record information matches the service requirement information, determine the data processing terminal as the training data processing terminal, so as to obtain the classification label of the labeled data object and the labeled terminal identity information in the training data processing terminal as the training sample data.

[0041] Further, the determining module further includes:

[0042] A sending unit, configured to send the service requirement information to the data processing terminal at a preset time interval, and instruct the data processing terminal to perform label marking processing based on the service requirement information;

[0043] The receiving unit is used to receive the label indication information that the data processing terminal completes the label labeling process, and execute the step of determining the training data processing terminal from the data processing terminal to update the training data classification model.

[0044] Furthermore, the determination module is specifically used to obtain a third-party application identifier for data transmission through the gateway data transmission interface, and determine a model verification indicator that matches the third-party application identifier based on a preset model verification relationship, wherein the preset model verification relationship includes a correspondence between multiple third-party application identifiers and multiple model verification indicators; if the model verification parameters of the trained data classification model match the model verification indicator, it is confirmed that the data classification model has passed the indicator verification, so that the data object can be classified and processed through the data classification model.

[0045] Furthermore, the processing module includes:

[0046] a first output unit, configured to output the data object to the target data processing terminal through the gateway data transmission interface if the data processing request is a data storage request, and instruct the target data processing terminal to store the data content of the data object;

[0047] A second output unit is configured to send the data query request of the data object to the target data processing terminal through the gateway data transmission interface if the data processing request is a data query request, and instruct the target data processing terminal to query and feedback the data content of the data object;

[0048] The third output unit is used to output the data object to the target data processing terminal through the gateway data transmission interface if the data processing request is a data update request, and instruct the target data processing terminal to replace and update the data content of the data object.

[0049] Furthermore, the device also includes:

[0050] An acquisition module, used to acquire data processing performance parameters of the target data processing terminal, wherein the data processing performance parameters include at least one of storage performance parameters, query performance parameters, terminal hardware operation parameters, and terminal operation line number;

[0051] A release module, used for releasing the binding relationship with the target data processing terminal if the data performance parameter does not match the gateway device transmission condition;

[0052] The instruction module is used for instructing to execute the step of outputting the data object to the target data processing terminal through the gateway data transmission interface if the data performance parameter matches the gateway device transmission condition.

[0053] According to another aspect of the present invention, a storage medium is provided, in which at least one executable instruction is stored, and the executable instruction causes a processor to perform operations corresponding to the above-mentioned artificial intelligence-based data processing method.

[0054] According to still another aspect of the present invention, a gateway device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0055] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above-mentioned artificial intelligence-based data processing method.

[0056] By means of the above technical solutions, the technical solutions provided by the embodiments of the present invention at least have the following advantages:

[0057] The present invention provides an artificial intelligence-based data processing method, device, medium, and gateway device. Compared with the prior art, in the embodiments of the present invention, a gateway data transmission interface is configured and at least one data processing terminal is bound, and the gateway data transmission interface is used to transmit data objects flowing through the gateway device; a data processing request is received through the gateway data transmission interface, and the data objects carried in the data processing request are classified based on a data classification model that has been trained, and the target data processing terminal of the data object is determined. The training sample data of the data classification model is a classification label for classifying different data objects to the target data processing terminal according to the data access times; through the gateway data transmission interface, the data object is processed in the target data processing terminal, so as to realize diversified and secure data processing of multi-object data in different business scenarios, improve the effectiveness of data processing such as data storage and query, reduce the data processing cost, and thus improve the efficiency of business processing based on data storage and query in different business scenarios.

[0058] The above description is only an overview of the technical solutions of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically described below. Description of the Drawings

[0059] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0060] Figure 1 shows a flowchart of a data processing method based on artificial intelligence provided by an embodiment of the present invention;

[0061] Figure 2 shows another flowchart of a data processing method based on artificial intelligence provided by an embodiment of the present invention;

[0062] Figure 3 shows still another flowchart of a data processing method based on artificial intelligence provided by an embodiment of the present invention;

[0063] Figure 4 shows yet another flowchart of a data processing method based on artificial intelligence provided by an embodiment of the present invention;

[0064] Figure 5 shows a schematic diagram of a data processing process through a gateway device provided by an embodiment of the present invention;

[0065] Figure 6 shows a block diagram of a data processing device based on artificial intelligence provided by an embodiment of the present invention;

[0066] Figure 7 shows a schematic diagram of the structure of a gateway device provided by an embodiment of the present invention. Detailed implementation manners

[0067] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0068] When different business parties store unstructured data, they usually store it in a single object storage system based on each unstructured data type. However, the single object storage system cannot meet the security requirements and high - efficiency storage performance requirements in different business scenarios, and the storage and query costs are relatively high, reducing the data processing efficiency of unstructured data, thus affecting the efficiency of business processing based on stored and queried data in different business scenarios.

[0069] An embodiment of the present invention provides a data processing method based on artificial intelligence, as Figure 1 shown, the method includes:

[0070] 101. Configure a gateway data transmission interface and bind at least one data processing terminal.

[0071] In an embodiment of the present invention, the current execution end is a gateway device. A gateway device, also known as an internetwork connector or protocol converter, is a computer system or device that provides data conversion services between multiple networks, including but not limited to network switches with layer-3 switching functions, routers, firewalls, and terminal devices with routing functions enabled through software. Thus, different application clients and application servers perform data transmission through the gateway device as the current execution end. Among them, a gateway data transmission interface is configured in the current gateway device. This gateway data transmission interface is used to transmit data objects flowing through the gateway device. That is, when an application client, such as an APP, generates data objects corresponding to different service scenarios, the data objects are transmitted to the current gateway device through the gateway data transmission interface. The embodiments of the present invention do not make specific limitations. At the same time, in order to enable the gateway device as the current execution end to cooperate with the data processing terminal to implement the data processing function, after configuring the gateway data transmission interface, the gateway data transmission interface is bound to at least one data processing terminal, so that the gateway device as the current execution end can transmit the data objects to the corresponding data processing terminals respectively for further data processing. Among them, the data objects include but are not limited to different data contents generated or requested by application clients and application servers. For example, video data recorded by an APP client. At this time, it can be transmitted in the form of a file. The embodiments of the present invention do not make specific limitations.

[0072] It should be noted that the application fields applicable to the application clients and application servers in the embodiments of the present invention include but are not limited to medical health, insurance product transactions, etc., so as to perform data processing on different data objects. In addition, since there may already be a data transmission interface or multiple data transmission interfaces in the gateway device as the current execution end, when configuring the gateway data transmission interface, an interface with a specified data transmission protocol can be selected from the gateway device for configuration, such as the gateway POST interface, or an interface with a specified data transmission protocol can be created, such as creating an upload interface. The embodiments of the present invention do not make specific limitations. At this time, after receiving a data processing request through the configured gateway data transmission interface, the steps of step 102 are executed, so as to implement the gateway service function in the gateway device.

[0073] 102. Receive a data processing request through the gateway data transmission interface, and classify the data object carried in the data processing request based on the data classification model that has been trained to determine the target data processing terminal of the data object.

[0074] In an embodiment of the present invention, after receiving a data processing request through a gateway data transmission interface, the gateway device serving as the current execution end extracts the data object carried in the data processing request, such as user identity information in the form of a file. Then, based on the data classification model that has completed model training, the data object is classified to determine the target data processing terminal. Among them, the classification label of the training sample data of the data classification model is determined based on the access times of different data processing terminals, so that the target data processing terminal determined by the trained data classification model is matched according to the data access times and classification labels, and is suitable for the terminal that processes the data object. For example, when the data object is user identity information, based on the data classification model, the target data processing terminal corresponding to classification label 1 is determined to be terminal a, and at this time, the data access times of the target data processing terminal match the classification label 1, so that processing the data object at the target data processing terminal can achieve the purpose of maximizing data processing efficiency.

[0075] It should be noted that since at least one data processing terminal bound in the embodiment of the present invention can be set in the form of a device cluster, a device cluster can include multiple sub-clusters. The data processing terminals in each sub-cluster can process data objects of one data classification type. When determining the target data processing terminal, it is to determine the sub-cluster corresponding to the data object that matches the data classification type from multiple sub-clusters, so as to determine a specific target data processing terminal in this sub-cluster, so as to relieve the processing pressure on other data processing terminals in a sub-cluster.

[0076] 103. Process the data object in the target data processing terminal through the gateway data transmission interface.

[0077] In an embodiment of the present invention, after determining the target data processing terminal, due to different data processing requests, the processing of the data object based on the target data processing terminal is also different. For example, if the data processing request is a storage request, the data object can be sent to the target data processing terminal through the gateway data transmission interface, so that the target data processing terminal stores this data object.

[0078] It should be noted that the data processing terminal can be a terminal device with different data processing functions, including but not limited to storage terminals (memories, storage bins, etc.), processing servers, etc., so as to achieve different data processing functions.

[0079] In another embodiment of the present invention, for further limitation and explanation, as Figure 2 shown, before the step of classifying the data object carried in the data processing request based on the data classification model that has completed training to determine the target data processing terminal of the data object, the method further includes:

[0080] 201. Determine the training data processing terminal from the data processing terminals, and obtain the training sample data in the training data processing terminal;

[0081] 202. Establish a data classification model, and perform model training on the data classification model based on the classification label and the training sample data to obtain a data classification model that has completed training.

[0082] In order to classify data objects based on the data classification model to determine a target data processing terminal from multiple data processing terminals, specifically, at least one training data processing terminal is determined in advance from multiple data processing terminals, and the classification labels of the labeled data objects and the labeled terminal identity information in this training data processing terminal are used as training sample data. Among them, the classification label in the training sample data is determined based on the data access times of each data processing terminal. The classification label is used to mark the data object types corresponding to different data classification types. For example, the medical and health information type corresponding to the video data classification type, and the insurance product evaluation information type corresponding to the model algorithm classification type. At this time, the terminal identifier in the labeled terminal identity is used as the identifier for marking the identity, and the data access times are the number of times the data processing terminal corresponding to this terminal identifier is accessed. The classification network model is trained using the labeled data object classification label, terminal identifier, and data access times as training sample data. Among them, the established data classification model can be a binary classification prediction model or a multi-classification prediction model in machine learning algorithms, such as Multi-class Classification, for model training. In the embodiments of the present invention, the specific process of model training is not specifically limited.

[0083] It should be noted that since it is based on the matching of the classification label of the labeled data object and the data access times of the data processing terminal. For example, the data access times of data processing terminal a exceed the preset threshold of 3, and the labeled data object is Tags tenant, then the classification label is 1. Based on this training sample data, the data classification model is trained, so that when the obtained data classification model classifies tenant 1, it determines that data processing terminal a with a classification label of 1 is processed, thereby constructing the training sample data.

[0084] In another embodiment of the present invention, for further limitation and explanation, as Figure 3 shown, the step of determining the training data processing terminal from the data processing terminals includes:

[0085] 301. Obtain business requirement information.

[0086] 302. Obtain the data processing record information of the data that has been processed in the data processing terminal;

[0087] 303. If the data processing record information matches the service requirement information, determine the data processing terminal as the training data processing terminal.

[0088] In an embodiment of the present invention, in order to accurately find the training data processing terminal storing the training sample data from multiple data processing terminals for accurate model training and improve the accuracy of data processing, first, obtain the service requirement information. The service requirement information includes the data classification types corresponding to data objects in different service scenarios and the data processing types. The data classification types include image data type, audio data type, video data type, and model algorithm data type in the unstructured data type. The data processing types include storage processing type, query processing type, and update processing type. When obtaining the service requirement information, it can be requested and loaded from the application client or the application server, so as to obtain the service requirement data that needs to be matched in the gateway device serving as the current execution end, accurately determine the training data processing terminal, and obtain the classification label of the labeled data object and the labeled terminal identity information in the training data processing terminal as the training sample data. At the same time, obtain the data processing record information of the data that has been processed in each data processing terminal. This data processing record information includes the specific information of storing, querying, and updating historical data objects, so as to match based on the data processing record information and the service requirement information, making the specific information of storing, querying, and updating historical data objects in the selected training data processing terminal applicable to the service requirement information, and thus training a data classification model that matches the service requirement information based on the training sample data in this training data processing terminal.

[0089] In another embodiment of the present invention, for further limitation and explanation, the steps further include:

[0090] Send the service requirement information to the data processing terminal at a preset time interval, and instruct the data processing terminal to perform label marking processing based on the service requirement information;

[0091] Receive the marking indication information of the data processing terminal after completing the label marking processing, and execute the step of determining the training data processing terminal from the data processing terminal to update and train the data classification model.

[0092] To achieve the accuracy of classifying data objects based on a data classification model, in the embodiments of the present invention, service requirement information is sent to a data processing terminal so that the data processing terminal performs label marking processing based on the service requirement information. Among them, the gateway device as the current execution end can request to obtain the latest service requirement information, send the latest service requirement information to the data processing terminal, and instruct the data processing terminal to perform label marking processing according to the service requirement information, so as to update as training sample data. At this time, in order to adaptively update the data classification model, the gateway device as the current execution end can send service requirement information to the data processing terminal at a preset time interval, that is, when the smallest service requirement information is obtained, the service requirement information is transmitted at a preset time interval such as 1 day or 2 days.

[0093] After each data processing terminal completes the label marking processing based on the received service requirement information, it feeds back the marking indication information indicating the completion of the label marking processing to inform the gateway device as the current execution end, so that the gateway device as the current execution end executes the steps of determining the training data processing terminal in the data processing terminal, achieving the purpose of updating and training the data classification model, greatly improving the effectiveness of the data classification model, meeting the data classification capabilities of the gateway device in different service requirement information update scenarios, and performing intelligent data processing.

[0094] In another embodiment of the present invention, for further limitation and illustration, as Figure 4 shown, after the step of establishing a data classification model and training the data classification model based on the classification label and the training sample data to obtain a trained data classification model, the steps further include:

[0095] 401. Obtain the third-party application identifier for data transmission through the gateway data transmission interface, and determine the model verification index matching the third-party application identifier based on the preset model verification relationship;

[0096] 402. If the model verification parameters of the trained data classification model match the model verification index, confirm that the data classification model passes the index verification, and classify the data object through the data classification model.

[0097] To improve the classification processing effectiveness of the data classification model, the gateway device, which is the current execution end, obtains the third-party application identifier, and determines the model verification metrics of the data classification model based on the preset model verification relationship, so as to verify the data classification model. Since different machine learning models have different functions and different verification metrics, the third-party application identifier is the identity identifier of the application client or application server that communicates with the gateway data transmission interface, so as to determine the model verification metrics that match the third-party application identifier, thereby adapting to different application scenarios. Among them, the preset model verification relationship includes the corresponding relationship between multiple third-party application identifiers and multiple model verification metrics. For example, if the third-party application identifier is a health check identifier, then based on the preset model verification relationship, the model verification metric matching the health check identifier is determined to be that the model accuracy is above 80%. After determining the model accuracy, it is judged whether the model verification parameters of the data classification model that has completed training match the model accuracy of above 80%. If they match, it means that the verification is passed, and then this data classification model can be applied to the classification processing, that is, it can be put into use in the gateway device.

[0098] It should be noted that in the embodiment of the present invention, the model verification metric is the index content that the expected machine learning model is preset to reach, including but not limited to the upper or lower limits of training duration, number of training times, model accuracy, model error, etc., so as to compare with the model verification parameters of the data classification model. The model verification parameters are the specific values of training duration, number of training times, model accuracy, model error, etc. calculated according to the data classification model for verification, so as to verify whether the data classification model has the ability of data classification processing and improve the accuracy of data classification.

[0099] In another embodiment of the present invention, for further limitation and explanation, the step of processing the data object in the target data processing terminal through the gateway data transmission interface includes:

[0100] If the data processing request is a data storage request, the data object is output to the target data processing terminal through the gateway data transmission interface, and the target data processing terminal is instructed to store the data content of the data object;

[0101] If the data processing request is a data query request, a data query request for the data object is sent to the target data processing terminal through the gateway data transmission interface, and the target data processing terminal is instructed to query and feedback the data content of the data object;

[0102] If the data processing request is a data update request, the data object is output to the target data processing terminal through the gateway data transmission interface, and the target data processing terminal is instructed to replace and update the data content of the data object.

[0103] In an embodiment of the present invention, in order for the gateway device as the current execution end to perform corresponding data processing for different data processing requests, the data processing requests include data storage requests, data query requests, and data update requests, and corresponding processing operations are performed through the gateway data transmission interface. Specifically, if the data processing request is a data storage request, after determining the target data processing terminal, the data object is output to the target data processing terminal through the gateway data transmission interface to store the data content of the data object. If the data processing request is a data query request, after determining the target data processing terminal, a data query request is sent to the target data processing terminal to find the data content of the data object in the target data processing terminal and feedback it to the current execution end. If the data processing request is a data update request, after determining the target data processing terminal, the data object is output to the target data processing terminal so that the target data processing terminal replaces and updates the data content of the data object.

[0104] It should be noted that in a specific application scenario, one of the multiple pre-defined data processing terminals can be used as the memory for storing training sample data. That is, when data processing is performed on each data processing terminal, the data access times of the target data processing terminal and the corresponding data tags are stored in this memory DB through the gateway data transmission interface, as Figure 5 shown, so that when obtaining the training data processing terminal, the training sample data in the memory DB can be directly retrieved for model training.

[0105] In another embodiment of the present invention, for further limitation and explanation, before the step of outputting the data object to the target data processing terminal through the gateway data transmission interface, the method further includes:

[0106] Obtain the data processing performance parameters of the target data processing terminal;

[0107] If the data performance parameters do not match the gateway device transmission conditions, the binding relationship with the target data processing terminal is released;

[0108] If the data performance parameters match the gateway device transmission conditions, it is instructed to execute the step of outputting the data object to the target data processing terminal through the gateway data transmission interface.

[0109] In an embodiment of the present invention, as Figure 5As shown in the figure, in order to improve the processing accuracy of the data processing terminal, before outputting the data object to the target data processing terminal, it is necessary to determine whether the data processing performance parameters of the target data processing terminal match the transmission conditions of the gateway device to ensure the transmission effectiveness. Among them, the data processing performance parameters include at least one of the storage performance parameter, the query performance parameter, the terminal hardware operation parameter, and the terminal operation line number. The storage performance parameter is the space size for the data processing terminal to store data, the query performance parameter is the speed for the data processing terminal to query data, the terminal hardware operation parameter is the operation environment, operation code type, etc. of the data processing terminal, and the terminal operation line number is the number of threads for data processing in the data processing terminal. Thus, for the data processing performance parameters of different data processing terminals, the transmission conditions of the gateway device are pre-configured. That is, when the data performance parameters do not match the transmission conditions of the gateway device, the binding relationship with the target data processing terminal is released, so that the current gateway device cannot output the data object through the gateway data transmission interface. When the data performance parameters match the transmission conditions of the gateway device, the current gateway device transmits the data object to the target data processing terminal through the gateway data transmission interface. Among them, the gateway data transmission condition is a pre-configured condition for limiting whether the data processing terminal can perform data processing on the data object. For example, the gateway data transmission condition is that the data processing terminal has a high-performance object storage ability. If it is determined to be a normal-performance object storage ability according to the storage performance parameter in the data performance parameters, it does not match, and the current gateway device releases the binding relationship between the gateway data transmission interface and the data processing terminal with the normal-performance object storage ability. The embodiments of the present invention do not make specific limitations.

[0110] It should be noted that since the data processing performance parameters in the data processing terminal change with the business processing scenario, therefore, before each transmission, the judgment of whether the current data performance parameters match the transmission conditions of the gateway device is executed, so as to improve the efficiency of data processing by outputting the data object to the data processing terminal based on the grid device.

[0111] The present invention provides a data processing method based on artificial intelligence. Compared with the prior art, in the embodiments of the present invention, a gateway data transmission interface is configured and at least one data processing terminal is bound. The gateway data transmission interface is used to transmit data objects flowing through the gateway device; a data processing request is received through the gateway data transmission interface, and the data objects carried in the data processing request are classified based on a data classification model that has been trained, and the target data processing terminal of the data object is determined. The training sample data of the data classification model is the classification label for classifying different data objects to the target data processing terminal according to the data access times; the data object is processed in the target data processing terminal through the gateway data transmission interface, so as to realize diversified and secure data processing of multi-object data in different business scenarios, improve the effectiveness of data processing such as data storage and query, reduce the data processing cost, and thus improve the efficiency of business processing based on data storage and query in different business scenarios.

[0112] Further, as an implementation of the method described above Figure 1 The embodiments of the present invention provide a data processing device based on artificial intelligence, as Figure 6 shown. The device includes:

[0113] A configuration module 51, configured to configure a gateway data transmission interface and bind at least one data processing terminal. The gateway data transmission interface is used to transmit data objects flowing through the gateway device;

[0114] A receiving module 52, configured to receive a data processing request through the gateway data transmission interface, and classify the data objects carried in the data processing request based on a data classification model that has been trained, and determine the target data processing terminal of the data object. The training sample data of the data classification model is the classification label for classifying different data objects to the target data processing terminal according to the data access times;

[0115] A processing module 53, configured to process the data object in the target data processing terminal through the gateway data transmission interface.

[0116] Further, the device further includes:

[0117] A determination module, configured to determine a training data processing terminal from the data processing terminals and obtain the training sample data in the training data processing terminal, where the classification label in the training sample data is determined based on the data access times of each data processing terminal;

[0118] A training module for establishing a data classification model and training the data classification model based on the classification labels and the training sample data to obtain a trained data classification model.

[0119] Further, the determination module includes:

[0120] A first acquisition unit for acquiring business requirement information, where the business requirement information includes data classification types and data processing types corresponding to data objects in different business scenarios. The data classification types include image data type, audio data type, video data type, and model algorithm data type in unstructured data types, and the data processing types include storage processing type, query processing type, and update processing type;

[0121] A second acquisition unit for acquiring data processing record information of the data that has been processed in the data processing terminal;

[0122] A determination unit for determining the data processing terminal as a training data processing terminal if the data processing record information matches the business requirement information, so as to obtain the classification labels of the labeled data objects and the labeled terminal identity information in the training data processing terminal as training sample data.

[0123] Further, the determination module further includes:

[0124] A sending unit for sending the business requirement information to the data processing terminal at a preset time interval and instructing the data processing terminal to perform label marking processing based on the business requirement information;

[0125] A receiving unit for receiving the marking indication information of the data processing terminal after completing the label marking processing and performing the step of determining the training data processing terminal from the data processing terminal to update and train the data classification model.

[0126] Further, the determination module is specifically further configured to obtain a third-party application identifier for data transmission through the gateway data transmission interface, and determine a model verification index matching the third-party application identifier based on a preset model verification relationship. The preset model verification relationship includes the corresponding relationship between multiple third-party application identifiers and multiple model verification indexes; if the model verification parameters of the trained data classification model match the model verification index, it is confirmed that the data classification model passes the index verification, so as to classify data objects through the data classification model.

[0127] Further, the processing module includes:

[0128] a first output unit, configured to output the data object to the target data processing terminal through the gateway data transmission interface if the data processing request is a data storage request, and instruct the target data processing terminal to store the data content of the data object;

[0129] A second output unit is configured to send the data query request of the data object to the target data processing terminal through the gateway data transmission interface if the data processing request is a data query request, and instruct the target data processing terminal to query and feedback the data content of the data object;

[0130] The third output unit is used to output the data object to the target data processing terminal through the gateway data transmission interface if the data processing request is a data update request, and instruct the target data processing terminal to replace and update the data content of the data object.

[0131] Furthermore, the device also includes:

[0132] An acquisition module, used to acquire data processing performance parameters of the target data processing terminal, wherein the data processing performance parameters include at least one of storage performance parameters, query performance parameters, terminal hardware operation parameters, and terminal operation line number;

[0133] A release module, configured to release the binding relationship with the target data processing terminal if the data performance parameter does not match the gateway device transmission condition;

[0134] The instruction module is used for instructing to execute the step of outputting the data object to the target data processing terminal through the gateway data transmission interface if the data performance parameter matches the gateway device transmission condition.

[0135] The present invention provides a data processing device based on artificial intelligence. Compared with the prior art, in the embodiments of the present invention, a gateway data transmission interface is configured and at least one data processing terminal is bound. The gateway data transmission interface is used to transmit data objects flowing through the gateway device; a data processing request is received through the gateway data transmission interface, and the data objects carried in the data processing request are classified based on a data classification model that has been trained. The target data processing terminal of the data object is determined. The training sample data of the data classification model is classification labels for classifying different data objects to the target data processing terminal according to the data access times; through the gateway data transmission interface, the data object is processed in the target data processing terminal, so as to realize diversified and secure data processing of multi-object data in different business scenarios, improve the effectiveness of data processing such as data storage and query, reduce the data processing cost, and thus improve the efficiency of business processing based on storing and querying data in different business scenarios.

[0136] According to an embodiment of the present invention, a storage medium is provided. The storage medium stores at least one executable instruction, and the computer executable instruction can execute the data processing method based on artificial intelligence in any of the above method embodiments.

[0137] Figure 7 The structural schematic diagram of a gateway device provided according to an embodiment of the present invention is shown. The specific implementation of the computer device is not limited in the specific embodiments of the present invention.

[0138] As Figure 7 shown, the computer device may include: a processor 602, a communications interface 604, a memory 606, and a communication bus 608.

[0139] Among them: the processor 602, the communications interface 604, and the memory 606 communicate with each other through the communication bus 608.

[0140] The communications interface 604 is used to communicate with network elements of other devices such as clients or other servers.

[0141] The processor 602 is used to execute the program 610, and specifically can execute the relevant steps in the above embodiments of the data processing method based on artificial intelligence.

[0142] Specifically, the program 610 may include program codes, and the program codes include computer operation instructions.

[0143] The processor 602 may be a central processing unit (CPU), or a specific application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computer device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0144] A memory 606 for storing a program 610. The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0145] The program 610 may specifically be used to cause the processor 602 to perform the following operations:

[0146] Configure a gateway data transmission interface and bind at least one data processing terminal, where the gateway data transmission interface is used to transmit data objects flowing through the gateway device;

[0147] Receive a data processing request through the gateway data transmission interface, and classify the data object carried in the data processing request based on a data classification model that has been trained to determine the target data processing terminal of the data object. The training sample data of the data classification model is the classification label for classifying different data objects to the target data processing terminal according to the data access times;

[0148] Process the data object in the target data processing terminal through the gateway data transmission interface.

[0149] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. Thus, the present invention is not limited to any specific combination of hardware and software.

[0150] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A data processing method based on artificial intelligence, characterized in that, Including: Configure a gateway data transmission interface and bind at least one data processing terminal. The gateway data transmission interface is used to transmit data objects flowing through the gateway device. Receive a data processing request through the gateway data transmission interface, and classify the data objects carried in the data processing request based on the trained data classification model to determine the target data processing terminal of the data object. The training sample data of the data classification model is the classification label for classifying different data objects to the target data processing terminal according to the data access times. Process the data object in the target data processing terminal through the gateway data transmission interface. Before classifying the data objects carried in the data processing request based on the trained data classification model to determine the target data processing terminal of the data object, the method further includes: Determine a training data processing terminal from the data processing terminals and obtain the training sample data in the training data processing terminal, where the classification label in the training sample data is determined based on the data access times of each data processing terminal. Establish a data classification model, and perform model training on the data classification model based on the classification label and the training sample data to obtain a trained data classification model. Wherein, determining the training data processing terminal from the data processing terminals includes: Obtain service requirement information, which includes the data classification types and data processing types corresponding to data objects in different service scenarios. The data classification types include image data type, audio data type, video data type, and model algorithm data type in the unstructured data type, and the data processing types include storage processing type, query processing type, and update processing type. Obtain the data processing record information of the data processing terminals that have completed data processing. If the data processing record information matches the service requirement information, determine the data processing terminal as the training data processing terminal, so as to obtain the classification label of the marked data object and the marked terminal identity information in the training data processing terminal as the training sample data.

2. The method according to claim 1, characterized in that, The method further includes: Send the service requirement information to the data processing terminals at a preset time interval, and instruct the data processing terminals to perform label marking processing based on the service requirement information. Receive the marking indication information that the data processing terminals have completed label marking processing, and execute the step of determining the training data processing terminal from the data processing terminals to update the training of the data classification model.

3. The method according to claim 1, characterized in that, After establishing the data classification model and performing model training on the data classification model based on the classification label and the training sample data to obtain a trained data classification model, the method further includes: Obtaining a third-party application identifier for data transmission through the gateway data transmission interface, and determining a model verification indicator matching the third-party application identifier based on a preset model verification relationship, wherein the preset model verification relationship includes a correspondence between multiple third-party application identifiers and multiple model verification indicators; If the model verification parameters of the trained data classification model match the model verification index, it is confirmed that the data classification model has passed the index verification so that the data object can be classified through the data classification model.

4. The method according to any one of claims 1-3, characterized in that, Processing the data object in the target data processing terminal through the gateway data transmission interface includes: If the data processing request is a data storage request, outputting the data object to the target data processing terminal through the gateway data storage interface, and instructing the target data processing terminal to store the data content of the data object; If the data processing request is a data query request, sending the data query request of the data object to the target data processing terminal through the gateway data transmission interface, and instructing the target data processing terminal to query and feedback the data content of the data object; If the data processing request is a data update request, the data object is output to the target data processing terminal through the gateway data transmission interface, and the target data processing terminal is instructed to replace and update the data content of the data object.

5. The method according to claim 4, characterized in that, Before outputting the data object to the target data processing terminal through the gateway data transmission interface, the method further includes: Acquire data processing performance parameters of the target data processing terminal, wherein the data processing performance parameters include at least one of storage performance parameters, query performance parameters, terminal hardware operation parameters, and terminal operation line number; If the data processing performance parameter does not match the gateway device transmission condition, then releasing the binding relationship with the target data processing terminal; If the data processing performance parameter matches the gateway device transmission condition, then the step of outputting the data object to the target data processing terminal through the gateway data transmission interface is instructed to be executed.

6. A data processing device based on artificial intelligence, characterized in that, include: A configuration module, used to configure a gateway data transmission interface and bind at least one data processing terminal, wherein the gateway data transmission interface is used to transmit data objects flowing through the gateway device; A receiving module, used to receive a data processing request through the gateway data transmission interface, and classify the data object carried in the data processing request based on the trained data classification model to determine the target data processing terminal of the data object, wherein the training sample data of the data classification model is a classification label for classifying different data objects into the target data processing terminal according to the number of data accesses; A processing module, used for processing the data object in the target data processing terminal through the gateway data transmission interface; The device also includes: A determination module, configured to determine a training data processing terminal from the data processing terminals, and obtain training sample data in the training data processing terminal, wherein the classification label in the training sample data is determined based on the data access times of each data processing terminal; A training module, configured to establish a data classification model, and perform model training on the data classification model based on the classification label and the training sample data to obtain a trained data classification model; The determination module includes: A first acquisition unit, configured to acquire service requirement information, where the service requirement information includes data classification types and data processing types corresponding to data objects in different service scenarios, and the data classification types include image data type, audio data type, video data type, and model algorithm data type in the unstructured data type, and the data processing types include storage processing type, query processing type, and update processing type; A second acquisition unit, configured to acquire data processing record information of the data that has been processed in the data processing terminal; A determination unit, configured to, if the data processing record information matches the service requirement information, determine the data processing terminal as the training data processing terminal, so as to obtain the classification label of the labeled data object and the labeled terminal identity information in the training data processing terminal as the training sample data.

7. A storage medium, characterized in that, At least one executable instruction is stored in the storage medium, and the executable instruction causes the processor to execute the operations corresponding to the artificial intelligence-based data processing method according to any one of claims 1-5.

8. A gateway device, characterized in that, Including: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the artificial intelligence-based data processing method according to any one of claims 1-5.

Images