Data processing method, electronic equipment and storage medium

Abstract

The invention discloses a data processing method, electronic equipment and a storage medium, and relates to the technical field of data processing.The data processing method comprises the steps that under the condition that first to-be-processed data is stored in a first storage space, whether an artificial intelligence model is in an idle state or not is judged, and if the artificial intelligence model is in the idle state, the first to-be-processed data is stored in the first storage space; if yes, the target number of the artificial intelligence models in the idle state is determined, the target number of second to-be-processed data is extracted from the second storage space according to the preset data priority, and then the target number of target to-be-processed data is extracted from the first to-be-processed data and the second to-be-processed data according to the preset data priority; and respectively inputting the target number of target to-be-processed data into the artificial intelligence model in the idle state, so that the to-be-processed data with relatively high preset data priority is processed in time.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, an electronic device, and a storage medium. Background Art

[0002] With the development of artificial intelligence technology, AI (Artificial Intelligence) models are gradually applied in various business scenarios. For example, in the audit business, some AI models are used to audit data.

[0003] However, these AI models consume a large amount of computing resources and can only process a small amount of data at the same time. In the related art, the data to be processed is written into an MQ (Message Queue) according to the production order, and the AI model processes the data. If the data production speed is faster than the processing speed of the AI model, the data in the MQ accumulates more and more. When the AI model processes the data in the production order of the data in the MQ, the data with higher priority, such as the latest produced data, cannot be processed in time. Summary of the Invention

[0004] In view of the above problems, this application provides a data processing method, an electronic device, and a storage medium, so that the data to be processed with a higher preset data priority can be processed in time. The specific solutions are as follows:

[0005] The first aspect of this application provides a data processing method, including:

[0006] When the first data to be processed is stored in the first storage space, determine whether there is an artificial intelligence model in an idle state;

[0007] If there is an artificial intelligence model in an idle state, determine the target number of artificial intelligence models in the idle state;

[0008] Extract the target number of second data to be processed from the second storage space according to the preset data priority;

[0009] Extract the target number of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority;

[0010] Input the target number of target data to be processed into the artificial intelligence models in the idle state respectively.

[0011] In a possible implementation, after extracting the target number of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority, the data processing method further includes:

[0012] Store the data other than the target data to be processed in the first data to be processed and the second data to be processed in the second storage space.

[0013] In a possible implementation, the data processing method further includes:

[0014] If no artificial intelligence model is in an idle state, store the first data to be processed in the second storage space.

[0015] In a possible implementation, the data processing method further includes:

[0016] Delete the second data to be processed when the storage time of the second data to be processed in the second storage space reaches a threshold.

[0017] In a possible implementation, when the storage time of the second data to be processed in the second storage space reaches a threshold, send the second data to be processed to a manual processing terminal.

[0018] In a possible implementation, the data processing method further includes:

[0019] If no artificial intelligence model is in an idle state, trigger a timing task, which is used to trigger the step of judging whether there is an artificial intelligence model in an idle state after the timing ends.

[0020] In a possible implementation, after inputting the target number of target data to be processed into the artificial intelligence model in an idle state respectively, the data processing method further includes:

[0021] When detecting that the artificial intelligence model switches to an idle state, extract the target data to be processed from the second storage space according to the preset data priority;

[0022] Input the target data to be processed into the artificial intelligence model in an idle state.

[0023] In a possible implementation, the preset data priority is time priority, importance priority, urgency priority, dependency priority, risk priority or user priority.

[0024] The second aspect of the present application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0025] The memory is used to store a computer program;

[0026] The processor is used to execute the computer program so that the electronic device can implement the data processing method of the first aspect or any implementation manner of the first aspect.

[0027] In a third aspect of the present application, a computer storage medium is provided. The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can perform the data processing method in the first aspect or any implementation manner of the first aspect.

[0028] With the above technical solution, in the data processing method provided by the present application, when the first data to be processed is stored in the first storage space, it is determined whether there is an artificial intelligence model in an idle state. If there is an artificial intelligence model in an idle state, the target number of the artificial intelligence models in the idle state is determined. The target number of the second data to be processed is extracted from the second storage space according to the preset data priority, and then the target number of the target data to be processed is extracted from the first data to be processed and the second data to be processed. The target number of the target data to be processed is respectively input into the artificial intelligence models in the idle state, so that the data to be processed with a higher preset data priority can be processed in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the original and elements are not necessarily drawn to scale.

[0030] Figure 1 It is a schematic diagram of a data review process provided by the present application;

[0031] Figure 2 It is a schematic flowchart of a data processing method provided by an embodiment of the present application;

[0032] Figure 3 It is a schematic flowchart of a data processing process provided by an embodiment of the present application;

[0033] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following describes the embodiments of the present application in combination with the drawings in the embodiments of the present application. The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0035] The following describes the embodiments of the present application in combination with the drawings. Those skilled in the art know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0036] In the description, claims and the above-mentioned drawings of this application, terms such as "first" and "second" are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinguishing objects with the same attributes when describing the embodiments of this application. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device including a series of units does not have to be limited to those units, but may include other units not clearly listed or inherent to these processes, methods, products or devices.

[0037] See Figure 1 , Figure 1 is a schematic diagram of a data review process. After multiple business parties produce data to be reviewed, the data to be reviewed is added to the review message queue. According to different data characteristics, the review message queue distributes the data to be reviewed into different queues, and different queues correspond to different computing resources behind. This method is essentially a business trade-off. Because in reality, computing resources cannot meet the review requirements, it is necessary to analyze the data characteristics and quickly process the data that wants to be processed first, and other data continues to queue.

[0038] However, according to the method of planning fast and slow queues based on data characteristics, the data characteristics can only be deduced from historical data. The data in the real business may change. Maybe this business party uploads more PC (Personal Computer) - end videos to be reviewed this week, and next week it becomes more mobile - end videos, which is difficult to accurately estimate. Once there is a deviation in the data estimation, the processing speed in the real business will be inconsistent with the prediction. For example: last week, there were few PC - end videos, which were put into the medium - speed queue and medium computing resources were allocated. Once there are more PC - end videos this week, the medium - speed queue will be blocked. As a result, the data processing speed of the medium - speed queue is even slower than that of the low - speed queue. Similarly, if the PC - end videos are a business that requires high - speed processing, once there are few PC - end videos this week, in fact, most of the computing resources corresponding to the fast - processing queue are idle most of the time. This is essentially a waste of AI computing resources and affects the processing speed of other businesses.

[0039] In addition, different - speed data processing queues are essentially still a first - in - first - out queue. That is to say, in a queue, the computing resources still have to process the old data first and then the new data. When there is a large amount of data in each queue, each computing resource is actually still processing data from a long time ago. This method only alleviates the timeliness problem and does not fundamentally solve the requirements of the review business for timeliness.

[0040] The embodiments of the present application provide a data processing method. The data processing method of the embodiments of the present application will be introduced in detail below with reference to the accompanying drawings.

[0041] Refer to Figure 2 , Figure 2 which is a schematic flowchart of a data processing method provided by the embodiments of the present application. As Figure 2 shown, the data processing method provided by the embodiments of the present application may include steps 201 to 205, and these steps will be described in detail below.

[0042] 201: When the first to-be-processed data is stored in the first storage space, determine whether there is an artificial intelligence model in an idle state;

[0043] After the business party produces the to-be-processed data, the to-be-processed data will be stored in the first storage space. After the to-be-processed data is stored in the first storage space, it is recorded as the first to-be-processed data.

[0044] The first storage space may be a data structure such as a queue, and when the first storage space is a queue, it may be MQ.

[0045] In different application scenarios, the types of to-be-processed data are different. For example: in the video resource review scenario, the to-be-processed data is the video resource to be reviewed; in the text review scenario, the to-be-processed data is the text data to be reviewed.

[0046] The artificial intelligence model is used to process the to-be-processed data.

[0047] There are various methods to determine whether the artificial intelligence model is in an idle state. For example: determine the state of the artificial intelligence model according to the status flag bit of the artificial intelligence model.

[0048] 202: If there is an artificial intelligence model in an idle state, determine the target number of artificial intelligence models in an idle state;

[0049] For example: The system has 5 artificial intelligence models, and 1 artificial intelligence model is running, then the target number of artificial intelligence models in an idle state is 4.

[0050] 203: Extract the target number of second to-be-processed data from the second storage space according to the preset data priority;

[0051] After the to-be-processed data is stored in the second storage space, it is recorded as the second to-be-processed data.

[0052] The second storage space can be a ZSet of Redis. The second storage space can also be other storage spaces, such as a database, etc., which is not specifically limited in this application. Redis is a remote dictionary service, an open-source log-type, Key-Value database written in ANSI C language, supporting networking, and can be memory-based or persistent, and provides APIs in multiple languages. Zset is a collection of strings without duplicate elements. Each member of Zset is associated with a score, and this score is used to sort the members in the set from the lowest score to the highest score. The members of the set are unique, but the scores can be repeated. In this embodiment, the score can be represented by a preset data priority.

[0053] Taking the preset data priority as the time priority as an example, each piece of second data to be processed in the second storage space corresponds to a timestamp, and the timestamp is used to represent the time priority. The closer the timestamp is to the current time, the higher the time priority. If the target quantity is 4, then 4 pieces of second data to be processed with the latest timestamp are extracted from the second storage space.

[0054] 204: Extract the target quantity of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority;

[0055] Taking the quantity of the first data to be processed as 3 and the target quantity as 4 as an example, then 4 pieces of target data to be processed are extracted from the 7 pieces of data composed of the first data to be processed and the second data to be processed according to the preset data priority.

[0056] Still taking the preset data priority as the time priority as an example, then 4 pieces of target data to be processed with the latest timestamp are extracted from the 7 pieces of data composed of the first data to be processed and the second data to be processed. It should be noted that since the target quantity of the second data to be processed and the target quantity of the target data to be processed are both extracted according to the preset data priority, the finally extracted target data to be processed has the highest global preset data priority.

[0057] 205: Input the target quantity of target data to be processed into the artificial intelligence model in the idle state respectively.

[0058] That is, the target quantity of artificial intelligence models in the idle state process the target data to be processed in parallel.

[0059] The data processing method provided in this embodiment, when the first data to be processed is stored in the first storage space, determines whether there is an artificial intelligence model in an idle state. If there is an artificial intelligence model in an idle state, it determines the target number of artificial intelligence models in the idle state, extracts the target number of second data to be processed from the second storage space according to the preset data priority, and then extracts the target number of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority, and inputs the target number of target data to be processed into the artificial intelligence models in the idle state respectively, so that the data to be processed with a higher preset data priority can be processed in a timely manner. When the preset data priority is the time priority, the data to be processed with the latest timestamp can be processed in a timely manner, fundamentally solving the requirement for timeliness in data processing compared to processing according to the production order of data in the first-in first-out queue.

[0060] In addition, each artificial intelligence model is equal. When there are a target number of artificial intelligence models in an idle state, it will extract the target number of target data to be processed with a higher preset data priority and input the target number of target data to be processed into the artificial intelligence models in the idle state, avoiding the waste of AI computing resources.

[0061] In a possible implementation, after extracting the target number of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority, the data other than the target data to be processed in the first data to be processed and the second data to be processed is stored in the second storage space. Taking the number of the first data to be processed as 3 and the target number as 4 as an example, 4 target data to be processed are extracted from the 7 pieces of data composed of the first data to be processed and the second data to be processed according to the preset data priority, and the remaining 3 pieces of data in the 7 pieces of data are stored in the second storage space, instead of being directly discarded, waiting for the next processing, avoiding a large number of requests being simultaneously accessed to the artificial intelligence model, resulting in the collapse of the artificial intelligence model.

[0062] In a possible implementation, if there is no artificial intelligence model in an idle state, the first data to be processed is stored in the second storage space, instead of being directly discarded, waiting for the next processing, avoiding the problem that if the first data to be processed is still stored in the first storage space, and if the first storage space extracts the first data to be processed in the first-in first-out order, the latest produced data cannot be processed in a timely manner.

[0063] In each of the above embodiments, if no artificial intelligence model is in an idle state, a timing task is triggered. The timing task is used to trigger the step of determining whether there is an artificial intelligence model in an idle state after the timing ends. Taking the timing duration of the timing task as 100 ms as an example, if no artificial intelligence model is in an idle state, it is delayed by 100 ms, and the step of re-determining whether there is an artificial intelligence model in an idle state is performed. If no artificial intelligence model is in an idle state, the timing task is continuously triggered; if there is an artificial intelligence model in an idle state, the target quantity of the artificial intelligence models in the idle state is determined, the target quantity of the second data to be processed is extracted from the second storage space according to the pre-set data priority, the target quantity of the second data to be processed is used as the target data to be processed, and the target quantity of the target data to be processed is respectively input into the artificial intelligence models in the idle state, so that the second data to be processed in the second storage space is processed.

[0064] In another possible implementation, after the target quantity of the target data to be processed is respectively input into the artificial intelligence models in the idle state, the artificial intelligence models switch from the idle state to the running state. When the artificial intelligence models output the processing results, the artificial intelligence models switch from the running state to the idle state. In the case where it is detected that the artificial intelligence models switch to the idle state, the target data to be processed is extracted from the second storage space according to the pre-set data priority, and the target data to be processed is input into the artificial intelligence models in the idle state, so as to realize that once it is detected that the artificial intelligence models switch to the idle state, the target data to be processed is allocated to them, ensure that the AI resources are always in use, avoid waste of AI resources, effectively utilize the artificial intelligence models, and enable the data in the second storage space to be processed in a timely manner.

[0065] The following provides a detailed introduction to a data processing method provided by the present application through the following specific examples.

[0066] Please refer to Figure 3The schematic diagram of the data processing flow shown. In this example, the first storage space is the audit message queue, the audit message queue is MQ, the second storage space is ZSet, and the preset data priority is the time priority. After the business party produces the data to be processed, it is stored in the audit message queue, and it is judged whether there is an AI model in an idle state, that is, whether the AI resources are idle. If there is an AI model in an idle state, that is, the AI resources are idle. For example: there are 5 AI models in the AI resources, and now 1 AI model is running and 4 AI models are in an idle state. Then, 4 data with the latest timestamps are extracted from ZSet. If there are 3 data in MQ, then compare the 3 data in MQ with the timestamps of the 4 smallest data extracted from ZSet, and perform AI detection on the 4 data with the latest timestamps to generate AI detection results. The remaining 3 old data are stored in ZSet. ZSet is an ordered set sorted in reverse order according to the timestamps. If the AI resources are not idle, that is, there is no AI model in an idle state, then the 3 data in MQ are stored in ZSet in the order of timestamps, and a timed task is triggered to notify the AI module to execute the detection process after a delay of 100 ms, that is, to judge whether the AI resources are idle.

[0067] The data processing method provided in this example avoids a large number of requests from accessing the AI module simultaneously, resulting in the collapse of the AI module. The data that cannot be processed simultaneously is stored in Zset and sorted by time. It will not be directly discarded, but will be repeatedly judged whether it can be executed through a timed task with a delay of 100 ms. Before performing AI detection, it is judged each time which data is newer between the latest data in MQ and the latest data in Zset, that is: the latest data is processed first to ensure the timeliness of the audit. By using an ordered set like Zset, when judging the latest data each time, only the head of the ordered queue needs to be taken out from the set, that is, the latest N data in the ordered set. Instead of comparing the submission times of the stored data to be audited each time, the comparison speed is accelerated. When MQ receives data, in the case of idle AI resources, the efficient query function of Zset is used to extract the target number of the latest data from Zset, and AI detection is performed on the target number of the latest data among the MQ data and the data extracted from Zset, rather than directly storing the MQ data in the database and performing a full table scan of the latest data when the AI resources are idle, avoiding the waste of computing resources caused by the full table scan of the latest data.

[0068] It should be noted that since the data to be processed with the highest preset data priority is processed when the artificial intelligence model is in an idle state, if the production speed of the data to be processed is greater than the processing speed of the data to be processed, then more and more data with a lower preset data priority will accumulate in the second storage space and cannot be processed. To solve this technical problem, the second data to be processed with a long storage time in the second storage space is processed, and the processing means are different in different application scenarios.

[0069] In a possible implementation, when the storage time of the second data to be processed in the second storage space reaches a threshold, the second data to be processed is deleted, reducing the storage amount of invalid data in the second storage space.

[0070] In another possible implementation, when the storage time of the second data to be processed in the second storage space reaches a threshold, the second data to be processed is sent to a manual processing terminal, and the received second data to be processed is processed manually at the manual processing terminal, so that the data with a lower preset data priority is processed.

[0071] It should also be noted that in different application scenarios, the preset data priority can be different and can be flexibly configured. Exemplarily, the preset data priority is time priority, importance priority, urgency priority, dependency priority, risk priority, or user priority.

[0072] Among them, in the time priority, the newer the timestamp, the higher the time priority;

[0073] The data to be processed is divided into importance priorities according to the degree of importance;

[0074] The data to be processed is divided into urgency priorities according to the degree of urgency;

[0075] The dependency priority is divided according to the dependency relationship between the data to be processed;

[0076] The risk priority is divided according to the possible risk degree of the data to be processed. The higher the risk degree, the higher the risk priority and the more it needs to be processed first;

[0077] The user priority is divided according to the user identity and permissions corresponding to the data to be processed.

[0078] The above introduces a data processing method provided by an embodiment of the present application. The following will introduce the device for executing the above data processing method.

[0079] A data processing device provided by an embodiment of the present application includes:

[0080] A judgment unit, configured to judge whether there is an artificial intelligence model in an idle state when the first data to be processed is stored in the first storage space;

[0081] A determination unit, configured to determine the target number of artificial intelligence models in an idle state if there is an artificial intelligence model in an idle state;

[0082] A first extraction unit, configured to extract the target number of second data to be processed from the second storage space according to the preset data priority;

[0083] A second extraction unit, configured to extract the target quantity of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority;

[0084] An input unit, configured to respectively input the target quantity of target data to be processed into an artificial intelligence model in an idle state.

[0085] In a possible implementation, the data processing device further includes:

[0086] A first storage unit, configured to store the data other than the target data to be processed in the first data to be processed and the second data to be processed into the second storage space after extracting the target quantity of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority.

[0087] In a possible implementation, the data processing device further includes:

[0088] A second storage unit, configured to store the first data to be processed into the second storage space if no artificial intelligence model is in an idle state.

[0089] In a possible implementation, the data processing device further includes:

[0090] A deletion unit, configured to delete the second data to be processed when the storage time of the second data to be processed in the second storage space reaches a threshold.

[0091] In a possible implementation, the data processing device further includes:

[0092] A sending unit, configured to send the second data to be processed to a manual processing terminal when the storage time of the second data to be processed in the second storage space reaches a threshold.

[0093] In a possible implementation, the data processing device further includes:

[0094] A triggering unit, configured to trigger a timing task if no artificial intelligence model is in an idle state, and the timing task is used to trigger the judgment unit after timing ends.

[0095] In a possible implementation, the data processing device further includes:

[0096] A third extraction unit, configured to, after respectively inputting the target quantity of target data to be processed into an artificial intelligence model in an idle state, extract the target data to be processed from the second storage space according to the preset data priority when it is detected that the artificial intelligence model switches to an idle state.

[0097] This embodiment discloses a data processing device. When the first data to be processed is stored in the first storage space, it is determined whether there is an artificial intelligence model in an idle state. If there is an artificial intelligence model in an idle state, the target number of the artificial intelligence models in the idle state is determined. By extracting the target number of second data to be processed from the second storage space according to the preset data priority, and then extracting the target number of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority, the target number of target data to be processed is respectively input into the artificial intelligence models in the idle state, so that the data to be processed with a higher preset data priority can be processed in time.

[0098] An electronic device is also provided in an embodiment of the present application. Refer to Figure 4 As shown, it shows a schematic structural diagram of an electronic device suitable for implementing the electronic device in the embodiment of the present application.

[0099] As Figure 4 shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage device 408 into the random access memory (RAM) 403. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0100] Generally, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a memory card, a hard disk, etc.; and a communication device 409. The communication device 409 can allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 shows an electronic device having various devices, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be alternatively implemented or had.

[0101] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions run on the electronic device, the electronic device implements any one of the data processing methods provided in the embodiment of the present application.

[0102] In an embodiment of the present application, a computer-readable storage medium is further provided. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any data processing method provided in the embodiment of the present application.

[0103] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided in the present application, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0104] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be various, such as analog circuits, digital circuits or dedicated circuits. However, for the present application, in more cases, software program implementation is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disc of a computer, etc., and includes several instructions for causing a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0105] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0106] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center by wired (such as coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a training device or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

Claims

1. A data processing method, characterized in that, Including: When storing first data to be processed in a first storage space, determining whether there is an artificial intelligence model in an idle state; If there is an artificial intelligence model in an idle state, determining a target quantity of the artificial intelligence models in the idle state; Extracting the target quantity of second data to be processed from a second storage space according to a preset data priority; Extracting the target quantity of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority; Inputting the target quantity of target data to be processed into the artificial intelligence models in the idle state respectively.

2. The data processing method according to claim 1, characterized in that, After extracting the target quantity of target data to be processed from the first data to be processed and the second data to be processed according to the preset data priority, the data processing method further includes: Storing the data other than the target data to be processed in the first data to be processed and the second data to be processed into the second storage space.

3. The data processing method according to claim 1, characterized in that, The data processing method further includes: If there is no artificial intelligence model in an idle state, storing the first data to be processed into the second storage space.

4. The data processing method according to any one of claims 1 - 3, characterized in that, The data processing method further includes: When the storage time of the second data to be processed in the second storage space reaches a threshold, deleting the second data to be processed.

5. The data processing method according to any one of claims 1 - 3, characterized in that, The data processing method further includes: When the storage time of the second data to be processed in the second storage space reaches a threshold, sending the second data to be processed to a manual processing terminal.

6. The data processing method according to claim 1 or 3, characterized in that, The data processing method further includes: If there is no artificial intelligence model in an idle state, triggering a timing task, where the timing task is used to trigger the execution of the step of determining whether there is an artificial intelligence model in an idle state after the timing ends.

7. The data processing method according to claim 1, characterized in that, After inputting the target quantity of target data to be processed into the artificial intelligence models in the idle state respectively, the data processing method further includes: When detecting that an artificial intelligence model switches to an idle state, extracting the target data to be processed from the second storage space according to the preset data priority; Inputting the target data to be processed into the artificial intelligence model in the idle state.

8. The data processing method according to claim 1, characterized in that, The preset data priority is time priority, importance priority, urgency priority, dependency priority, risk priority or user priority.

9. An electronic device, characterized in that, Including at least one processor and a memory connected to the processor, where: The memory is used for storing a computer program; The processor is used for executing the computer program so that the electronic device can implement the data processing method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the data processing method according to any one of claims 1 to 8.

Images