Sample data preprocessing method and device, equipment and storage medium
By dividing it into parallel and batch areas in sample data preprocessing, and performing preprocessing steps separately using CPU and GPU, the problem of low sample data preprocessing efficiency in the prior art is solved, and a more efficient model training process is realized.
Patent Information
- Application Number
- CN202310699479.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-13
- Publication Date
- 2024-12-13
AI Technical Summary
In the prior art, sample data preprocessing efficiency is low, and multiple preprocessing steps are required to be performed separately for each sample data, resulting in inefficiency.
By displaying the preprocessing execution diagram, it is divided into a parallel execution area and a batch execution area. The CPU and GPU respectively execute the preprocessing steps of parallel and batch processing, giving full play to the advantages of computing resources.
It improves the efficiency of sample data preprocessing, reduces redundant calls to CPU and GPU resources, and improves the efficiency of model training.
Smart Images
Figure CN119127456A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning, and in particular to a method, device, equipment and storage medium for preprocessing sample data. Background Art
[0002] With the development of computer technology, the application scenarios of artificial intelligence are becoming more and more abundant. For example, face recognition and voice recognition are scenarios that we often encounter in life. The functions of artificial intelligence are often realized through a model. The training process of the model requires the use of a large amount of sample data. In order to improve the effect of model training, the sample data often needs to be preprocessed.
[0003] In the related art, preprocessing usually includes multiple preprocessing steps, and each sample data needs to be individually executed with the multiple preprocessing steps, resulting in low efficiency in preprocessing the sample data. Summary of the invention
[0004] The embodiments of the present application provide a sample data preprocessing method, device, equipment, and storage medium, which can improve the efficiency of preprocessing sample data. The technical solution is as follows:
[0005] In one aspect, a method for preprocessing sample data is provided, the method comprising:
[0006] In response to a preprocessing instruction for a plurality of sample data of a target type, a preprocessing execution graph corresponding to the target type is displayed, the preprocessing execution graph is used to indicate a flow of executing a plurality of preprocessing steps, the preprocessing execution graph includes a first execution area and a second execution area, the preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner;
[0007] In response to a confirmation instruction of the preprocessing execution graph, calling the CPU and the GPU to execute the plurality of preprocessing steps on the plurality of sample data according to the process indicated by the preprocessing execution graph;
[0008] The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area.
[0009] In one aspect, a sample data preprocessing device is provided, the device comprising:
[0010] A display module, configured to display a preprocessing execution diagram corresponding to a target type in response to a preprocessing instruction for a plurality of sample data of a target type, wherein the preprocessing execution diagram is used to indicate a flow of executing a plurality of preprocessing steps, and the preprocessing execution diagram includes a first execution area and a second execution area, wherein the preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner;
[0011] A calling module, configured to respond to a confirmation instruction of the preprocessing execution graph and call the CPU and the GPU to execute the plurality of preprocessing steps on the plurality of sample data according to the process indicated by the preprocessing execution graph;
[0012] The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area.
[0013] On the one hand, a computer device is provided, which includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the sample data preprocessing method.
[0014] On the one hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the sample data preprocessing method.
[0015] On the one hand, a computer program product or a computer program is provided, which includes a program code, and the program code is stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computer device performs the above-mentioned sample data preprocessing method.
[0016] Through the technical solution provided by the embodiment of the present application, in response to the preprocessing instruction for multiple sample data of the target type, the preprocessing execution graph corresponding to the target type is displayed, and the preprocessing execution graph is used to indicate the process of executing multiple preprocessing steps, that is, to indicate the execution method and execution order of multiple preprocessing steps. The preprocessing execution graph includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch manner. In response to the confirmation instruction of the preprocessing execution graph, according to the process indicated by the preprocessing execution graph, the CPU and GPU are called to execute the multiple preprocessing steps on the multiple sample data, so as to make full use of the CPU and GPU and improve the efficiency of preprocessing the sample data. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 It is a schematic diagram of an implementation environment of a sample data preprocessing method provided in an embodiment of the present application;
[0019] Figure 2 is a flow chart of a sample data preprocessing method provided in an embodiment of the present application;
[0020] Figure 3 is a flow chart of another sample data preprocessing method provided in an embodiment of the present application;
[0021] Figure 4 This is a flowchart of initial preprocessing of an image provided by an embodiment of the present application;
[0022] Figure 5 is a flowchart of another sample data preprocessing method provided in an embodiment of the present application;
[0023] Figure 6 is a flowchart of another sample data preprocessing method provided in an embodiment of the present application;
[0024] Figure 7 is a structural schematic diagram of a sample data preprocessing device provided in an embodiment of the present application;
[0025] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0027] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on quantity and execution order.
[0028] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0029] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge sub-models to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.
[0030] Computer Vision (CV): Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further do graphic processing to make the computer process images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain "information" from images or multidimensional data.
[0031] Preprocessing: refers to some processing of data before the main processing. In the embodiment of the present application, preprocessing refers to the process of processing sample data used for model training to improve the model training effect.
[0032] Batch processing: Batch processing is the processing of a certain object in batches. In the embodiment of the present application, it refers to the process of performing the same preprocessing step on multiple sample data at the same time.
[0033] CPU (Central Processing Unit): It is one of the main devices of electronic computers and the core component of computers. Its main function is to interpret computer instructions and process data in computer software. CPU is the core component of the computer responsible for reading instructions, decoding instructions and executing instructions. The central processing unit mainly consists of two parts, namely the controller and the arithmetic unit, which also includes a cache memory and a data and control bus that realizes the connection between them. The three core components of electronic computers are CPU, internal memory, and input / output devices. The functions of the central processing unit are mainly to process instructions, execute operations, control time, and process data.
[0034] GPU (Graphics Processing Unit): A graphics processor that uses graphics processing tasks to calculate general computing tasks that were originally handled by the central processing unit. These general computing tasks often have nothing to do with graphics processing. Due to the powerful parallel processing capabilities and programmable pipelines of modern graphics processors, stream processors can process non-graphics data.
[0035] Computational graph: A computational graph is a data structure used to represent the computational process. Specifically, a computational graph is a graph consisting of a set of nodes and edges, where nodes represent computational units and edges represent data flows.
[0036] Figure 1 is a schematic diagram of an implementation environment of a sample data preprocessing method provided in an embodiment of the present application, see Figure 1 , including a computer device 100, the computer device 100 includes a CPU 101 and a GPU 102. In an embodiment of the present application, the CPU 101 is used to process preprocessing steps executed in parallel, and the GPU 102 is used to process preprocessing steps executed in batch mode. In some embodiments, the CPU 101 includes multiple cores, each of which can independently complete calculations, and the embodiment of the present application does not limit the number of cores in the CPU 101. The GPU 102 includes multiple computing units, each of which can independently complete calculations, and the embodiment of the present application does not limit the number of computing units in the GPU 102.
[0037] After introducing the implementation environment of the embodiment of the present application, the preprocessing method of the sample data provided by the embodiment of the present application will be described in combination with the above-mentioned implementation environment. In the following description, the CPU is CPU101 in the above-mentioned implementation environment, and the GPU is GPU102 in the above-mentioned implementation environment as an example.
[0038] The technical solution provided in the embodiments of the present application can be applied to scenarios of data preprocessing before training various models, for example, it can be applied to scenarios of preprocessing sample images before training image recognition models, or it can be applied to scenarios of preprocessing sample audio before training speech recognition models, or it can be applied to scenarios of preprocessing sample text before training text recognition models, or it can be applied to scenarios of preprocessing sample videos before training video recognition models.
[0039] Taking the application of the technical solution provided in the embodiment of the present application in the scenario of preprocessing sample images before training the image recognition model as an example, after adopting the technical solution provided in the embodiment of the present application, in response to the preprocessing instruction for multiple sample images, the preprocessing execution diagram corresponding to the image is displayed, and the preprocessing execution diagram is used to indicate the process of executing multiple preprocessing steps. The preprocessing execution diagram includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner. Each sample image will execute the multiple preprocessing steps to complete the preprocessing. In response to the confirmation instruction of the preprocessing execution diagram, according to the process indicated by the preprocessing execution diagram, the CPU is called to execute the preprocessing steps in the first execution area for the multiple sample data, and the GPU is called to execute the preprocessing steps in the second execution area for the multiple sample data, that is, the CPU is called to execute the preprocessing steps that need to be executed in parallel, and the GPU is called to execute the preprocessing steps that need to be executed sequentially in a batch processing manner, so as to give full play to the capabilities of the CPU and GPU and improve the efficiency of preprocessing multiple sample images.
[0040] The above is an example of the application of the technical solution provided in the embodiment of the present application in the scenario of preprocessing the sample image. The application methods in the above other scenarios belong to the same inventive concept as the above description. The implementation process is referred to the above description and will not be repeated here. Of course, in addition to the above described scenarios, it can also be applied in other scenarios.
[0041] After introducing the implementation environment and application scenarios of the embodiments of the present application, the following describes the sample data preprocessing method provided in the embodiments of the present application. Figure 2 Taking the execution subject as an electronic device as an example, the method includes the following steps.
[0042] 201. In response to a preprocessing instruction for a plurality of sample data of a target type, the electronic device displays a preprocessing execution diagram corresponding to the target type.
[0043] Among them, the preprocessing execution diagram is used to indicate the process of executing multiple preprocessing steps, and the preprocessing execution diagram includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner. Parallel execution refers to the synchronous execution of multiple preprocessing steps, and the multiple preprocessing steps are independent of each other, or not dependent on each other, that is, the execution of each preprocessing step in the multiple preprocessing steps does not depend on the execution results of other preprocessing steps. Execution in a batch processing manner means that the same preprocessing step is executed on multiple sample data at the same time. The multiple preprocessing steps are distributed in the first execution area and the second execution area. The target type is set by the technician according to the actual situation, that is, it is determined according to the type of model to be trained. In some embodiments, the target type is any one of image, audio, text and video. Sample data is data used for model training. The preprocessing instructions for multiple sample data are triggered by the preprocessing operation of the technician, or automatically triggered after meeting certain conditions, and the embodiments of the present application are not limited to this.
[0044] 202. In response to a confirmation instruction for the preprocessing execution graph, the electronic device calls the CPU and the GPU to execute the plurality of preprocessing steps on the plurality of sample data according to the process indicated by the preprocessing execution graph.
[0045] The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area. That is, the CPU is called to execute multiple preprocessing steps in the first execution area in parallel, and the GPU is called to execute multiple preprocessing steps in the second execution area in a batch processing manner, giving full play to the performance of the CPU and GPU. The confirmation instruction is triggered by the confirmation operation performed by the technician.
[0046] Through the technical solution provided by the embodiment of the present application, in response to the preprocessing instruction for multiple sample data of the target type, the preprocessing execution graph corresponding to the target type is displayed, and the preprocessing execution graph is used to indicate the process of executing multiple preprocessing steps, that is, to indicate the execution method and execution order of multiple preprocessing steps. The preprocessing execution graph includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch manner. In response to the confirmation instruction of the preprocessing execution graph, according to the process indicated by the preprocessing execution graph, the CPU and GPU are called to execute the multiple preprocessing steps on the multiple sample data, so as to make full use of the CPU and GPU and improve the efficiency of preprocessing the sample data.
[0047] It should be noted that the above steps 201-202 are a simple description of the technical solution provided in the embodiment of the present application. The technical solution provided in the embodiment of the present application will be more clearly described below with reference to some examples. Figure 3 Taking the execution subject as an electronic device as an example, the method includes the following steps.
[0048] 301. The electronic device obtains a preprocessing execution graph of a target type.
[0049] Among them, the target type is the type of sample data, and the target type is set by the technician according to the actual situation, that is, it is determined according to the type of model to be trained. The preprocessing execution diagram is used to indicate the process of executing multiple preprocessing steps, that is, it is used to indicate the execution method and execution order of the multiple preprocessing steps. The multiple preprocessing steps are used to realize the preprocessing of the sample data of the target type, that is, the complete preprocessing process includes the execution of the multiple preprocessing steps. The preprocessing execution diagram includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner. Parallel execution refers to the synchronous execution of multiple preprocessing steps, and the multiple preprocessing steps are independent of each other, or not dependent on each other, that is, the execution of each preprocessing step in the multiple preprocessing steps does not depend on the execution results of other preprocessing steps. Execution in a batch processing manner means that the same preprocessing step is executed on multiple sample data at the same time. The multiple preprocessing steps are distributed in the first execution area and the second execution area.
[0050] In a possible implementation, the electronic device obtains an initial preprocessing flow of the target type, and the initial preprocessing flow includes an initial execution order of the multiple preprocessing steps. The electronic device divides the multiple preprocessing steps into a first preprocessing step set and a second preprocessing step set based on the types of the preprocessing steps, the first preprocessing step set corresponds to parallel execution, and the second preprocessing step set corresponds to sequential execution in a batch processing manner. The electronic device generates the preprocessing execution graph corresponding to the target type based on the initial execution order, the first preprocessing step set, and the second preprocessing step set.
[0051] Among them, the initial preprocessing flow is the initial execution order of the multiple preprocessing steps. In the initial preprocessing flow, the multiple preprocessing steps are executed in sequence, that is, one preprocessing step is executed after another preprocessing step is completed, and the execution of each preprocessing step depends on the completion of the previous preprocessing step. The types of preprocessing steps include the first type and the second type. The first type of preprocessing steps are preprocessing steps that do not depend on the execution results of other preprocessing steps when they are executed. For example, when the target type is an image, the execution of the two preprocessing steps of generating noise and generating a mask are not mutually dependent, and generating noise and generating a mask are two first type preprocessing steps. Correspondingly, the second type of preprocessing steps are preprocessing steps that depend on the execution results of other preprocessing steps when they are executed. For example, when the target type is an image, the execution of the preprocessing step of adding noise depends on the execution result of the preprocessing step of generating noise, and adding noise is a second type of preprocessing step. The preprocessing steps in the first preprocessing step set are preprocessing steps of the first type, and the preprocessing steps in the second preprocessing step set are preprocessing steps of the second type.
[0052] In this implementation, an initial preprocessing flow of the target type is obtained, and based on the types of each preprocessing step in the initial preprocessing flow, multiple preprocessing steps are divided into a first preprocessing step set and a second preprocessing step set, wherein the first preprocessing step set corresponds to parallel execution, and the second preprocessing step set corresponds to sequential execution in a batch processing manner. Based on the initial execution order, the first preprocessing step set and the second preprocessing step set, the preprocessing execution graph corresponding to the target type is generated to achieve rearrangement of the multiple preprocessing steps, which helps to improve the execution efficiency of the multiple preprocessing steps.
[0053] For example, the electronic device obtains an initial preprocessing flow of the target type, and the initial preprocessing flow includes an initial execution order of the multiple preprocessing steps. The electronic device adds the preprocessing steps of the first type among the multiple preprocessing steps to the first preprocessing step set, and adds the preprocessing steps of the second type among the multiple preprocessing steps to the second preprocessing step set. The electronic device sets the multiple first preprocessing steps in the first preprocessing step set to be executed in parallel to form the first execution area. Based on the initial execution order, the electronic device sets the execution order of the multiple second preprocessing steps in the second preprocessing step set to form the second execution area. The electronic device generates the preprocessing execution graph based on the first execution area and the second execution area. In some embodiments, the execution order of the preprocessing steps in the first execution area is earlier than the execution order of the preprocessing steps in the second execution area.
[0054] The following will take the target type as image, combined with Figure 4 and Figure 5 The above-mentioned embodiment is described.
[0055] See also Figure 4 , shows an initial preprocessing process for preprocessing an image, the initial preprocessing process includes multiple preprocessing steps, and the multiple preprocessing steps are executed in sequence. The initial preprocessing process includes sampling image A to obtain images B to X. Then, the same preprocessing steps are performed on image A and images B to X, including adding black level, scaling, blurring, adding noise, format conversion, and adding masks. During the execution process, it is necessary to first generate images B to X based on image A, and then perform the above complete preprocessing steps on image A to complete the preprocessing of image A. After completing the preprocessing of image A, perform the above complete preprocessing steps on image B to complete the preprocessing of image B, and so on.
[0056] See also Figure 5 After the initial preprocessing process is processed by the above-mentioned implementation method, a preprocessing execution graph can be formed, that is, the preprocessing steps that can be executed in parallel are set in the first execution area, and the preprocessing steps that are executed sequentially in batch mode are set in the second execution area. The first execution area is also called the parallel execution graph area, and the second execution area is also called the batched execution graph area. For example, the steps of sampling image A to obtain images B to X and fusing image A with images B to X are set in the first execution area, and the steps of generating black levels, creating noise, and creating masks for different images are also set in the first execution area. The steps of adding the generated black level, scaling, blurring and adding the generated noise, format conversion, and adding the generated mask in image A and images B to X are set in the second execution area.
[0057] Figure 5 The corresponding logical graphical configuration is shown in the following code:
[0058] {$imag----------------------------------I
[0059] $imag->sampling(random_seed)-I->concat(dim)
[0060] $imag->sampling(random_seed)-I->add->resize(h,w,type)->blur(type)
[0061] create_black_level I----------------------------->add
[0062] create noise(random seed)->---------------------------------------RGB2GBRG-
[0063] I->mul
[0064] create mask(randomseed)----------------------------------------------------------------
[0065] }.
[0066] Each line in the logical graph configuration represents an execution flow. "$" represents a variable, the variable in parentheses represents the parameter passed into the operator, and -> represents the execution direction. imag represents image, sampling represents sampling, random_seed represents random sampling, concat represents fusion, add represents addition, resize represents scaling, blur represents blurring, create represents creation, black_level represents black level, noise represents noise, and mask represents mask.
[0067] In addition to the above implementation, the present application embodiment also provides another implementation of the above step 301.
[0068] In a possible implementation, the electronic device performs a query based on the target type to obtain a preprocessing execution graph of the target type.
[0069] The previous implementation is a way of executing step 301 when there is no preprocessing execution graph of the target type, and this implementation is a way of executing step 301 when there is a preprocessing execution graph of the target type.
[0070] It should be noted that the above steps are optional steps. The electronic device may first execute the above step 301 and then execute the following step 302, or may directly execute the following step 302. The embodiment of the present application does not limit this.
[0071] 302. In response to a preprocessing instruction for a plurality of sample data of a target type, the electronic device displays a preprocessing execution diagram corresponding to the target type.
[0072] The target type is any one of image, audio, text and video. Sample data is data used for model training. The preprocessing instructions for multiple sample data are triggered by the preprocessing operation of the technician, or automatically triggered after meeting certain conditions, which is not limited in the embodiments of the present application.
[0073] In a possible implementation, in response to a preprocessing operation on a plurality of sample data of a target type, the electronic device triggers a preprocessing instruction on the plurality of sample data. In response to the preprocessing instruction, the electronic device displays a preprocessing execution diagram corresponding to the target type.
[0074] Under this implementation, by executing the preprocessing operation on the multiple sample data, the preprocessing instructions for the multiple sample data can be triggered, thereby controlling the electronic device to display the preprocessing execution diagram, making it convenient for technicians to view the process of preprocessing the multiple sample data and realize the visualization of the preprocessing process.
[0075] Optionally, after step 302, the electronic device may perform the following step 303 or 304 according to actual conditions, which is not limited in this embodiment of the present application.
[0076] 303. In response to a confirmation instruction for the preprocessing execution graph, the electronic device calls the CPU and the GPU to execute the multiple preprocessing steps on the multiple sample data according to the process indicated by the preprocessing execution graph.
[0077] The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area. That is, the CPU is called to execute multiple preprocessing steps in the first execution area in parallel, and the GPU is called to execute multiple preprocessing steps in the second execution area in a batch processing manner, giving full play to the performance of the CPU and GPU. The confirmation instruction is triggered by the confirmation operation performed by the technician.
[0078] In a possible implementation, the execution order of the preprocessing steps in the first execution region is earlier than that of the preprocessing steps in the second execution region, and in response to a confirmation instruction of the preprocessing execution graph, the electronic device calls the CPU to execute the preprocessing steps in the first execution region in parallel on the multiple sample data. The electronic device calls the GPU to execute the preprocessing steps in the second execution region sequentially on the multiple sample data in a batch processing manner.
[0079] In this embodiment, in response to a confirmation instruction for the preprocessing execution graph, the CPU and GPU are called to execute the preprocessing steps in the first execution area and the second execution area in different ways, respectively, so as to give full play to the performance of the CPU and GPU and improve the efficiency of preprocessing multiple sample data.
[0080] In order to explain the above implementation more clearly, the above implementation will be explained in two parts below.
[0081] The first part, in response to a confirmation instruction of the preprocessing execution diagram, the electronic device calls the CPU to execute the preprocessing steps in the first execution area in parallel on the plurality of sample data.
[0082] In a possible implementation, the first execution area includes a plurality of first preprocessing steps. For any sample data among the plurality of sample data, in response to a confirmation instruction of the preprocessing execution diagram, the electronic device calls the CPU to synchronously execute the plurality of first preprocessing steps on the sample data.
[0083] The multiple first preprocessing steps are independent of each other, or are not dependent on each other, that is, the execution of each first preprocessing step in the multiple first preprocessing steps does not depend on the execution results of other first preprocessing steps.
[0084] In this implementation, the CPU is called to synchronously execute the multiple first preprocessing steps on the sample data, changing the original sequential execution into parallel execution, giving full play to the computing power of the CPU and improving the efficiency of preprocessing the sample data.
[0085] For example, the first execution area includes a plurality of first preprocessing steps. For any sample data among the plurality of sample data, in response to a confirmation operation on the preprocessing execution graph, the electronic device triggers a confirmation instruction on the preprocessing execution graph. In response to the confirmation instruction on the preprocessing execution graph, the electronic device calls multiple cores of the CPU through the graph execution engine to synchronously execute the plurality of first preprocessing steps on the sample data.
[0086] In the second part, the electronic device calls the GPU to sequentially execute the preprocessing steps in the second execution area on the multiple sample data in a batch processing manner.
[0087] In a possible implementation, the second execution area includes a plurality of second preprocessing steps, and for any second preprocessing step among the plurality of second preprocessing steps, the electronic device calls the GPU to synchronously execute the second preprocessing step on the plurality of sample data.
[0088] The execution of the second preprocessing step depends on the execution results of other preprocessing steps, and the other preprocessing steps mentioned here include both the first preprocessing step and the second preprocessing step.
[0089] In this implementation, the GPU is called to synchronously execute the second preprocessing step on multiple sample data, thereby achieving synchronous preprocessing of multiple sample data and improving the efficiency of preprocessing the sample data.
[0090] For example, the second execution area includes multiple second preprocessing steps. For any second preprocessing step among the multiple second preprocessing steps, the electronic device calls the GPU to call the operator corresponding to the second preprocessing step, and the multiple computing units of the GPU execute the second preprocessing step on the multiple sample data based on the operator, thereby giving full play to the computing resource advantages of the GPU.
[0091] In the above implementation, the CPU can synchronously execute multiple first preprocessing steps on one sample data, and the GPU can synchronously execute one second preprocessing step on multiple sample data, which gives full play to the computing advantages of the CPU and GPU and improves the efficiency of preprocessing the sample data. In addition, the CPU and GPU can perform operations simultaneously during the preprocessing of multiple sample data, and the overlap of operation time will also improve the efficiency of preprocessing.
[0092] After step 303, the electronic device can use the preprocessed plurality of sample data to perform model training.
[0093] 304. In response to an adjustment instruction for the preprocessing execution graph, the electronic device adjusts the preprocessing steps in the first execution area and / or the second execution area in a manner indicated by the adjustment instruction to update the preprocessing execution graph.
[0094] In a possible implementation, when the adjustment instruction indicates to add a preprocessing step in the first execution area or the second execution area, in response to the adjustment instruction, the electronic device adds a preprocessing step corresponding to the adjustment instruction in the first execution area or the second execution area.
[0095] The preprocessing steps newly added in the first execution region will be executed in parallel, and the preprocessing steps newly added in the second execution region will be executed in batch processing.
[0096] In this implementation, a preprocessing step can be added to the first execution region or the second execution region by adjusting the preprocessing execution graph, and the efficiency of the added preprocessing step is relatively high.
[0097] The above implementation is described below through two examples.
[0098] Example 1: When the adjustment instruction indicates to add a preprocessing step in the first execution area, in response to the adjustment instruction, the electronic device adds a preprocessing step corresponding to the adjustment instruction in the first execution area. Since the preprocessing steps in the first execution area are executed in parallel and there is no sequential execution order, the preprocessing step can be directly added to the first execution area.
[0099] Example 2: When the adjustment instruction indicates to add a preprocessing step in the second execution region, in response to the adjustment instruction, the electronic device determines the execution order of the preprocessing steps corresponding to the adjustment instruction in the second execution region, and adds the preprocessing step at a position corresponding to the execution order.
[0100] In a possible implementation, when the adjustment instruction indicates to delete the preprocessing step in the first execution area or the second execution area, in response to the adjustment instruction, the electronic device deletes the preprocessing step corresponding to the adjustment instruction in the first execution area or the second execution area.
[0101] In this implementation mode, the preprocessing steps in the preprocessing execution graph can be quickly deleted by adjusting the instructions, which is highly efficient.
[0102] In a possible implementation, when the adjustment instruction indicates adjusting the execution order of any preprocessing step in the first execution area or the second execution area, in response to the adjustment instruction, the electronic device adjusts the execution order of the preprocessing step in the first execution area or the second execution area.
[0103] In this implementation, the execution order of the preprocessing steps in the preprocessing execution graph can be adjusted by adjusting the instructions, which is more efficient.
[0104] In a possible implementation, when the adjustment instruction indicates adjusting the execution order of any preprocessing step in the preprocessing execution diagram, in response to the adjustment instruction, the electronic device adjusts the preprocessing step from the first execution area to the second execution area, or adjusts the preprocessing step from the second execution area to the first execution area.
[0105] Among them, adjusting the preprocessing step from the first execution area to the second execution area, or adjusting the preprocessing step from the second execution area to the first execution area is to adjust the execution mode of the preprocessing step.
[0106] In this implementation mode, the execution mode of the preprocessing steps in the preprocessing execution diagram can be adjusted by adjusting the instructions, which is more efficient.
[0107] Through step 304, the preprocessing process is displayed in the form of a diagram, which can not only facilitate the sorting out of the processing process and improve the processing efficiency, but also configure the flow chart to facilitate the adjustment of the preprocessing steps, facilitate experiments, and improve the development efficiency of technicians.
[0108] 305. In response to a confirmation instruction for the updated preprocessing execution graph, the electronic device calls the CPU and the GPU to preprocess the multiple sample data according to the process indicated by the updated preprocessing execution graph.
[0109] The preprocessing process described in step 305 and the preprocessing process described in step 303 belong to the same inventive concept. For the implementation process, please refer to the relevant description of step 303 and will not be repeated here.
[0110] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0111] Through the technical solution provided by the embodiment of the present application, in response to the preprocessing instruction for multiple sample data of the target type, the preprocessing execution graph corresponding to the target type is displayed, and the preprocessing execution graph is used to indicate the process of executing multiple preprocessing steps, that is, to indicate the execution method and execution order of multiple preprocessing steps. The preprocessing execution graph includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch manner. In response to the confirmation instruction of the preprocessing execution graph, according to the process indicated by the preprocessing execution graph, the CPU and GPU are called to execute the multiple preprocessing steps on the multiple sample data, so as to make full use of the CPU and GPU and improve the efficiency of preprocessing the sample data.
[0112] After adopting the technical solution provided in the embodiment of the present application, the CPU and GPU resources are fully utilized, the processing logic suitable for the CPU is still placed on the CPU, and the logic placed on the GPU is completely batched to improve performance. For example, if the batch size is 16, the number of operator calls is reduced by at least 15 times (16 calls to each operator becomes 1 call). In addition, in the preprocessing of image enhancement, it is necessary to randomly sample and shift the original image to obtain multiple sampled images, and then perform transformation operations together with the original image. For example, the slice is sampled 4 times, so combined with the internal image small batch sampling and the entire training batch, the number of calls to the general preprocessing operator is more significantly reduced (5*16=80→1). See Figure 6 Taking image data enhancement as an example, preprocessing the batched data uniformly for image preprocessing can give full play to the computing resource advantages of the GPU, and make full use of the time overlap between data reading and GPU computing asynchronously to improve the overall performance of the training system.
[0113] In addition, the configuration of the preprocessing flow chart also improves the efficiency of algorithm model development iteration. It can basically achieve "zero code" preprocessing, without spending a lot of development time on preprocessing.
[0114] Figure 7 is a schematic diagram of the structure of a sample data preprocessing device provided in an embodiment of the present application, see Figure 7 The device includes: a display module 701 and a calling module 702.
[0115] Display module 701 is used to respond to preprocessing instructions for multiple sample data of a target type and display a preprocessing execution graph corresponding to the target type. The preprocessing execution graph is used to indicate the process of executing multiple preprocessing steps. The preprocessing execution graph includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch manner.
[0116] The calling module 702 is used to respond to a confirmation instruction of the preprocessing execution graph and call the CPU and GPU to execute the multiple preprocessing steps on the multiple sample data according to the process indicated by the preprocessing execution graph.
[0117] The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area.
[0118] In a possible implementation, the execution order of the preprocessing steps in the first execution region is earlier than that of the preprocessing steps in the second execution region, and the calling module 702 is used to call the CPU to execute the preprocessing steps in the first execution region in parallel on the multiple sample data in response to a confirmation instruction of the preprocessing execution graph, and call the GPU to execute the preprocessing steps in the second execution region sequentially on the multiple sample data in a batch processing manner.
[0119] In one possible implementation, the first execution area includes a plurality of first preprocessing steps, and the calling module 702 is used to call the CPU to synchronously execute the plurality of first preprocessing steps on any sample data among the plurality of sample data in response to a confirmation instruction of the preprocessing execution graph.
[0120] In a possible implementation, the second execution area includes a plurality of second preprocessing steps, and the calling module 702 is used to call the GPU to synchronously execute the second preprocessing step on the plurality of sample data for any second preprocessing step among the plurality of second preprocessing steps.
[0121] In a possible embodiment, the device also includes an adjustment module for adjusting the preprocessing steps in the first execution area and / or the second execution area in response to an adjustment instruction to the preprocessing execution graph based on the manner indicated by the adjustment instruction to update the preprocessing execution graph.
[0122] The calling module 702 is further configured to, in response to a confirmation instruction of the updated preprocessing execution graph, call the CPU and GPU to preprocess the plurality of sample data according to the process indicated by the updated preprocessing execution graph.
[0123] In a possible implementation, the adjustment module is used to, when the adjustment instruction indicates that a preprocessing step is to be added in the first execution area or the second execution area, respond to the adjustment instruction, add the preprocessing step corresponding to the adjustment instruction in the first execution area or the second execution area. When the adjustment instruction indicates that a preprocessing step in the first execution area or the second execution area is to be deleted, respond to the adjustment instruction, delete the preprocessing step corresponding to the adjustment instruction in the first execution area or the second execution area. When the adjustment instruction indicates that the execution order of any preprocessing step in the first execution area or the second execution area is to be adjusted, respond to the adjustment instruction, adjust the execution order of the preprocessing step in the first execution area or the second execution area. When the adjustment instruction indicates that the execution order of any preprocessing step in the preprocessing execution diagram is to be adjusted, respond to the adjustment instruction, adjust the preprocessing step from the first execution area to the second execution area, or adjust the preprocessing step from the second execution area to the first execution area.
[0124] In a possible implementation, the device further includes a generation module for acquiring an initial preprocessing flow of the target type, the initial preprocessing flow including an initial execution order of the plurality of preprocessing steps. Based on the types of the respective preprocessing steps, the plurality of preprocessing steps are divided into a first preprocessing step set and a second preprocessing step set, the first preprocessing step set corresponding to parallel execution, and the second preprocessing step set corresponding to sequential execution in a batch processing manner. Based on the initial execution order, the first preprocessing step set and the second preprocessing step set, the preprocessing execution graph corresponding to the target type is generated.
[0125] In a possible implementation, the generating module is used to set the plurality of first preprocessing steps in the first preprocessing step set to be executed in parallel to form the first execution region. Based on the initial execution order, the execution order of the plurality of second preprocessing steps in the second preprocessing step set is set to form the second execution region. Based on the first execution region and the second execution region, the preprocessing execution graph is generated.
[0126] It should be noted that: the sample data preprocessing device provided in the above embodiment only uses the division of the above functional modules as an example when preprocessing the sample data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the sample data preprocessing device provided in the above embodiment and the sample data preprocessing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0127] Through the technical solution provided by the embodiment of the present application, in response to the preprocessing instruction for multiple sample data of the target type, the preprocessing execution graph corresponding to the target type is displayed, and the preprocessing execution graph is used to indicate the process of executing multiple preprocessing steps, that is, to indicate the execution method and execution order of multiple preprocessing steps. The preprocessing execution graph includes a first execution area and a second execution area. The preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch manner. In response to the confirmation instruction of the preprocessing execution graph, according to the process indicated by the preprocessing execution graph, the CPU and GPU are called to execute the multiple preprocessing steps on the multiple sample data, so as to make full use of the CPU and GPU and improve the efficiency of preprocessing the sample data.
[0128] Figure 8 800 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device 800 may be: Generally, the computer device 800 includes: one or more processors 801 and one or more memories 802 .
[0129] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0130] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one computer program, which is used to be executed by the processor 801 to implement the sample data preprocessing method provided in the method embodiment of the present application.
[0131] In some embodiments, the computer device 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807 and a power supply 808.
[0132] The peripheral device interface 803 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
[0133] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like.
[0134] The display screen 805 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on the surface or above the surface of the display screen 805. The touch signal may be input as a control signal to the processor 801 for processing. At this time, the display screen 805 may also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards.
[0135] The camera assembly 806 is used to collect images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Usually, the front camera is arranged on the front panel of the computer device, and the rear camera is arranged on the back of the computer device.
[0136] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them to the processor 801 for processing, or input them to the radio frequency circuit 804 to achieve voice communication.
[0137] The power supply 808 is used to supply power to various components in the computer device 800. The power supply 808 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery.
[0138] In some embodiments, the computer device 800 further includes one or more sensors 809 , including but not limited to: an acceleration sensor 810 , a gyroscope sensor 811 , a pressure sensor 812 , an optical sensor 813 , and a proximity sensor 814 .
[0139] The acceleration sensor 810 can detect the magnitude of acceleration on three coordinate axes of the coordinate system established by the computer device 800 .
[0140] The gyro sensor 811 can detect the body direction and rotation angle of the computer device 800 . The gyro sensor 811 can cooperate with the acceleration sensor 810 to collect the user's 3D actions on the computer device 800 .
[0141] The pressure sensor 812 may be disposed on the side frame of the computer device 800 and / or the lower layer of the display screen 805. When the pressure sensor 812 is disposed on the side frame of the computer device 800, it may detect the user's gripping signal of the computer device 800, and the processor 801 may perform left and right hand recognition or shortcut operations according to the gripping signal collected by the pressure sensor 812. When the pressure sensor 812 is disposed on the lower layer of the display screen 805, the processor 801 may control the operable controls on the UI interface according to the user's pressure operation on the display screen 805.
[0142] The optical sensor 813 is used to collect the ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 according to the ambient light intensity collected by the optical sensor 813.
[0143] The proximity sensor 814 is used to collect the distance between the user and the front of the computer device 800 .
[0144] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the computer device 800, and the computer device 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.
[0145] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program, and the computer program can be executed by a processor to complete the sample data preprocessing method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.
[0146] In an exemplary embodiment, a computer program product or a computer program is also provided, which includes a program code, and the program code is stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computer device performs the above-mentioned sample data preprocessing method.
[0147] In some embodiments, the computer program involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.
[0148] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0149] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A sample data preprocessing method, It is characterized in that The method comprises: In response to a preprocessing instruction for a plurality of sample data of a target type, a preprocessing execution graph corresponding to the target type is displayed, the preprocessing execution graph is used to indicate a flow of executing a plurality of preprocessing steps, the preprocessing execution graph includes a first execution area and a second execution area, the preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner; In response to a confirmation instruction of the preprocessing execution graph, calling the CPU and the GPU to perform the plurality of preprocessing steps on the plurality of sample data according to the process indicated by the preprocessing execution graph; The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area.
2. The method according to claim 1, It is characterized in that The execution order of the preprocessing steps in the first execution region is earlier than that of the preprocessing steps in the second execution region, and in response to the confirmation instruction of the preprocessing execution graph, calling the CPU and the GPU to execute the plurality of preprocessing steps on the plurality of sample data according to the process indicated by the preprocessing execution graph includes: In response to a confirmation instruction on the preprocessing execution graph, calling the CPU to execute the preprocessing steps in the first execution area in parallel on the plurality of sample data; The GPU is called to sequentially execute the preprocessing steps in the second execution region on the plurality of sample data in a batch processing manner.
3. The method according to claim 2, It is characterized in that The first execution region includes a plurality of first preprocessing steps, and in response to a confirmation instruction on the preprocessing execution graph, calling the CPU to execute the preprocessing steps in the first execution region in parallel on the plurality of sample data includes: For any sample data among the plurality of sample data, in response to a confirmation instruction on the pre-processing execution graph, the CPU is called to synchronously execute the plurality of first pre-processing steps on the sample data.
4. The method according to claim 2, It is characterized in that The second execution region includes a plurality of second preprocessing steps, and calling the GPU to sequentially execute the preprocessing steps in the second execution region on the plurality of sample data in a batch processing manner includes: For any second preprocessing step among the plurality of second preprocessing steps, the GPU is called to synchronously execute the second preprocessing step on the plurality of sample data.
5. The method according to claim 1, It is characterized in that After the preprocessing execution graph corresponding to the target type is displayed in response to the preprocessing instruction for the plurality of sample data of the target type, the method further includes: In response to an adjustment instruction for the preprocessing execution graph, adjusting the preprocessing steps in the first execution region and / or the second execution region based on a manner indicated by the adjustment instruction to update the preprocessing execution graph; In response to a confirmation instruction for the updated preprocessing execution graph, the CPU and the GPU are called to preprocess the plurality of sample data according to a process indicated by the updated preprocessing execution graph.
6. The method according to claim 5, It is characterized in that In response to the adjustment instruction for the preprocessing execution graph, adjusting the preprocessing steps in the first execution region and / or the second execution region based on the manner indicated by the adjustment instruction includes: When the adjustment instruction indicates to add a preprocessing step in the first execution region or the second execution region, in response to the adjustment instruction, add a preprocessing step corresponding to the adjustment instruction in the first execution region or the second execution region; In a case where the adjustment instruction indicates to delete a preprocessing step in the first execution region or the second execution region, in response to the adjustment instruction, deleting the preprocessing step corresponding to the adjustment instruction in the first execution region or the second execution region; In a case where the adjustment instruction indicates to adjust the execution order of any preprocessing step in the first execution area or the second execution area, in response to the adjustment instruction, adjusting the execution order of the preprocessing step in the first execution area or the second execution area; When the adjustment instruction indicates adjusting the execution order of any preprocessing step in the preprocessing execution diagram, in response to the adjustment instruction, the preprocessing step is adjusted from the first execution area to the second execution area, or the preprocessing step is adjusted from the second execution area to the first execution area.
7. The method according to claim 1, It is characterized in that Before displaying the preprocessing execution graph corresponding to the target type in response to the preprocessing instruction for the plurality of sample data of the target type, the method further includes: Acquire an initial preprocessing process of the target type, wherein the initial preprocessing process includes an initial execution order of the plurality of preprocessing steps; Based on the types of the respective preprocessing steps, the plurality of preprocessing steps are divided into a first preprocessing step set and a second preprocessing step set, wherein the first preprocessing step set corresponds to parallel execution, and the second preprocessing step set corresponds to sequential execution in a batch processing manner; The preprocessing execution graph corresponding to the target type is generated based on the initial execution order, the first preprocessing step set, and the second preprocessing step set.
8. The method according to claim 7, It is characterized in that The generating the preprocessing execution graph corresponding to the target type based on the initial execution order, the first preprocessing step set and the second preprocessing step set comprises: Arrange multiple first preprocessing steps in the first preprocessing step set to be executed in parallel to form the first execution region; Based on the initial execution order, setting the execution order of the plurality of second preprocessing steps in the second preprocessing step set to form the second execution region; The pre-processing execution graph is generated based on the first execution region and the second execution region.
9. A sample data preprocessing device, It is characterized in that The device comprises: A display module, configured to display a preprocessing execution diagram corresponding to a target type in response to a preprocessing instruction for a plurality of sample data of a target type, wherein the preprocessing execution diagram is used to indicate a flow of executing a plurality of preprocessing steps, and the preprocessing execution diagram includes a first execution area and a second execution area, wherein the preprocessing steps in the first execution area are executed in parallel, and the preprocessing steps in the second execution area are executed sequentially in a batch processing manner; A calling module, configured to respond to a confirmation instruction of the preprocessing execution graph and call the CPU and the GPU to execute the plurality of preprocessing steps on the plurality of sample data according to the process indicated by the preprocessing execution graph; The CPU executes the preprocessing steps in the first execution area, and the GPU executes the preprocessing steps in the second execution area.
10. A computer device, It is characterized in that The computer device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the sample data preprocessing method according to any one of claims 1 to 8.
11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores at least one computer program, and the computer program is loaded and executed by a processor to implement the sample data preprocessing method according to any one of claims 1 to 8.
12. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the sample data preprocessing method according to any one of claims 1 to 8 is implemented.