A data processing method and device, computer equipment and a storage medium

Abstract

The present disclosure provides a data processing method and device, computer equipment and a storage medium, wherein the method comprises: obtaining sequencing sample information corresponding to a sequencing chip and hardware parameter information of a hardware device used for performing sequencing data processing; wherein the sequencing sample information is used to represent the characteristics of a sample to be sequenced in the sequencing chip; inputting the sequencing sample information and the hardware parameter information into a pre-trained target model to obtain target software parameter information output by the target model and used for performing sequencing data processing; and performing data processing on sequencing data corresponding to the sample to be sequenced based on the target software parameter information.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] During operation, the sequencer can add various reagents to the sequencing chip through a syringe pump to achieve gene sequencing of the corresponding sample. During gene sequencing, a large amount of sequencing data that needs to be processed in a timely manner is often generated.

[0003] In related technologies, in order to improve the sequencing speed of gene sequencing, the sequencing data is often processed first to convert the data format. Then, data compression and other processing steps can be performed on the converted sequencing data to complete the gene sequencing of the sample. However, when converting the data format, various related parameters need to be set. Manually setting these parameter values ​​or using default parameter values ​​may result in slow format conversion due to unreasonable parameter settings, which will make the entire gene sequencing time longer and the efficiency need to be improved. Summary of the Invention

[0004] This disclosure provides at least one data processing method, apparatus, computer device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a data processing method, including:

[0006] Obtain sequencing sample information corresponding to the sequencing chip, as well as hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip;

[0007] The sequencing sample information and the hardware parameter information are input into the pre-trained target model to obtain the target software parameter information output by the target model for performing sequencing data processing;

[0008] Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is processed.

[0009] In one possible implementation, the sequencing sample information includes at least one of the following:

[0010] Number of sequencing samples, number of sequencing cycles, sequencing method, sequencing throughput, adapter sequence length, and sequencing mismatch data.

[0011] In one possible implementation, the hardware device information includes at least one of the following:

[0012] Number of processor cores, processor frequency, hard disk data transfer bandwidth, and hard disk cache capacity.

[0013] In one possible implementation, the target software parameter information includes at least one of the following:

[0014] Number of read threads, number of write threads, number of compute threads.

[0015] In one possible implementation, the target model is a decision tree model, and the method further includes constructing the decision tree model according to the following method;

[0016] Obtain a training dataset; wherein the training dataset contains sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data;

[0017] Based on the decision tree model, the sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data in the training dataset are used to model the target model.

[0018] In one possible implementation, the method further includes training the target model according to the following method:

[0019] Obtain a validation dataset; wherein the validation dataset contains sequencing sample information, validation hardware parameter information, and validation software parameter information corresponding to the validation data;

[0020] The test sample information and verification hardware parameter information corresponding to the verification data are input into the target model to be trained to obtain the sample software parameter information output by the target model.

[0021] Based on the sample software parameter information and the verification software parameter information, the target loss value for this training is determined, and the network parameters of the target model are adjusted based on the target loss value.

[0022] In one possible implementation, the data processing of the sequencing data corresponding to the sample to be sequenced based on the target software parameter information includes:

[0023] Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is split into sequencing data to convert the data format of the sequencing data from the initial data format to the target data format, which is a data format used for data compression processing.

[0024] Secondly, embodiments of this disclosure also provide a data processing apparatus, including:

[0025] The acquisition module is used to acquire sequencing sample information corresponding to the sequencing chip, as well as hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip.

[0026] The input module is used to input the sequencing sample information and the hardware parameter information into the pre-trained target model to obtain the target software parameter information output by the target model for performing sequencing data processing;

[0027] The processing module is used to process the sequencing data corresponding to the sample to be sequenced based on the target software parameter information.

[0028] In one possible implementation, the sequencing sample information includes at least one of the following:

[0029] Number of sequencing samples, number of sequencing cycles, sequencing method, sequencing throughput, adapter sequence length, and sequencing mismatch data.

[0030] In one possible implementation, the hardware device information includes at least one of the following:

[0031] Number of processor cores, processor frequency, hard disk data transfer bandwidth, and hard disk cache capacity.

[0032] In one possible implementation, the target software parameter information includes at least one of the following:

[0033] Number of read threads, number of write threads, number of compute threads.

[0034] In one possible implementation, the target model is a decision tree model, and the input module is further configured to construct the decision tree model according to the following steps;

[0035] Obtain a training dataset; wherein the training dataset contains sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data;

[0036] Based on the decision tree model, the sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data in the training dataset are used to model the target model.

[0037] In one possible implementation, the input module is further configured to train the target model according to the following method:

[0038] Obtain a validation dataset; wherein the validation dataset contains sequencing sample information, validation hardware parameter information, and validation software parameter information corresponding to the validation data;

[0039] The test sample information and verification hardware parameter information corresponding to the verification data are input into the target model to be trained to obtain the sample software parameter information output by the target model.

[0040] Based on the sample software parameter information and the verification software parameter information, the target loss value for this training is determined, and the network parameters of the target model are adjusted based on the target loss value.

[0041] In one possible implementation, the processing module, when processing the sequencing data corresponding to the sample to be sequenced based on the target software parameter information, is used to:

[0042] Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is split into sequencing data to convert the data format of the sequencing data from the initial data format to the target data format, which is a data format used for data compression processing.

[0043] Thirdly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0044] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation of the first aspect.

[0045] The data processing method, apparatus, computer equipment, and storage medium provided in this disclosure, through pre-training, are used to output target software parameter information for data processing of test data based on input hardware parameter information and sequencing sample information related to software parameter settings. This allows for the automatic determination of target software parameter information for data processing based on the current hardware conditions and the conditions of the sequencing samples. Compared to manually setting relevant parameters or using default parameters, the target model can accurately combine the hardware conditions of the processing equipment and the sample conditions of the sequencing samples to determine more suitable software parameter information, thereby improving the data processing speed of sequencing data and increasing the overall efficiency of gene sequencing.

[0046] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0048] Figure 1 A flowchart of a data processing method provided by an embodiment of this disclosure is shown;

[0049] Figure 2 A schematic diagram of the architecture of a data processing apparatus provided in an embodiment of this disclosure is shown;

[0050] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0052] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0053] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0054] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0055] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0056] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0057] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0058] Studies have found that in order to improve the sequencing speed of gene sequencing, sequencing data is often processed first to convert the data format. This allows for data compression and other processing steps to complete the gene sequencing of the sample. However, data format conversion requires setting various related parameters. Manually setting these parameters or using default values ​​may result in slow format conversion due to unreasonable parameter settings, thus making the entire gene sequencing process time-consuming and inefficient.

[0059] Based on the above research, this disclosure provides a data processing method, apparatus, computer device, and storage medium. By pre-training a target model that outputs target software parameter information for processing test data based on input hardware parameter information related to software parameter settings and sequencing sample information, the target model can automatically determine the target software parameter information for data processing based on the current hardware conditions and the conditions of the sequencing samples. Compared to manually setting relevant parameters or using default parameters, the target model can accurately combine the hardware conditions of the processing equipment and the sample conditions of the sequencing samples to determine more suitable software parameter information, thereby improving the data processing speed of sequencing data and increasing the overall efficiency of gene sequencing.

[0060] To facilitate understanding of this embodiment, a data processing method disclosed in this disclosure will first be described in detail. The execution entity of the data processing method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this data processing method can be implemented by a processor calling computer-readable instructions stored in memory.

[0061] See Figure 1 The diagram shows a flowchart of a data processing method provided in an embodiment of this disclosure. The method includes steps S101 to S103, wherein:

[0062] S101: Obtain the sequencing sample information corresponding to the sequencing chip, and the hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip.

[0063] S102: Input the sequencing sample information and the hardware parameter information into the pre-trained target model to obtain the target software parameter information output by the target model for performing sequencing data processing.

[0064] S103: Based on the target software parameter information, perform data processing on the sequencing data corresponding to the sample to be sequenced.

[0065] The following is a detailed explanation of the steps described above.

[0066] For S101,

[0067] The sequencing chip can be used to perform gene sequencing on the sample to be sequenced. The sequencing chip can have multiple flow channels (lanes). When processing the sequencing data corresponding to the sample to be sequenced, data processing can be performed on the sequencing data corresponding to each lane. The sequencing sample information is used to characterize the features of the test sample, and the sequencing sample information corresponding to different lanes can be different. The hardware parameter information of the hardware device is used to characterize the parameters of the hardware device that performs sequencing data processing. When processing the sequencing data, different computing resources can be used to process the sequencing data corresponding to each lane.

[0068] In one possible implementation, the sequencing sample information may include at least one of the following:

[0069] Number of sequencing samples, number of sequencing cycles, sequencing method, sequencing throughput, adapter sequence length, and sequencing mismatch data.

[0070] The sequence number of samples is used to characterize the number of samples to be sequenced in the sequencing chip; the number of sequencing cycles is used to characterize the number of sequencing cycles executed by the sequencing chip during gene sequencing, whereby one base of the sample to be sequenced is read in one sequencing cycle to complete the gene sequencing of the sample; the sequencing method is used to characterize the gene sequencing method adopted by the sequencing chip during gene sequencing of the sample, such as single-end sequencing, paired-end sequencing, etc.; the sequencing throughput is used to characterize the amount of data generated by the sequencing chip per unit time during gene sequencing of the sample, and can be used to measure the sequencing speed during gene sequencing; the adapter sequence length is used to characterize the adapter sequence used by the sequencing chip during gene sequencing of the sample. The sequence length of r, the adapter sequence is a short nucleotide sequence with known components, used to link unknown sequencing gene fragments, to cut or mask the sequencing sample to complete the gene sequencing of the sample to be sequenced; the sequencing mismatch data is used to characterize the number of permissible sequencing mismatch bases corresponding to different sample category identifiers barcodes during the gene sequencing process of the sequencing chip. Taking wheat as an example, the permissible number of sequencing mismatch bases corresponding to wheat is 1. If 15 out of the 16 bases obtained during gene sequencing match wheat bases and 1 base does not match wheat, and this does not exceed the permissible number of sequencing mismatch bases corresponding to wheat, then the gene fragment currently being sequenced can be considered to match wheat.

[0071] In one possible implementation, the hardware device information may include at least one of the following:

[0072] Number of processor cores, processor frequency, hard disk data transfer bandwidth, and hard disk cache capacity.

[0073] The number of processor cores refers to the number of processing cores in the Central Processing Unit (CPU) of the hardware device that performs sequencing data processing. For example, the number of processor cores can be 4, 8, or 16. The processor frequency refers to the clock frequency of the CPU's processing cores, i.e., the operating frequency of the processing cores when performing calculations. A higher processor frequency results in faster processing speeds. The hard disk data transfer bandwidth refers to the data transfer bandwidth of the storage hard disk of the hardware device that performs sequencing data processing. A larger hard disk data transfer bandwidth theoretically allows for the simultaneous transfer of more data, resulting in higher data transfer speeds. The hard disk cache capacity refers to the cache size of the storage hard disk of the hardware device that performs sequencing data processing. A larger hard disk cache capacity indicates a larger amount of data that can be transferred at high speed. When the data transfer volume exceeds the hard disk cache capacity, it becomes difficult to continue using the theoretically maximum data transfer speed for high-speed sequencing data transfer. Therefore, a larger hard disk cache capacity results in shorter data transfer times.

[0074] Specifically, when obtaining the sequencing sample information corresponding to the sequencing chip and the hardware parameter information of the hardware device used to perform sequencing data processing, a preset detection program or detection script can be used.

[0075] For example, taking the use of a detection program to obtain hardware parameter information of a hardware device used to perform sequencing data processing as an example, the detection program can obtain the hardware parameter information of the hardware device from the target registry corresponding to the hardware device.

[0076] For S102,

[0077] Here, the target model can be a pre-built and trained neural network model, and the network type of the target model can be, for example, a decision tree model, a convolutional neural network (CNN), etc.

[0078] In one possible implementation, the target software parameter information may include at least one of the following:

[0079] Number of read threads, number of write threads, number of compute threads.

[0080] The number of read threads represents the number of threads used by the target software for performing sequencing data processing during operation to read and process the sequencing data; the number of write threads represents the number of threads used by the target software for performing sequencing data processing during operation to write the processed sequencing data; and the number of computation threads represents the number of threads used by the target software for performing sequencing data processing during operation to perform computational processing on the sequencing data.

[0081] Specifically, since the hardware resources of the terminal device are limited, the number of threads that the target software can use during operation is also limited. It is necessary to reasonably allocate the number of read threads, write threads, and computing threads in order to quickly complete the sequencing data processing of the sample to be sequenced.

[0082] In practical applications, the type of target model can be selected according to actual needs. The following will use a decision tree model as an example to introduce the creation and training methods of the target model:

[0083] 1. Create a decision tree model

[0084] Here, when creating the decision tree model, the following steps A1 to A2 can be used:

[0085] A1: Obtain the training dataset; wherein, the training dataset contains sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data.

[0086] A2: Based on the decision tree model, perform data modeling on the sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data in the training dataset to construct the target model.

[0087] Here, during the data modeling process, a decision tree model can be generated layer by layer recursively. In generating the decision tree model, target information can be selected as the optimal feature from various types of information corresponding to the sample data in the training dataset, according to criteria such as the minimum squared error criterion and the minimum Gini coefficient criterion. The training dataset is then divided into multiple subsets based on the optimal features. For any subset, it is determined whether the subset is correctly classified. If it is correctly classified, a leaf node in the decision tree is constructed based on that subset. If the subset is not correctly classified, the above subset partitioning (i.e., dataset splitting and subset generation) steps are recursively executed until the partitioned subsets can be correctly classified. Whether a subset is correctly classified can be determined by calculating whether the Gini coefficient corresponding to the subset is less than a preset Gini coefficient threshold, or by whether the number of data points in the subset is less than a preset sample size threshold.

[0088] In this way, a target model for predicting the parameter information of the target software can be constructed using the sample data in the training dataset.

[0089] 2. Training the decision tree model

[0090] Here, when training the decision tree model, the following steps B1 to B3 can be used:

[0091] B1: Obtain the validation dataset; wherein the validation dataset contains sequencing sample information, validation hardware parameter information, and validation software parameter information corresponding to the validation data.

[0092] B2: Input the test sample information and verification hardware parameter information corresponding to the verification data into the target model to be trained, and obtain the sample software parameter information output by the target model.

[0093] B3: Based on the sample software parameter information and the verification software parameter information, determine the target loss value for this training, and adjust the network parameters of the target model based on the target loss value.

[0094] Here, the target loss value is used to characterize the difference between the sample software parameter information and the verification software parameter information.

[0095] Specifically, when determining the target loss value for this training based on the sample software parameter information and the validation software parameter information, the target loss value for this training can be determined based on the sample software parameter information, the validation software parameter information, and the target loss function; after determining the target loss value, the parameters of the target model can be updated according to the parameter update method that matches the model type of the target model.

[0096] For example, taking the target model type as a decision tree model and the parameter update method matching the decision tree model as the back propagation (BP) algorithm, after determining the target loss value, the parameters of each model in the decision tree model can be updated by the back propagation (BP) algorithm. Since the decision tree model can be constructed by the genetic algorithm (GA), the final target model can also be called a GA-BP neural network model. Here, GA indicates that the overall architecture of the model is a decision tree architecture constructed using a genetic algorithm, and BP indicates that the parameter update method of the model is to update the parameters using the back propagation algorithm.

[0097] Furthermore, after training the decision tree model, it can be tested using a test dataset to ensure the accuracy of the trained decision tree model.

[0098] Specifically, the data sources of the training dataset, the validation dataset, and the test dataset can be the same dataset, and the data types of the training dataset, the validation dataset, and the test dataset can be the same; cross-validation can be used in the process of building the target model, training the target model, and validating the target model, and the ratio of data used in the process of building the target model, training the target model, and validating the target model can be 8:1:1.

[0099] The same dataset can be a target dataset that is representative and stable, constructed based on the k-means clustering algorithm. After completing the construction-training-validation process of the target model, the target model can be used to process the input sequencing sample information and hardware parameter information to obtain the target software parameter information, and the obtained target software parameter information can be used to process the sequencing data.

[0100] Furthermore, in order to ensure that the performance of the target model can match the real-time status of hardware devices such as sequencing chips, the data in the target dataset can be updated based on the data obtained in subsequent data processing, and the updated target dataset can be used to perform incremental learning (i.e., retraining) on ​​the target model, so that the model performance of the target model can match the real-time status of each hardware device.

[0101] It should be noted that the decision tree model is only one possible model type for the target model. In actual use, the target model can also be a convolutional neural network. When the network type of the target model is a convolutional neural network, the input of the target model is still the sequencing sample information and the hardware parameter information, and the output of the target model is still the target software parameter information. The training method of the target model can be supervised training. More details about the network model of the target model being a convolutional neural network will not be elaborated here.

[0102] For S103,

[0103] Here, when processing the sequencing data corresponding to the sample to be sequenced based on the target software parameter information, the target software parameter information can be input into the target software to instruct the target software to process the sequencing data corresponding to the sample to be sequenced according to the target software parameter information; wherein, the target software may be, for example, gene splitting software.

[0104] In practical applications, the amount of sequencing data obtained from the samples to be sequenced during gene sequencing is often very large. To improve the processing speed of sequencing data, the data format of the sequencing data can be converted so that data compression can be performed based on the converted data format, thereby improving the processing speed of sequencing data.

[0105] In one possible implementation, when processing the sequencing data corresponding to the sample to be sequenced based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced can be split based on the target software parameter information to convert the data format of the sequencing data from the initial data format to the target data format, wherein the target data format is a data format used for data compression processing.

[0106] Here, the initial data format can be base call and base score (baq) data format, and the target data format can be fastq data format. By converting the data format of the sequencing data corresponding to the sample to be sequenced from the initial data format to the target data format that can be used for data compression processing, the sequencing data after sequencing data splitting can be compressed, thereby improving the processing speed of the sequencing data of the sample to be sequenced.

[0107] The data processing method provided in this disclosure pre-trains a target model that outputs target software parameter information for processing test data based on input hardware parameter information related to software parameter settings and sequencing sample information. This allows the target software parameter information for data processing to be automatically determined based on the current hardware conditions and the conditions of the sequencing samples. Compared to manually setting relevant parameters or using default parameters, the target model can accurately combine the hardware conditions of the processing equipment and the sample conditions of the sequencing samples to determine more suitable software parameter information, thereby improving the data processing speed of sequencing data and increasing the overall efficiency of gene sequencing.

[0108] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0109] Based on the same inventive concept, this disclosure also provides a data processing device corresponding to the data processing method. Since the principle of the device in this disclosure for solving the problem is similar to the data processing method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and repeated details will not be repeated.

[0110] Reference Figure 2 The diagram shown is an architectural schematic of a data processing device provided in an embodiment of this disclosure. The device includes: an acquisition module 201, an input module 202, and a processing module 203; wherein,

[0111] The acquisition module 201 is used to acquire sequencing sample information corresponding to the sequencing chip, as well as hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip.

[0112] Input module 202 is used to input the sequencing sample information and the hardware parameter information into the pre-trained target model to obtain the target software parameter information output by the target model for performing sequencing data processing;

[0113] The processing module 203 is used to process the sequencing data corresponding to the sample to be sequenced based on the target software parameter information.

[0114] In one possible implementation, the sequencing sample information includes at least one of the following:

[0115] Number of sequencing samples, number of sequencing cycles, sequencing method, sequencing throughput, adapter sequence length, and sequencing mismatch data.

[0116] In one possible implementation, the hardware device information includes at least one of the following:

[0117] Number of processor cores, processor frequency, hard disk data transfer bandwidth, and hard disk cache capacity.

[0118] In one possible implementation, the target software parameter information includes at least one of the following:

[0119] Number of read threads, number of write threads, number of compute threads.

[0120] In one possible implementation, the target model is a decision tree model, and the input module 202 is further configured to construct the decision tree model according to the following steps;

[0121] Obtain a training dataset; wherein the training dataset contains sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data;

[0122] Based on the decision tree model, the sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data in the training dataset are used to model the target model.

[0123] In one possible implementation, the input module 202 is further configured to train the target model according to the following method:

[0124] Obtain a validation dataset; wherein the validation dataset contains sequencing sample information, validation hardware parameter information, and validation software parameter information corresponding to the validation data;

[0125] The test sample information and verification hardware parameter information corresponding to the verification data are input into the target model to be trained to obtain the sample software parameter information output by the target model.

[0126] Based on the sample software parameter information and the verification software parameter information, the target loss value for this training is determined, and the network parameters of the target model are adjusted based on the target loss value.

[0127] In one possible implementation, the processing module 203, when processing the sequencing data corresponding to the sample to be sequenced based on the target software parameter information, is used to:

[0128] Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is split into sequencing data to convert the data format of the sequencing data from the initial data format to the target data format, which is a data format used for data compression processing.

[0129] The data processing apparatus provided in this disclosure pre-trains a target model that outputs target software parameter information for processing test data based on input hardware parameter information and sequencing sample information related to software parameter settings. This allows the target software parameter information for data processing to be automatically determined based on the current hardware conditions and the conditions of the sequencing samples. Compared to manually setting relevant parameters or using default parameters, the target model can accurately combine the hardware conditions of the processing device and the sample conditions of the sequencing samples to determine more suitable software parameter information, thereby improving the data processing speed of sequencing data and increasing the overall efficiency of gene sequencing.

[0130] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0131] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 3 The diagram shows the structure of a computer device 300 provided in this embodiment, including a processor 301, a memory 302, and a bus 303. The memory 302 stores execution instructions and includes a main memory 3021 and an external memory 3022. The main memory 3021, also called internal memory, is used to temporarily store computational data in the processor 301 and data exchanged with external memory such as a hard disk. The processor 301 exchanges data with the external memory 3022 through the main memory 3021. When the computer device 300 is running, the processor 301 and the memory 302 communicate through the bus 303, causing the processor 301 to execute the following instructions:

[0132] Obtain sequencing sample information corresponding to the sequencing chip, as well as hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip;

[0133] The sequencing sample information and the hardware parameter information are input into the pre-trained target model to obtain the target software parameter information output by the target model for performing sequencing data processing;

[0134] Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is processed.

[0135] In one possible implementation, the instructions of the processor 301 include at least one of the following:

[0136] Number of sequencing samples, number of sequencing cycles, sequencing method, sequencing throughput, adapter sequence length, and sequencing mismatch data.

[0137] In one possible implementation, the hardware device information in the instructions of the processor 301 includes at least one of the following:

[0138] Number of processor cores, processor frequency, hard disk data transfer bandwidth, and hard disk cache capacity.

[0139] In one possible implementation, the target software parameter information in the instructions of the processor 301 includes at least one of the following:

[0140] Number of read threads, number of write threads, number of compute threads.

[0141] In one possible implementation, the processor 301's instructions specify that the target model is a decision tree model, and further include constructing the decision tree model according to the following method;

[0142] Obtain a training dataset; wherein the training dataset contains sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data;

[0143] Based on the decision tree model, the sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data in the training dataset are used to model the target model.

[0144] In one possible implementation, the instructions of the processor 301 further include training the target model according to the following method:

[0145] Obtain a validation dataset; wherein the validation dataset contains sequencing sample information, validation hardware parameter information, and validation software parameter information corresponding to the validation data;

[0146] The test sample information and verification hardware parameter information corresponding to the verification data are input into the target model to be trained to obtain the sample software parameter information output by the target model.

[0147] Based on the sample software parameter information and the verification software parameter information, the target loss value for this training is determined, and the network parameters of the target model are adjusted based on the target loss value.

[0148] In one possible implementation, the instructions of the processor 301, specifically the step of processing the sequencing data corresponding to the sample to be sequenced based on the target software parameter information, include:

[0149] Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is split into sequencing data to convert the data format of the sequencing data from the initial data format to the target data format, which is a data format used for data compression processing.

[0150] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the data processing method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0151] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the data processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0152] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0156] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data processing method, characterized in that, include: Obtain sequencing sample information corresponding to the sequencing chip, as well as hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip; The sequencing sample information and the hardware parameter information are input into the pre-trained target model to obtain the target software parameter information output by the target model for performing sequencing data processing; Based on the target software parameter information, the sequencing data corresponding to the sample to be sequenced is split into sequencing data to convert the data format of the sequencing data from the initial data format to the target data format. The target software parameter information includes at least one of the following: Number of read threads, number of write threads, number of compute threads.

2. The method according to claim 1, characterized in that, The sequencing sample information includes at least one of the following: Number of sequencing samples, number of sequencing cycles, sequencing method, sequencing throughput, adapter sequence length, and sequencing mismatch data.

3. The method according to claim 1, characterized in that, The hardware device information includes at least one of the following: Number of processor cores, processor frequency, hard disk data transfer bandwidth, and hard disk cache capacity.

4. The method according to claim 1, characterized in that, The target model is a decision tree model, and the method further includes constructing the decision tree model according to the following method; Obtain a training dataset; wherein the training dataset contains sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data; Based on the decision tree model, the sequencing sample information, sample hardware parameter information, and sample software parameter information corresponding to the sample data in the training dataset are used to model the target model.

5. The method according to claim 4, characterized in that, The method further includes training the target model according to the following method: Obtain a validation dataset; wherein the validation dataset contains sequencing sample information, validation hardware parameter information, and validation software parameter information corresponding to the validation data; The test sample information and verification hardware parameter information corresponding to the verification data are input into the target model to be trained to obtain the sample software parameter information output by the target model. Based on the sample software parameter information and the verification software parameter information, the target loss value for this training is determined, and the network parameters of the target model are adjusted based on the target loss value.

6. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire sequencing sample information corresponding to the sequencing chip, as well as hardware parameter information of the hardware device used to perform sequencing data processing; wherein, the sequencing sample information is used to characterize the features of the sample to be sequenced in the sequencing chip. An input module is used to input the sequencing sample information and the hardware parameter information into a pre-trained target model to obtain target software parameter information output by the target model for performing sequencing data processing; wherein, the target software parameter information includes at least one of the following: number of read threads, number of write threads, and number of computation threads; The processing module is used to perform sequencing data splitting processing on the sequencing data corresponding to the sample to be sequenced based on the target software parameter information, so as to convert the data format of the sequencing data from the initial data format to the target data format.

7. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the data processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the data processing method as described in any one of claims 1 to 5.

Images