Laboratory data processing method and system based on big data

Abstract

The invention discloses a laboratory data processing method and system based on big data, and relates to the technical field of laboratory data process.The method comprises the following steps that current laboratory historical big data are collected, a cloud server is selected, and a laboratory database is established in the cloud server; the data processing module is used for uploading, storing and transmitting laboratory associated data, and marking and pairing data of the same batch based on a time interval of data uploading in a laboratory database and a time interval threshold value; for the data directly transmitted by the user from the cloud server, the transmission speed and the transmission data size are calculated in combination with the cloud based on the sending sequence of the sending side data. According to the invention, through the remark and marking technology based on the uploading sequence, the problem that the sequences of the two parties of the laboratory data in the cloud server are inconsistent is solved, the accuracy and efficiency of data processing are improved, the operation and maintenance cost is reduced, and reliable and efficient cloud service support is provided for the laboratory.

Description

Technical Field

[0001] The present invention relates to the technical field of laboratory data processing, and in particular to a laboratory data processing method and system based on big data. Background Art

[0002] Cloud computing is a technology and service model that provides computing resources such as servers, storage, and databases on demand through the Internet. Cloud computing technology is widely used in laboratory data processing. A large amount of laboratory-related data is stored in cloud servers, and laboratory personnel frequently send and retrieve data.

[0003] The existing laboratory data processing process is mostly parallel data transmission, that is, the transmission speed is evenly divided and data is transmitted simultaneously, which easily leads to the data with small storage occupancy being transmitted first, and the data with large storage occupancy being transmitted later, causing the data order on the sending and receiving sides to be disordered, and information errors are very likely to occur, which affects the normal operation of the laboratory, and further affects the accuracy of the data and the normal development of the business. At the same time, in the process of extracting data from the laboratory cloud server, the operators on the extraction end need to spend a lot of time and resources to verify the order of the extracted data, which increases the complexity and cost of data processing.

[0004] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0005] In response to the problems in the related art, the present invention proposes a laboratory data processing method and system based on big data to overcome the data disorder problem existing in the existing related technology.

[0006] To this end, the specific technical solutions adopted in the present invention are as follows:

[0007] A laboratory data processing method based on big data, the method comprising the following steps:

[0008] S1. Collect the current laboratory's historical big data, select a cloud server, and establish a laboratory database in the cloud server to upload, store, and transmit laboratory-related data. Based on the time interval between data uploads in the laboratory database, mark and pair the same batch of data based on the time interval threshold, and annotate the data with a number to distinguish the priority of the data in the same batch;

[0009] S2. For data transmitted directly from the cloud server by users, the data is synchronized and marked during the transmission process based on the sending order of the sending side data, combined with the cloud computing transmission speed and the size of the transmitted data. Different transmission strategies are selected according to the current cloud server transmission rate and the size of different transmitted data. At the same time, the remaining size of the data during transmission is calculated to adjust the data transmission strategy in real time and unify the sending side data and the receiving side data.

[0010] S3. For the laboratory data extracted by the user from the cloud server, combined with the tags of the extracted data, the same batch of data in the extracted data is grouped according to their association, and the speed of extracting the associated data is controlled to meet the extraction order of the associated data.

[0011] As a preferred embodiment, the S1 comprises the following steps:

[0012] S11. Calculate the daily data volume, data processing, and storage requirements of the current laboratory over the next two years. Based on the upper limit of the current laboratory's daily data volume, data processing, and storage requirements, provide a cloud server and create a laboratory database.

[0013] S12. For the data uploaded by the current laboratory to the laboratory database in the cloud server, batch marking is performed on the uploaded data based on the time interval of the uploaded data to determine the priority of the uploaded data in the same batch.

[0014] As a preferred embodiment, the S12 includes the following steps:

[0015] S121: For data uploaded by the user to the cloud server laboratory database, determine the batch of data based on the time interval threshold θ between consecutive data uploads by the user. The specific steps are as follows:

[0016] For the data set D uploaded by the user, D = {d1, d2, ..., d n}, d i Represents the i-th data in the data set, each data d i The upload time is recorded as t i , for two adjacent data d i and d i+1 , the time interval T i for:

[0017] T i =t i+1 -t i ;

[0018] Set the labeled batch set B={b1,b2,...,b n}, b iRepresents the batch to which the i-th data belongs, initializes the batch counter k=1, traverses the data set D, assigns the first data d1 to batch b1=k, and determines and assigns batches to subsequent data based on the time interval and time interval threshold:

[0019] For the i-th data d i , where i>1, calculate its difference with the previous data d i-1 The time interval T i-1 , when T i-1 <θ, represents d i with d i-1 Belong to the same batch, that is, b i =k;

[0020] When T i-1 When ≥θ, it represents d i with d i-1 Do not belong to the same batch, update the batch counter k=k+1, and change d i Assign a new batch b i =k;

[0021] S122. For the data in the laboratory database of the same batch, sort them in chronological order based on the upload time of different data in the same batch, and number them sequentially in the notes of the same batch of data to obtain the priority of the same batch of data. The note format of each data is XX-XXXX, where XX is the batch of the current data and XXXX represents the number of the data under the current batch.

[0022] As a preferred embodiment, the S2 comprises the following steps:

[0023] S21. For data transmitted directly by the user in the cloud server, based on the sending order of the data on the sending side, the data sequence is synchronously noted during the data transmission process. The specific steps are as follows:

[0024] Based on the sending side data sequence S={s1,s2,...,s m}, where s i Indicates the data sent by the i-th sending side. When sending each data s i Add a sequence note p i , where p i Increment in the order of sending, i.e. p1=1, p2=1, ..., p m =m;

[0025] S22. Based on the data remark sequence during the sending process, combined with the current data file size and transmission speed, the transmission rate of different data in the current transmission process is adjusted through an intelligent algorithm to unify the data sequence on the sending side and the data sequence on the receiving side.

[0026] As a preferred embodiment, the S22 includes the following sub-steps:

[0027] S221, according to the sending side data sequence S={s1,s2,...,s m}, obtain the data size H in the data sequence on the sending side = {h1,h2,...,h m}, combined with the current maximum cloud computing transmission speed V0, calculate the data sequence S={s1,s2,...,s m}Receive side data time:

[0028]

[0029] Among them, U i represents the transmission time of the i-th data in the data sequence, h i Indicates the size of the i-th data, and arranges the transmission time of each data in the sending side data sequence in ascending order. When the ascending order is consistent with the transmission time of each data s i Additional sequence notes p i If they are consistent, it means that the current data sequence on the sending side is consistent with the data sequence on the receiving side, and parallel data transmission is directly adopted;

[0030] S222, when the ascending order is the same as sending each data s i Additional sequence notes p i If the data is inconsistent, it means that the size and order of the data on the current sending side do not match. i Data transmission and obtain real-time transmission process s i The remaining data size r i :

[0031] When r i <h i+1 When , it represents the current data s i 、s i+1 The sequential transmission requirements in S221 have been met, and the data s i With s i+1 Perform parallel data transmission and evenly distribute the transmission speed V0 to s i With s i+1 Data transmission in

[0032] When r i <r i+1 <h i+2 When , it represents the current data s i 、s i+1 、s i+2 The sequential transmission requirements in S221 have been met, and the data s i 、s i+1 、s i+2Perform parallel data transmission and evenly distribute the transmission speed V0 to s i 、s i+1 、s i+2 Data is transmitted in the data sequence, and all data in the data sequence are compared in this way;

[0033] S223, repeat step S222 until the sending side data sequence S={s1,s2,...,s m}Transfer completed.

[0034] As a preferred embodiment, the S3 includes the following steps:

[0035] S31. Based on the data extracted by the user from the cloud server laboratory database, extract notes for the same batch of extracted data and group the data by relevance. The specific steps are as follows:

[0036] According to the total data set extracted by the user from the laboratory database in the cloud server, based on the notes of each data in the total data set, the data of the same batch in the total data set are divided into the same subset and arranged in ascending order based on the note number to obtain the subset. Where α represents the number of subsets, that is, the number of different batches of data in the total set, represents the data unit in subset α;

[0037] S32. Based on the data notes in each subset, control the data extraction speed to adjust the data extraction order so that the data in the association group meets the note numbering requirements.

[0038] As a preferred embodiment, the S32 includes the following steps:

[0039] S321, based on each subset Get the data size set of each subset

[0040] S322, for each set The data in the system is extracted in sequence and the real-time extraction process is obtained. The remaining data size

[0041] when When, it represents the current data Already meets the requirements for sequential extraction, data and Perform parallel data extraction and evenly distribute the transmission speed from V0 to and Data extraction was performed in

[0042] when When, it represents the current data Already meets the requirements for sequential extraction, data Perform parallel data extraction and evenly distribute the transmission speed from V0 to , and so on, all the data in the data sequence are compared until the data extraction in all subsets is completed in sequence.

[0043] A laboratory data processing system based on big data, including a data marking module, a sequence unification module, and an association adjustment module;

[0044] The data marking module batch-marks the data uploaded by the current laboratory to the laboratory database in the cloud server based on the time interval of the uploaded data, determines the priority of the data uploaded in the same batch, and adds notes to the data in the laboratory database;

[0045] The sequence unification module, for data transmitted directly from the cloud server by users, marks the data in a synchronized sequence during the transmission process based on the sending order of the data on the sending side, combined with the cloud computing transmission speed and the size of the transmitted data. Different transmission strategies are selected according to the current cloud server transmission rate and the size of different transmitted data. At the same time, the remaining size of the data in the transmission process is calculated to adjust the data transmission strategy in real time and unify the sending and receiving side data.

[0046] The association adjustment module groups the same batch of data in the laboratory data extracted by the user from the cloud server according to their association, combined with the tags of the extracted data, and controls the extraction speed of the associated data to meet the extraction order of the associated data.

[0047] The beneficial effects of the present invention are:

[0048] 1. The present invention optimizes the transmission of data sent by users from the cloud server based on remarks, giving priority to data transmission of the first-ranked data. When the remaining data size of the first-ranked data is smaller than that of the second-ranked data, the first-ranked data and the second-ranked data are transmitted in parallel. Similarly, the sizes of all data in all sending-side data sequences are compared to unify the order of the sending-side data with the receiving-side data, thus avoiding information errors caused by unequal data ordering between the sender and the receiver during the data transmission process, thereby enhancing practicality.

[0049] 2. The present invention divides the current data set into subsets based on the data batch number, arranges the subsets in ascending order based on the batch number, and compares the remaining data size of the data with the batch number in the extraction process with the size of the data with the batch number in the end. This allows the data set extracted by the current user to be sequentially extracted based on the order of the data entered into the laboratory database. This allows the receiving end to perform subsequent data processing directly based on the order of the extracted data without the need to verify whether the data order is correct.

[0050] 3. The present invention annotates and marks the laboratory data in the cloud server based on the upload order, so that when the cloud server data is sent or data is extracted in the laboratory, the sent or extracted data are in the same order on the sending side and the receiving side, avoiding data confusion or errors caused by sequence disorder, and reducing the time for manual data verification afterwards, thereby improving the overall data processing efficiency, ensuring the smoothness of the business process, and reducing rework or delays caused by data sequence problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 is a flow chart of a laboratory data processing method based on big data according to an embodiment of the present invention;

[0053] Figure 2 4 is a block diagram of a laboratory data processing system based on big data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0055] According to an embodiment of the present invention, a laboratory data processing method and system based on big data are provided.

[0056] The present invention will now be further described with reference to the accompanying drawings and specific embodiments:

[0057] Example 1:

[0058] like Figure 1 As shown, a laboratory data processing method based on big data according to an embodiment of the present invention includes the following steps:

[0059] S1. Collect the current laboratory's historical big data, select a cloud server, and establish a laboratory database in the cloud server to upload, store, and transmit laboratory-related data. Based on the time interval between data uploads in the laboratory database, mark and pair the same batch of data based on the time interval threshold, and annotate the data with a number to distinguish the priority of the data in the same batch;

[0060] S11. Calculate the daily data volume, data processing, and storage requirements of the current laboratory over the next two years. Based on the upper limit of the current laboratory's daily data volume, data processing, and storage requirements, provide a cloud server and create a laboratory database.

[0061] S12. For the data uploaded by the current laboratory to the laboratory database in the cloud server, batch-mark the uploaded data based on the time interval of the uploaded data, and determine the priority of the uploaded data in the same batch;

[0062] S121: For data uploaded by the user to the cloud server laboratory database, determine the batch of data based on the time interval threshold θ between consecutive data uploads by the user. The specific steps are as follows:

[0063] For the data set D uploaded by the user, D = {d1, d2, ..., d n}, d i Represents the i-th data in the data set, each data d i The upload time is recorded as t i , for two adjacent data d i and d i+1 , the time interval T i for:

[0064] T i =t i+1 -t i ;

[0065] Set the labeled batch set B={b1,b2,...,b n}, b i Represents the batch to which the i-th data belongs, initializes the batch counter k=1, traverses the data set D, assigns the first data d1 to batch b1=k, and determines and assigns batches to subsequent data based on the time interval and time interval threshold:

[0066] For the i-th data d i , where i>1, calculate its difference with the previous data d i-1 The time interval T i-1, when T i-1 <θ, represents d i with d i-1 Belong to the same batch, that is, b i =k;

[0067] When T i-1 When ≥θ, it represents d i with d i-1 Do not belong to the same batch, update the batch counter k=k+1, and change d i Assign new batch b i =k;

[0068] It should be noted that each data d i Each batch will be assigned a batch number b i , to indicate the batch to which it belongs, and the value of the batch number k will be dynamically adjusted according to the upload time interval and time interval threshold, thereby completing the batch assignment of the user uploaded data, that is:

[0069]

[0070] S122. For the data in the laboratory database of the same batch, sort them in chronological order based on the upload time of different data in the same batch, and number them sequentially in the notes of the same batch of data to obtain the priority of the same batch of data. The note format of each data is XX-XXXX, where XX is the batch of the current data and XXXX represents the number of the data under the current batch.

[0071] It should be noted that the data in the same batch are numbered sequentially, that is, the format of each data is XX-XXXX, which means that the data in the same batch XX is numbered XXXX.

[0072] S2. For data transmitted directly from the cloud server by users, the data is synchronized and marked during the transmission process based on the sending order of the sending side data, combined with the cloud computing transmission speed and the size of the transmitted data. Different transmission strategies are selected according to the current cloud server transmission rate and the size of different transmitted data. At the same time, the remaining size of the data during transmission is calculated to adjust the data transmission strategy in real time and unify the sending side data and the receiving side data.

[0073] S21. For data transmitted directly from the cloud server by the user, the data sequence is noted during the data transmission process based on the transmission order of the data on the sending side. The specific steps are as follows:

[0074] Based on the sending side data sequence S={s1,s2,...,s m}, where s i Indicates the data sent by the i-th sending side. When sending each data si Add a sequence note p i , where p i Increment in the order of sending, i.e. p1=1, p2=1, ..., p m =m;

[0075] S22. Based on the order of data notes during the transmission process, combined with the size of the current data file and the transmission speed, an intelligent algorithm is used to adjust the transmission rate of different data during the current transmission process, and unify the data order on the sending side and the data order on the receiving side;

[0076] S221, based on the sending side data sequence S={s1,s2,...,s m}, obtain the data size H in the data sequence on the sending side = {h1,h2,...,h m}, combined with the current maximum cloud computing transmission speed V0, calculate the data sequence S={s1,s2,...,s m}Receive side data time:

[0077]

[0078] Among them, U i represents the transmission time of the i-th data in the data sequence, h i Indicates the size of the i-th data, and arranges the transmission time of each data in the sending side data sequence in ascending order. When the ascending order is consistent with the transmission time of each data s i Additional sequence notes p i If they are consistent, it means that the current data sequence on the sending side is consistent with the data sequence on the receiving side, and parallel data transmission is directly adopted;

[0079] It should be noted that based on the maximum transmission speed V0 of cloud computing, the transmission time of each data in the sending side data sequence is calculated. When the ascending order is the same as the transmission time of each data s i Additional sequence notes p i When they are consistent, it means that the data size on the sending side is consistent with the data order on the sending side, that is, the earlier the data is in the order, the smaller the data size. Based on the parallel data transmission strategy, the current cloud computing maximum transmission rate V0 is evenly divided, and all data on the sending side are transmitted at the same time.

[0080] S222, when the ascending order is the same as sending each data s i Additional sequence notes p i If the data is inconsistent, it means that the size and order of the data on the current sending side do not match. i Data transmission and obtain real-time transmission process s i The remaining data size r i :

[0081] When r i <h i+1 When , it represents the current data s i 、s i+1 The sequential transmission requirements in S221 have been met, and the data s i With s i+1 Perform parallel data transmission and evenly distribute the transmission speed V0 to s i With s i+1 Data transmission in

[0082] When r i <r i+1 <h i+2 When , it represents the current data s i 、s i+1 、s i+2 The sequential transmission requirements in S221 have been met, and the data s i 、s i+1 、s i+2 Perform parallel data transmission and evenly distribute the transmission speed V0 to s i 、s i+1 、s i+2 Data is transmitted in the data sequence, and all data in the data sequence are compared in this way;

[0083] S223, repeat step S222 until the sending side data sequence S={s1,s2,...,s m}Transfer completed.

[0084] It should be noted that, based on the order of the data on the sending side, the data that is ranked first is given priority for data transmission. When the remaining data size of the data ranked first is smaller than that of the data ranked second, the data ranked first and the data ranked second are transmitted in parallel. Similarly, the sizes of all data in the data sequence on the sending side are compared to make the order of the data on the sending side consistent with the order of the data on the receiving side, thereby avoiding information errors caused by unequal data ordering between the sender and the receiver during the data transmission process.

[0085] Example 2:

[0086] S3: For the laboratory data extracted by the user from the cloud server, the same batch of data in the extracted data is grouped according to their association, and the speed of extracting the associated data is controlled to ensure the order of extracting the associated data.

[0087] S31. Based on the data extracted by the user from the cloud server laboratory database, extract notes for the same batch of extracted data and group the data by relevance. The specific steps are as follows:

[0088] According to the total data set extracted by the user from the laboratory database in the cloud server, based on the notes of each data in the total data set, the data of the same batch in the total data set are divided into the same subset and arranged in ascending order based on the note number to obtain the subset. Where α represents the number of subsets, that is, the number of different batches of data in the total set, represents the data unit in subset α;

[0089] S32. Based on the data notes in each subset, control the data extraction speed to adjust the data extraction order so that the data in the relevance group meets the note numbering requirements;

[0090] S32 includes the following steps:

[0091] S321, based on each subset Get the data size set of each subset

[0092] S322, for each set The data in the system is extracted in sequence and the real-time extraction process is obtained. The remaining data size

[0093] when When, it represents the current data Already meets the sequential extraction requirements, the data and Perform parallel data extraction and evenly distribute the transmission speed from V0 to and Data extraction was performed in

[0094] when When, it represents the current data Already meets the sequential extraction requirements, the data Perform parallel data extraction and evenly distribute the transmission speed from V0 to , and so on, all the data in the data sequence are compared until the data extraction in all subsets is completed in sequence.

[0095] It should be noted that the current total data set is divided into subsets based on the data batch number, and arranged in ascending order based on the numbers in the subsets. The remaining data size of the data with earlier numbers during the extraction process is compared with the size of the data with later numbers, so as to achieve sequential extraction of the total data set extracted by the current user based on the order of data entered into the laboratory database, so that the receiving end does not need to additionally verify whether the order of the data is correct, and can directly perform subsequent data processing based on the order of the extracted data.

[0096] Example 3:

[0097] like Figure 2 As shown, a laboratory data processing system based on big data includes a data marking module, a sequence unification module, and an association adjustment module;

[0098] The data marking module batch-marks the data uploaded by the current laboratory to the laboratory database in the cloud server based on the time interval of the uploaded data, determines the priority of the data uploaded in the same batch, and adds notes to the data in the laboratory database;

[0099] The sequence unification module synchronizes the data transmitted directly from the cloud server to the sending side based on the sending order of the data, combined with the cloud computing transmission speed and the size of the transmitted data. It selects different transmission strategies based on the current cloud server transmission rate and the size of the different transmitted data. It also calculates the remaining size of the data during transmission to adjust the data transmission strategy in real time and unify the sending and receiving side data.

[0100] The association adjustment module groups the same batch of data in the laboratory data extracted by users from the cloud server according to their association, combined with the tags of the extracted data, and controls the extraction speed of the associated data to meet the extraction order of the associated data.

[0101] In summary, the present invention optimizes the transmission of data sent by users from the cloud server based on the remarks, prioritizes the data in the first place, and when the remaining data size of the data in the first place is smaller than that of the data in the second place, transmits the data in the first place and the data in the second place in parallel. Similarly, the sizes of all data in all sending-side data sequences are compared, so that the order of the sending-side data is unified with the order of the receiving-side data, thereby avoiding the occurrence of information errors caused by unequal data ordering between the sender and the receiver during the data transmission process, thereby enhancing practicality.

[0102] By annotating and marking the laboratory data in the cloud server based on the upload order, when the cloud server data is sent or extracted in the laboratory, the order of the sent or extracted data is consistent on the sending and receiving sides, avoiding data confusion or errors caused by sequence disorder, and reducing the time for manual data verification afterwards, improving the overall data processing efficiency, ensuring the smoothness of the business process, and reducing rework or delays caused by data sequence problems.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A laboratory data processing method based on big data, characterized in that: The method comprises the following steps: S1. Collect the current laboratory's historical big data, select a cloud server, and establish a laboratory database in the cloud server to upload, store, and transmit laboratory-related data. Based on the time interval between data uploads in the laboratory database, mark and pair the same batch of data based on the time interval threshold, and annotate the data with a number to distinguish the priority of the data in the same batch; S2. For data transmitted directly from the cloud server by users, the data is synchronized and marked during the transmission process based on the sending order of the sending side data, combined with the cloud computing transmission speed and the size of the transmitted data. Different transmission strategies are selected according to the current cloud server transmission rate and the size of different transmitted data. At the same time, the remaining size of the data during transmission is calculated to adjust the data transmission strategy in real time and unify the sending side data and the receiving side data. S3. For the laboratory data extracted by the user from the cloud server, combined with the tags of the extracted data, the same batch of data in the extracted data is grouped according to their association, and the speed of extracting the associated data is controlled to meet the extraction order of the associated data.

2. A laboratory data processing method based on big data according to claim 1, characterized in that: Said S1 comprises the following steps: S11. Calculate the daily data volume, data processing, and storage requirements of the current laboratory over the next two years. Based on the upper limit of the current laboratory's daily data volume, data processing, and storage requirements, provide a cloud server and create a laboratory database. S12. For the data uploaded by the current laboratory to the laboratory database in the cloud server, batch marking is performed on the uploaded data based on the time interval of the uploaded data to determine the priority of the uploaded data in the same batch.

3. A laboratory data processing method based on big data according to claim 2, characterized in that: The S12 includes the following steps: S121: For data uploaded by the user to the cloud server laboratory database, determine the batch of data based on the time interval threshold θ between consecutive data uploads by the user. The specific steps are as follows: For the data set D uploaded by the user, D = {d1, d2, ..., d n }, d i Represents the i-th data in the data set, each data d i The upload time is recorded as t i , for two adjacent data d i and d i+1 , the time interval T i for: T i =t i+1 -t i ; Set the labeled batch set B={b1,b2,...,b n }, b i Represents the batch to which the i-th data belongs, initializes the batch counter k=1, traverses the data set D, assigns the first data d1 to batch b1=k, and determines and assigns batches to subsequent data based on the time interval and time interval threshold: For the i-th data d i , where i>1, calculate its difference with the previous data d i-1 The time interval T i-1 , when T i-1 <θ, represents d i with d i-1 Belong to the same batch, that is, b i =k; When T i-1 When ≥θ, it represents d i with d i-1 Do not belong to the same batch, update the batch counter k=k+1, and change d i Assign a new batch b i =k; S122. For the data in the laboratory database of the same batch, sort them in chronological order based on the upload time of different data in the same batch, and number them sequentially in the notes of the same batch of data to obtain the priority of the same batch of data. The note format of each data is XX-XXXX, where XX is the batch of the current data and XXXX represents the number of the data under the current batch.

4. The method for processing laboratory data based on big data according to claim 1, characterized in that: The S2 comprises the following steps: S21. For data transmitted directly by the user in the cloud server, based on the sending order of the data on the sending side, the data sequence is synchronously noted during the data transmission process. The specific steps are as follows: Based on the sending side data sequence S={s1,s2,...,s m }, where s i Indicates the data sent by the i-th sending side. When sending each data s i Add a sequence note p i , where p i Increment in the order of sending, i.e. p1=1, p2=1, ..., p m =m; S22. Based on the data remark sequence during the sending process, combined with the current data file size and transmission speed, the transmission rate of different data in the current transmission process is adjusted through an intelligent algorithm to unify the data sequence on the sending side and the data sequence on the receiving side.

5. The method for processing laboratory data based on big data according to claim 4, characterized in that: The S22 includes the following sub-steps: S221, according to the sending side data sequence S={s1,s2,...,s m }, obtain the data size H in the data sequence on the sending side = {h1,h2,...,h m }, combined with the current maximum cloud computing transmission speed V0, calculate the data sequence S={s1,s2,...,s m }Receive side data time: Among them, U i represents the transmission time of the i-th data in the data sequence, h i Indicates the size of the i-th data, and arranges the transmission time of each data in the sending side data sequence in ascending order. When the ascending order is consistent with the transmission time of each data s i Additional sequence notes p i If they are consistent, it means that the current data sequence on the sending side is consistent with the data sequence on the receiving side, and parallel data transmission is directly adopted; S222, when the ascending order is the same as sending each data s i Additional sequence notes p i If the data is inconsistent, it means that the size and order of the data on the current sending side do not match. i Data transmission and obtain real-time transmission process s i The remaining data size r i : When r i <h i+1 When , it represents the current data s i 、s i+1 The sequential transmission requirements in S221 have been met, and the data s i With s i+1 Perform parallel data transmission and evenly distribute the transmission speed V0 to s i With s i+1 Data transmission in When r i <r i+1 <h i+2 When , it represents the current data s i 、s i+1 、s i+2 The sequential transmission requirements in S221 have been met, and the data s i 、s i+1 、s i+2 Perform parallel data transmission and evenly distribute the transmission speed V0 to s i 、s i+1 、s i+2 Data is transmitted in the data sequence, and all data in the data sequence are compared in this way; S223, repeat step S222 until the sending side data sequence S={s1,s2,...,s m }Transfer completed.

6. The method for processing laboratory data based on big data according to claim 3, characterized in that: The S3 includes the following steps: S31. Based on the data extracted by the user from the cloud server laboratory database, extract notes for the same batch of extracted data and group the data by relevance. The specific steps are as follows: According to the total data set extracted by the user from the laboratory database in the cloud server, based on the notes of each data in the total data set, the data of the same batch in the total data set are divided into the same subset and arranged in ascending order based on the note number to obtain the subset. Where α represents the number of subsets, that is, the number of different batches of data in the total set, represents the data unit in subset α; S32. Based on the data notes in each subset, control the data extraction speed to adjust the data extraction order so that the data in the association group meets the note numbering requirements.

7. The method for processing laboratory data based on big data according to claim 6, characterized in that: The S32 includes the following steps: S321, based on each subset Get the data size set of each subset S322, for each set The data in the system is extracted in sequence and the real-time extraction process is obtained. The remaining data size when When, it represents the current data Already meets the requirements for sequential extraction, data and Perform parallel data extraction and evenly distribute the transmission speed from V0 to and Data extraction was performed in when When, it represents the current data Already meets the sequential extraction requirements, the data Perform parallel data extraction and evenly distribute the transmission speed from V0 to , and so on, all the data in the data sequence are compared until the data extraction in all subsets is completed in sequence.

8. A laboratory data processing system based on big data, characterized in that: The system adopts the laboratory data processing method based on big data as described in any one of claims 1 to 7, including a data marking module, a sequence unification module, and an association adjustment module; The data marking module batch-marks the data uploaded by the current laboratory to the laboratory database in the cloud server based on the time interval of the uploaded data, determines the priority of the data uploaded in the same batch, and adds notes to the data in the laboratory database; The sequence unification module, for data transmitted directly from the cloud server by users, marks the data in a synchronized sequence during the transmission process based on the sending order of the data on the sending side, combined with the cloud computing transmission speed and the size of the transmitted data. Different transmission strategies are selected according to the current cloud server transmission rate and the size of different transmitted data. At the same time, the remaining size of the data in the transmission process is calculated to adjust the data transmission strategy in real time and unify the sending and receiving side data. The association adjustment module groups the same batch of data in the laboratory data extracted by the user from the cloud server according to their association, combined with the tags of the extracted data, and controls the extraction speed of the associated data to meet the extraction order of the associated data.

Images