Base identification method and device, electronic equipment and storage medium

Abstract

According to the base identification method and device, the electronic equipment and the storage medium provided by the invention, the image data is acquired, the image data is converted to obtain the signal intensity data, and the signal intensity data is integrated to obtain the input data; performing data conversion on the input data according to the trained preset detection model to obtain a to-be-detected conversion vector; performing spatial feature extraction on the to-be-detected conversion vector to obtain a spatial feature vector, and performing time feature extraction on the spatial feature vector to obtain a to-be-detected feature vector; and carrying out dimensionality reduction on the to-be-detected feature vector to obtain a to-be-classified vector, and carrying out base classification according to the to-be-classified vector to obtain a base identification result. Compared with the prior art, the embodiment of the invention has the advantages that by adopting the preset detection model, from the perspective of time and space, not only can the spatial association between the to-be-detected data be extracted, but also the time sequence information of the to-be-detected data can be extracted, the feature extraction capability of the data features is enhanced, the calculation cost is reduced, and the accuracy of base identification is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method and device for base identification, an electronic device, and a storage medium. Background Art

[0002] Base calling (base recognition technology) is an algorithm (software) that uses computer vision to identify base types (DNA sequences) from row images (raw images), writes the results to a cal file, and ultimately generates a sequencing report and FastQ data. Each shot of the sequencer's camera captures a square area called the field of view (FOV). This FOV displays the fluorescent image of tens of thousands of DNA nanoballs (DNBs) on the sequencing chip or other amplified targets on other sequencing chips, called base fluorescence. The base calling algorithm processes this fluorescent image and converts the fluorescent signal of each DNB on the sequencing chip into a base sequence.

[0003] Currently, there are multiple base calling algorithms for base detection based on different sequencing platforms and technologies, such as base calling algorithms based on statistical models and machine learning methods (Bustard and Alta-Cyclic), algorithms in TorrentSuite software, and algorithms in Zebracall software. In summary, the above methods often encounter problems such as signal crosstalk and high error rates. Therefore, machine learning models or electrical signal processing are currently the main methods used to address noise factors.

[0004] However, the base calling algorithms based on machine learning models in the existing technology require the use of support vector machines (SVM) or the maximum expectation algorithm and clustering algorithm to detect bases. When using SVM, supervised learning is required to optimize the SVM parameters on the reference sequence, and the SVM is trained for each grid point on the reference sequence. When using the maximum expectation algorithm and clustering algorithm, accuracy is sacrificed in order to improve speed. As a result, the existing base calling algorithms based on machine learning models have high computational costs and low accuracy. Summary of the Invention

[0005] The present disclosure provides a method and apparatus for base calling, an electronic device, and a storage medium. The main purpose of the present disclosure is to solve the problems of high computational cost and low accuracy of existing base calling algorithms based on machine learning models.

[0006] According to a first aspect of the present disclosure, a method for base identification is provided, comprising:

[0007] Acquire a first preset amount of image data in each field of view, perform format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrate the signal strength data according to position information of the object to be detected in the image data to obtain input data, wherein each input data includes a second preset amount of signal strength data;

[0008] Performing data conversion processing on the input data according to the trained preset detection model to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected;

[0009] Based on the trained preset detection model, performing spatial feature extraction processing on the conversion vector to be detected to obtain a spatial feature vector, and performing temporal feature extraction processing on the spatial feature vector to obtain a feature vector to be detected;

[0010] Based on the trained preset detection model, the feature vector to be detected is subjected to dimensionality reduction processing to obtain a corresponding vector to be classified of a second preset dimension, and base classification is performed according to the vector to be classified to obtain a base recognition result.

[0011] Optionally, the performing format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrating the signal strength data according to position information of the object to be detected in the image data to obtain input data includes:

[0012] Performing bleeding processing on the image data using a preset image processing algorithm, and extracting the signal strength data corresponding to each of the objects to be detected from the processed image data;

[0013] Obtaining an index of each of the objects to be detected in the image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0014] Determine the position information of each of the objects to be detected in the image data according to the index;

[0015] Based on the position information, the signal strength data corresponding to the second preset number of objects to be detected are respectively integrated with each of the objects to be detected as the center until all the signal strength data are processed to obtain the input data.

[0016] Optionally, performing data conversion processing on the input data according to the trained preset detection model to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected includes:

[0017] Performing data conversion on the input data using a preset full-connection algorithm to obtain an object vector of a first preset dimension corresponding to each object to be detected;

[0018] Randomly assigning a position coding vector to each of the objects to be detected, and adding the object vector corresponding to the object to be detected to the assigned position coding vector to obtain a calculation vector to be detected, wherein the position coding vector is used to determine spatial feature information in the calculation vector to be detected;

[0019] The calculation vector to be detected is subjected to data reshaping processing by a preset reshaping algorithm to obtain the conversion vector to be detected.

[0020] Optionally, performing spatial feature extraction processing on the to-be-detected conversion vector based on the trained preset detection model to obtain a spatial feature vector includes:

[0021] Performing self-attention calculation on the transformation vector to be detected through a preset attention algorithm to obtain an attention vector;

[0022] Inputting the attention vector into a first preset fully connected structure, performing data operation with a first fully connected parameter, and activating the structure through a preset function to obtain a first spatial vector, wherein the first fully connected parameter is used to extract spatial features from the attention vector;

[0023] Inputting the first space vector into a second preset fully connected structure, performing data operation with a second fully connected parameter to obtain a second space vector, wherein the second fully connected parameter is used to further extract spatial features from the first space vector;

[0024] After calculating the second space vector using a first preset residual function, the first space vector is added to the second space vector to obtain an initial space vector;

[0025] The initial spatial vector is pooled using a preset pooling algorithm to obtain the spatial feature vector.

[0026] Optionally, performing temporal feature extraction processing on the spatial feature vector to obtain the feature vector to be detected includes:

[0027] Performing self-attention calculation on the spatial feature vector through a preset attention algorithm to obtain a spatial attention vector;

[0028] Inputting the spatial attention vector into a third preset fully connected structure, performing data operation with a third fully connected parameter, and activating the structure through a preset function to obtain a first time vector, wherein the third fully connected parameter is used to extract time features from the spatial attention vector;

[0029] Inputting the first time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second time vector, wherein the fourth fully connected parameter is used to further extract time features from the first time vector;

[0030] After the second time vector is calculated by a second preset residual function, the second time vector is added to the second time vector to obtain the feature vector to be detected.

[0031] Optionally, performing dimensionality reduction processing on the feature vector to be detected based on the trained preset detection model to obtain a corresponding vector to be classified of a second preset dimension includes:

[0032] Performing a regularization operation on the feature vector to be detected by a preset regularization algorithm to obtain a regularized feature vector;

[0033] The regularized feature vector is subjected to dimensionality reduction processing through a fifth preset fully connected structure to obtain the vector to be classified of the second preset dimension.

[0034] According to a second aspect of the present disclosure, a method for training a preset detection model is provided, wherein the preset detection model can be applied to the method described in the first aspect, including:

[0035] Acquiring a third preset number of training image data in each field of view, performing format conversion processing on the training image data respectively to obtain training signal strength data corresponding to each training object to be detected, and integrating the training signal strength data according to position information of the training object to be detected in the training image data to obtain training input data, wherein each training input data includes a fourth preset number of training signal strength data;

[0036] Obtaining a true label corresponding to each of the training objects to be detected, wherein the true label is a known base recognition result of the training object to be detected;

[0037] Performing data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each training to-be-detected object;

[0038] Based on the preset detection model, performing spatial feature extraction processing on the training to-be-detected conversion vector to obtain a training spatial feature vector, and performing temporal feature extraction processing on the training spatial feature vector to obtain a training to-be-detected feature vector;

[0039] Based on the preset detection model, performing dimensionality reduction processing on the training feature vector to be detected to obtain a corresponding training vector to be classified of a fourth preset dimension, and performing base classification according to the training vector to be classified to obtain a training base recognition result;

[0040] Based on the training base recognition results and the true labels, the preset detection model is optimized by a preset loss function and a preset optimization function until a preset convergence condition is met, thereby obtaining a trained preset detection model.

[0041] Optionally, performing format conversion processing on the training image data respectively to obtain training signal strength data corresponding to each training object to be detected, and integrating the training signal strength data according to position information of the training object to be detected in the training image data to obtain the training input data includes:

[0042] Performing bleeding processing on the training image data using a preset image processing algorithm, and extracting the training signal strength data corresponding to each of the training objects to be detected from the processed image data;

[0043] Obtaining an index of each training object to be detected in the training image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0044] Determine the position information of each of the training objects to be detected in the training image data according to the index;

[0045] Based on the position information, the training signal strength data corresponding to the fourth preset number of training objects to be detected are integrated with each of the training objects to be detected as the center until all the training signal strength data are processed to obtain the training input data.

[0046] Optionally, performing data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each training to-be-detected object includes:

[0047] Performing data conversion on the training input data using a preset full-connection algorithm to obtain a training object vector of the third preset dimension corresponding to each training object to be detected;

[0048] Randomly assigning a training position coding vector to each of the training objects to be detected, and adding the training object vector corresponding to the training object to be detected to the assigned training position coding vector to obtain a training calculation vector to be detected, wherein the training position coding vector is used to determine spatial feature information in the training calculation vector to be detected;

[0049] The training calculation vector to be detected is subjected to data reshaping processing through a preset reshaping algorithm to obtain the training conversion vector to be detected.

[0050] Optionally, performing spatial feature extraction processing on the training to-be-detected conversion vector based on the preset detection model to obtain the training spatial feature vector includes:

[0051] Performing self-attention calculation on the training transformation vector to be detected through a preset attention algorithm to obtain a training attention vector;

[0052] Inputting the training attention vector into a first preset fully connected structure, performing data operation with a fifth training fully connected parameter, and activating the training attention vector through a preset function to obtain a first training space vector, wherein the first training fully connected parameter is used to extract spatial features from the training attention vector;

[0053] Inputting the first training space vector into a second preset fully connected structure, performing data operation with a second training fully connected parameter to obtain a second training space vector, wherein the second training fully connected parameter is used to further extract spatial features from the first training space vector;

[0054] After calculating the second training space vector using a first preset residual function, the vector is added to the second training space vector to obtain an initial training space vector;

[0055] The initial space vector for training is pooled using a preset pooling algorithm to obtain the space feature vector for training.

[0056] Optionally, performing temporal feature extraction processing on the training spatial feature vector to obtain a training feature vector to be detected includes:

[0057] Performing self-attention calculation on the training spatial feature vector through a preset attention algorithm to obtain a training spatial attention vector;

[0058] Inputting the training spatial attention vector into a third preset fully connected structure, performing data operation with the third fully connected parameter, and activating it through a preset function to obtain a first time vector of the line, wherein the third fully connected parameter for training is used to extract time features from the training spatial attention vector;

[0059] Inputting the first training time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second training time vector, wherein the fourth training fully connected parameter is used to further extract time features from the first training time vector;

[0060] After the second time vector for training is calculated by a second preset residual function, the second time vector for training is added to the second time vector for training to obtain the feature vector to be detected for training.

[0061] Optionally, performing dimensionality reduction processing on the training feature vector to be detected based on the preset detection model to obtain a corresponding training vector to be classified of a fourth preset dimension includes:

[0062] Performing a regularization operation on the training feature vector to be detected by a preset regularization algorithm to obtain a training regularized feature vector;

[0063] The training regularized feature vector is subjected to dimensionality reduction processing through a fifth preset fully connected structure to obtain the training vector to be classified of the fourth preset dimension.

[0064] According to a third aspect of the present disclosure, there is provided a base recognition apparatus, comprising:

[0065] a conversion unit, configured to obtain a first preset amount of image data in each field of view, and perform format conversion processing on the image data to obtain signal strength data corresponding to each object to be detected;

[0066] an integration unit, configured to integrate the signal strength data according to position information of the object to be detected in the image data to obtain input data, wherein each input data includes a second preset number of the signal strength data;

[0067] a conversion unit, configured to perform data conversion processing on the input data according to a preset detection model that has been trained, to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected;

[0068] an extraction unit, configured to perform spatial feature extraction processing on the to-be-detected conversion vector based on the trained preset detection model to obtain a spatial feature vector;

[0069] The extraction unit is further configured to perform time feature extraction processing on the spatial feature vector to obtain a feature vector to be detected;

[0070] A dimensionality reduction unit, configured to perform dimensionality reduction processing on the feature vector to be detected based on the trained preset detection model to obtain a corresponding vector to be classified of a second preset dimension;

[0071] The classification unit is used to perform base classification according to the vector to be classified to obtain a base recognition result.

[0072] Optionally, the conversion unit is further configured to perform bleeding processing on the image data using a preset image processing algorithm, and extract the signal intensity data corresponding to each of the objects to be detected from the processed image data;

[0073] The integration unit comprises:

[0074] an acquisition module, configured to acquire an index of each of the objects to be detected in the image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0075] a determination module, configured to determine position information of each of the objects to be detected in the image data according to the index;

[0076] The integration module is used to integrate the signal strength data corresponding to the second preset number of objects to be detected based on the position information and with each of the objects to be detected as the center, until all the signal strength data are processed to obtain the input data.

[0077] Optionally, the conversion unit includes:

[0078] a conversion module, configured to perform data conversion on the input data using a preset full-connection algorithm to obtain an object vector of a first preset dimension corresponding to each object to be detected;

[0079] a calculation module, configured to randomly assign a position coding vector to each of the objects to be detected, and to add the object vector corresponding to the object to be detected to the assigned position coding vector to obtain a calculation vector to be detected, wherein the position coding vector is used to determine spatial feature information in the calculation vector to be detected;

[0080] The reshaping module is used to perform data reshaping processing on the calculation vector to be detected through a preset reshaping algorithm to obtain the conversion vector to be detected.

[0081] Optionally, the extraction unit includes:

[0082] A first calculation module is used to perform self-attention calculation on the conversion vector to be detected through a preset attention algorithm to obtain an attention vector;

[0083] a processing module, configured to input the attention vector into a first preset fully connected structure, perform data operation on the attention vector and a first fully connected parameter, and then activate the attention vector through a preset function to obtain a first spatial vector, wherein the first fully connected parameter is used to extract spatial features from the attention vector;

[0084] The processing module is further configured to input the first space vector into a second preset fully connected structure, perform data operation on the first space vector and a second fully connected parameter to obtain a second space vector, wherein the second fully connected parameter is used to further extract spatial features from the first space vector;

[0085] a second calculation module, configured to calculate the second spatial vector using a first preset residual function, and then add the calculated second spatial vector to the second spatial vector to obtain an initial spatial vector;

[0086] An operation module is used to perform a pooling operation on the initial spatial vector through a preset pooling algorithm to obtain the spatial feature vector.

[0087] Optionally, the extraction unit is further used to:

[0088] Performing self-attention calculation on the spatial feature vector through a preset attention algorithm to obtain a spatial attention vector;

[0089] Inputting the spatial attention vector into a third preset fully connected structure, performing data operation with a third fully connected parameter, and activating the structure through a preset function to obtain a first time vector, wherein the third fully connected parameter is used to extract time features from the spatial attention vector;

[0090] Inputting the first time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second time vector, wherein the fourth fully connected parameter is used to further extract time features from the first time vector;

[0091] After the second time vector is calculated by a second preset residual function, the second time vector is added to the second time vector to obtain the feature vector to be detected.

[0092] Optionally, the dimensionality reduction unit includes:

[0093] An operation module, configured to perform a regularization operation on the feature vector to be detected by using a preset regularization algorithm to obtain a regularized feature vector;

[0094] A dimensionality reduction module is used to perform dimensionality reduction processing on the regularized feature vector through a fifth preset fully connected structure to obtain the vector to be classified of the second preset dimension.

[0095] According to a fourth aspect of the present disclosure, a training device for a preset detection model is provided, comprising:

[0096] a conversion unit, configured to obtain a third preset number of training image data in each field of view, and perform format conversion processing on each of the training image data to obtain training signal strength data corresponding to each training object to be detected;

[0097] an integration unit, configured to integrate the training signal strength data according to position information of the training object to be detected in the training image data to obtain training input data, wherein each training input data includes a fourth preset number of the training signal strength data;

[0098] An acquiring unit, configured to acquire a true label corresponding to each of the training objects to be detected, wherein the true label is a known base recognition result of the training object to be detected;

[0099] a conversion unit, configured to perform data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each of the training to-be-detected objects;

[0100] an extraction unit, configured to perform spatial feature extraction processing on the training to-be-detected conversion vector based on the preset detection model to obtain a training spatial feature vector;

[0101] The extraction unit is further configured to perform temporal feature extraction processing on the training spatial feature vector to obtain a training feature vector to be detected;

[0102] a dimensionality reduction unit, configured to perform dimensionality reduction processing on the training feature vector to be detected based on the preset detection model to obtain a corresponding training vector to be classified of a fourth preset dimension;

[0103] A classification unit, configured to perform base classification based on the training vector to be classified to obtain a training base recognition result;

[0104] An optimization unit is used to optimize the preset detection model based on the training base recognition results and the true labels through a preset loss function and a preset optimization function until a preset convergence condition is met, thereby obtaining a trained preset detection model.

[0105] Optionally, the conversion unit is further configured to perform bleeding processing on the training image data using a preset image processing algorithm, and extract the training signal strength data corresponding to each of the training objects to be detected from the processed image data;

[0106] The integration unit comprises:

[0107] an acquisition module, configured to acquire an index of each object to be detected for training in the training image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0108] a determination module, configured to determine position information of each of the training objects to be detected in the training image data according to the index;

[0109] An integration module is used to integrate the training signal strength data corresponding to the fourth preset number of training objects to be detected based on the position information and with each of the training objects to be detected as the center, until all the training signal strength data are processed to obtain the training input data.

[0110] Optionally, the conversion unit includes:

[0111] a conversion module, configured to perform data conversion on the training input data using a preset full-connection algorithm to obtain a training object vector of the third preset dimension corresponding to each training object to be detected;

[0112] a calculation module, configured to randomly assign a training position coding vector to each of the training objects to be detected, and add the training object vector corresponding to the training object to be detected to the assigned training position coding vector to obtain a training calculation vector to be detected, wherein the training position coding vector is used to determine spatial feature information in the training calculation vector to be detected;

[0113] The reshaping module is used to perform data reshaping processing on the training calculation vector to be detected through a preset reshaping algorithm to obtain the training conversion vector to be detected.

[0114] Optionally, the extraction unit includes:

[0115] A first calculation module is used to perform self-attention calculation on the training to-be-detected conversion vector through a preset attention algorithm to obtain a training attention vector;

[0116] a processing module, configured to input the training attention vector into a first preset fully connected structure, perform data operation with a fifth training fully connected parameter, and then activate the training attention vector through a preset function to obtain a first training space vector, wherein the first training fully connected parameter is used to extract spatial features from the training attention vector;

[0117] The processing module is further configured to input the first training space vector into a second preset fully connected structure, perform data operation with a second training fully connected parameter, and obtain a second training space vector, wherein the second training fully connected parameter is used to further extract spatial features from the first training space vector;

[0118] a second calculation module, configured to calculate the second training space vector using a first preset residual function, and then add the calculated value to the second training space vector to obtain an initial training space vector;

[0119] An operation module is used to perform a pooling operation on the initial space vector for training through a preset pooling algorithm to obtain the space feature vector for training.

[0120] Optionally, the extraction unit is further used to:

[0121] Performing self-attention calculation on the training spatial feature vector through a preset attention algorithm to obtain a training spatial attention vector;

[0122] Inputting the training spatial attention vector into a third preset fully connected structure, performing data operation with the third fully connected parameter, and activating it through a preset function to obtain a first time vector of the line, wherein the third fully connected parameter for training is used to extract time features from the training spatial attention vector;

[0123] Inputting the first training time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second training time vector, wherein the fourth training fully connected parameter is used to further extract time features from the first training time vector;

[0124] After the second time vector for training is calculated by a second preset residual function, the second time vector for training is added to the second time vector for training to obtain the feature vector to be detected for training.

[0125] Optionally, the dimensionality reduction unit includes:

[0126] An operation module, configured to perform a regularization operation on the training feature vector to be detected by using a preset regularization algorithm to obtain a training regularized feature vector;

[0127] A dimensionality reduction module is used to perform dimensionality reduction processing on the regularized feature vector for training through a fifth preset fully connected structure to obtain the vector to be classified for training with the fourth preset dimension.

[0128] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0129] at least one processor; and

[0130] a memory communicatively connected to the at least one processor; wherein,

[0131] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect or the method described in the second aspect.

[0132] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect or the method described in the second aspect.

[0133] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method described in the first aspect or the method described in the second aspect.

[0134] The method and apparatus, electronic device and storage medium for base identification provided by the present disclosure obtain a first preset number of image data in each field of view, perform format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrate the signal strength data according to the position information of the object to be detected in the image data to obtain input data, wherein each input data contains a second preset number of signal strength data; perform data conversion processing on the input data according to a preset detection model that has been trained to obtain a conversion vector to be detected of a first preset dimension corresponding to each object to be detected; based on the preset detection model that has been trained, perform spatial feature extraction processing on the conversion vector to be detected to obtain a spatial feature vector, and perform temporal feature extraction processing on the spatial feature vector to obtain a feature vector to be detected; based on the preset detection model that has been trained, perform dimensionality reduction processing on the feature vector to be detected to obtain a corresponding vector to be classified of a second preset dimension, and perform base classification based on the vector to be classified to obtain a base identification result. Compared with related technologies, the embodiments of the present disclosure adopt a preset detection model, which can extract both the spatial correlation between the data to be tested and the time series information of the data to be tested from the perspective of time and space, thereby enhancing the feature extraction capability of data features, reducing computing costs, and improving the accuracy of base recognition.

[0135] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0136] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0137] Figure 1 A schematic flow chart of a base recognition method provided in an embodiment of the present disclosure;

[0138] Figure 2 A schematic diagram of the structure of a preset detection model provided in an embodiment of the present disclosure;

[0139] Figure 3 A schematic diagram of an example of base recognition provided by an embodiment of the present disclosure;

[0140] Figure 4 A schematic diagram of a data preprocessing process provided by an embodiment of the present disclosure;

[0141] Figure 5 A schematic diagram of a process for searching a neighboring object to be detected provided by an embodiment of the present disclosure;

[0142] Figure 6 A schematic diagram of a data conversion process provided by an embodiment of the present disclosure;

[0143] Figure 7 A schematic diagram of a data space feature extraction process according to an embodiment of the present disclosure;

[0144] Figure 8 A schematic diagram of the structure of a multi-head attention mechanism layer provided in an embodiment of the present disclosure;

[0145] Figure 9 A schematic diagram of a pooling operation according to an embodiment of the present disclosure;

[0146] Figure 10 A schematic diagram of a data dimensionality reduction process provided by an embodiment of the present disclosure;

[0147] Figure 11 A flowchart of a method for training a preset detection model provided in an embodiment of the present disclosure;

[0148] Figure 12 A schematic structural diagram of a base recognition device provided in an embodiment of the present disclosure;

[0149] Figure 13 A schematic structural diagram of another base recognition device provided by an embodiment of the present disclosure;

[0150] Figure 14 A schematic diagram of the structure of a training device for a preset detection model provided in an embodiment of the present disclosure;

[0151] Figure 15 A schematic diagram of the structure of another training device for a preset detection model provided in an embodiment of the present disclosure;

[0152] Figure 16 A schematic block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0153] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0154] The following describes the base calling method and apparatus, electronic device, and storage medium according to embodiments of the present disclosure with reference to the accompanying drawings.

[0155] Figure 1A schematic flow chart of a base recognition method provided in an embodiment of the present disclosure.

[0156] like Figure 1 As shown, the method comprises the following steps:

[0157] Step 101: Acquire a first preset amount of image data in each field of view, perform format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrate the signal strength data according to the position information of the object to be detected in the image data to obtain input data, wherein each input data contains a second preset amount of signal strength data.

[0158] In an embodiment of the present disclosure, the image data is a fluorescent image of an object to be detected on a sequencing chip displayed in a field of view (FOV), the first preset number is the number of all image data in the FOV, and the second preset number is a custom-set number, for example, 9, 16, etc., representing the number of signal intensity data corresponding to the object to be detected contained in each input data, wherein the signal intensity data contained in each input data is the signal intensity data corresponding to adjacent objects to be detected. Specifically, the embodiment of the present disclosure does not limit the size of the first preset number and the second preset number.

[0159] The signal intensity data is the fluorescence intensity of the object to be detected on the image data, and the expression form includes but is not limited to: fluorescence intensity numerical form, matrix form, vector form, etc. The expression form of the input data includes but is not limited to: matrix form, vector form, etc. The object to be detected includes but is not limited to: DNA nanoball (DNB), etc. The DNB is a spherical structure self-assembled by DNA molecules, which is usually used in single-molecule sequencing technology. Specifically, the expression form of the input data, the data expression form of the signal intensity data, and the object to be detected are not limited in the embodiments of the present disclosure.

[0160] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, an example is provided for illustration: the Raw intensity data (signal strength data) X∈R corresponding to each DNB (object to be detected) is obtained. N×T×2 , perform neighbor data search operation on the Raw intensity data ×, find the neighbor DNBs around according to the real spatial location information of DNB, obtain the Raw intensity data of the neighbor DNB, integrate the Raw intensity data corresponding to DNB and neighbor DNB to construct the input data X′, X′∈R N×T×9×2, N is the number of DNBs (objects to be detected) in each image data, T represents the length of the sequence Cycles, the number of light source intensity channels is 2, wherein 9 is the second preset number, indicating that the input data includes 9 signal strength data, i.e., the Raw intensity data of the DNB and the Raw intensity data corresponding to 8 neighboring DNBs. For example, the second preset number is 10, indicating that the input data includes 10 signal strength data, i.e., the Raw intensity data of the DNB and the Raw intensity data corresponding to 9 neighboring DNBs. The input data is expressed as X′∈R N×T×10×2 The length of the Cycles sequence is the number of image data in the FOV (the first preset number). The number of light source intensity channels is a property of the image data itself. For example, if the image data has three light source intensity channels and the number of light source intensity channels is 3, the signal intensity data is expressed as X∈R N×T×3 , the input data is expressed as X′∈R N×T×9×3 , specifically, the embodiments of the present disclosure are not limited.

[0161] Step 102 : performing data conversion processing on the input data according to the trained preset detection model to obtain a detection conversion vector of a first preset dimension corresponding to each of the objects to be detected.

[0162] In the embodiment of the present disclosure, the first preset dimension is a vector dimension of a custom setting, for example: 64, 128, etc. The trained preset detection model includes multiple module structures, including but not limited to: Embedding module, spatial ViT module, temporal ViT module, classification module, etc. In order to more intuitively understand the preset detection model, the embodiment of the present disclosure provides a structural diagram of the preset detection model, such as Figure 2 As shown, the model network structure diagram of the preset detection model (left), the network structure diagram of spatial ViT and temporal ViT (middle) and the internal structure diagram of Transformer (right). Specifically, the embodiment of the present disclosure does not limit the internal module structure of the preset detection model that has completed training and the size of the first preset dimension.

[0163] In some embodiments, the definition of each module in the trained preset detection model may include but is not limited to the following forms:

[0164] 1) Embedding module:

[0165] In the preset detection model that has been trained, the first is the Embedding module. The input of the Embedding module is: according to the index corresponding to each DNB and the real spatial position of the DNB, the subscript index of the surrounding 8 neighboring DNBs is found to construct the input data X′ containing the neighbor information, where X′∈R N×T×9×2 , the operation included in the Embedding module is the fully connected layer operation f e , the operation process is as follows:

[0166] f e (X′) = w·X′ + bias

[0167] Where w is the weight in the fully connected layer operation, w∈R 2×d , bias is the bias in the full connection layer operation, and the output after the full connection layer operation is f e (X′), f e (X′)∈R N×T×9×d , the dimension of the input data after the fully connected layer operation is increased from 2 dimensions to d dimensions.

[0168] 2) Spatial ViT module:

[0169] Spatial ViT module f space It mainly consists of two parts: the 4-layer Transformer structure and pooling. The input data X after the full connection layer operation e After the self-attention mechanism of Transformer, the mutual correlation between neighbors is learned. After a special pooling, the connection between the central DNB and the surrounding can be learned, which can extract more feature information for base recognition. The output result is recorded as X space ∈R N×T×d , the calculation process is as follows:

[0170] X space =f space (X e )

[0171] About Transformer structure and pooling process:

[0172] Transformer structure:

[0173] The internal structure of the Transformer consists of two key components: the multi-head attention mechanism and the fully connected layer operation. Multi-head attention introduces the concept of multiple heads based on self-attention, indicating that the preset detection model can observe from multiple angles. The Transformer operation process is as follows:

[0174] output transformer =Transformer(X e )

[0175] where output transformer ∈R N×T×9×d .

[0176] The implementation of Multi-Head Attention is to split the transformation vector to be detected into n_heads parts in the d dimension, where n_heads represents the number of self-attentions (calculating attention separately), and finally re-join the results of n_heads self-attentions along the d dimension.

[0177] The conversion of the input data is performed in the Embedding module of the preset detection model that has been trained. The Embedding module mainly maps the input X′ into a DNB vector of embed_size dimension, that is, converts the input image block data into a conversion vector to be detected of the first preset dimension. For example, the input data X′∈R N×T×9×2 , converted to the transformation vector to be detected e∈R of the first preset dimension N×T×9×d , d is the first preset dimension. At the same time, in order to facilitate the spatial ViT module to further calculate the calculation results of the Embedding module, it is also necessary to reshape the calculation results of the Embedding module to obtain the transformation vector X to be detected e ∈R (N*T)×9×d .

[0178] The calculation process of Multi-Head Attention is as follows:

[0179] First, X e Input into three fully connected layers (FC) to obtain Q, K, and V respectively:

[0180] Q=FC(X e )Q∈R N×T×d

[0181] K=FC(X e )K∈R N×T×d

[0182] V=FC(X e )V∈R N×T×d

[0183] Then split Q, K, and V in the d dimension into n_heads parts:

[0184]

[0185]

[0186]

[0187] Then calculate self-attention for n_heads parts Q, K, V respectively:

[0188]

[0189]

[0190] Finally, the n_heads self-attentions are concatenated along the d dimension as the output of Multi-HeadAttention:

[0191] Attention=Concat(Attention n_heads )

[0192] Attention∈R N×T×d

[0193] Concat refers to the concatenate operation in the d dimension.

[0194] Pooling process:

[0195] The pooling method used is to simply select the DNB vector at the center and output it. Due to the self-attention mechanism of the Transformer, each DNB can learn the ratio relationship with the surrounding DNBs. Combined with the back-propagation mechanism of deep learning network training, selecting the vector of the center DNB will strengthen the model to pay more attention to the characteristics of the center DNB and probabilistically adjust the influence ratio of the surrounding DNBs on it, which is more in line with the impact of fluorescence crosstalk in real situations. The overall description is as follows Figure 9 As shown, it can be seen that the DNB vector of the center position P4 is taken. The calculation process is as follows:

[0196] (output pooling=Pooling(output transformer )

[0197] where output pooling ∈R N×T×d

[0198] 3). Time ViT module.

[0199] Time ViT module f temporal It is composed of a 4-layer Transformer structure. Its structure is the same as that of the spatial ViT module. The difference is that the temporal ViT module does not have a pooling process. The objects targeted by the spatial ViT module are converted from the neighbor dimension to the Cycles dimension, while the temporal ViT module learns the association relationship between time series.

[0200] X temporal =f temporal (X space ),

[0201] where X temporal ∈R N×T×d .

[0202] 4). Classification module.

[0203] After the feature extraction of the spatial ViT module and the temporal ViT module, the model learns the relationship between DNB and its neighbors, as well as the association relationship between DNB in ​​the Cycles dimension, and finally connects to a classification module f class The classification module includes LayerNorm regularization operation and fully connected layer operation (mapping the d dimension to 4 dimensions, representing the 4 categories of ATGC).

[0204] Y=f class (X temporal ),

[0205] where Y∈R N×T×4 .

[0206] The calculation formula of LayerNorm is as follows:

[0207]

[0208] where μ and σ are respectively X vit The mean and standard deviation of , γ, β are learnable parameters of LayerNorm, and θ is the minimum value to prevent division by 0.

[0209] Step 103: Based on the trained preset detection model, perform spatial feature extraction processing on the conversion vector to be detected to obtain a spatial feature vector, and perform temporal feature extraction processing on the spatial feature vector to obtain a feature vector to be detected.

[0210] In the disclosed embodiment, the feature extraction of the conversion vector to be detected is performed in the ViT module of the preset detection model that has been trained. The ViT module includes a spatial ViT module and a temporal ViT module. The spatial ViT module and the temporal ViT module are mainly composed of a Transformer encoding structure. After the conversion vector to be detected passes through the self-attention mechanism of the Transformer of the spatial ViT module, it can learn the mutual correlation between the object to be detected and the neighboring objects to be detected, that is, the correlation relationship between the spaces. At the same time, in the spatial ViT module, the data will also be pooled (pooling pooling), and the connection between the central object to be detected and the surroundings can be learned, so that the spatial feature vector can extract more feature information for subsequent base recognition.

[0211] The self-attention mechanism in the Transformer encoding structure of the spatial ViT module enables each DNB to learn the ratio association with the surrounding DNBs. Combined with the back-propagation mechanism of deep learning network training, selecting the vector of the central DNB for pooling operation will strengthen the model's focus on the characteristics of the central DNB and probabilistically adjust the influence ratio of the surrounding DNBs on it, which is more in line with the impact of fluorescence crosstalk in real situations.

[0212] After the spatial feature vector passes through the self-attention mechanism of the Transformer of the temporal ViT module, the target object is converted from the neighbor dimension to the Cycles dimension. That is, the temporal ViT module learns the correlation between the time series of data. At the same time, the internal structure of the temporal ViT module is roughly the same as that of the spatial ViT module. Both are composed of the Transformer encoding structure. The main internal structure includes: multi-head attention mechanism (Muti-Head Attention), feed forward neural network (Feed forward) and residual structure. In the spatial VIT module, the transformation vector to be detected needs to perform pooling operation on the data, while pooling is not required in the temporal VIT module. For example: X e After the self-attention mechanism of the spatial ViT module, a pooling operation is performed to learn the correlation between spaces. The output result is recorded as X space ∈R N×T×d , if the process of performing spatial feature extraction on the transformation vector to be detected is represented by a function, it can be expressed as X space =f space (X e ), then Xspace After the self-attention mechanism of the time ViT module, the correlation between time is learned and the output result is recorded as X temporal ∈R N×T×d , if the process of extracting the temporal features from the spatial feature vector is represented by a function, it can be expressed as X temporal =f temporal (X space ), wherein the spatial feature vector and the feature vector to be detected are also vectors of the first preset dimension.

[0213] Step 104: Based on the trained preset detection model, the feature vector to be detected is subjected to dimensionality reduction processing to obtain a corresponding vector to be classified of a second preset dimension, and base classification is performed according to the vector to be classified to obtain a base recognition result.

[0214] In an embodiment of the present disclosure, the second preset dimension is a custom-set vector dimension, and the setting of the second preset dimension corresponds to the type of base. For example, if there are 4 types of bases in total, the second preset dimension is also set to 4. The base recognition result refers to the base type corresponding to each object to be detected.

[0215] Among them, the dimensionality reduction processing of the feature vector to be detected and the classification of the vector to be classified are both carried out in the classification module of the preset detection model that has completed training. Since the feature vector to be detected has learned the relationship between DNB (object to be detected) and its neighbors, as well as the association relationship between DNB in ​​the Cycles (number of image data) dimension, after the feature vector to be detected is reduced to the dimension of base type, that is, the second preset dimension, classification can be performed to obtain the base type result of each DNB (object to be detected), that is, the base recognition result.

[0216] In order to facilitate the understanding of the embodiments of the present disclosure, a schematic diagram of an example of base recognition is provided, such as Figure 3 As shown, where base Fluorescence is the image data, Cycle is the number of image data, DNB is the object to be detected, Raw intensity is the signal intensity data, input is the input data, ViViT model is the preset detection model that has been trained, output is the base recognition result, and N is the number of objects to be detected in each image data. For ease of display, Figure 3 There is one object to be detected in each image data, that is, N=1.

[0217] The present disclosure provides a method for base identification, which obtains a first preset number of image data in each field of view, performs format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrates the signal strength data according to the position information of the object to be detected in the image data to obtain input data, wherein each input data contains a second preset number of signal strength data; performs data conversion processing on the input data according to a preset detection model that has been trained to obtain a conversion vector to be detected of a first preset dimension corresponding to each object to be detected; performs spatial feature extraction processing on the conversion vector to be detected based on the preset detection model that has been trained to obtain a spatial feature vector, and performs temporal feature extraction processing on the spatial feature vector to obtain a feature vector to be detected; performs dimensionality reduction processing on the feature vector to be detected based on the preset detection model that has been trained to obtain a corresponding vector to be classified of a second preset dimension, and performs base classification based on the vector to be classified to obtain a base identification result. Compared with related technologies, the embodiments of the present disclosure adopt a preset detection model, which can extract both the spatial correlation between the data to be tested and the time series information of the data to be tested from the perspective of time and space, thereby enhancing the feature extraction capability of data features, reducing computing costs, and improving the accuracy of base recognition.

[0218] In one possible implementation of the embodiment of the present disclosure, as a refinement of the above step 101, regarding the processing of the image data, the embodiment of the present disclosure provides a flow chart of data preprocessing, such as Figure 4 Shown, including:

[0219] Step 401 : performing bleeding processing on the image data using a preset image processing algorithm, and extracting the signal strength data corresponding to each of the objects to be detected from the processed image data.

[0220] In the embodiment of the present disclosure, the preset image processing algorithm is a custom-set image processing algorithm, for example, an algorithm in a base recognition software (such as Litecall software, etc.). After the image data is subjected to bleeding processing by the software, it is extracted and processed, and the signal intensity data obtained is called Raw intensity.

[0221] The functions of image data bleeding processing include, but are not limited to: overflow removal: ensuring that image content does not overflow into areas that should not be displayed, thereby maintaining image integrity; aesthetic enhancement: improving the overall appearance of the image, making it more aesthetically pleasing and reducing visual disharmony caused by improper edge processing; and clarity enhancement: improving image clarity by reducing edge blur or bleeding.

[0222] Step 402: Obtain an index of each of the objects to be detected in the image data, where the index is used to determine position information of the object to be detected in the image data.

[0223] In the embodiment of the present disclosure, the index is a kind of positioning information carried by each object to be detected. The position of the corresponding object to be detected can be located in the image data through the index. The expression form of the index includes but is not limited to: numerical value, coordinates, etc. Specifically, the embodiment of the present disclosure does not limit the expression form of the index.

[0224] Step 403: Determine the position information of each of the objects to be detected in the image data according to the index.

[0225] In the embodiment of the present disclosure, the position information is used to indicate the position of the object to be detected in the image data, including but not limited to: coordinate information of the object to be detected in the image data, etc. The positions of all objects to be detected in the image data can be determined by the index.

[0226] Step 404 : Based on the position information, the signal strength data corresponding to the second preset number of objects to be detected are respectively integrated with each of the objects to be detected as the center, until all the signal strength data are processed to obtain the input data.

[0227] In the embodiment of the present disclosure, the neighboring objects to be detected are searched for based on the real spatial position information of the object to be detected, and the signal strength data of the neighboring objects to be detected are obtained. The signal strength data corresponding to the object to be detected and the neighboring objects to be detected are integrated to form input data. It should be noted that for the object to be detected existing at the edge of the image data, when searching for neighbor data with the object to be detected as the center, the neighboring object to be detected closest to the object to be detected will be searched to ensure that the object to be detected can be associated with the neighboring objects to be detected. Specifically, Figure 5 As shown, Figure 5 A schematic diagram of a search process for a neighboring object to be detected provided by an embodiment of the present disclosure, wherein each DNB (object to be detected) has its own unique identification index. Based on the index of each DNB and its position in two-dimensional space, the raw intensity data of the eight neighbors closest to the current DNB are searched to construct the input data X containing the neighbor information. ′ .

[0228] In one possible implementation of the embodiment of the present disclosure, as a refinement of the above step 102, regarding the data conversion of the input data, the embodiment of the present disclosure provides a flow chart of data conversion processing, such as Figure 6 Shown, including:

[0229] Step 601 : performing data conversion on the input data through a preset full-connection algorithm to obtain an object vector of a first preset dimension corresponding to each object to be detected.

[0230] In the embodiment of the present disclosure, the preset fully connected algorithm is a fully connected algorithm with custom settings, such as: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), etc. Specifically, the embodiment of the present disclosure does not limit the preset fully connected algorithm.

[0231] Therefore, the implementation process of the embodiment of the present disclosure includes but is not limited to: N×T×9×2 , after full connection operation, the input data is converted into the object vector to be detected (DNB vector) e∈R with dimension d N×T×9×d , if this process is recorded as f e , then the process is f e Convert the input data into a DNB vector e∈R of dimension d N×T×9×d , wherein the object vector to be detected is a first preset dimension vector.

[0232] In this disclosure, the fully connected algorithm is uniformly referred to as the fully connected operation. Its function is to map the last dimension of the input vector to other dimensions. The purpose is to enable the model to learn more abstract features and improve the model's nonlinear transformation capabilities. The operation process is as follows:

[0233] f e (X′) = w·X′ + bias

[0234] where X′∈R N×T×9×2 , w is the weight in the fully connected layer operation, w∈R 2×d , bias is the bias in the full connection layer operation, and the output after the full connection layer operation is f e (X′), f e (X′)∈R N×T×9×d , it can be seen that the final dimension is mapped from 2 to d.

[0235] Step 602: randomly assign a position coding vector to each of the objects to be detected, and add the object vector corresponding to the object to be detected to the assigned position coding vector to obtain a calculation vector to be detected, wherein the position coding vector is used to determine the spatial feature information in the calculation vector to be detected.

[0236] In the embodiment of the present disclosure, the position encoding vector is a position encoding about T, that is, the position encoding vector is related to the amount of the image data in the field of view, and can be expressed as: e ∈R 1×T×9×d The position encoding vector can make up for the lack of perception of data order in the self-attention mechanism when processing the transformation vector to be detected. Since the self-attention mechanism is adopted in the VIT module (Transformer structure), this mechanism focuses on the relationship between each element in the input sequence, that is, the transformation vector to be detected, but does not take into account the position information of the transformation vector to be detected in the sequence. The position encoding vector can enable the VIT module to use position information to distinguish the transformation vectors to be detected at different positions in the sequence, so that when the preset detection model that has been trained processes the input data, it can better understand the order and relationship of the objects to be detected in the input data.

[0237] It should be noted here that the position coding vector is a trainable vector, the initial position coding vector is an all-zero matrix, and the position coding vector will be trained synchronously with the training of the preset detection model. Therefore, in the preset detection model that has completed training, the position coding vector is also a trained position coding vector and can be directly called for allocation and use.

[0238] Step 603 : reshape the calculation vector to be detected using a preset reshaping algorithm to obtain the conversion vector to be detected.

[0239] In the embodiment of the present disclosure, in order to facilitate the spatial ViT module to further calculate the calculation results of the Embedding module, the calculation results of the Embedding module need to be reshaped (reshape operation). The reshape operation is a mathematical operation performed on a multidimensional array (such as a tensor, matrix, etc.). It changes the layout of the elements inside the array and converts it into an array with different dimensions without changing the number and value of the elements in the array.

[0240] The overall implementation process of data conversion in the embodiment of the present disclosure includes but is not limited to: N×T×9×2 , after full connection operation, the input data is converted into the object to be detected (DNB vector) e∈R with dimension d N×T×9×d , if this process is recorded as f e , then the process is f e Convert the input data into a DNB vector e∈R of dimension d N×T×9×d , and then assign a position encoding vector p to each DNB e ∈R 1×T×9×d, add the position encoding vector and the DNB vector, and reshape the calculated result to get the final result, X e =reshape(f e (X′)+p e ), where X e ∈R (N*T)×9×d .

[0241] In one possible implementation of the embodiment of the present disclosure, as a refinement of the above step 102, regarding the spatial feature extraction process of the conversion vector to be detected, the embodiment of the present disclosure provides a flow chart of the data spatial feature extraction process, as shown in FIG. Figure 7 Shown, including:

[0242] Step 701: Perform self-attention calculation on the conversion vector to be detected through a preset attention algorithm to obtain an attention vector.

[0243] In the embodiment of the present disclosure, the attention algorithm is all the algorithms used in the process of using Muti-Head Attention. When performing self-attention calculation on the conversion vector to be detected, the conversion vector to be detected is split into n_heads parts on the d dimension (the first preset dimension), where n_heads represents the number of self-attentions (attention is calculated separately). Finally, the results of n_heads self-attentions are re-spliced ​​back along the d dimension to obtain the attention vector.

[0244] In order to facilitate understanding of the implementation process of the embodiment of the present disclosure, a structural diagram of a multi-head attention mechanism layer is provided, as shown in FIG. Figure 8 As shown, Figure 8 The left is the structure diagram of self-attention. Figure 8 The right is the structure diagram of Multi-Head Attention. Specifically, the process of calculating the self-attention result of the transformation vector to be detected by Multi-Head Attention includes but is not limited to: e The (transformation vector to be detected) is input into three fully connected layers (FC) to obtain Q, K, and V respectively. This process can be expressed by formula (1), formula (2), and formula (3) respectively:

[0245] Q=FC(X e )Q∈R N×T×d Formula (1)

[0246] K=FC(X e )K∈R N×T×dFormula (2)

[0247] V=FC(X e )V∈R N×T×d Formula (3)

[0248] Then, Q, K, and V are split into n_heads parts in the d dimension. This process can be expressed by formula (4), formula (5), and formula (6) respectively:

[0249]

[0250]

[0251]

[0252] Then, the self-attention results are calculated for n_heads parts Q, K, and V respectively. This process can be expressed by formula (7):

[0253]

[0254] in,

[0255] Finally, the n_heads self-attentions are concatenated along the d dimension to obtain the attention vector. This process can be expressed by formula (8):

[0256] Attention=Concat(Attention n_heads )Formula (8)

[0257] Among them, Attention is the attention vector, Attention∈R N×T×d , Concat refers to the concatenate operation in the d dimension.

[0258] Step 702: Input the attention vector into a first preset fully connected structure, perform data operations with the first fully connected parameters, and activate it through a preset function to obtain a first spatial vector, wherein the first fully connected parameters are used to extract spatial features of the attention vector.

[0259] In the disclosed embodiment, the first preset fully connected structure is a fully connected structure of a custom setting, such as: a feedforward neural network (Feed forward), etc., and the second preset function is a function of a custom setting, such as: a GELU activation function, etc. The role of the second preset function is to learn more abstract features so that the interaction relationship between the objects to be detected (DNB) is strengthened. In the Muti-Head Attention calculation process, most of the operations are performed using matrix multiplication, and these operations are linear transformations. The introduction of Feed forward increases the model's ability to perform nonlinear transformations.

[0260] The calculation process of the first spatial eigenvector includes but is not limited to obtaining it through formula (9):

[0261] FFN(x1)=GELU(AttentionW1+b1) Formula (9)

[0262] Among them, x1 is the first eigenvector, FFN represents a feedforward neural network, b1 is a bias value, and the method for determining the bias value includes but is not limited to: custom setting, random generation, etc. W1 is a parameter of the feedforward neural network, namely the first fully connected parameter, which mainly plays the role of connection and weight. The first fully connected parameter exists in the form of weight between each level in the neural network, and determines the mapping relationship between the input and output of the neuron. By learning and adjusting the first fully connected parameter, the neural network can realize various complex nonlinear function approximation and classification tasks, that is, the first fully connected parameter plays the role of connecting input and output, extracting features, realizing nonlinear mapping, fitting optimization model and providing interpretability in the feedforward neural network.

[0263] It should be noted that the first fully connected parameter is a trainable parameter. The initial first fully connected parameter is a randomly generated parameter, and the first fully connected parameter will be trained synchronously with the training of the preset detection model. Therefore, in the preset detection model that has completed training, the first fully connected parameter is also a trained fully connected parameter and can be directly called for use.

[0264] Step 703: Input the first space vector into a second preset fully connected structure, perform data operation with a second fully connected parameter, and obtain a second space vector, wherein the second fully connected parameter is used to further extract spatial features from the first space vector.

[0265] In the embodiment of the present disclosure, the second preset fully connected structure is a fully connected structure with custom settings, such as a feedforward neural network (Feed forward), etc. It can be the same structure as the first fully connected structure or a different structure. Specifically, the embodiment of the present disclosure does not limit it.

[0266] The calculation process of the second spatial eigenvector includes but is not limited to obtaining it through formula (10):

[0267] FFN(x2)=GELU(xW1+b1)W2+b2 formula (10)

[0268] Among them, x2 is the second spatial eigenvector, GELU(xW1+b1) is the calculation formula of the first spatial eigenvector, that is, the second spatial eigenvector is calculated by the first spatial eigenvector, b2 is the bias value, W2 is the parameter of the feedforward neural network, that is, the second fully connected parameter. Specifically, for the specific description of b2 and W2, please refer to the description of b1 and W1 in the above step 702, so they will not be repeated here.

[0269] Step 704 : After calculating the second spatial vector using a first preset residual function, the second spatial vector is added to the first spatial vector to obtain an initial spatial vector.

[0270] In the embodiment of the present disclosure, the preset residual function is all functions in the residual structure, such as: identity mapping function (Identity Mapping), skip connection (Skip Connections), batch normalization (BatchNormalization), weighted residual (Weighted Residual), depthwise separable convolution (Depthwise SeparableConvolution), etc. Specifically, the preset residual function can be determined according to the actual residual structure, and the embodiment of the present disclosure does not limit it.

[0271] The calculation process of the feature vector to be detected includes but is not limited to obtaining it through formula (11):

[0272] X3=F(x2)+x2 Formula (11)

[0273] Among them, X3 is the spatial eigenvector to be initialized, x2 is the second spatial eigenvector, and F(x2) is the result of calculating the second spatial eigenvector using a preset residual function.

[0274] Step 705 : Performing a pooling operation on the initial spatial vector using a preset pooling algorithm to obtain the spatial feature vector.

[0275] In the embodiment of the present disclosure, the preset pooling algorithm is a custom-selected pooling algorithm, such as: Max Pooling, Average Pooling, Bilinear Pooling, Region of Interest Pooling (ROI Pooling), etc. Specifically, the embodiment of the present disclosure does not limit the pooling algorithm.

[0276] In order to make each object to be detected learn the ratio association with the surrounding objects to be detected, it is necessary to perform a pooling operation on the initial space vector. When performing the pooling operation, only the vector of the object to be detected at the center position is selected for output, such as Figure 9 As shown, Figure 9 This is a schematic diagram of a pooling operation provided by an embodiment of the present disclosure. After the pooling operation, the spatial feature vector X can be obtained. space ∈R N×T×d .

[0277] In one implementation of the embodiment of the present disclosure, performing temporal feature extraction processing on the spatial feature vector to obtain the feature vector to be detected includes:

[0278] Performing self-attention calculation on the spatial feature vector through a preset attention algorithm to obtain a spatial attention vector;

[0279] Inputting the spatial attention vector into a third preset fully connected structure, performing data operation with a third fully connected parameter, and activating the structure through a preset function to obtain a first time vector, wherein the third fully connected parameter is used to extract time features from the spatial attention vector;

[0280] Inputting the first time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second time vector, wherein the fourth fully connected parameter is used to further extract time features from the first time vector;

[0281] After the second time vector is calculated by a second preset residual function, the second time vector is added to the second time vector to obtain the feature vector to be detected.

[0282] Specifically, regarding the implementation process of the embodiment of the present disclosure, reference may be made to the description in steps 701 to 704 above, and therefore will not be repeated here.

[0283] In one possible implementation of the embodiment of the present disclosure, as a refinement of the above step 104, regarding the dimensionality reduction processing of the feature vector to be detected, the embodiment of the present disclosure provides a flow chart of data dimensionality reduction processing, such as Figure 10Shown, including:

[0284] Step 1001: Regularize the feature vector to be detected using a preset regularization algorithm to obtain a regularized feature vector.

[0285] In the embodiment of the present disclosure, the preset regularization algorithm is a custom-set algorithm, and the feature vector to be detected can be calculated by a formula to complete the regularization operation (LayerNorm). Therefore, the method of performing LayerNorm on the feature vector to be detected includes but is not limited to performing it by formula (12):

[0286]

[0287] Among them, μ, σ are respectively temporal The mean and standard deviation of , γ, β are learnable parameters of LayerNorm, and θ is the minimum value to prevent division by 0.

[0288] Step 1002: Perform dimensionality reduction processing on the regularized feature vector through a third preset fully connected structure to obtain a vector to be classified of the second preset dimension.

[0289] In the embodiment of the present disclosure, the third preset fully connected structure is a fully connected structure with custom settings, such as: feed forward neural network (Feed forward), recurrent neural network (Recurrent Neural Network, RNN), multilayer perceptron (Multilayer Perceptron, MLP), etc. Specifically, the embodiment of the present disclosure does not limit the third preset fully connected structure.

[0290] Regarding the second preset dimension, please refer to the description in the above step 104, so it will not be described in detail here.

[0291] Regarding the dimensionality reduction processing of the feature vector to be detected, it can be expressed as: Y = f class (X temporal ), where Y∈R N×T×4 .

[0292] Corresponding to the above-mentioned base recognition method, Figure 11 A flow chart of a method for training a preset detection model provided in an embodiment of the present disclosure is shown as follows: Figure 11 Shown, including:

[0293] Step 1101: Acquire a third preset number of training image data in each field of view, perform format conversion processing on the training image data respectively to obtain training signal strength data corresponding to each training object to be detected, and integrate the training signal strength data according to the position information of the training object to be detected in the training image data to obtain training input data, wherein each of the training input data contains a fourth preset number of the training signal strength data.

[0294] Specifically, regarding the implementation process of the embodiment of the present disclosure, please refer to the description in the above step 101, so it will not be described here one by one.

[0295] Step 1102 : Obtain a true label corresponding to each of the training objects to be detected, where the true label is a known base recognition result of the training object to be detected.

[0296] In the embodiment of the present disclosure, when training the preset prediction model, a training set and a verification set are required. The training set is used to train the preset prediction model, and the verification set is used to optimize the preset prediction model. Therefore, it is necessary to obtain the real label corresponding to each object to be detected for training as a verification set.

[0297] Step 1103 : performing data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each of the training to-be-detected objects.

[0298] Step 1104 : Based on the preset detection model, perform spatial feature extraction processing on the training transformation vector to be detected to obtain a training spatial feature vector, and perform temporal feature extraction processing on the training spatial feature vector to obtain a training feature vector to be detected.

[0299] Step 1105: Based on the preset detection model, the training feature vector to be detected is subjected to dimensionality reduction processing to obtain a corresponding training vector to be classified of a fourth preset dimension, and base classification is performed according to the training vector to be classified to obtain a training base recognition result.

[0300] Specifically, regarding the implementation process of steps 1103 to 1105, please refer to the description of steps 102 to 104 above, so they will not be described here one by one.

[0301] Step 1106: Based on the training base recognition results and the true labels, the preset detection model is optimized by a preset loss function and a preset optimization function until a preset convergence condition is met, thereby obtaining a trained preset detection model.

[0302] In the embodiment of the present disclosure, the preset loss function is a custom-selected loss function, such as the cross entropy function, etc., and the preset optimization function is a custom-selected optimization function, such as the AdamW function, etc. Specifically, the embodiment of the present disclosure does not limit the selection of the preset loss function and the preset optimization function.

[0303] Calculating classification loss with the preset loss function The method includes but is not limited to formula (13):

[0304]

[0305] in, is the output of the model prediction, i.e., the training base recognition result, and y is the one-hot form of the true category (true label).

[0306] In one implementation of the embodiment of the present disclosure, the format conversion processing is performed on the training image data to obtain training signal strength data corresponding to each training object to be detected, and the training signal strength data is integrated according to the position information of the training object to be detected in the training image data to obtain the training input data, including:

[0307] Performing bleeding processing on the training image data using a preset image processing algorithm, and extracting the training signal strength data corresponding to each of the training objects to be detected from the processed image data;

[0308] Obtaining an index of each training object to be detected in the training image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0309] Determine the position information of each of the training objects to be detected in the training image data according to the index;

[0310] Based on the position information, the training signal strength data corresponding to the fourth preset number of training objects to be detected are integrated with each of the training objects to be detected as the center until all the training signal strength data are processed to obtain the training input data.

[0311] Specifically, regarding the implementation process of the embodiment of the present disclosure, reference may be made to the description in steps 401 to 404 above, and therefore will not be repeated here.

[0312] In one implementation of the embodiment of the present disclosure, performing data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each training to-be-detected object includes:

[0313] Performing data conversion on the training input data using a preset full-connection algorithm to obtain a training object vector of the third preset dimension corresponding to each training object to be detected;

[0314] A training position coding vector is randomly assigned to each of the training objects to be detected, and the training object vector corresponding to the training object to be detected is added to the assigned training position coding vector to obtain a training calculation vector to be detected, and the training position coding vector is used to determine the spatial feature information in the training calculation vector to be detected.

[0315] The training calculation vector to be detected is subjected to data reshaping processing through a preset reshaping algorithm to obtain the training conversion vector to be detected.

[0316] Specifically, regarding the implementation process of the embodiment of the present disclosure, reference may be made to the description in steps 601-603 above, and therefore will not be repeated here.

[0317] In one implementation of the embodiment of the present disclosure, performing spatial feature extraction processing on the training to-be-detected conversion vector based on the preset detection model to obtain the training spatial feature vector includes:

[0318] Performing self-attention calculation on the training transformation vector to be detected through a preset attention algorithm to obtain a training attention vector;

[0319] Inputting the training attention vector into a first preset fully connected structure, performing data operation with a fifth training fully connected parameter, and activating the training attention vector through a preset function to obtain a first training space vector, wherein the first training fully connected parameter is used to extract spatial features from the training attention vector;

[0320] Inputting the first training space vector into a second preset fully connected structure, performing data operation with a second training fully connected parameter to obtain a second training space vector, wherein the second training fully connected parameter is used to further extract spatial features from the first training space vector;

[0321] After calculating the second training space vector using a first preset residual function, the vector is added to the second training space vector to obtain an initial training space vector;

[0322] The initial space vector for training is pooled using a preset pooling algorithm to obtain the space feature vector for training.

[0323] Specifically, regarding the implementation process of the embodiment of the present disclosure, reference may be made to the description in steps 701-705 above, and therefore details will not be repeated here.

[0324] In one possible implementation of the embodiment of the present disclosure, performing temporal feature extraction processing on the training spatial feature vector to obtain a training feature vector to be detected includes:

[0325] Performing self-attention calculation on the training spatial feature vector through a preset attention algorithm to obtain a training spatial attention vector;

[0326] Inputting the training spatial attention vector into a third preset fully connected structure, performing data operation with the third fully connected parameter, and activating it through a preset function to obtain a first time vector of the line, wherein the third fully connected parameter for training is used to extract time features from the training spatial attention vector;

[0327] Inputting the first training time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second training time vector, wherein the fourth training fully connected parameter is used to further extract time features from the first training time vector;

[0328] After the second time vector for training is calculated by a second preset residual function, the second time vector for training is added to the second time vector for training to obtain the feature vector to be detected for training.

[0329] Specifically, regarding the implementation process of the embodiment of the present disclosure, reference may be made to the description in steps 701 to 704 above, and therefore will not be repeated here.

[0330] In one implementation of the embodiment of the present disclosure, performing dimensionality reduction processing on the training feature vector to be detected based on the preset detection model to obtain a corresponding training vector to be classified of the fourth preset dimension includes:

[0331] Performing a regularization operation on the training feature vector to be detected by a preset regularization algorithm to obtain a training regularized feature vector;

[0332] The training regularized feature vector is subjected to dimensionality reduction processing through a fifth preset fully connected structure to obtain the training vector to be classified of the fourth preset dimension.

[0333] Specifically, regarding the implementation process of the embodiment of the present disclosure, reference may be made to the description in steps 1001-1002 above, and therefore will not be repeated here.

[0334] In combination with all the above embodiments, in order to verify the performance of the method described in this disclosure, this disclosure also provides a method for evaluating the performance of base recognition of different models. At the same time, in order to more intuitively evaluate the performance of joint embedding of data of different modalities, it can also be implemented in but not limited to the following ways:

[0335] Dataset: The training dataset contains 10M DNBs, all of which are raw intensity data with neighbors. The training sample size is 8,232,295, and the validation sample size is 2,767,705, equivalent to the total data volume of one FOV. The training set mainly includes the model input raw intensity data and the true labels, with the shapes of (batch size, cycles, neighbors, 2) and (batch size, cycles), respectively, where the cycles size is 150 and the number of neighbors is 9. The prediction data is made using six FOVs that the model has never seen: L02-C020R001, L02-C020R003, L02-C036R003, L01-C020R003, L02-C009R003, and L02-C024R004. These only contain the model input raw intensity data and do not include the true labels.

[0336] Parameter Settings: Both the spatial ViT module and the temporal ViT module in this paper use a 4-layer, 3-head TransformerBlock, with an embedding layer dimension of 192 and a class size of 4. The batch size for training is 384, the learning rate is 5e-4, and a linear learning rate decay strategy with warmup is introduced during training. The warmup step size is set to 0.1 of the total step size. The AdamW optimizer is used for parameter updates, with an epoch count of 500 and an early stopping round count of 50.

[0337] Compared to traditional base calling algorithms, X_call (the base calling method described in this proposal) achieves significantly improved MappingRate and AvgErrorRate performance. We trained the model on 10MB of raw intensity data and applied the trained model to predictions under six FOVs never seen by the model. Table 1 shows a comparison of X_call's MappingRate and AvgErrorRate performance with a base calling software.

[0338] Table 1

[0339]

[0340] Among them, the higher the MappingRate (%) and the lower the AvgErrorRate (%), the better the base recognition performance.

[0341] X_call is the name of the method described in this disclosure, and ViViT model is the name of the model structure referenced by the method described in this disclosure (ViViT model is a method in an article [1]). Embedding module, spatial ViT module, temporal ViT module, and classification module are all part of the X_call method (such as Figure 2 As shown in Figure 3, the spatial ViT module and the temporal ViT module are designed with reference to the ViViT model, but they are not exactly the same.

[0342] In summary, the embodiments of the present disclosure can achieve the following effects:

[0343] 1. The disclosed embodiment adopts a preset detection model, which can extract both the spatial correlation between the test data and the time series information of the test data from the perspective of time and space, thereby enhancing the feature extraction capability of data features, reducing computing costs, and improving the accuracy of base recognition.

[0344] 2. The embodiment of the present disclosure uses the spatial ViT module and the temporal ViT module in the preset detection model to capture the spatial and temporal information in the input data when performing base recognition, and adopts the self-attention mechanism to capture the long-range dependencies and relationships between different objects to be detected, so as to better understand the content of the input data.

[0345] 3. The embodiment of the present disclosure uses a preset detection model to process the objects to be detected when performing base recognition, similar to an image-based Transformer. The objects to be detected can be processed in parallel, which is computationally more efficient than the traditional method of processing the objects to be detected sequentially.

[0346] 4. The disclosed embodiment can process different amounts of image data when performing base recognition by adopting a preset detection model without additional modification, can effectively capture time dependencies on different time scales, and is applicable to different amounts of image data.

[0347] 5. The preset detection model provided in the embodiment of the present disclosure is a model pre-trained on a large-scale set of objects to be detected, and can be fine-tuned in specific data detection tasks to achieve better generalization and performance.

[0348] Corresponding to the above-described base recognition method and method for training a preset detection model, the present invention also provides a base recognition device and a device for training a preset detection model. Since the device embodiments of the present invention correspond to the above-described method embodiments, details not disclosed in the device embodiments can be referred to the above-described method embodiments and will not be further described in this invention.

[0349] Figure 12 A schematic diagram of the structure of a base recognition device provided in an embodiment of the present disclosure is shown in FIG. Figure 12 Shown, including:

[0350] The conversion unit 1201 is configured to obtain a first preset amount of image data in each field of view, and perform format conversion processing on the image data to obtain signal strength data corresponding to each object to be detected;

[0351] an integration unit 1202, configured to integrate the signal strength data according to the position information of the object to be detected in the image data to obtain input data, wherein each input data includes a second preset number of signal strength data;

[0352] The conversion unit 1203 is configured to perform data conversion processing on the input data according to the trained preset detection model to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected;

[0353] An extraction unit 1204 is configured to perform spatial feature extraction processing on the to-be-detected conversion vector based on the trained preset detection model to obtain a spatial feature vector;

[0354] The extraction unit 1204 is further configured to perform time feature extraction processing on the spatial feature vector to obtain a feature vector to be detected;

[0355] A dimensionality reduction unit 1205 is configured to perform dimensionality reduction processing on the feature vector to be detected based on the trained preset detection model to obtain a corresponding vector to be classified of a second preset dimension;

[0356] The classification unit 1206 is configured to perform base classification according to the vector to be classified to obtain a base recognition result.

[0357] The base recognition device provided by the present disclosure obtains a first preset number of image data in each field of view, performs format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrates the signal strength data according to the position information of the object to be detected in the image data to obtain input data, wherein each input data contains a second preset number of signal strength data; performs data conversion processing on the input data according to a preset detection model that has been trained to obtain a conversion vector to be detected of a first preset dimension corresponding to each object to be detected; based on the preset detection model that has been trained, performs spatial feature extraction processing on the conversion vector to be detected to obtain a spatial feature vector, and performs temporal feature extraction processing on the spatial feature vector to obtain a feature vector to be detected; based on the preset detection model that has been trained, performs dimensionality reduction processing on the feature vector to be detected to obtain a corresponding vector to be classified of a second preset dimension, and performs base classification according to the vector to be classified to obtain a base recognition result. Compared with related technologies, the embodiments of the present disclosure adopt a preset detection model, which can extract both the spatial correlation between the data to be tested and the time series information of the data to be tested from the perspective of time and space, thereby enhancing the feature extraction capability of data features, reducing computing costs, and improving the accuracy of base recognition.

[0358] Furthermore, in a possible implementation of the embodiment of the present disclosure, the conversion unit 1201 is further configured to perform bleeding processing on the image data using a preset image processing algorithm, and extract the signal strength data corresponding to each of the objects to be detected from the processed image data;

[0359] like Figure 13 As shown, the integration unit 1202 includes:

[0360] An acquisition module 12021 is configured to acquire an index of each of the objects to be detected in the image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0361] a determination module 12022, configured to determine position information of each of the objects to be detected in the image data according to the index;

[0362] The integration module 12023 is configured to integrate the signal strength data corresponding to the second preset number of objects to be detected based on the position information and with each object to be detected as the center, until all the signal strength data are processed to obtain the input data.

[0363] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 13 As shown, the conversion unit 1203 includes:

[0364] The conversion module 12031 is configured to perform data conversion on the input data using a preset full-connection algorithm to obtain an object vector of a first preset dimension corresponding to each object to be detected;

[0365] A calculation module 12032 is configured to randomly assign a position coding vector to each of the objects to be detected, and to add the object vector corresponding to the object to be detected to the assigned position coding vector to obtain a calculation vector to be detected, wherein the position coding vector is used to determine spatial feature information in the calculation vector to be detected;

[0366] The reshaping module 12033 is configured to perform data reshaping processing on the calculation vector to be detected using a preset reshaping algorithm to obtain the conversion vector to be detected.

[0367] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 13 As shown, the extraction unit 1204 includes:

[0368] A first calculation module 12041 is configured to perform self-attention calculation on the to-be-detected conversion vector through a preset attention algorithm to obtain an attention vector;

[0369] Processing module 12042 is configured to input the attention vector into a first preset fully connected structure, perform data operation on the attention vector and first fully connected parameters, and then activate the attention vector using a preset function to obtain a first spatial vector, wherein the first fully connected parameters are used to extract spatial features from the attention vector.

[0370] The processing module 12042 is further configured to input the first space vector into a second preset fully connected structure, perform data operation on the first space vector and a second fully connected parameter to obtain a second space vector, wherein the second fully connected parameter is used to further extract spatial features from the first space vector;

[0371] A second calculation module 12043 is configured to calculate the second spatial vector using a first preset residual function, and then add the calculated second spatial vector to the first spatial vector to obtain an initial spatial vector;

[0372] The operation module 12044 is configured to perform a pooling operation on the initial spatial vector using a preset pooling algorithm to obtain the spatial feature vector.

[0373] Furthermore, in a possible implementation of the embodiment of the present disclosure, the extracting unit 1204 is further configured to:

[0374] Performing self-attention calculation on the spatial feature vector through a preset attention algorithm to obtain a spatial attention vector;

[0375] Inputting the spatial attention vector into a third preset fully connected structure, performing data operation with a third fully connected parameter, and activating the structure through a preset function to obtain a first time vector, wherein the third fully connected parameter is used to extract time features from the spatial attention vector;

[0376] Inputting the first time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second time vector, wherein the fourth fully connected parameter is used to further extract time features from the first time vector;

[0377] After the second time vector is calculated by a second preset residual function, the second time vector is added to the second time vector to obtain the feature vector to be detected.

[0378] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 13 As shown, the dimension reduction unit 1205 includes:

[0379] An operation module 12051 is configured to perform a regularization operation on the feature vector to be detected using a preset regularization algorithm to obtain a regularized feature vector;

[0380] The dimensionality reduction module 12052 is configured to perform dimensionality reduction processing on the regularized feature vector through a fifth preset fully connected structure to obtain the vector to be classified of the second preset dimension.

[0381] Figure 14 A schematic diagram of a structure of a training device for a preset detection model provided in an embodiment of the present disclosure, such as Figure 14 Shown, including:

[0382] The conversion unit 1401 is configured to obtain a third preset number of training image data in each field of view, and perform format conversion processing on the training image data to obtain training signal strength data corresponding to each training object to be detected;

[0383] an integration unit 1402, configured to integrate the training signal strength data according to position information of the training object to be detected in the training image data to obtain training input data, wherein each training input data includes a fourth preset number of training signal strength data;

[0384] An acquiring unit 1403 is configured to acquire a true label corresponding to each of the training objects to be detected, where the true label is a known base recognition result of the training object to be detected;

[0385] The conversion unit 1404 is configured to perform data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each training to-be-detected object;

[0386] An extraction unit 1405 is configured to perform spatial feature extraction processing on the training to-be-detected transformation vector based on the preset detection model to obtain a training spatial feature vector;

[0387] The extraction unit 1405 is further configured to perform temporal feature extraction processing on the training spatial feature vector to obtain a training feature vector to be detected;

[0388] A dimensionality reduction unit 1406 is configured to perform dimensionality reduction processing on the training feature vector to be detected based on the preset detection model to obtain a corresponding training vector to be classified of a fourth preset dimension;

[0389] A classification unit 1407 is configured to perform base classification based on the training vector to be classified to obtain a training base recognition result;

[0390] The optimization unit 1408 is used to optimize the preset detection model based on the training base recognition results and the true labels through a preset loss function and a preset optimization function until a preset convergence condition is met, thereby obtaining a trained preset detection model.

[0391] Furthermore, in a possible implementation of the embodiment of the present disclosure, the conversion unit 1401 is further configured to perform bleeding processing on the training image data using a preset image processing algorithm, and extract the training signal strength data corresponding to each of the training objects to be detected from the processed image data;

[0392] like Figure 15 As shown, the integration unit 1402 includes:

[0393] An acquisition module 14021 is configured to acquire an index of each object to be detected for training in the training image data, wherein the index is used to determine position information of the object to be detected in the image data;

[0394] a determination module 14022, configured to determine position information of each of the training objects to be detected in the training image data according to the index;

[0395] The integration module 14023 is used to integrate the training signal strength data corresponding to the fourth preset number of training objects to be detected based on the position information, with each of the training objects to be detected as the center, until all the training signal strength data are processed to obtain the training input data.

[0396] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 15 As shown, the conversion unit 1404 includes:

[0397] The conversion module 14041 is configured to perform data conversion on the training input data using a preset full-connection algorithm to obtain a training object vector of the third preset dimension corresponding to each training object to be detected;

[0398] A calculation module 14042 is configured to randomly assign a training position coding vector to each of the training objects to be detected, and to add the training object vector corresponding to the training object to be detected to the assigned training position coding vector to obtain a training calculation vector to be detected, wherein the training position coding vector is used to determine spatial feature information in the training calculation vector to be detected;

[0399] The reshaping module 14043 is used to perform data reshaping processing on the training calculation vector to be detected through a preset reshaping algorithm to obtain the training conversion vector to be detected.

[0400] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 15 As shown, the extraction unit 1405 includes:

[0401] A first calculation module 14051 is configured to perform self-attention calculation on the training detection conversion vector using a preset attention algorithm to obtain a training attention vector;

[0402] Processing module 14052 is configured to input the training attention vector into a first preset fully connected structure, perform data operation with a fifth training fully connected parameter, and then activate the training attention vector using a preset function to obtain a first training space vector, wherein the first training fully connected parameter is used to extract spatial features from the training attention vector.

[0403] The processing module 14052 is further configured to input the first training space vector into a second preset fully connected structure, perform data operation with a second training fully connected parameter, and obtain a second training space vector, wherein the second training fully connected parameter is used to further extract spatial features from the first training space vector.

[0404] A second calculation module 14053 is configured to calculate the second training space vector using a first preset residual function, and then add the calculated value to the second training space vector to obtain an initial training space vector.

[0405] The operation module 14054 is used to perform a pooling operation on the initial training space vector through a preset pooling algorithm to obtain the training space feature vector.

[0406] Furthermore, in a possible implementation of the embodiment of the present disclosure, the extraction unit 1405 is further configured to:

[0407] Performing self-attention calculation on the training spatial feature vector through a preset attention algorithm to obtain a training spatial attention vector;

[0408] Inputting the training spatial attention vector into a third preset fully connected structure, performing data operation with the third fully connected parameter, and activating it through a preset function to obtain a first time vector of the line, wherein the third fully connected parameter for training is used to extract time features from the training spatial attention vector;

[0409] Inputting the first training time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second training time vector, wherein the fourth training fully connected parameter is used to further extract time features from the first training time vector;

[0410] After the second time vector for training is calculated by a second preset residual function, the second time vector for training is added to the second time vector for training to obtain the feature vector to be detected for training.

[0411] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 15 As shown, the dimension reduction unit 1406 includes:

[0412] An operation module 14061 is configured to perform a regularization operation on the training feature vector to be detected using a preset regularization algorithm to obtain a training regularized feature vector;

[0413] The dimensionality reduction module 14062 is configured to perform dimensionality reduction processing on the training regularized feature vector through a fifth preset fully connected structure to obtain the training vector to be classified of the fourth preset dimension.

[0414] It should be noted that the above explanation of the method embodiment is also applicable to the device of the embodiment of the present disclosure, and the principles are the same, which is no longer limited in the embodiment of the present disclosure.

[0415] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0416] Figure 16A schematic block diagram of an example electronic device 1600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0417] like Figure 16 As shown, the device 1600 includes a computing unit 1601, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1602 or a computer program loaded from a storage unit 1608 into a RAM (Random Access Memory) 1603. Various programs and data required for the operation of the device 1600 can also be stored in the RAM 1603. The computing unit 1601, the ROM 1602, and the RAM 1603 are connected to each other via a bus 1604. An I / O (Input / Output) interface 1605 is also connected to the bus 1604.

[0418] Various components in device 1600 are connected to I / O interface 1605, including an input unit 1606, such as a keyboard and mouse; an output unit 1607, such as various types of displays and speakers; a storage unit 1608, such as a magnetic disk and optical disk; and a communication unit 1609, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1609 allows device 1600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0419] The computing unit 1601 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1601 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1601 performs the various methods and processes described above, such as a method for base identification or a training method for a preset detection model. For example, in some embodiments, the method for base identification or the training method for a preset detection model can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1600 via ROM 1602 and / or communication unit 1609. When the computer program is loaded into RAM 1603 and executed by computing unit 1601, one or more steps of the method described above may be performed. Alternatively, in other embodiments, computing unit 1601 may be configured to perform the aforementioned base calling method or the training method of the preset detection model by any other appropriate means (e.g., by means of firmware).

[0420] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0421] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0422] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0423] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0424] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0425] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0426] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0427] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0428] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for base recognition, characterized in that: include: Acquire a first preset amount of image data in each field of view, perform format conversion processing on the image data respectively to obtain signal strength data corresponding to each object to be detected, and integrate the signal strength data according to position information of the object to be detected in the image data to obtain input data, wherein each input data includes a second preset amount of signal strength data; Performing data conversion processing on the input data according to the trained preset detection model to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected; Based on the trained preset detection model, performing spatial feature extraction processing on the conversion vector to be detected to obtain a spatial feature vector, and performing temporal feature extraction processing on the spatial feature vector to obtain a feature vector to be detected; Based on the trained preset detection model, the feature vector to be detected is subjected to dimensionality reduction processing to obtain a corresponding vector to be classified of a second preset dimension, and base classification is performed according to the vector to be classified to obtain a base recognition result.

2. The method according to claim 1, characterized in that The image data are format converted to obtain signal strength data corresponding to each object to be detected, and the signal strength data are integrated according to the position information of the object to be detected in the image data to obtain input data, including: Performing bleeding processing on the image data using a preset image processing algorithm, and extracting the signal strength data corresponding to each of the objects to be detected from the processed image data; Obtaining an index of each of the objects to be detected in the image data, wherein the index is used to determine position information of the object to be detected in the image data; Determine the position information of each of the objects to be detected in the image data according to the index; Based on the position information, the signal strength data corresponding to the second preset number of objects to be detected are respectively integrated with each of the objects to be detected as the center until all the signal strength data are processed to obtain the input data.

3. The method according to claim 1, characterized in that The step of performing data conversion processing on the input data according to the trained preset detection model to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected includes: Performing data conversion on the input data using a preset full-connection algorithm to obtain an object vector of a first preset dimension corresponding to each object to be detected; Randomly assigning a position coding vector to each of the objects to be detected, and adding the object vector corresponding to the object to be detected to the assigned position coding vector to obtain a calculation vector to be detected, wherein the position coding vector is used to determine spatial feature information in the calculation vector to be detected; The calculation vector to be detected is subjected to data reshaping processing by a preset reshaping algorithm to obtain the conversion vector to be detected.

4. The method according to claim 1, wherein The spatial feature extraction process is performed on the to-be-detected conversion vector based on the trained preset detection model to obtain a spatial feature vector, which includes: Performing self-attention calculation on the transformation vector to be detected through a preset attention algorithm to obtain an attention vector; Inputting the attention vector into a first preset fully connected structure, performing data operation with a first fully connected parameter, and activating the structure through a preset function to obtain a first spatial vector, wherein the first fully connected parameter is used to extract spatial features from the attention vector; Inputting the first space vector into a second preset fully connected structure, performing data operation with a second fully connected parameter to obtain a second space vector, wherein the second fully connected parameter is used to further extract spatial features from the first space vector; After calculating the second spatial vector using a first preset residual function, the first spatial vector is added to the second spatial vector to obtain an initial spatial vector; The initial spatial vector is pooled using a preset pooling algorithm to obtain the spatial feature vector.

5. The method according to claim 1, wherein The performing time feature extraction processing on the spatial feature vector to obtain the feature vector to be detected includes: Performing self-attention calculation on the spatial feature vector through a preset attention algorithm to obtain a spatial attention vector; Inputting the spatial attention vector into a third preset fully connected structure, performing data operation with a third fully connected parameter, and activating the structure through a preset function to obtain a first time vector, wherein the third fully connected parameter is used to extract time features from the spatial attention vector; Inputting the first time vector into a fourth preset fully connected structure, performing data operation with a fourth fully connected parameter to obtain a second time vector, wherein the fourth fully connected parameter is used to further extract time features from the first time vector; After the second time vector is calculated by a second preset residual function, the second time vector is added to the second time vector to obtain the feature vector to be detected.

6. The method according to claim 1, characterized in that The step of performing dimensionality reduction processing on the feature vector to be detected based on the trained preset detection model to obtain a corresponding vector to be classified of a second preset dimension includes: Performing a regularization operation on the feature vector to be detected by a preset regularization algorithm to obtain a regularized feature vector; The regularized feature vector is subjected to dimensionality reduction processing through a fifth preset fully connected structure to obtain the vector to be classified of the second preset dimension.

7. A method for training a preset detection model, characterized in that: The preset detection model can be applied to the method according to any one of claims 1 to 6, including: Acquiring a third preset number of training image data in each field of view, performing format conversion processing on the training image data respectively to obtain training signal strength data corresponding to each training object to be detected, and integrating the training signal strength data according to position information of the training object to be detected in the training image data to obtain training input data, wherein each training input data includes a fourth preset number of training signal strength data; Obtaining a true label corresponding to each of the training objects to be detected, wherein the true label is a known base recognition result of the training object to be detected; Performing data conversion processing on the training input data according to a preset detection model to obtain a training to-be-detected conversion vector of a third preset dimension corresponding to each training to-be-detected object; Based on the preset detection model, performing spatial feature extraction processing on the training to-be-detected conversion vector to obtain a training spatial feature vector, and performing temporal feature extraction processing on the training spatial feature vector to obtain a training to-be-detected feature vector; Based on the preset detection model, performing dimensionality reduction processing on the training feature vector to be detected to obtain a corresponding training vector to be classified of a fourth preset dimension, and performing base classification according to the training vector to be classified to obtain a training base recognition result; Based on the training base recognition results and the true labels, the preset detection model is optimized by a preset loss function and a preset optimization function until a preset convergence condition is met, thereby obtaining a trained preset detection model.

8. A base recognition device, characterized in that: include: a conversion unit, configured to obtain a first preset amount of image data in each field of view, and perform format conversion processing on the image data to obtain signal strength data corresponding to each object to be detected; an integration unit, configured to integrate the signal strength data according to position information of the object to be detected in the image data to obtain input data, wherein each input data includes a second preset number of the signal strength data; a conversion unit, configured to perform data conversion processing on the input data according to a preset detection model that has been trained, to obtain a detection conversion vector of a first preset dimension corresponding to each object to be detected; an extraction unit, configured to perform spatial feature extraction processing on the to-be-detected conversion vector based on the trained preset detection model to obtain a spatial feature vector; The extraction unit is further configured to perform time feature extraction processing on the spatial feature vector to obtain a feature vector to be detected; A dimensionality reduction unit, configured to perform dimensionality reduction processing on the feature vector to be detected based on the trained preset detection model to obtain a corresponding vector to be classified of a second preset dimension; The classification unit is used to perform base classification according to the vector to be classified to obtain a base recognition result.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6 or the method of claim 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6 or the method according to claim 7.

11. A computer program product, characterized in that The method comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6 or the method according to claim 7.

Images