Intelligent auxiliary system for medical scientific research training based on artificial intelligence large model

Abstract

The application discloses an intelligent auxiliary system for medical scientific research training based on an artificial intelligence large model, and particularly relates to the field of artificial intelligence and scientific research data processing, and is used for solving the problems of multi-source dispersion, non-uniform structure, missing steps difficult to identify and dependence on manual verification of experimental records in the existing medical scientific research process; a digital experimental record file is constructed by image acquisition on an experimental table record carrier and in combination with target detection and text recognition, logical breakpoints are recognized based on numerical jump and semantic co-occurrence, and jump record recognition is realized by using phenotype feature distribution and optimal transport calculation; missing steps are further reasoned and completed by dynamic programming, and finally, a complete experimental process draft is generated in combination with a structured template, automatic analysis, abnormal identification, process completion and multi-modal data fusion processing of scientific research data are realized, and the intelligent level of experimental data processing and the standardization of the scientific research process are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and scientific research data processing technology, and more specifically, to an intelligent auxiliary system for medical scientific research training based on a large artificial intelligence model. Background Technology

[0002] With the rapid development of artificial intelligence technology, data processing methods based on deep learning and large models have been widely applied in the field of medical research, playing a crucial role, especially in experimental data acquisition, analysis, and result recording. Currently, in medical research, experimental records still largely rely on manual writing, scattered image acquisition, and independent storage of instrument data, resulting in diverse data sources and inconsistent structures, making efficient integration and automated analysis difficult.

[0003] In actual experiments, researchers need to simultaneously monitor the execution of experimental steps, changes in instrument readings, and the evolution of biological sample states. This information typically exists in multimodal forms, including text, images, and numerical values, making it difficult for traditional data processing methods to perform unified modeling and correlation analysis. Furthermore, due to operational oversights or incomplete recording during experiments, issues such as missing steps, incorrect sequences, or data anomalies are prone to occur. Existing technologies largely rely on post-experiment manual review, lacking the ability for real-time structured analysis and automatic error correction of the experimental process. While some technologies have attempted to digitize experimental records using image recognition and natural language processing, most remain at the level of single-modal analysis or simple information extraction, lacking cross-modal fusion, temporal logic modeling, and abnormal behavior recognition capabilities, making it difficult to support the automated management and traceability analysis of complex research processes.

[0004] Therefore, there is an urgent need for a medical research assistance method that integrates multimodal data analysis, temporal reasoning, and intelligent completion capabilities to improve the intelligence level of experimental data processing and the standardization of the research process. Summary of the Invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, embodiments of the present invention provide a medical research training intelligent assistance system based on a large artificial intelligence model to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A medical research training intelligent assistance system based on a large-scale artificial intelligence model includes:

[0008] The acquisition module is used to automatically acquire images of the recording medium on the experimental platform, and obtain raw experimental record image data including handwritten notes, instrument screens and biological samples.

[0009] The recording module is used to input raw image data into the target detection network, locate and crop out the instrument's digital display area, handwritten text area and biological sample imaging area, and generate digital experimental record archives.

[0010] The breakpoint identification module is used to extract the recording timestamps of digital experimental record archives, construct the experimental event sequence according to the physical recording order, and identify candidate abnormal locations where logical breakpoints exist by calculating the numerical jump gradient of adjacent instrument readings and the semantic coherence of the operation description.

[0011] The skip step identification module is used to extract the first sample image slice and the second sample image slice from the sample imaging region before and after the candidate anomaly position, calculate the offset of the sample morphology distribution, and identify the existing skip step records.

[0012] The step reasoning module is used to extract instrument readings and operation descriptions before and after the skipped step record position from the event sequence, perform missing step reasoning, and generate the reasoning results of missing steps based on the contextual statistical patterns of complete event sequences in the same experimental batch.

[0013] The completion module is used to call blank templates from the experimental structured template library, fill in the operation descriptions, skip records and missing step inference results in the event sequence into the blank templates, and create an experimental process draft containing a complete operation chain.

[0014] As a further aspect of the present invention, the automatic image acquisition of the experimental platform recording medium in the acquisition module specifically includes:

[0015] Acquire a pre-scanned image of the current experimental platform. Based on the texture features and reflected light intensity distribution of different regions in the pre-scanned image, divide the page into a handwritten note area, an instrument screen area, and a biological sample area.

[0016] The above-mentioned areas were scanned in a focused manner, and grayscale images of the handwritten note area, the instrument screen area, and the biological sample area were acquired simultaneously. All the acquired images were stitched together and fused according to their original spatial positions on the experimental platform to generate the original experimental record image data.

[0017] As a further aspect of the present invention, the recording module specifically includes the following steps for generating digital experimental record archives:

[0018] The original experimental record image data is input into the target detection network, and the bounding box coordinates and category labels of the instrument digital display area, handwritten text area and biological sample imaging area are output. The images of each region are cropped from the original image based on the bounding box coordinates, and redundant regions are removed by sorting them in descending order of confidence.

[0019] Input the image of the instrument's digital display area into the optical character recognition model to output a sequence of digital characters; input the image of the handwritten text area into the text recognition model to generate a text sequence word by word.

[0020] The file paths of the digital character sequence, text sequence, and biological sample imaging area image are indexed and associated according to the spatial location of each area in the original image and the order of acquisition time, generating a digital experimental record archive containing area images, recognized text, and location coordinates.

[0021] As a further embodiment of the present invention, the target detection network uses a feature pyramid structure to extract multi-scale image features, and after processing by classification and regression branches, outputs the bounding box coordinates and category labels of the instrument digital display area, handwritten text area and biological sample imaging area.

[0022] As a further aspect of the present invention, the breakpoint identification module specifically includes identifying candidate abnormal locations where logical breakpoints exist, including:

[0023] Extract the timestamps of each record item from the digital experimental record archive, sort them in chronological order, and construct an experimental event sequence.

[0024] Traverse the sequence of experimental events and find two adjacent records. Calculate the rate of change of the difference between each pair of adjacent instrument readings with respect to the time interval, and use this rate of change as the numerical jump gradient.

[0025] The verb-object collocations in two adjacent operation descriptions are input into the experimental operation semantic co-occurrence statistical model built based on the same experimental batch, and the frequency score of the current collocation is calculated.

[0026] A joint anomaly score is performed based on the numerical jump gradient and semantic co-occurrence score. The boundary position of adjacent records where the anomaly score reaches the preset anomaly threshold is marked as a candidate anomaly position with a logical breakpoint.

[0027] As a further aspect of the present invention, the skip step identification module specifically includes identifying existing skip step records as follows:

[0028] Based on the time period corresponding to the candidate anomaly location in the experimental event sequence, the sample image corresponding to the nearest previous record of the candidate anomaly location is extracted from the biological sample imaging area image stored in the digital experimental record archive as the first sample image slice, and the sample image corresponding to the nearest next record is extracted as the second sample image slice.

[0029] The first and second sample image slices are input into the morphological difference analysis network to extract the morphological parameters and optical density parameters of each target object in the two image slices, and encode them into a phenotypic feature distribution representation.

[0030] Based on the phenotypic feature distribution representation, the optimal transmission calculation is performed on the first distribution corresponding to the first sample image slice and the second distribution corresponding to the second sample image slice to obtain the distribution migration relationship from the first distribution to the second distribution and the corresponding minimum transmission cost, which is used as the morphological distribution offset.

[0031] The morphological distribution offset between all adjacent sample image slices in the same experimental batch is statistically analyzed to construct a morphological distribution offset sequence. Based on the relative change level of the morphological distribution offset sequence, the morphological distribution offset of candidate anomaly locations is compared and analyzed to identify skipped records.

[0032] As a further aspect of the present invention, the optical density parameter is obtained by logarithmic transformation of the pixel grayscale value and the background grayscale value of the grayscale image of the biological sample imaging region; the morphological parameter is represented by the numerical value corresponding to the geometric structure of the segmented contour of the target object in the biological sample imaging region.

[0033] As a further aspect of the present invention, the step reasoning module specifically includes generating the reasoning result for the missing step as follows:

[0034] Extract the operation description corresponding to the previous record of the skip record position from the experimental event sequence as the predecessor operation, extract the operation description corresponding to the next record as the successor operation, and extract all event sequences in the same experimental batch that do not have skip records from the digital experimental record archive to form a reference sequence library.

[0035] The preceding and succeeding operations are used as sequence endpoint constraints. All possible subsequences in the reference sequence library located between these endpoints are used as candidate filling paths. Each candidate filling path contains operation descriptions of several intermediate steps arranged in sequence and corresponding instrument reading ranges.

[0036] The sum of the semantic vector similarity values ​​of each step in the candidate filling path is used as the path benefit, and the degree of agreement between the instrument reading range corresponding to each step in the candidate filling path and the actual reading before and after the skip step is used as the path cost. The dynamic programming algorithm is used to search for the optimal filling path with the maximum benefit and the minimum cost in the candidate filling path space.

[0037] The sequence of intermediate steps described in the optimal filling path and their corresponding instrument reading ranges are output as the inference results for the missing steps.

[0038] As a further aspect of the present invention, the completion module, specifically including the following: Establishing an experimental process draft containing a complete operational chain, includes:

[0039] Call the corresponding blank template from the experimental structured template library. The blank template contains sequentially arranged step fields.

[0040] The operation descriptions of each record in the event sequence are filled into the matching step fields in the blank template in chronological order, and the skipped records are marked in the gaps between the corresponding step fields. At the same time, the operation descriptions in the reasoning results of the missing steps are inserted into the skipped positions as completion steps.

[0041] The filled-in fields are then bound to the coordinates of the corresponding regions in the original image and the instrument readings to generate a draft of the experimental procedure containing the complete operation chain and anomaly marker locations.

[0042] The technical effects and advantages of the intelligent auxiliary system for medical research training based on a large artificial intelligence model of the present invention are as follows:

[0043] By introducing a large-scale artificial intelligence model and a multimodal data fusion processing mechanism, the automated analysis and structured reconstruction of medical research experimental processes are achieved. Compared with the traditional method that relies on manual recording and post-event verification, this approach can complete the unified integration and correlation modeling of image, text, and instrument data during the experimental record collection stage, effectively improving data processing efficiency and consistency. By constructing a breakpoint identification mechanism based on numerical jumps and semantic co-occurrence, and a step-by-step identification method based on phenotypic feature distribution and optimal transmission calculation, abnormal steps and potential missing links in the experimental process can be accurately located, enhancing the ability to judge the integrity of the experimental process. Furthermore, by combining a missing step reasoning model based on dynamic programming, the missing operations can be reasonably completed with the support of multiple experimental batches of data, improving the continuity and logical consistency of experimental records. At the same time, through structured template automatic filling technology, discrete multi-source data is transformed into a unified and standardized experimental process draft, significantly reducing manual processing costs and improving the traceability and reusability of scientific research data, thereby comprehensively improving the intelligence level and data management quality of the medical research training process. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the intelligent auxiliary system for medical research training based on a large artificial intelligence model, as described in this invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] Figure 1 This invention presents an intelligent auxiliary system for medical research training based on a large-scale artificial intelligence model, comprising:

[0048] The acquisition module is used to automatically acquire images of the recording medium on the experimental platform, and obtain raw experimental record image data including handwritten notes, instrument screens and biological samples.

[0049] The recording module is used to input raw image data into the target detection network, locate and crop out the instrument's digital display area, handwritten text area and biological sample imaging area, and generate digital experimental record archives.

[0050] The breakpoint identification module is used to extract the recording timestamps of digital experimental record archives, construct the experimental event sequence according to the physical recording order, and identify candidate abnormal locations where logical breakpoints exist by calculating the numerical jump gradient of adjacent instrument readings and the semantic coherence of the operation description.

[0051] The skip step identification module is used to extract the first sample image slice and the second sample image slice from the sample imaging region before and after the candidate anomaly position, calculate the offset of the sample morphology distribution, and identify the existing skip step records.

[0052] The step reasoning module is used to extract instrument readings and operation descriptions before and after the skipped step record position from the event sequence, perform missing step reasoning, and generate the reasoning results of missing steps based on the contextual statistical patterns of complete event sequences in the same experimental batch.

[0053] The completion module is used to call blank templates from the experimental structured template library, fill in the operation descriptions, skip records and missing step inference results in the event sequence into the blank templates, and create an experimental process draft containing a complete operation chain.

[0054] The acquisition module automatically acquires images of the recording medium on the experimental platform.

[0055] A pre-scanned image of the current experimental platform is acquired. Based on the texture features and reflected light intensity distribution of different regions in the pre-scanned image, the page is divided into a handwritten note area, an instrument screen area, and a biological sample area. In practice, the imaging unit is fixed at a preset height above the experimental platform, for example, within a range of 40cm to 60cm from the platform. A full-frame exposure acquisition is performed before shooting to obtain a pre-scanned image covering the entire platform. The pre-scanned image is uniformly grayscaled, and local contrast enhancement is performed to highlight the texture differences between different regions. Subsequently, the grayscale co-occurrence matrix features and edge density features of each region in the image are extracted. The handwritten note area exhibits a high-frequency fine line structure with continuous grayscale changes, the instrument screen area is a bright area with obvious rectangular boundaries, while the biological sample area exhibits an irregular shape and localized grayscale aggregation. Regarding reflected light intensity, by statistically analyzing the average grayscale value and distribution range of each region, areas with a brightness higher than 180 were marked as high reflectance candidate areas. Regions with concentrated grayscale distribution and regular edges were further identified as instrument screen areas; regions with drastic grayscale changes and fine line texture distribution were identified as handwritten note areas; and regions with irregular patchy grayscale distribution and continuous spatial expansion were identified as biological sample areas. If grayscale and texture features were not obvious, preliminary division was performed by manually marking fixed regions beforehand. After preliminary division, morphological closing operations were performed on the boundaries of each region to smooth them out and eliminate boundary spikes. Regions were then filtered based on area thresholds, for example, setting the minimum effective region area to 5% of the total image area to remove mis-segmented small areas. Finally, stable division results for the three types of regions were formed, and the position coordinates of each region in the original image were recorded.

[0056] The aforementioned regions were scanned in a focused manner, simultaneously acquiring grayscale images of the handwritten note area, the instrument screen area, and the biological sample area. All acquired images were then stitched together according to their original spatial positions on the experimental platform to generate the original experimental record image data. Based on the obtained regional coordinates, the focusing module of the imaging unit was controlled at the regional level, setting different focusing parameters for the handwritten note area, the instrument screen area, and the biological sample area. A shallow depth of field was used for the handwritten note area to enhance text clarity; a fixed focal length was used for the instrument screen area combined with an exposure suppression strategy to avoid overexposure; and a medium depth of field was used for the biological sample area to balance overall shape and local details. Each region was scanned sequentially at high resolution, while a time synchronization mechanism ensured that image acquisition for each region occurred within the same time window, for example, completing all region acquisitions within 200 milliseconds to prevent changes in the experimental platform from affecting image consistency. After acquisition, the images of each region were normalized to a uniform size, and mapped to the same coordinate system based on pre-recorded spatial coordinates. A boundary-aligned stitching method was used for fusion processing. During stitching, a weighted average fusion was performed on overlapping areas, with weights allocated based on the distance to the region edges (e.g., 0.3 for edge regions and 0.7 for middle regions) to ensure a smooth stitching transition. For regions with slight positional deviations, fine-tuning alignment was performed using feature point matching, selecting corner points or edge intersections as matching benchmarks to ensure the overall image maintained spatial continuity after stitching. The final output is a high-resolution grayscale image containing complete experimental platform information, serving as the original experimental record image data for subsequent object detection and information analysis.

[0057] The recording module generates digital experimental record archives.

[0058] The original experimental record image data is input into the target detection network, which outputs the bounding box coordinates and category labels for the instrument digital display area, handwritten text area, and biological sample imaging area. Based on the bounding box coordinates, images of each region are cropped from the original image. The original experimental record image is then normalized to a uniform size and input into the backbone network of the target detection network for feature extraction. This backbone network uses a multi-layer convolutional structure to extract feature information at different scales and fuses shallow detail features with deep semantic features through a feature pyramid structure, forming a multi-scale feature map set. Based on this, candidate region sets covering the entire image are generated at different scales, according to a pre-defined multi-scale distribution, including sizes such as 32×32, 64×64, and 128×128, to accommodate target regions of different sizes. Subsequently, the candidate regions are input into the classification and regression branches. The classification branch determines the category of the candidate region and outputs category labels, including those for the instrument digital display area, handwritten text area, and biological sample imaging area. The regression branch fine-tunes the bounding boxes of the candidate regions and outputs the corresponding bounding box coordinates. For multiple overlapping bounding boxes generated at the same location, an overlap-based filtering method is used. For example, an overlap threshold of 0.5 is set. When the proportion of the overlapping area of ​​two bounding boxes to the area of ​​the smaller bounding box exceeds this threshold, only the bounding box with higher confidence is retained, and redundant detection results are removed. Finally, the retained bounding boxes are sorted from high to low confidence, and the corresponding region images are cropped from the original image based on the bounding box coordinates, resulting in a set of region images with clear structure and well-defined categories.

[0059] For images of the digital display area of ​​an instrument, an optical character recognition model based on a combination of convolutional neural networks and recurrent neural networks is adopted. First, features are extracted from the input image through convolutional layers, converting the two-dimensional image into a one-dimensional feature sequence. Multi-layer convolution and pooling operations are then used to enhance the edge and contour features of the digits and characters. This feature sequence is then input into a bidirectional recurrent neural network to model the contextual information within the sequence to recognize consecutive digit characters. At the output layer, a connection-based temporal classification decoding method is used to convert the consecutive feature sequence into the final digit character sequence, ensuring accurate recognition even without character spacing annotations. For images of handwritten text areas, an encoder-decoder structure based on an attention mechanism is adopted. The encoder part consists of multi-layer convolutional networks used to extract spatial features of the image and generate a feature map sequence. The decoder part uses a recurrent neural network structure, and at each time step, the feature map output by the encoder is weighted and aggregated through an attention mechanism to focus on the character region that needs to be recognized, outputting the corresponding text content word by word. During model training, handwritten samples and instrument reading samples containing experimental record scenarios were selected as training data. The number of training iterations was set to more than 30 rounds, and the cross-entropy loss function was used to optimize the prediction results in each round of training until the recognition accuracy of the model on the validation set stabilized at more than 95%, thereby ensuring the recognition stability and accuracy of the model in practical applications.

[0060] Each detected region is assigned a unique identifier, and its top-left corner coordinates and width / height information in the original image are recorded as its spatial positioning basis. Simultaneously, the timestamp information recorded during image acquisition is read, and all regions are sorted according to chronological order. Then, the numerical character sequences corresponding to the instrument's digital display areas are bound to their spatial coordinates, and the text sequences corresponding to the handwritten text areas are bound to their spatial coordinates. The image data of the biological sample imaging areas are also stored in an indexed manner. The image data is encoded and saved in a unified format, and its access path in the storage medium is recorded as index information. When establishing the index association, the regions are organized according to a principle combining spatial location and chronological order; that is, they are grouped according to their relative position in the image, and then arranged chronologically within each group to ensure that the record structure is consistent with the actual experimental table layout. Finally, the image data index, recognized text content, and spatial coordinate information of all regions are uniformly encapsulated into structured data records. Each record contains a region category identifier, location coordinates, text or numerical content, and time information, thus forming a complete digital experimental record archive.

[0061] The breakpoint identification module identifies candidate abnormal locations where logical breakpoints exist.

[0062] The digital experimental record archive is accessed uniformly, and the corresponding timestamp information is parsed from each record. The timestamps are derived from the system time recorded during image acquisition or the device's embedded time stamp, with a time precision uniform to the millisecond level. The timestamps of all records are sorted to form a sequential sequence of experimental events. After the sequence is constructed, adjacent records are iterated sequentially, and the corresponding instrument readings for each pair are extracted, while simultaneously acquiring the time interval between the two records. Instrument readings are uniformly normalized, for example, by linearly normalizing the original readings according to the historical reading range of the batch, ensuring comparability of data with different dimensions. Subsequently, the ratio of the reading difference between two records to the corresponding time interval is calculated to obtain the reading change amplitude per unit time. To avoid interference from single-point fluctuations, a sliding window averaging process is applied to the change amplitudes of three consecutive adjacent records, i.e., the average of the current change amplitude and the adjacent change amplitudes is calculated to obtain a smoothed trend value. The smoothed trend value is used as the numerical gradient between adjacent records and is recorded at the corresponding record boundary to characterize the intensity and rate of change of experimental parameters within this time period.

[0063] Semantic parsing is performed on the operation description text of two adjacent records in an experimental event sequence. First, the operation descriptions are segmented into words, and verb-object combinations are identified based on a pre-built domain dictionary, such as typical experimental operation expressions like "add reagent," "heat treatment," and "centrifugation." For each operation description, its core verb and corresponding object are extracted to form a standardized verb-object collocation unit, which serves as the basic unit for semantic analysis. Subsequently, this collocation unit is input into an experimental operation semantic co-occurrence statistical model for matching calculations. This model is built based on complete historical records of the same experimental batch. By statistically analyzing operation descriptions in a large number of experimental records, a co-occurrence frequency matrix between verbs and objects is established, and the frequency distribution of different collocations appearing consecutively in adjacent steps is recorded. During model construction, at least 100 complete experimental sequences are selected as training samples, and all adjacent operation pairs are statistically analyzed to form stable co-occurrence relationship data. For the verb-object combination to be analyzed, its historical occurrence count is retrieved from the co-occurrence matrix, and combined with the consecutive occurrence frequency of the combination in adjacent steps, the corresponding occurrence frequency score is calculated. This score reflects the rationality of the current operation combination in the historical experimental process; the higher the score, the more frequently the operation combination appears in the actual experimental process, and the stronger its semantic coherence.

[0064] This study employs a joint analysis of numerical jump gradients and semantic co-occurrence scores to identify logical breakpoints in experimental event sequences. First, both numerical jump gradients and semantic co-occurrence scores are scaled uniformly, mapped to a range of 0 to 1. The numerical jump gradient is normalized based on the statistical range of all changes in the batch, while the semantic co-occurrence score is scaled proportionally based on the historical maximum frequency. Then, a joint anomaly scoring rule is constructed, combining the normalized numerical jump gradient and semantic co-occurrence score. The numerical jump gradient reflects the degree of anomaly in physical quantity changes, while the semantic co-occurrence score reflects the rationality of operational logic. Each has a fixed weight in the score; for example, the weight of the numerical jump gradient is set to 0.6, and the weight of the semantic co-occurrence score is set to 0.4. These weights are determined through statistical analysis of historical experimental data and remain constant during implementation. Based on these weights, the two indicators are weighted and summed to obtain the joint anomaly score. An anomaly judgment threshold is set for the joint anomaly score; for example, a score greater than 0.7 is considered an anomaly interval. This threshold is determined by statistically analyzing the score distribution of historical normal experimental data. Traverse the entire sequence of experimental events, calculate the joint anomaly score for each boundary position of adjacent records, and when the score exceeds the set threshold, mark the position as a candidate anomaly position with a logical breakpoint, and record the corresponding time position and related record information.

[0065] The skip step identification module identifies existing skip step records.

[0066] Based on the temporal location information of candidate anomaly locations within the experimental event sequence, the corresponding biological sample imaging region image data stored in the digital experimental record archive is retrieved. The record index corresponding to the candidate anomaly location is read, and the nearest valid record is retrieved backwards. The biological sample imaging region image corresponding to that record is extracted as the first sample image slice. Simultaneously, the nearest valid record is retrieved sequentially backwards, and its corresponding biological sample imaging region image is extracted as the second sample image slice. During the retrieval process, time intervals are constrained, for example, limiting the time interval between consecutive records to no more than twice the average sampling interval of the experimental batch, ensuring continuity between the selected image slices. After extraction, both image slices undergo uniform size normalization, unifying the resolution to a fixed specification, such as 256×256 pixels, and grayscale normalization is performed to ensure a consistent grayscale distribution range for images acquired under different conditions. Subsequently, the images undergo mild denoising processing, using median filtering to eliminate random noise points while preserving edge structure integrity. To ensure spatial consistency in subsequent analysis, keypoint-based alignment was performed on the two images. Stable structural points in the images were selected as references, and the two images were adjusted to a unified coordinate frame through affine transformation, thereby obtaining a first sample image slice and a second sample image slice with consistent spatial position and stable quality.

[0067] The morphological difference analysis network adopts a dual-branch structure, with each branch containing a convolutional feature extraction layer, a segmentation sub-network, and a feature encoding layer. In each branch, multi-scale features of the image are extracted through multi-layer convolutional operations and input into the segmentation sub-network to segment the target objects in the image. The segmentation sub-network employs an encoder-decoder structure, extracting semantic information through downsampling and restoring spatial resolution through upsampling, outputting a segmentation mask for the target objects. Based on the segmentation mask, the contour information of each target object is extracted, and geometric analysis is performed on the contours, calculating morphological parameters including area, perimeter, major axis length, and minor axis length. Simultaneously, curvature statistics are performed on the contour boundaries to obtain boundary variation distribution characteristics. For obtaining optical density parameters, the pixel grayscale values ​​in the image are normalized to the mean grayscale value of the corresponding background region. For example, the average grayscale value of the non-target region in the image is selected as the background reference value. Then, a logarithmic transformation is performed on the normalization result to obtain the optical density value of each pixel. The average value and distribution range of the optical density within the target object region are calculated as the optical features of the object. After parameter extraction, the morphological parameters and optical density parameters of each target object are combined to form a feature vector. The feature vectors of all target objects are statistically encoded, for example, by dividing them into several segments according to the feature value range and counting the number of target objects in each segment. Finally, a phenotypic feature distribution representation describing the overall sample state is formed.

[0068] The two sets of phenotypic feature distributions are converted into discrete distributions, each consisting of several feature segments and their corresponding number of target objects. Then, the transmission relationship between the distributions is constructed, establishing a correspondence between each feature segment in the first distribution and each feature segment in the second distribution, and calculating the degree of difference between different segments. The difference is determined based on the combined difference between morphological parameters and optical density parameters. Next, the transmission scheme with the smallest total difference is selected from all possible transmission paths. This scheme describes the minimum overall change required to transform from the first distribution to the second distribution, and this minimum change is defined as the morphological distribution offset between the two images. In the same experimental batch, the above calculation process is repeated for all adjacent sample image slices, resulting in a set of morphological distribution offset sequences arranged in chronological order. Statistical analysis is performed on this sequence to calculate the magnitude of change between adjacent offsets, and a sliding window averaging is applied to three consecutive offsets to obtain a stable trend. For candidate abnormal locations, the corresponding morphological distribution offset is compared with the offset of its adjacent locations before and after it. When the offset of the location is significantly higher than the average level of the adjacent locations, such as exceeding 1.5 times the average of the two adjacent locations, it is determined that there is an abnormal jump at the location, and thus it is identified as a skip record.

[0069] In the step-by-step reasoning module, the reasoning result for the missing step is generated.

[0070] The process involves reading the index position of a skip record in the event sequence, retrieving the nearest valid record at that position, extracting its corresponding operation description text, and standardizing it into a standardized operation expression. This standardization is achieved by unifying verb forms and removing redundant modifiers to maintain structural consistency, serving as the predecessor operation. Simultaneously, the process moves backward to retrieve the nearest valid record at that position, extracting and standardizing its operation description in the same way, serving as the successor operation. After extracting endpoint operations, the process filters all complete event sequences within the same experimental batch that are not marked as skip records from the digitized experimental record archive. These sequences are traversed one by one, selecting those containing subsequence fragments that are semantically consistent with the predecessor operation and subsequently contain subsequence fragments that are semantically consistent with the successor operation. During the selection process, a semantic matching model is used to vectorize the operation descriptions. This model employs a two-layer encoding structure: a convolutional encoding layer for extracting local semantic features and a recurrent encoding layer for modeling contextual relationships. The model is trained iteratively with at least 200 sets of experimental record texts for 40 training epochs to ensure stable output of semantic vector representations. For each candidate sequence, the semantic similarity between its starting operation and its predecessor operation and the semantic similarity between its ending operation and its successor operation are calculated. When the similarity is higher than 0.8, the sequence is included in the reference sequence library, thereby constructing a sequence set containing multiple valid reference paths.

[0071] Each sequence in the reference sequence library is scanned step by step to locate the starting position that semantically matches the preceding operation and the ending position that semantically matches the succeeding operation, and the continuous operation segments between them are extracted as candidate subsequences. During the extraction process, the operation order in the subsequence is ensured to maintain the chronological order of the original record. At the same time, the instrument reading range corresponding to each step is extracted. This reading range is obtained by statistically analyzing the reading distribution of the step in the historical records, for example, recording the minimum and maximum values ​​of the step in all historical occurrences as its valid interval. To ensure the rationality of the candidate paths, the length of the subsequences is constrained, for example, the number of intermediate steps is limited to no less than one step and no more than five steps, to avoid excessively long or short paths interfering with the inference results. For each candidate filling path, its operation description sequence is bound to the corresponding instrument reading range to form a complete path description structure. At the same time, the semantic consistency of the operation descriptions in the path is checked. By calculating the semantic similarity between adjacent operations, paths with obvious semantic jumps are eliminated. For example, when the semantic similarity between adjacent steps is less than 0.3, the path is excluded.

[0072] For each candidate path, the operation descriptions are semantically vectorized. A semantic encoding model is used to convert each operation into a corresponding semantic vector representation, and the semantic coherence between steps in the path is calculated. The similarity of semantic vectors of adjacent steps is accumulated as the semantic benefit index of the path. Simultaneously, the instrument reading range corresponding to each step in the path is compared with the instrument readings before and after the actual jump position. The deviation between the reading range and the actual reading is calculated, and the deviation is accumulated as the cost index of the path. During dynamic programming, the sequence of steps in the path is used as a stage division, with each step as a decision node. At each node, the cumulative benefit and cumulative cost of the current path are recorded simultaneously, and the path branch with higher benefit and lower cost is retained in subsequent node selections. To ensure the stability of the results, a fixed ratio is set between benefit and cost, for example, a benefit weight of 0.6 and a cost weight of 0.4. This ratio is obtained through statistical analysis of historical experimental data and remains unchanged during implementation. After the dynamic programming search is completed, the path with the highest comprehensive evaluation is selected from all candidate paths as the optimal filling path. The intermediate step operation description sequence contained in the path and its corresponding instrument reading range are output as the reasoning result of the missing steps, so as to achieve complete completion of the skip position.

[0073] The completion module creates a draft of the experimental process containing a complete operation chain.

[0074] A blank template matching the current experiment type is retrieved from the experimental structured template library, and the template structure is instantiated and loaded. The templates are stored in the template library in a standardized structure. Each template contains step fields arranged in the order of the experimental procedure. Each step field predefines an operation description area, an instrument reading area, and an additional marker area. During execution, the corresponding template is first located from the template library based on the type identifier of the experimental batch. For example, for chemical reaction experiments, a template containing step fields such as "reagent addition—reaction control—result recording" is called. Then, the sequence information of each step field in the template is read, and the step fields are numbered and mapped to correspond to the time sequence in the experimental event sequence. Next, each record in the experimental event sequence is traversed, and the operation descriptions in each record are filled into the corresponding step fields in the template in chronological order. During the filling process, the operation descriptions of each record are standardized, converting the original descriptions into a standardized expression form, and ensuring that each step field only contains operation content that semantically matches the description. For identified skipped step record locations, markers are inserted between the corresponding step fields in the template. For example, an "abnormal gap marker" is set between adjacent step fields, and the index information of that location is recorded. Subsequently, the operation descriptions in the missing step inference results are inserted one by one into the corresponding abnormal gap locations according to their order in the inference path. When inserting, a source identifier is added to each completion step, such as labeled "inference generation step," and its corresponding instrument reading range is recorded. After completing the template filling in the above manner, a complete step sequence structure containing the original operation record, abnormal markers, and completion steps is obtained.

[0075] The completed template structure undergoes data binding, associating each step field with the spatial coordinates and instrument readings of the corresponding region in the original image. Specifically, all step fields in the template are first traversed, reading the operation description for each field and locating the corresponding original image region coordinates using index information from the digital experimental record archive, including the region's location range and boundary information within the original image. Simultaneously, the instrument reading data corresponding to that step is extracted from the experimental record, and the readings are uniformly formatted, for example, converting readings from different units to a unified dimension while retaining the original reading range information. Subsequently, the operation description, spatial coordinates, and instrument readings in the step fields are bound together to form structured data units. Each data unit contains a step number, operation content, location coordinates, and reading information. For completion steps, since they originate from inference results, they are associated with the corresponding reading range during the binding process and marked as "inference-generated" in the spatial coordinate field, while retaining their insertion position in the template. For anomaly marker positions, the step numbers before and after the anomaly and the time interval information are recorded in the corresponding field to pinpoint the specific location where the anomaly occurred. After completing the data binding of all fields, a completeness check is performed on the entire template structure to ensure that each field contains an operation description and at least one piece of related information. Finally, the structured template is output as an experimental procedure draft. This draft records the complete operation chain, anomaly marker locations, and supplementary step information in a uniform format, achieving a structured expression of the entire experimental process.

[0076] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0077] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0078] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0080] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0082] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0084] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A medical research training intelligent auxiliary system based on a large-scale artificial intelligence model, characterized in that: include: The acquisition module is used to automatically acquire images of the recording medium on the experimental platform, and obtain raw experimental record image data including handwritten notes, instrument screens and biological samples. The recording module is used to input raw image data into the target detection network, locate and crop out the instrument's digital display area, handwritten text area and biological sample imaging area, and generate digital experimental record archives. The breakpoint identification module is used to extract the recording timestamps of digital experimental record archives, construct the experimental event sequence according to the physical recording order, and identify candidate abnormal locations where logical breakpoints exist by calculating the numerical jump gradient of adjacent instrument readings and the semantic coherence of the operation description. The skip step identification module is used to extract the first sample image slice and the second sample image slice from the sample imaging region before and after the candidate anomaly position, calculate the offset of the sample morphology distribution, and identify the existing skip step records. The step reasoning module is used to extract instrument readings and operation descriptions before and after the skipped step record position from the event sequence, perform missing step reasoning, and generate the reasoning results of missing steps based on the contextual statistical patterns of complete event sequences in the same experimental batch. The completion module is used to call blank templates from the experimental structured template library, fill in the operation descriptions, skip records and missing step inference results in the event sequence into the blank templates, and create an experimental process draft containing a complete operation chain.

2. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 1, characterized in that, The acquisition module specifically includes the automatic image acquisition of the recording medium on the experimental platform, which includes: Acquire a pre-scanned image of the current experimental platform. Based on the texture features and reflected light intensity distribution of different regions in the pre-scanned image, divide the page into a handwritten note area, an instrument screen area, and a biological sample area. The above-mentioned areas were scanned in a focused manner, and grayscale images of the handwritten note area, the instrument screen area, and the biological sample area were acquired simultaneously. All the acquired images were stitched together and fused according to their original spatial positions on the experimental platform to generate the original experimental record image data.

3. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 1, characterized in that, The recording module specifically includes generating digital experimental record archives, including: The original experimental record image data is input into the target detection network, and the bounding box coordinates and category labels of the instrument digital display area, handwritten text area and biological sample imaging area are output. The images of each region are cropped from the original image based on the bounding box coordinates, and redundant regions are removed by sorting them in descending order of confidence. Input the image of the instrument's digital display area into the optical character recognition model to output a sequence of digital characters; input the image of the handwritten text area into the text recognition model to generate a text sequence word by word. The file paths of the digital character sequence, text sequence, and biological sample imaging area image are indexed and associated according to the spatial location of each area in the original image and the order of acquisition time, generating a digital experimental record archive containing area images, recognized text, and location coordinates.

4. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 3, characterized in that, The target detection network uses a feature pyramid structure to extract multi-scale image features. After processing by classification and regression branches, it outputs the bounding box coordinates and category labels for the instrument digital display area, handwritten text area, and biological sample imaging area.

5. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 1, characterized in that, The breakpoint identification module specifically identifies candidate anomaly locations where logical breakpoints exist, including: Extract the timestamps of each record item from the digital experimental record archive, sort them in chronological order, and construct an experimental event sequence. Traverse the sequence of experimental events and find two adjacent records. Calculate the rate of change of the difference between each pair of adjacent instrument readings with respect to the time interval, and use this rate of change as the numerical jump gradient. The verb-object collocations in two adjacent operation descriptions are input into the experimental operation semantic co-occurrence statistical model built based on the same experimental batch, and the frequency score of the current collocation is calculated. A joint anomaly score is performed based on the numerical jump gradient and semantic co-occurrence score. The boundary position of adjacent records where the anomaly score reaches the preset anomaly threshold is marked as a candidate anomaly position with a logical breakpoint.

6. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 1, characterized in that, The skip step identification module specifically identifies existing skip step records by including: Based on the time period corresponding to the candidate anomaly location in the experimental event sequence, the sample image corresponding to the nearest previous record of the candidate anomaly location is extracted from the biological sample imaging area image stored in the digital experimental record archive as the first sample image slice, and the sample image corresponding to the nearest next record is extracted as the second sample image slice. The first and second sample image slices are input into the morphological difference analysis network to extract the morphological parameters and optical density parameters of each target object in the two image slices, and encode them into a phenotypic feature distribution representation. Based on the phenotypic feature distribution representation, the optimal transmission calculation is performed on the first distribution corresponding to the first sample image slice and the second distribution corresponding to the second sample image slice to obtain the distribution migration relationship from the first distribution to the second distribution and the corresponding minimum transmission cost, which is used as the morphological distribution offset. The morphological distribution offset between all adjacent sample image slices in the same experimental batch is statistically analyzed to construct a morphological distribution offset sequence. Based on the relative change level of the morphological distribution offset sequence, the morphological distribution offset of candidate anomaly locations is compared and analyzed to identify skipped records.

7. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 6, characterized in that, The optical density parameter is obtained by logarithmic transformation of the pixel gray values ​​and background gray values ​​of the grayscale image of the biological sample imaging region; the morphological parameter is based on the numerical representation of the geometric structure of the segmented contour of the target object in the biological sample imaging region.

8. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 1, characterized in that, In the step-by-step reasoning module, generating the reasoning result for the missing step specifically includes: Extract the operation description corresponding to the previous record of the skip record position from the experimental event sequence as the predecessor operation, extract the operation description corresponding to the next record as the successor operation, and extract all event sequences in the same experimental batch that do not have skip records from the digital experimental record archive to form a reference sequence library. The preceding and succeeding operations are used as sequence endpoint constraints. All possible subsequences in the reference sequence library located between these endpoints are used as candidate filling paths. Each candidate filling path contains operation descriptions of several intermediate steps arranged in sequence and corresponding instrument reading ranges. The sum of the semantic vector similarity values ​​of each step in the candidate filling path is used as the path benefit, and the degree of agreement between the instrument reading range corresponding to each step in the candidate filling path and the actual reading before and after the skip step is used as the path cost. The dynamic programming algorithm is used to search for the optimal filling path with the maximum benefit and the minimum cost in the candidate filling path space. The sequence of intermediate steps described in the optimal filling path and their corresponding instrument reading ranges are output as the inference results for the missing steps.

9. The intelligent auxiliary system for medical research training based on a large artificial intelligence model according to claim 1, characterized in that, The completion module, specifically including the creation of an experimental procedure draft containing a complete operation chain, includes: Call the corresponding blank template from the experimental structured template library. The blank template contains sequentially arranged step fields. The operation descriptions of each record in the event sequence are filled into the matching step fields in the blank template in chronological order, and the skipped records are marked in the gaps between the corresponding step fields. At the same time, the operation descriptions in the reasoning results of the missing steps are inserted into the skipped positions as completion steps. The filled-in fields are then bound to the coordinates of the corresponding regions in the original image and the instrument readings to generate a draft of the experimental procedure containing the complete operation chain and anomaly marker locations.

Images