A deep learning-based bone puncture specimen real-time pathological recognition system and method

Abstract

The application provides a deep learning-based bone puncture specimen real-time pathological identification system and method, relates to the technical field of bone puncture pathological identification, and comprises: an image density feature extraction module, which is used for acquiring an image frame sequence in a bone puncture specimen channel and outputting a tissue density stable structure domain set; a boundary contour calibration module, which is used for generating an image sliding boundary structure atlas based on the tissue density stable structure domain set; an atlas time sequence construction module, which is used for extracting the corresponding position of a structure point on a puncture path by using the image sliding boundary structure atlas, judging the position relationship between a spectrum frame sampling position and the structure point, and outputting an atlas and spectrum frame synchronous pairing set; and a spectrum peak evolution section extraction module, which is used for extracting the spectrum data in each pairing according to a synchronous order by using the atlas and spectrum frame synchronous pairing set, identifying the position of a main peak and the peak symmetry form of left and right regions, and outputting an atlas evolution consistency image paragraph set.

Description

Technical Field

[0001] This invention relates to the field of bone marrow biopsy pathology identification technology, and in particular to a deep learning-based real-time pathology identification system and method for bone marrow biopsy specimens. Background Technology

[0002] The field of bone marrow aspiration pathology identification technology falls under the category of medical artificial intelligence applications that integrate medical image recognition and pathological analysis. This technology primarily involves image acquisition, data analysis, and pathological feature identification of tissue specimens obtained through bone marrow aspiration. Core aspects include specimen image acquisition methods, image feature extraction techniques, automatic classification methods for pathological feature identification, and auxiliary assessment of specimen quality. In clinical applications, this technology is typically used to assist in the diagnosis of pathological conditions such as hematological diseases and bone marrow lesions, providing diagnostic references through the analysis results of bone marrow aspiration specimens. Traditional real-time pathological identification of bone marrow aspiration specimens involves fixing, embedding, sectioning, and staining manually collected bone marrow aspiration specimens before sending them to the pathology department. Pathologists then observe the tissue structure under a microscope to determine the presence of pathological changes, interpreting image features and pathological morphology through manual microscopic observation. The technical challenge addressed here is how to achieve real-time pathological identification of the specimen during the bone marrow aspiration process to assist in specimen sampling and diagnostic confirmation. Traditional methods involve sending the specimen to a pathology laboratory for processing and awaiting the pathologist's manual diagnostic results.

[0003] Current technologies rely on pathologists manually interpreting images of stained sections under a microscope. This is limited by the lag in image acquisition and subjective judgment, resulting in deficiencies in real-time performance and consistency of judgment standards. During bone marrow aspiration, it is impossible to capture continuous changes in tissue structure in real time, making it difficult to implement auxiliary judgments based on dynamic changes in structural features at the puncture site. Some minor lesions or areas with blurred boundaries may be missed or misjudged due to observation delays or subjective identification. At the same time, there is a lack of effective correspondence between images and subsequent multimodal data such as spectra, which restricts the depth of multidimensional information fusion analysis and the timeliness and accuracy of diagnostic support. Summary of the Invention

[0004] To address the limitations of existing technologies, such as image acquisition delays and subjective judgment discrepancies leading to deficiencies in real-time performance and consistency of judgment standards, the inability to capture continuous changes in tissue structure during bone aspiration, the difficulty in implementing auxiliary judgments based on dynamic changes in structural features at the puncture site, the potential for missed or misjudged lesions or areas with blurred boundaries due to observation delays or subjective identification, and the lack of effective correspondence between images and subsequent multimodal data such as spectra, which restricts the depth of multidimensional information fusion analysis and the timeliness and accuracy of diagnostic support, this invention provides a deep learning-based real-time pathological identification system and method for bone aspiration specimens.

[0005] The technical solutions provided by the embodiments of the present invention are as follows: A first aspect of this invention provides a deep learning-based real-time pathological identification system for bone aspiration specimens, comprising: The image density feature extraction module is used to acquire the image frame sequence within the bone puncture specimen channel, arrange the gray-level dots of the longitudinal puncture area in each frame image in a column direction, compare the gray-level change direction of pixels between adjacent frames, identify the point columns with consistent change direction, and output the set of stable tissue density structure surface regions. The boundary contour calibration module is used to detect gray-level difference changes in the edge direction of the surface region based on the stable structure surface region set of tissue density, and to mark the boundary points of the pixel positions where the gray-level difference change direction is reversed, thereby generating an image slip boundary structure map. The image temporal construction module is used to extract the corresponding positions of structural points on the puncture path using the image sliding boundary structure map, extract Raman spectrum sampling record frames in the path in sequence, determine the positional relationship between the sampling positions of the spectral frames and the structural points in the image, and output a synchronous pairing set of the image and spectral frames. The peak evolution segment extraction module is used to extract spectral data from each pair of synchronously paired sets of spectral images and spectral frames in a synchronous order, identify the location of the main peak and the peak symmetry of the left and right regions, and output a set of image segments with consistent spectral evolution.

[0006] A second aspect of this invention provides a deep learning-based method for real-time pathological identification of bone marrow specimens, applied to the deep learning-based system for real-time pathological identification of bone marrow specimens described in the first aspect, comprising: S1: Obtain the image frame sequence within the bone aspiration specimen channel, call the gray-level pixel matrix of the longitudinal puncture region in each frame image, arrange the gray values ​​according to the column order, calculate the gray-level direction change trend of pixels between adjacent image frames, filter the pixel columns with consistent direction changes, and generate a set of tissue density stable structural surface regions. S2: Based on the stable structure surface region set of tissue density, call the gray-level difference in the edge direction of the structure region, detect the trend of gray-level difference in the edge direction, mark the pixel coordinates at the position where the gray-level difference direction is reversed, compare whether the vertical position of the reversed coordinates in adjacent image frames is continuous, and generate the image slip boundary structure map. S3: Call the image slip boundary structure map, obtain the Raman spectrum sampling frame in the corresponding puncture path, and determine whether it is located in the vicinity of the structure point based on the spatial position relationship between the coordinates of the map structure point and the coordinates of the spectrum sampling point. Pair the map frame and spectrum frame that meet the proximity criterion to generate a synchronous pairing set of map and spectrum frames. S4: Call the synchronized pairing set of the map and spectral frames, identify the frequency domain position corresponding to the wavenumber of the main peak, obtain the half-peak width data of the regions on both sides of the main peak and perform symmetry difference calculation, determine whether the main peak offset direction and the symmetry difference change direction are consistent in adjacent spectral frames, extract and merge the map frame numbers corresponding to the spectral groups that show consistent changes continuously, and generate a map evolution consistency image segment set. S5: Call the image segment set of the map evolution consistency, obtain three types of image structure parameters in the corresponding frame image: the gray density of the nuclear region, the directionality of cytoplasmic texture, and the rate of change of curvature of the tissue boundary in the puncture area, and compare them with the spectral main peak shift direction to generate a map and spectral joint recognition training label set.

[0007] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: By identifying the stability of point regions with consistent gray-level change directions in image sequences, a structural region with tissue continuity is constructed. The gray-level reversal trajectory at the boundary is detected, the spatial extension relationship of the structural boundary is obtained, and a synchronous pairing between spectral sampling frames and spectral structural points is established to enhance the correspondence accuracy between spectral data and image features. By combining the trend of spectral main peak changes with the continuous judgment of image structural evolution, the accurate association between spectral evolution stages and image structural segments is achieved. Furthermore, through the linkage analysis of three types of image feature directions—nuclear region density, cytoplasmic texture continuity, and boundary curvature trend—with spectral peak trends, a multi-dimensional mutually verifiable spectral identification label system is constructed to improve the discrimination stability of real-time identification in the puncture path, the structural analysis capability of spectral information, and the trend consistency recognition accuracy of pathological features. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of the structure of a deep learning-based real-time pathological identification system for bone aspiration specimens, provided in an embodiment of the present invention.

[0010] Figure 2 This is a flowchart illustrating a method for real-time pathological identification of bone marrow aspiration specimens based on deep learning, provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0012] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0013] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0014] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0015] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0016] Reference manual attached Figure 1 The diagram shows a schematic representation of a deep learning-based real-time pathological identification system for bone aspiration specimens provided in an embodiment of the present invention.

[0017] This invention provides a deep learning-based real-time pathological identification system for bone marrow aspiration specimens, the system comprising: The image density feature extraction module is used to acquire the image frame sequence within the bone puncture specimen channel. It arranges the grayscale dots of the longitudinal puncture region in each frame image in a column direction, compares the grayscale change direction of pixels between adjacent frames, identifies the dots with consistent change direction, and marks the persistent dots in the image sequence as structurally stable regions, outputting a set of tissue density stable structural surface regions.

[0018] The boundary contour calibration module is used to detect gray-level difference changes in the edge direction of the surface region based on the stable structure surface region set of tissue density, and to mark the boundary points of the pixel positions where the gray-level difference change direction is reversed. It determines whether the boundary points in the image frame sequence form a continuous trajectory along the puncture direction. If a continuous trajectory exists, it is determined to be a slip boundary line, and an image slip boundary structure map is generated.

[0019] The image-sliding boundary structure map module is used to extract the corresponding positions of structure points on the puncture path using the image-sliding boundary structure map. It extracts Raman spectral sampling and recording frames in the path in sequence, and judges the positional relationship between the sampling position of the spectral frame and the structure point in the image. If the position of the spectral point falls within the vicinity of any structure point, a set of image-spectral synchronization pairings is formed, and the image-spectral synchronization pairing set is output.

[0020] The spectral peak evolution segment extraction module is used to extract spectral data from each pair of synchronously paired sets of spectra and spectral frames in a synchronous order, identify the location of the main peak and the peak symmetry of the left and right regions, and make a consistency judgment on the main peak shift trend and symmetry change trend in adjacent spectra. If multiple consecutive spectral groups maintain consistent changes, the corresponding spectral frame number is recorded as an evolution marker segment, and the spectral evolution consistency image segment set is output.

[0021] The pathological recognition module is used to extract puncture area image data from a set of image segments with consistent spectral evolution. It performs orientation judgment on three types of structural features in the puncture area image: gray density of the nuclear region, continuity of cytoplasmic texture, and curvature trend of tissue boundaries. It compares the change direction of each feature with the change direction of the main peak in the corresponding spectrum. If multiple features show a consistent trend, a positive recognition label is assigned to the image segment; otherwise, a negative label is assigned. The module outputs a training label set for joint spectral and image recognition.

[0022] The set of stable structural regions for tissue density includes a series of points with consistent gray-level changes, markers for structurally stable regions, and a sequence of vertical gray-level arrays. The image gravitational boundary structure map includes gravitational boundary lines, boundary point trajectories, and edge gray-level inversion markers. The set of synchronized pairing of image and spectral frames includes the location of image structural points, spectral sampling frame numbers, and the pairing relationship between image and spectral positions. The set of image segments with consistent image evolution includes the main peak shift trend, consistent symmetrical change segments, and spectral peak evolution segment numbers. The training label set for joint recognition of image and spectral data includes nuclear region gray-level density direction labels, cytoplasmic texture continuity direction labels, and tissue boundary curvature trend direction labels.

[0023] In one possible implementation, the image frame acquisition submodule is used to acquire the image frame sequence within the bone puncture specimen channel, sequentially read the complete pixel array of the longitudinal puncture area in the frame image, number the image frames according to their order on the time axis, and map the gray values ​​of the pixels to their spatial positions to generate an image frame sequence gray matrix set.

[0024] Specifically, images within the needle tract are continuously captured using a microscopic imaging device with a fixed puncture needle tract perspective. This process is synchronized with bone marrow puncture, and the acquisition frequency can be set to 30 frames per second to obtain sufficient temporally continuous image samples. In practice, the acquisition of image sequences requires setting the image storage path in the image acquisition software and recording the timestamp information of each frame for sequence sorting. Pixel array extraction of the longitudinal puncture area is performed on each frame of image data. If the puncture area is fixed within the column index range in the middle of the image, the grayscale value of each pixel within the range needs to be read sequentially starting from the upper left corner of the image. The grayscale value comes from the pixel information in the 8-bit grayscale image. During the reading process, image processing software such as OpenCV is used to array the frame images. After arraying, each frame image is numbered according to its time sequence, and each frame image is bound to the corresponding time index to form a time sequence structure. The pixel grayscale values ​​are mapped to a unified spatial location system according to the row and column coordinates in the image. The grayscale matrix data corresponding to the longitudinal puncture area in the image frame is structured and organized to generate an image frame sequence grayscale matrix set.

[0025] The grayscale column construction submodule is used to extract pixels in the same column within the vertical puncture region of an image based on the grayscale matrix set of the image frame sequence, aggregate and arrange the grayscale values ​​in the order of the frame sequence, call the inter-frame difference sequence of grayscale values ​​in the aggregation result, compare the change amplitude and change direction of grayscale values ​​between adjacent frames, and obtain the column-oriented grayscale change trend set.

[0026] Specifically, each column of the vertical puncture region in the image needs to be processed frame by frame. The system identifies the fixed coordinate range of multiple columns in the image grayscale matrix, corresponding to the puncture region range set during acquisition. Starting from the first frame, the grayscale values ​​of the same column position are extracted sequentially, and a grayscale time series is constructed according to the image frame acquisition order. If the 210th column in the image frame is set to be extracted, then the grayscale values ​​from the first frame to the last frame need to be extracted, resulting in a set of column-to-grayscale evolution sequences. Such sequences can be used to analyze the grayscale change trend in local areas, and the extracted grayscale values ​​are used to calculate inter-frame differences. The grayscale values ​​between adjacent frames are directionally determined, i.e., whether the grayscale is increasing, decreasing, or remaining stable. The continuity and stability on the time axis can be determined by the arrangement structure of the directional changes. If the column-wise grayscale changes show a consistent directional trend in multiple consecutive frames, and the grayscale is set to continuously increase, then the region is considered to have a continuous density change trend. By comparing the amplitude of the grayscale changes, invalid fluctuations such as random jitter can be eliminated, enhancing the ability to judge the actual change trend. After processing, the grayscale change directions sorted out in each column are combined to form a column-wise grayscale change direction set.

[0027] The stable structure recognition submodule is used to identify points and columns in continuous frames that maintain a consistent direction of change based on the set of column-oriented grayscale change directions, filter column-oriented sequences whose consistent change direction has reached a stable threshold for a certain number of consecutive frames, call the corresponding frame position information, mark the column-oriented regions that meet the stability requirements, and generate a set of stable structure surface regions for tissue density.

[0028] Specifically, the continuity analysis of the grayscale change direction of each column in the time series is performed to identify the trend sequence of grayscale change direction in each column. It is searched to find whether there are long segments of frame sequences that maintain a consistent grayscale change direction. If the grayscale value of a certain column consistently shows an upward or downward trend for 15 consecutive frames without any reversal, then the region can be preliminarily identified as a continuous structure with a consistent change direction. The trend segments are filtered to determine whether they meet the preset frame number threshold requirement. If the frame number of a trend segment does not exceed the threshold, it is not included in the stable structure marking category. The column regions that meet the trend consistency and have a sufficient number of continuous frames are located in the original image frames. The system extracts the corresponding frame number and column coordinate position, and marks them as stable regions in the image using two-dimensional coordinates. The region is represented by a continuous and stable grayscale structure on the time axis and a continuous column distribution in space. By integrating the image position and frame sequence range of stable column segments, a set of stable structure surface regions with tissue density is generated.

[0029] In one possible implementation, the edge gray difference detection submodule is used to extract the pixel gray values ​​in the area region by region based on the stable structure area region set of tissue density, calculate the gray difference between adjacent pixels in the column direction, identify the changing trend of continuous gray difference, and record the gray change direction for pixels whose absolute gray difference value exceeds the set edge response threshold, and generate a gray change direction map of the region edge.

[0030] Specifically, each area region undergoes pixel-level processing, extracting the grayscale values ​​of pixels and organizing them according to the image column direction. During the process, the system sequentially traverses the pixels within a specified column range in each area region, extracting the grayscale value differences between adjacent pixels arranged from top to bottom. Based on the vertical grayscale difference of pixels within a column, the system calculates the difference sequence for that column. Then, a sliding comparison method is used to identify the trend direction of grayscale changes in the difference sequence. By setting a continuous increase or decrease in the grayscale difference of pixels continuously moving downwards within a column, it can be determined that the region exhibits a clear edge transition trend. During the judgment process, an edge response threshold is set, and the absolute value of each grayscale difference is compared. If the absolute value of a difference exceeds the threshold, the pixel is determined to be an edge position. The direction of the pixel difference change is recorded, i.e., determining whether the grayscale change is from dark to bright or bright to dark. The directional state of each pixel exceeding the threshold is marked. The edge pixel trend results in the column direction are integrated to generate a region edge grayscale change direction map.

[0031] The boundary point identification submodule is used to call the gray-level change direction map of the region edge, locate the column position where the gray-level change direction between adjacent pixels is reversed, mark the boundary point at the position in the surface domain that has the feature of reversed change direction, and combine the corresponding image frame index to map the boundary point to the spatial coordinates in the frame sequence to generate the image frame sequence boundary point position set.

[0032] Specifically, the grayscale change direction of adjacent pixels in each column is analyzed to find the position where the grayscale change direction reverses. The current pixel is set to bright to dark and the next pixel to dark to bright, or the two pixels before and after each other are edge trends in different directions. This is regarded as the turning point of edge change. Pixels often represent inter-tissue interfaces or density transition areas in the image structure, so they need to be specially marked. The system scans the column direction change sequence in the area to find the above-mentioned abrupt change positions and marks the positions as boundary points to form a preliminary set of boundary points. After the marking is completed, the system calls the number information of the image frame to which the area belongs, and pairs the column index of the boundary point with the index of the frame to obtain the specific spatial position of the boundary point in the temporal frame structure. Not only are the positions of the boundary points marked, but the temporal positions in the image frame sequence are also mapped to form a set of boundary point positions in the image frame sequence.

[0033] The slip trajectory determination submodule is used to detect whether the boundary points in the frame sequence maintain the continuity of the column position in the puncture direction based on the boundary point position set of the image frame sequence, and to determine whether the boundary point index forms a serialized arrangement structure. If the continuous column position matching condition is met, it is determined to be a slip structure, the corresponding boundary point chain is marked and the image is reconstructed to generate an image slip boundary structure map.

[0034] Specifically, the boundary points are sorted according to the time axis of the frame sequence. The positions of boundary points within the same frame are paired and analyzed according to the column coordinates. It is observed whether there is a continuous arrangement of boundary points located at the same or adjacent column coordinates in multiple consecutive frames. If a certain column position in five consecutive frames of images has boundary point markings, it means that the position always shows obvious structural boundary features throughout the puncture process. After the system identifies this kind of temporal continuity, it strings the consecutively appearing boundary points together into a trajectory structure. During the identification process, the system judges whether the continuity of the trajectory meets the set minimum number of consecutive frames and the stability range of column displacement. If the column displacement between any two consecutive frame boundary points exceeds the allowable threshold, the trajectory marking is interrupted. Otherwise, it is identified as a real slip structure. The boundary point sequence that meets the conditions will be uniformly classified under a slip boundary line. The system reconstructs the image for each slip boundary line and redraws the boundary points on the trajectory in the original image frame to form a visual structure, generating an image slip boundary structure map.

[0035] In one possible implementation, the structure path localization submodule is used to extract the intra-frame column direction and frame order index coordinates of the structure points on the puncture path based on the image slip boundary structure map, record the spatial positioning information of the structure points in the image sequence, and arrange the positions of the structure points according to the frame order index to obtain the puncture path structure point sequence set.

[0036] Specifically, structural points representing the slip boundary lines are extracted frame by frame, and the column position and corresponding frame index coordinates of each structural point in the image frame are recorded. After uniformly numbering the structural points, they are arranged sequentially in the image sequence to form a temporally continuous structural trajectory dataset. During the extraction process, the boundary line data of the frames in the atlas need to be traversed, and the actual column coordinate position of the boundary points in the image needs to be identified. At the same time, the corresponding position in the temporal structure is recorded through the frame index. If the boundary point in the 20th frame is located in the 240th column of the image, it is recorded as structural point S1 = (frame order = 20, column = 240), and so on. The structural point information in the image frames is collected and encoded. Combining the column coordinates of the structural points with the frame index, a spatial-temporal joint coordinate sequence is constructed. During the data processing, in order to ensure the continuity of the structural point sequence, the continuity of structural points between frames needs to be checked to eliminate data interruptions caused by positioning errors or missing frames, so as to obtain the puncture path structural point sequence set.

[0037] The image and spectrum pairing submodule is used to call the puncture path structure point sequence set, extract the corresponding Raman spectrum sampling record frame in the puncture path frame by frame, obtain the position coordinates of the spectral frame sampling point, determine whether the spectral sampling point is in the vicinity of any structure point, and if the vicinity interval position determination condition is met, then the frame index and spatial pairing relationship between the structure point and the spectral point are established, and the image and spectral frame synchronous pairing set is generated.

[0038] Specifically, the coordinates of the structural points corresponding to each frame are sequentially obtained from the sequence. Within the same frame, the sampling record frame of the Raman spectrum is extracted. By reading the data output by the spectrometer recording device, the sampling position of the corresponding spectral sampling point in the image coordinate system is obtained. Generally, the spatial position of the spectral sampling point is determined by sampling control parameters. Combined with the spatial resolution of the Raman probe and the image acquisition angle, it can be converted into column coordinate values. After completing the spatial position mapping operation, it is determined whether the spectral sampling point is located within the neighboring region of any structural point. The neighboring region is set based on the column coordinate of the structural point in the image and a fixed column tolerance range. For example, if the column coordinate of the structural point is 240 and the tolerance range is ±2 columns, it is only necessary to determine whether the spectral sampling point falls between column coordinates 238 and 242. If the condition is met, a synchronous pairing relationship is established between the spectral sampling point and the structural point. Each pairing records the frame number and spatial coordinate information, and a mapping channel between the spectral structural point and the spectral sampling point is formed using the frame index as a connecting line. The structural point-spectral point combinations that meet the pairing conditions are integrated to generate a synchronous pairing set of the spectrum and spectral frames.

[0039] In one possible implementation, the main peak morphology recognition submodule is used to extract spectral data from each pairing according to the synchronization order based on the spectrum and spectral frame synchronous pairing set, locate the global peak value of the signal intensity in each pair of spectra as the main peak position, extract the spectral line intensity distribution in the specified intervals to the left and right of the main peak, analyze the peak symmetry degree of the left and right segments, and generate the main peak symmetry feature sequence.

[0040] Specifically, the system extracts and analyzes the spectral data of each group in the pairing order, identifying the global peak position of the signal intensity in each spectral curve. This peak represents the main Raman response of the target molecule at the current sampling point. After establishing the correspondence between spectral intensity and frequency for each spectral data, the system scans the frequency position corresponding to the maximum signal intensity in the curve and records it as the main peak position. Fixed frequency intervals are set on the left and right sides of the main peak, with 15 frequency points offset to the left and 15 frequency points offset to the right. The spectral line intensity distribution data in the two intervals are extracted and used as inputs for peak symmetry calculation. During the analysis, the system calculates the amplitude difference, slope change, and distribution pattern consistency of the spectral lines on the left and right sides, forming a set of data indicators describing the peak shape symmetry characteristics. The system sets the core feature quantities based on the difference in integral area on both sides, the difference in mean, or the difference in slope change. The process is executed sequentially for the spectral pairing groups, gradually establishing a data sequence containing the main peak position and symmetry characteristics of each group, and constructing the main peak symmetry feature sequence.

[0041] The spectral peak trend analysis submodule is used to call the main peak symmetry feature sequence, combine it with the main peak frequency offset value, construct the joint trend sequence of main peak frequency position and symmetry feature in adjacent spectral pairing groups, calculate the main peak position offset direction and symmetry change direction between consecutive pairing groups, determine whether the change direction is consistent within the specified sliding window length, and generate a spectral peak trend consistency index sequence.

[0042] Specifically, by combining the frequency offset value of the main peak of each spectrum, the system jointly judges the peak change trend between adjacent spectral pairs. The system calculates the difference in the main peak frequency position in any two adjacent spectral pairs to represent the offset direction of the main peak in the time series. At the same time, it extracts the change value of the peak symmetry feature between the two groups, that is, judges whether the symmetry is enhanced, weakened or has no significant change. These two changes constitute the joint evolution trend feature of the main peak in the time series. In data processing, the system traverses the entire pairing sequence in the form of a sliding window. In each window, it judges whether the direction of the main peak frequency change is consistent with the direction of the symmetry change. If both directions show continuous increase, continuous decrease or remain stable, it is considered that there is a consistent peak change trend within the window. The judgment process will gradually slide the entire sequence according to the preset window length to generate a peak trend consistency index sequence.

[0043] The evolutionary segment labeling submodule is used to filter the consecutive index labels as consistent based on the spectral peak trend consistency index sequence, extract the corresponding spectral frame number set, and use the number as the segment boundary to record the start and end frame indices of consecutive consistent segments, thereby generating a spectral evolution consistency image segment set.

[0044] Specifically, the system filters paired groups marked as having consistent trends, identifies index segments where consistent trend markers appear consecutively in the sequence, and sets a threshold where five consecutive paired groups are marked as having consistent trends, recording these consecutive segments as candidate evolutionary segments. The system extracts the frame number information corresponding to the paired groups in the graph, and forms a segment boundary index by combining the start and end frame numbers of each segment. This index is used to construct the evolutionary trajectory interval in the graph time series. To prevent misjudging short-term trend fluctuations as evolutionary segments, the system sets a minimum threshold for the number of consistent frames, requiring at least three consecutive consistent indices to be recognized as a segment. After meeting the conditions, each segment number is stored and archived. The system then summarizes the set of graph frame indices that meet the consistent trend conditions and outputs a set of graph evolution consistency image segments.

[0045] In one possible implementation, the structural feature extraction submodule is used to extract the corresponding puncture region image data in the image segment based on the image segment set of map evolution consistency, calculate the gray density distribution of the cell nucleus region, the gradient direction of the texture edge of the cytoplasm region and the curvature change path of the image edge contour frame by frame, obtain the arrangement trend of the three types of structural features in the image segment respectively, and generate the structural feature direction sequence of the image segment.

[0046] Specifically, the system analyzes and processes the puncture region images corresponding to each image segment. This involves extracting the puncture region image data corresponding to the image frames within the segment, cropping image frames using a fixed column index range to obtain the target region image sequence, and performing cell structure feature extraction on the image content of each frame. During the extraction of gray-level density distribution in the nuclear region, the degree of clustering of pixel gray-level values ​​in the nuclear region is statistically analyzed using a gray-level histogram to reflect dense features. In the texture analysis of the cytoplasmic region, the system extracts the arrangement direction and continuity trend of texture by calculating the local edge gradient direction of the image and combining it with the edge amplitude. In the analysis of the curvature change path of the image edge contour, a curve contour is constructed using the coherent coordinates of the boundary contour points of the puncture region. The curvature direction is calculated by extracting the angle changes of continuous boundary points to form a spatial structure change trend. The three types of structural features are respectively formed into frame-level direction markers, which are then arranged in the order of frame index to obtain the structural direction trend. The directional data of the three types of structures in the frame are summarized according to the image segment number to generate an image segment structural feature direction sequence.

[0047] The feature orientation comparison submodule is used to call the image segment structural feature orientation sequence, obtain the structural feature orientation within the image segment, extract the direction of change of the main peak frequency in the corresponding Raman spectrum within the segment, compare the structural feature orientation with the direction of change of the main peak frequency according to the segment number, determine whether the structural feature orientation is consistent with the direction of change of the main peak, and generate a result showing the consistency between the feature and the spectral orientation.

[0048] Specifically, the arrangement direction information of the three types of structural features in each segment is obtained according to the segment number. These include the direction of gray density trend in the core region, the direction of the dominant gradient of the cytoplasmic texture edge, and the direction of edge curvature. The corresponding direction of Raman spectrum main peak frequency change is also obtained. Within each image segment, the direction of the three types of image structural features is compared with the direction of main peak frequency change one by one. If the main peak frequency in the segment shows an upward trend, it is checked whether the gray density is enhanced, whether the texture direction tends to be consistent, and whether the curvature path increases frame by frame. By matching the direction change status of the three types of structures, it is determined whether each item is consistent with the direction of spectral main peak change. For consistent items, a positive relationship is recorded, and inconsistent items are marked as direction deviation. The process is executed cyclically within the segment and the comparison results are recorded to generate a result of consistency between features and spectral direction.

[0049] The label recognition submodule is used to count the number of features with consistent orientation among the three types of structural features in an image segment based on the consistency results of features and spectral orientation. If the number of consistent features reaches the orientation consistency threshold, a positive label is assigned to the segment; otherwise, a negative label is assigned. The segment number and label value are summarized to generate a training label set for joint recognition of the image and spectrum.

[0050] Specifically, statistical processing is performed on the consistency of the three types of structural features within an image segment with the direction of the spectral peak change. The comparison results of the three types of features in each segment are extracted, and it is determined whether the number of features with consistent direction reaches a preset threshold. If the threshold for consistent direction is set to 2, then at least two structural features must have the same direction as the direction of the peak change for the segment to be judged as a positive label; otherwise, it is a negative label. The judgment results are bound to the segment number to form a set of structural label pairs for the segment. After all segments have undergone the judgment process, the system will summarize and output the judgment results to generate a training label set for joint recognition of the image and spectrum.

[0051] Reference manual attached Figure 2 The diagram shows a flowchart of a deep learning-based method for real-time pathological identification of bone marrow aspiration specimens provided by the present invention.

[0052] This invention also provides a deep learning-based method for real-time pathological identification of bone marrow specimens, applied to the aforementioned deep learning-based real-time pathological identification system for bone marrow specimens, comprising: S1: Obtain the image frame sequence within the bone aspiration specimen channel, call the gray-level pixel matrix of the longitudinal puncture region in each frame image, arrange the gray values ​​in column order, calculate the gray-level direction change trend of pixels between adjacent image frames, filter the pixel columns with consistent direction changes, and generate a set of tissue density stable structural surface regions.

[0053] S2: Based on the stable structure surface region set of tissue density, call the gray-level difference in the edge direction of the structure region, detect the trend of gray-level difference in the edge direction, mark the pixel coordinates at the position where the gray-level difference direction is reversed, compare whether the vertical position of the reversed coordinates in adjacent image frames is continuous, and generate the image slip boundary structure map.

[0054] S3: Call the image slip boundary structure map, obtain the Raman spectrum sampling frame in the corresponding puncture path, and determine whether it is located in the vicinity of the structure point based on the spatial position relationship between the coordinates of the map structure point and the coordinates of the spectrum sampling point. Pair the map frame and spectrum frame that meet the proximity criterion to generate a synchronous pairing set of map and spectrum frames.

[0055] S4: Call the synchronized pairing set of the map and spectral frames, identify the frequency domain position corresponding to the wavenumber of the main peak, obtain the half-peak width data of the regions on both sides of the main peak and perform symmetry difference calculation, determine whether the main peak offset direction and the symmetry difference change direction are consistent in adjacent spectral frames, extract and merge the map frame numbers corresponding to the spectral groups that show consistent changes, and generate a map evolution consistency image segment set.

[0056] S5: Call the image segment set of the map evolution consistency, obtain three types of image structure parameters in the corresponding frame image: the gray density of the nuclear region, the directionality of cytoplasmic texture, and the rate of change of curvature of the tissue boundary in the puncture area, and compare them with the spectral main peak shift direction to generate a map and spectral joint recognition training label set.

[0057] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0058] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0059] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0060] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0061] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0062] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0063] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0066] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0068] The following points need to be explained: (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0069] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0070] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0071] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based real-time pathological identification system for bone marrow aspiration specimens, characterized in that, include: The image density feature extraction module is used to acquire the image frame sequence within the bone puncture specimen channel, arrange the gray-level dots of the longitudinal puncture area in each frame image in a column direction, compare the gray-level change direction of pixels between adjacent frames, identify the point columns with consistent change direction, and output the set of stable tissue density structure surface regions. The boundary contour calibration module is used to detect gray-level difference changes in the edge direction of the surface region on a region-by-region basis based on the set of stable structure surface regions of tissue density, mark the boundary points of the pixel positions where the direction of gray-level difference change is reversed, and generate an image slip boundary structure map. The image time sequence construction module is used to extract the corresponding positions of structural points on the puncture path using the image slip boundary structure map, extract Raman spectrum sampling record frames in the path in sequence, determine the positional relationship between the sampling positions of the spectral frames and the structural points in the image, and output a synchronous pairing set of the image and spectral frames. The peak evolution segment extraction module is used to extract spectral data from each pair according to the synchronous pairing set of the spectrum and spectral frame, identify the location of the main peak and the peak symmetry of the left and right regions, and output a set of image segments with consistent spectrum evolution.

2. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 1, characterized in that, The set of stable structural surface regions with tissue density includes a consistent gray-level change point array, structurally stable region markers, and a vertical gray-level array sequence. The image slip boundary structure map includes slip boundary lines, boundary point trajectories, and edge grayscale inversion markers; The synchronized pairing set of the spectrum and spectral frames includes the location of the spectrum structure points, the number of the spectral sampling frame, and the pairing relationship between the spectrum and spectral positions. The set of image segments representing the evolution of the spectrum includes the main peak shift trend, the segments showing consistent symmetry changes, and the segment numbers representing the evolution of the spectral peaks.

3. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 1, characterized in that, The image density feature extraction module specifically includes: The image frame acquisition submodule is used to acquire the image frame sequence within the bone puncture specimen channel. It sequentially reads the complete pixel array of the longitudinal puncture area in the frame image, numbers the image frames according to their order on the time axis, and maps the gray values ​​of the pixels to their spatial positions to generate an image frame sequence gray matrix set. The grayscale column construction submodule is used to extract pixels in the same column within the vertical puncture area of ​​the image based on the grayscale matrix set of the image frame sequence, aggregate and arrange the grayscale values ​​according to the frame sequence order, call the inter-frame difference sequence of grayscale values ​​in the aggregation result, compare the change amplitude and change direction of grayscale values ​​between adjacent frames, and obtain the column-oriented grayscale change trend set. The stable structure identification submodule is used to identify points and columns with consistent change directions in consecutive frames based on the set of column-direction grayscale change directions, filter column-direction sequences whose consistent change direction has reached a stable threshold for a continuous number of frames, call the corresponding frame position information, mark the column-direction regions that meet the stability requirements, and generate a set of stable structure surface regions for tissue density.

4. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 1, characterized in that, The boundary contour calibration module specifically includes: The edge gray difference detection submodule is used to extract the pixel gray values ​​in the surface region one by one based on the set of stable structure surface regions of the tissue density, calculate the gray difference between adjacent pixels in the column direction, identify the changing trend of continuous gray difference, and record the gray change direction for pixels whose absolute value of gray difference exceeds the set edge response threshold, and generate a gray change direction map of the region edge. The boundary point identification submodule is used to call the gray-scale change direction map of the region edge, locate the column position where the gray-scale change direction between adjacent pixels is reversed, mark the boundary point at the position in the surface domain that has the feature of reversed change direction, and combine the corresponding image frame index to map the boundary point to the spatial coordinates in the frame sequence to generate the image frame sequence boundary point position set. The slip trajectory determination submodule is used to detect whether the boundary points in the frame sequence maintain the continuity of the column position in the puncture direction according to the boundary point position set of the image frame sequence, and to determine whether the boundary point index forms a serialized arrangement structure. If the continuous column position matching condition is met, it is determined to be a slip structure, the corresponding boundary point chain is marked and the image is reconstructed to generate an image slip boundary structure map.

5. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 1, characterized in that, The time series map construction module specifically includes: The structure path localization submodule is used to extract the intra-frame column direction and frame order index coordinates of the structure points on the puncture path based on the image slip boundary structure map, record the spatial positioning information of the structure points in the image sequence, and arrange the positions of the structure points according to the frame order index to obtain the puncture path structure point sequence set. The image and spectrum pairing submodule is used to call the puncture path structure point sequence set, extract the corresponding Raman spectrum sampling record frame in the puncture path frame by frame, obtain the position coordinates of the spectral frame sampling point, determine whether the spectral sampling point is in the vicinity of any structure point, and if the vicinity interval position determination condition is met, then the frame index and spatial pairing relationship between the structure point and the spectral point are established, and the image and spectral frame synchronous pairing set is generated.

6. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 1, characterized in that, The spectral peak evolution segment extraction module includes: The main peak morphology recognition submodule is used to extract spectral data from each pairing according to the synchronization order based on the spectrum and spectral frame synchronous pairing set, locate the global peak value of the signal intensity in each pair of spectra as the main peak position, extract the spectral line intensity distribution in the specified intervals to the left and right of the main peak, analyze the peak symmetry degree of the left and right segments, and generate the main peak symmetry feature sequence. The spectral peak trend analysis submodule is used to call the main peak symmetry feature sequence, combine the main peak frequency shift value, construct the joint trend sequence of main peak frequency position and symmetry feature in adjacent spectral pairing groups, calculate the main peak position shift direction and symmetry change direction between consecutive pairing groups, determine whether the change direction is consistent within the specified sliding window length, and generate a spectral peak trend consistency index sequence. The evolutionary segment labeling submodule is used to filter the consecutive index labels as consistent pairing sequences according to the spectral peak trend consistency index sequence, extract the corresponding spectral frame number set, and use the number as the segment boundary to record the start and end frame indices of consecutive consistent segments, thereby generating a spectral evolution consistency image segment set.

7. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 6, characterized in that, The number of frames with an odd-numbered specified sliding window length; The specific process for determining whether the direction of change remains consistent within a specified sliding window length is as follows: count the number of times the direction vector of the offset direction of the main peak position between adjacent paired groups within the sliding window range is consistent with the direction vector of the symmetry change direction of the main peak. When the proportion of the number of consistent occurrences to the total number of frames in the sliding window is not less than a preset proportion, mark the frames covered by the real-time sliding window as consistent trend segments. In the spectral peak trend consistency index sequence, the index value corresponds to a frame of spectral data. The index value is marked with a binary label. The frame index label corresponding to the consistent trend segment is 1, and the remaining frame index labels are 0.

8. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 1, characterized in that, It also includes a pathology recognition module; The pathological recognition module is used to extract puncture area image data using the image segment set with consistent atlas evolution. It performs orientation judgment on three types of structural features in the puncture area image: gray density of the nuclear region, continuity of cytoplasmic texture, and curvature trend of tissue boundary. It compares the change direction of each feature with the change direction of the main peak in the corresponding spectrum. If multiple features show a consistent trend, a positive recognition label is assigned to the image segment. If there is a deviation, a negative label is assigned. The module outputs a training label set for joint recognition of atlas and spectrum. The joint identification training label set of the map and the spectrum includes nuclear region gray density direction label, cytoplasmic texture continuity direction label, and tissue boundary curvature trend direction label.

9. The deep learning-based real-time pathological identification system for bone marrow aspiration specimens according to claim 8, characterized in that, The pathology identification module specifically includes: The structural feature extraction submodule is used to extract the corresponding puncture area image data in the image segment based on the image segment set of the atlas evolution consistency, calculate the gray density distribution of the cell nucleus region, the gradient direction of the texture edge of the cytoplasm region and the curvature change path of the image edge contour in the image frame by frame, obtain the arrangement trend of the three types of structural features in the image segment respectively, and generate the structural feature direction sequence of the image segment. The feature orientation comparison submodule is used to call the image segment structural feature orientation sequence, obtain the structural feature orientation within the image segment, extract the change direction of the main peak frequency in the corresponding Raman spectrum within the segment, compare the structural feature orientation with the change direction of the main peak frequency according to the segment number, determine whether the structural feature orientation is consistent with the change direction of the main peak, and generate a result showing consistency between the feature and the spectral orientation. The label recognition submodule is used to count the number of features with consistent orientation among the three types of structural features in the image segment based on the consistency results of the features and spectral orientation. If the number of consistent features reaches the consistency threshold, a positive label is assigned to the segment; otherwise, a negative label is assigned. The segment number and label value are summarized to generate a training label set for joint recognition of the image and spectrum.

10. A method for real-time pathological identification of bone marrow aspiration specimens based on deep learning, characterized in that, The deep learning-based real-time pathological identification system for bone aspiration specimens, as described in any one of claims 1 to 9, comprises: S1: Obtain the image frame sequence within the bone aspiration specimen channel, call the gray-level pixel matrix of the longitudinal puncture region in each frame image, arrange the gray values ​​according to the column order, calculate the gray-level direction change trend of pixels between adjacent image frames, filter the pixel columns with consistent direction changes, and generate a set of tissue density stable structural surface regions. S2: Based on the set of stable structural surface regions of the tissue density, call the gray-level difference in the edge direction of the structural region, detect the trend of gray-level difference in the edge direction, mark the pixel coordinates at the position where the gray-level difference direction is reversed, compare whether the vertical position of the reversed coordinates in adjacent image frames is continuous, and generate an image slip boundary structure map. S3: Call the image sliding boundary structure map, obtain the Raman spectrum sampling frame in the corresponding puncture path, and determine whether it is located in the vicinity of the structure point based on the spatial position relationship between the coordinates of the map structure point and the coordinates of the spectrum sampling point. Pair the map frame and spectrum frame that meet the proximity criterion to generate a synchronous pairing set of map and spectrum frames. S4: Call the synchronized pairing set of the spectrum and spectral frames, identify the frequency domain position corresponding to the wavenumber of the main peak, obtain the half-peak width data of the regions on both sides of the main peak and perform symmetry difference calculation, determine whether the main peak offset direction and the symmetry difference change direction are consistent in adjacent spectral frames, extract and merge the corresponding spectrum frame numbers of the spectral groups that show consistent changes, and generate a set of image segments with consistent spectrum evolution. S5: Call the image segment set of the map evolution consistency, obtain three types of image structure parameters in the corresponding frame image: the gray density of the nuclear region, the directionality of cytoplasmic texture, and the rate of change of curvature of the tissue boundary in the puncture area, and compare them with the spectral main peak shift direction to generate a map and spectral joint recognition training label set.

Images