Embryo development stage positioning method, device and medium

Abstract

The application discloses an embryo development stage positioning method and device and a medium, and relates to the technical field of assisted reproduction, and comprises the following steps: acquiring a frame sequence of an original embryo development video, and constructing a positive sample pair and a negative sample pair; adopting a contrast loss function to optimize parameters of a feature encoder; calculating an inter-frame similarity matrix by using the optimized feature encoder; calculating variances in different time windows based on the inter-frame similarity matrix, dividing a similarity region by taking multiple optimal change points detected on a time axis by dynamic programming as time sequence boundaries, setting different optimal change point quantity sets, and generating multi-scale time sequence nominations; extracting video-level features from the original embryo development video; cutting nomination-level features from the video-level features according to the time sequence nominations; inputting the video-level features and the nomination-level features into a neural network after splicing, and outputting a development stage category and a boundary regression result of each time sequence nomination; and the method improves the accurate positioning capability of the time sequence boundary region.

Description

Technical Field

[0001] This invention relates to the field of assisted reproductive technology, and in particular to a method, device and medium for locating embryonic development stages. Background Technology

[0002] Embryos exhibit specific morphodynamic characteristics at different developmental stages. Accurately dividing each stage is of great scientific value for morphological analysis and viability assessment. Figure 3 This study illustrates the developmental process of an embryo throughout its culture cycle. Typically, within 26-28 hours after fertilization, the embryo sequentially progresses through the pronuclear stage (D1), cleavage into 2-cell, 4-cell (D2), and 8-cell (D3) stages, at which point the embryo is in the cleavage stage. Subsequently, the embryo enters the multicellular fusion stage, forming a morula (D4), and develops into a blastocyst (D5) on days 5-6. Due to the continuity and heterogeneity of embryonic development, the morphological evolution between adjacent stages exhibits dynamic, gradual changes, and subtle differences in critical morphological structures exacerbate the ambiguity of temporal boundaries. This process possesses nonlinear bifurcation dynamics; the morphological changes in transitional regions between stages are gradual, similar to the gradual cooling or warming process of semantic features, increasing the difficulty of accurate content localization. Existing methods have limited ability to capture the subtle structural changes in this dynamic evolutionary process, making it difficult to effectively identify the start and end points of stages, leading to significant deviations in the division of embryonic development stages and affecting accurate identification. Summary of the Invention

[0003] Based on the technical problems existing in the background art, the present invention proposes a method, device and medium for locating embryonic development stages, which improves the accuracy of locating temporal boundary regions.

[0004] The present invention proposes a method for locating embryonic developmental stages, comprising: The frame sequence of the original embryonic development video is obtained, and positive sample pairs of anchor frames and positive sample frames, as well as negative sample pairs of anchor frames and negative sample frames, are constructed. Fine-grained features of each frame are extracted using a feature encoder. The parameters of the feature encoder are optimized using a contrastive loss function. The inter-frame similarity matrix is ​​calculated using the optimized feature encoder. Based on the inter-frame similarity matrix, the variance within different time windows is calculated to identify candidate temporal boundaries; multiple optimal change points detected on the time axis are used as temporal boundaries between different developmental stages to divide similar regions; and different sets of optimal change point numbers are set to generate multi-scale temporal nominations. Video-level features are extracted from the original embryonic development video; nomination-level features are extracted from the video-level features based on the temporal nomination; the video-level features and nomination-level features are concatenated and input into the action localization network, and the developmental stage category and boundary regression results of each temporal nomination are output.

[0005] Furthermore, the feature encoder is constructed by cascading multiple deformable real-valued pulse layers. Each deformable real-valued pulse layer includes deformable pulse features, deformable real-valued features, and deformable fused features. The specific calculation formula is as follows: ; ; ; in, For activation function, Represents the input image matrix. Represents the number of image channels. For the first Image data of the channel, and They represent the first The first in the passage The weight parameters of the nth filter and the nth filter The bias term of the pulse feature mapping. and Let represent the deformable pulse characteristic and the deformable real-valued characteristic, respectively, which is the th The filters are located at coordinate positions Deformable pulse eigenvalues ​​and deformable real-value eigenvalues ​​at the location, For use as constraints Local adaptive scaling factor of response range The offset of the deformable convolution represents the position in space. Above, the first Channel, First The offset of each sampling point It is a deformable fusion feature. This indicates deformable convolution, which adapts to the diversity of the internal structure of the embryo.

[0006] Furthermore, the contrastive loss function for: ; in, For predefined similarity boundary values, For anchor frames, For positive sample frames, For difficult negative sample frames, For cosine similarity, This refers to the hinge function. For negative sample frames, For difficult negative sample expansion factor, This is an indicator function.

[0007] Furthermore, the calculation of variance within different time windows to identify candidate temporal boundaries, wherein, for the similarity matrix S, at time... variance within the interval for: ; in, Let the th element in the similarity matrix S be... Frame and the Frame similarity.

[0008] Furthermore, the objective function of the dynamic programming is set to minimize the similarity loss after partitioning, and multiple optimal change points are detected through recursive segmentation. The formula for the objective function is: ; in, For from the first Frame to the The optimal loss for dividing the time interval of a frame. The location of the candidate change point. left subinterval The optimal partitioning loss, The right subinterval The optimal partitioning loss.

[0009] Furthermore, the video-level features are extracted from the original embryonic development video sequence by the I3D network, and the nomination-level features are obtained by mapping the start and end times of the temporal nominations onto the video-level features.

[0010] An embryonic development stage positioning device, comprising: The contrastive learning module for hard negative sample mining is used to acquire frame sequences of the original embryonic development video, construct positive sample pairs between anchor frames and positive sample frames, and negative sample pairs between anchor frames and negative sample frames; extract fine-grained features of each frame using a feature encoder; optimize the parameters of the feature encoder using a contrastive loss function; and calculate the inter-frame similarity matrix using the optimized feature encoder. The boundary-sensitive temporal nomination generation module calculates the variance within different time windows based on the inter-frame similarity matrix to identify candidate temporal boundaries; it uses dynamic programming to divide similar regions into temporal boundaries between different developmental stages by detecting multiple optimal change points on the time axis; and it sets different sets of optimal change point numbers to generate multi-scale temporal nominations. The feature fusion and action localization module is used to extract video-level features from the original embryonic development video; extract nomination-level features from the video-level features according to the temporal nomination; and input the video-level features and nomination-level features into the action localization network after concatenation, and output the developmental stage category and boundary regression results for each temporal nomination.

[0011] Furthermore, the contrastive learning module for negative sample mining includes: The sample partitioning unit is used to construct the positive sample pool and the negative sample pool; The fine-grained feature encoder uses a ResNet network constructed with deformable real-valued pulse layers as the feature encoder. The contrastive learning unit uses a contrastive loss function to optimize the feature encoder. The formula for the contrastive loss function is as follows: ; in, For predefined similarity boundary values, For anchor frames, For positive sample frames, For difficult negative sample frames, For cosine similarity, This refers to the hinge function. For the current sample, For difficult negative sample expansion factor, This is an indicator function.

[0012] Furthermore, the video-level features are extracted from the original embryonic development video sequence by the I3D network, and the nomination-level features are obtained by mapping the start and end times of the temporal nominations onto the video-level features.

[0013] A computer-readable storage medium having stored thereon a plurality of computer programs for being invoked by a processor and executing the method as described above.

[0014] The advantages of the embryonic development stage localization method, device, and medium provided by this invention are as follows: First, it achieves effective extraction of fine-grained semantic features from negative sample frames (especially difficult negative sample frames). By introducing bivariate feature (positive sample pairs and negative sample pairs) modeling and contrastive learning strategies, high-order fine-grained features in subtle structural changes are mined, thereby significantly improving the method's ability to distinguish and identify transitional states between adjacent developmental stages. Second, a boundary-sensitive temporal nomination generation method is proposed. This method is based on inter-frame similarity matrix modeling and uses variance as a key statistic to automatically detect and generate multi-scale temporal nominations, thereby enhancing the temporal sensitivity of this embodiment to boundary regions. Finally, a precise temporal content localization mechanism that integrates multi-source features is designed. By jointly modeling truncated nomination-level features and video-level features, collaborative perception of global and local semantics is achieved, thereby improving the precise localization capability of temporal boundary regions. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the structure of the present invention; Figure 2 This is a schematic diagram of the structure and flow of a feature encoder; Figure 3 This refers to the developmental process of an embryo during a culture cycle. Detailed Implementation

[0016] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0017] like Figure 1 and Figure 2 As shown, the present invention proposes a method for locating embryonic development stages, comprising: Step 1: Obtain the frame sequence of the original embryonic development video, construct positive sample pairs between anchor frames and positive sample frames, and negative sample pairs between anchor frames and negative sample frames; extract fine-grained features of each frame using a feature encoder; optimize the parameters of the feature encoder using a contrastive loss function; calculate the inter-frame similarity matrix using the optimized feature encoder. Step 2: Based on the inter-frame similarity matrix, calculate the variance within different time windows to identify candidate temporal boundaries; use dynamic programming to divide similar regions into temporal boundaries between different developmental stages by detecting multiple optimal change points on the time axis; set different sets of optimal change point numbers to generate multi-scale temporal nominations. Step 3: Extract video-level features from the original embryonic development video; extract nomination-level features from the video-level features based on the temporal nominations; concatenate the video-level features and nomination-level features and input them into the action localization network (e.g., ActionFormer), and output the developmental stage category and boundary regression results for each temporal nomination.

[0018] This embodiment first achieves efficient extraction of fine-grained semantic features from negative sample frames (especially difficult negative sample frames). By introducing bivariate feature (positive sample pairs and negative sample pairs) modeling and contrastive learning strategies, high-order fine-grained features in subtle structural changes are mined, thereby significantly improving the ability of this embodiment to distinguish and identify transitional states between adjacent developmental stages. Second, a boundary-sensitive temporal nomination generation method is proposed. This method is based on inter-frame similarity matrix modeling and uses variance as the key statistic to automatically detect and generate multi-scale temporal nominations, thereby enhancing the temporal sensitivity of this embodiment to boundary regions. Finally, a mechanism for accurate temporal content localization that integrates multi-source features is designed. By jointly modeling truncated nomination-level features and video-level features, collaborative perception of global and local semantics is achieved, thereby improving the accurate localization capability of temporal boundary regions.

[0019] In embryonic developmental timing analysis, a primitive embryonic developmental video of a complete embryo culture cycle is used. Its actual annotation can be represented as a set of action instances. ,in , and Representing action instances respectively The start time, end time, and corresponding developmental stage categories, Videos representing the development of primitive embryos Total number of action instances This is an index for action instances. An action instance is a specific stage in embryonic development; in this embodiment, each different developmental stage is referred to as a different action instance, for example... Figure 3 The diagram shows the pronuclear stage, cleavage stage (2-cell, 4-cell, 8-cell), morula stage, blastocyst stage, etc.

[0020] The goal of embryonic development stage localization is based on inputting raw embryonic development videos. Predict the set of developmental stages within it. Due to the continuity of morphological dynamics between adjacent developmental stages, temporal boundaries often exhibit high uncertainty. Existing methods fail to effectively distinguish between positive samples and difficult negative samples, making it difficult for traditional methods to learn fine-grained differences in visual semantics. To address this issue, this embodiment proposes a boundary-sensitive two-stage embryonic developmental stage analysis framework, which consists of three core modules: a contrastive learning module for mining difficult negative samples, a boundary-sensitive temporal nomination generation module, and a feature fusion and action localization module. This method focuses on mining temporal heterogeneity features between developmental stages, breaking through the traditional localization paradigm based on static classification labels. By constructing a boundary-enhanced contrastive learning mechanism, this framework guides the method of this embodiment to capture critical state dynamics features in the developmental process at both global and local scales, thereby improving the ability to analyze key temporal boundaries.

[0021] 1. A contrastive learning module for mining difficult negative samples; (1-1) Dividing positive and negative samples; First, videos of primitive embryonic development. The time interval is Uniform sampling to obtain contents Video sequence of frames , For the first A video sequence of frames. Positive and negative sample pools are constructed based on a sampling strategy, where each anchor frame... and positive samples Constitutes positive sample pairs, while anchor frames and negative samples Form negative sample pairs.

[0022] Negative samples are further classified using a loss ranking calculated based on the I3D network. In this embodiment, a pre-trained I3D network is used, and negative samples are further divided into simple negative samples and hard negative samples, denoted as follows: and Among them, positive samples and simple negative samples Significant differences exist at both the pixel and semantic levels, making them easily distinguishable by action recognition networks; however, difficult negative samples... Compared with positive samples While highly similar at the pixel level, these sequences are semantically completely different, often existing in the "warm-up" or "cool-down" phase of the semantic sphere, significantly increasing the difficulty of differentiation. This embodiment further delineates negative samples, enabling efficient learning guidance for video sequences that are difficult to classify and locate.

[0023] (1-2) Fine-grained feature encoder; To accurately capture subtle morphological differences at each stage of embryonic development, this embodiment employs a deformable bivariate hybrid feature encoder to acquire fine-grained visual semantic features of the embryo. This feature encoder uses a deformable real-valued pulse layer (ssLayer) as its basic building block, and utilizes multiple deformable real-valued pulse layers (such as...) Figure 2 Higher-order feature representation is achieved by stacking ssLayer (e.g., ssLayerN, etc.) in series, mapping input data to a specific feature space to extract visual semantic information of embryos with varying morphologies. Based on the original Restricted Boltzmann Machine (Spike-and-Slab RBM), each hidden unit has a pair of (spike, slab) variables; this embodiment extends this to include both types of information in each feature map. Furthermore, this embodiment replaces standard convolution with deformable convolution, enabling the sampling grid to dynamically adjust its sampling position according to the embryo's morphological changes. This effectively adapts to the rapid evolution of cell cluster morphology, stretching and rotation of embryonic polarity, and irregular deformation caused by blastocoel expansion during embryonic development. Here, slab is a real-valued variable used to capture continuous, overall intensity changes within the embryonic image patch (e.g., zona pellucida thickness, blastomere uniformity); spike is a binary variable used to control the activity or presence of its corresponding real-valued variable, simulating the presence or absence of key embryonic structures (e.g., pronucleus, polar body). The two work together to enable the network to accurately locate discriminative micro-regions within the embryo.

[0024] The deformable real-valued pulse layer introduces the squared term after deformable convolution when calculating pulse features. In statistics, the squared term and the cross term represent the covariance, which allows the deformable real-valued pulse layer to capture not only the average appearance of the embryo (first-order statistics) but also its internal texture and structural variations (second-order statistics). Simultaneously, due to the spatially adaptive nature of deformable convolution, the model can actively deform the sampling grid to align with the geometrical differences between different embryo samples, overcoming the registration challenges caused by embryo rotation, scaling, and asymmetric development.

[0025] The deformable real-valued pulse layer can extract second-order features from embryo images with only a single layer and has stronger geometric adaptive feature expression capabilities, enabling higher-order features to exhibit better discriminative power in tasks such as embryo quality assessment, developmental stage discrimination, and implantation success rate prediction. Each deformable real-valued pulse layer includes a pulse feature layer, a real-valued feature layer, and a fusion feature layer, used to obtain pulse features, real-valued features, and fusion features, respectively. The calculation formulas are shown in equations (1) to (3). The feature encoder is constructed by cascading multiple deformable real-valued pulse layers. Each deformable real-valued pulse layer includes deformable pulse features, deformable real-valued features, and deformable fusion features. The specific calculation formulas are as follows: ,(1); (2); (3); in, For activation function, Represents the input image matrix. Represents the number of image channels. For the first Image data of the channel, and They represent the first The first in the passage The weight parameters of the nth filter and the nth filter The bias term of the pulse feature mapping. and Let represent the deformable pulse characteristic and the deformable real-valued characteristic, respectively, which is the th The filters are located at coordinate positions Deformable pulse eigenvalues ​​and deformable real-value eigenvalues ​​at the location, For use as constraints Local adaptive scaling factor of response range The offset of the deformable convolution represents the position in space. Above, the first Channel, First The offset of each sampling point This represents a deformable fusion feature. In the formula... This indicates deformable convolution, which adapts to the diversity of the internal structure of the embryo.

[0026] (1-3) Comparative learning; like Figure 2 As shown, the mapping process of the feature encoder can be formally expressed as: , Representative feature encoder, For video sequences The fused features are obtained through feature encoder mapping. To measure the visual semantic differences between different frames in the original embryonic development video, a contrastive learning-based loss function is further constructed to calculate the visual semantic similarity matrix between samples. Taking positive sample pairs as an example, their similarity can be measured using cosine similarity. ,in, Indicates anchor frame With positive sample frames Similarity in feature space. A higher similarity value indicates a greater degree of visual semantic similarity between the two samples. Similarly, semantic similarity calculation for negative sample pairs... ,in, Indicates anchor frame With negative sample frames Similarity in the feature space. The lower the value, the greater the difference in visual semantics between the two. Finally, the optimization process of the feature encoder is constrained by the contrastive loss function of formula (4) to ensure that the model can effectively learn the fine-grained semantic differences between samples, thereby improving the accurate recognition ability of the embryonic development stage.

[0027] (4); in, These are predefined similarity boundary values ​​used to measure the similarity constraints between the anchor frame and positive and negative sample frames. This represents the hinge function (the loss function for support vector machines). Its mechanism of action is determined by this boundary value. In this embodiment, it refers to a parameter that is not explicitly stated. ; For the expansion factor of difficult negative samples ( >1), if and only if the indicator function When the value is 1, that is, when the current sample is a difficult negative sample, the loss is... Conversely, the loss is .

[0028] Overall, the core objective of contrastive learning is to reduce the distance between positive sample pairs in the feature space while increasing the distance between negative sample pairs, so as to achieve effective differentiation and semantic representation of positive and negative samples.

[0029] During the inference phase, the trained feature encoder is used to calculate the inter-frame similarity matrix among all frames in the original embryonic development video. , The first The video sequence corresponding to each frame, of which This serves as an important basis for subsequent time-series nomination generation. To simplify the following description, sample pairs... Similarity between .

[0030] 2. Boundary-sensitive timing nomination generation module; The core of temporal nomination lies in constructing a set of video frames that are visually semantically consistent and temporally continuous. This is based on an inter-frame similarity matrix. This embodiment proposes a concise and efficient semantic similarity measurement method to accurately capture temporal features. This method analyzes the inter-frame similarity matrix and concatenates visually semantically similar and temporally continuous frames to achieve efficient and robust temporal nomination generation. Using video frames as the basic unit, it can accurately measure semantic similarity at any given time while ensuring the sensitivity of the nomination results to dynamic temporal changes.

[0031] To further measure the global relationship of inter-frame similarity over long time periods, temporal structure is identified by calculating changes in the statistical characteristics of the time series. This is to characterize the inter-frame similarity matrix. The dispersion of inter-frame similarity is analyzed, and the semantic differences between anchor frames and positive and negative samples are distinguished. Variance is selected as the core statistical attribute. The specific process employs a temporal nomination generation method based on change point detection, with the following steps: (2-1) Variance calculation; For a given inter-frame similarity matrix At any time variance within the interval It can be calculated by summing similarities, see formula (5) for details: (5); in, Let the th element in the similarity matrix S be... Frame and the Frame similarity.

[0032] This variance is used to measure the similarity fluctuations within different time windows, thereby identifying possible time series boundaries as candidate time series boundaries.

[0033] The purpose of identifying candidate temporal boundaries through variance calculation in this embodiment is to quickly and roughly locate time regions where semantic changes may occur, serving as attention cues or heuristic information for subsequent dynamic programming.

[0034] Furthermore, the candidate temporal boundaries are not directly output as boundaries, but rather provide an initial, weighted search guide for dynamic programming. For example, change points are more likely to be set in regions with high variance. This avoids dynamic programming performing an indiscriminate exhaustive search across the entire time axis, reducing computational overhead.

[0035] (2-2) Dynamic programming optimization; The similarity matrix is ​​optimized using dynamic programming, with the goal of finding the optimal change point on the time axis that minimizes the similarity loss of each segment after interval partitioning. The optimization objective function is as follows: (6); in, In order to be in The optimal loss for dividing the time interval. The location of the candidate change point. left subinterval The optimal partitioning loss, The right subinterval The optimal partitioning loss.

[0036] This process can effectively partition the inter-frame similarity matrix. Several similar regions are defined so that frames within the same region share relatively consistent semantic information.

[0037] The goal of dynamic programming optimization is to accurately partition similar regions in a globally optimal sense, with the objective of minimizing the similarity loss after partitioning. Dynamic programming can operate independently of candidate temporal boundaries, but if we know beforehand which regions are more likely to have boundaries through these candidate boundaries, we can focus the search space on these regions while still maintaining the ability to optimize globally. Therefore, candidate temporal boundaries improve the computational efficiency and enhance the boundary sensitivity of dynamic programming optimization.

[0038] (2-3) Multi-scale temporal nomination generation: Based on the aforementioned variance analysis method, the inter-frame similarity matrix can be partitioned multiple times using different numbers of variable points to generate temporal nominations at different time scales. Specifically, this is achieved by setting an optimal set of variable points. A complete video sequence can be divided into multiple temporal nominations with different intersection-union ratios, among which... For the first The optimal variable point This is the optimal variable point index. For each variable point, the resulting start time is set. and end time Common definition corresponding to the first Timing nomination of each frame This method can deeply mine the internal statistical patterns of the similarity matrix, enabling the generated temporal nominations to flexibly adapt to action intervals with different time spans, thereby enhancing the adaptability and generalization ability of the method in this embodiment in complex temporal tasks.

[0039] 3. Feature fusion and motion localization module; To generate boundary-sensitive temporal nominations and obtain action category information, nomination-level features are derived based on temporal nominations and video-level features, acquiring fine-grained visual semantic information from a local perspective. Specifically, the I3D network is used to extract video-level features from raw embryonic development videos. Among them, I3D network, short for Two-Stream Inflated 3DConvNets, was proposed by the DeepMind team in 2017 and is mainly used for tasks such as action recognition in video understanding.

[0040] The starting point that has already received a timeline nomination and end time Mapping to video-level features The specific moment in the video is used to determine whether the nominated feature belongs to the video-level feature. Which part, and from video-level features The features extracted during this period are used as the nomination-level features for this time series nomination, that is, the nomination-level features of this video. Compared to video-level features that are more sensitive to action categories. In comparison, nomination-level features It tends to focus on the local visual semantics of videos, and is therefore more sensitive to the temporal boundaries of actions. To combine features from these two different scales to integrate information from different perspectives, video-level features... and nomination-level features These features are concatenated together and used as input to the subsequent action localization network for action classification and regression. This multi-scale feature effectively balances action temporal boundaries and action categories, while providing both local and global information.

[0041] Based on the above description of the embodiments, those skilled in the art will understand that the embryonic development stage positioning method, device, and medium described in this embodiment can be implemented in pure software or deployed and run on a general-purpose or dedicated computing hardware platform. Based on this essence, the technical solution of this embodiment can be specifically implemented in the form of a software product containing program instructions. This software product can be stored on various non-volatile storage media or directly deployed as a local or cloud service. The program instructions are used to cause computer devices with processing capabilities—including but not limited to personal computers, server clusters, mobile terminals, or other network devices—to execute the steps described in this embodiment.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for locating embryonic developmental stages, characterized in that, include: Obtain the frame sequence of the original embryonic development video, and construct positive sample pairs of anchor frames and positive sample frames, as well as negative sample pairs of anchor frames and negative sample frames. Fine-grained features of each frame are extracted using a feature encoder; the parameters of the feature encoder are optimized using a contrastive loss function; and the inter-frame similarity matrix is ​​calculated using the optimized feature encoder. Based on the inter-frame similarity matrix, the variance within different time windows is calculated to identify candidate temporal boundaries; multiple optimal change points detected on the time axis are used as temporal boundaries between different developmental stages to divide similar regions; and different sets of optimal change point numbers are set to generate multi-scale temporal nominations. Video-level features are extracted from the original embryonic development video; nomination-level features are extracted from the video-level features based on the temporal nomination; the video-level features and nomination-level features are concatenated and input into the action localization network, and the developmental stage category and boundary regression results of each temporal nomination are output.

2. The method according to claim 1, characterized in that, The feature encoder is constructed by cascading multiple deformable real-valued pulse layers. Each deformable real-valued pulse layer includes deformable pulse features, deformable real-valued features, and deformable fused features. The specific calculation formula is as follows: ； ； ； in, For activation function, Represents the input image matrix. Represents the number of image channels. For the first Channel image data, and They represent the first The first in the passage The weight parameters of the nth filter and the nth filter The bias term of the pulse feature mapping. and Let represent the deformable pulse characteristic and the deformable real-valued characteristic, respectively, which is the th The filters are located at coordinate positions Deformable pulse eigenvalues ​​and deformable real-value eigenvalues ​​at the location, For use as constraints Local adaptive scaling factor of response range The offset of the deformable convolution represents the position in space. Above, the first Channel, First The offset of each sampling point It is a deformable fusion feature. This indicates deformable convolution, which adapts to the diversity of the internal structure of the embryo.

3. The method according to claim 1, characterized in that, The contrast loss function for: ； in, For predefined similarity boundary values, For anchor frames, For positive sample frames, For difficult negative sample frames, For cosine similarity, This refers to the hinge function. For negative sample frames, For difficult negative sample expansion factor, This is an indicator function.

4. The method according to claim 1, characterized in that, The calculation of variance within different time windows to identify candidate temporal boundaries, wherein, for the similarity matrix S, at time... variance within the interval for: ； in, The first element in the similarity matrix S is... Frame and the Frame similarity.

5. The method according to claim 4, characterized in that, The objective function of the dynamic programming is set to minimize the similarity loss after partitioning, and multiple optimal change points are detected through recursive segmentation. The formula for the objective function is: ； in, For from the first Frame to the The optimal loss for dividing the time interval of a frame. The location of the candidate change point. left subinterval The optimal partitioning loss, The right subinterval The optimal partitioning loss.

6. The method according to claim 1, characterized in that, The video-level features are extracted from the original embryonic development video sequence by the I3D network, and the nomination-level features are obtained by mapping the start and end times of the temporal nominations onto the video-level features.

7. A device for locating embryonic development stages, characterized in that, include: The contrastive learning module for hard negative sample mining is used to acquire frame sequences of the original embryonic development video, construct positive sample pairs between anchor frames and positive sample frames, and negative sample pairs between anchor frames and negative sample frames; extract fine-grained features of each frame using a feature encoder; optimize the parameters of the feature encoder using a contrastive loss function; and calculate the inter-frame similarity matrix using the optimized feature encoder. The boundary-sensitive temporal nomination generation module calculates the variance within different time windows based on the inter-frame similarity matrix to identify candidate temporal boundaries; it uses dynamic programming to divide similar regions into temporal boundaries between different developmental stages by detecting multiple optimal change points on the time axis; and it sets different sets of optimal change point numbers to generate multi-scale temporal nominations. The feature fusion and action localization module is used to extract video-level features from the original embryonic development video; extract nomination-level features from the video-level features according to the temporal nomination; and input the video-level features and nomination-level features into the action localization network after concatenation, and output the developmental stage category and boundary regression results for each temporal nomination.

8. The positioning device according to claim 7, characterized in that, The contrastive learning module for negative sample mining includes: The sample partitioning unit is used to construct the positive sample pool and the negative sample pool; The fine-grained feature encoder uses a ResNet network constructed with deformable real-valued pulse layers as the feature encoder. The contrastive learning unit uses a contrastive loss function to optimize the feature encoder. The formula for the contrastive loss function is as follows: ； in, For predefined similarity boundary values, For anchor frames, For positive sample frames, For difficult negative sample frames, For cosine similarity, This refers to the hinge function. For the current sample, For difficult negative sample expansion factor, This is an indicator function.

9. The positioning device according to claim 7, characterized in that, The video-level features are extracted from the original embryonic development video sequence by the I3D network, and the nomination-level features are obtained by mapping the start and end times of the temporal nominations onto the video-level features.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs, which are used to be invoked by a processor and to execute the method as described in any one of claims 1-6.

Images