A video processing method, apparatus, electronic device, and storage medium

Abstract

This invention discloses a video processing method, apparatus, electronic device, and storage medium. The video processing method includes: acquiring a video stream of a target embryo, the video stream comprising multiple video frames of the target embryo at different time points; processing the multiple video frames based on a cystic formation prediction model to obtain a cystic formation prediction result for the target embryo; wherein the cystic formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module; each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame; the temporal feature extraction module extracts temporal features from the spatial features corresponding to the multiple video frames to obtain target features; and the classification module classifies the target features to obtain the cystic formation prediction result for the target embryo, achieving accurate prediction of embryo cystic formation and providing accurate data support for subsequent decision-making.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a video processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of assisted reproductive technology, embryo selection is crucial for successful clinical pregnancy. The ability of an embryo to form a blastocyst directly reflects its developmental potential and is an important basis for selecting high-quality embryos and improving pregnancy rates. Accurate blastocyst formation prediction can reduce the need for multiple embryo transfers, increase the success rate of single embryo transfers, and reduce maternal and infant risks.

[0003] In existing technologies, convolutional neural networks are typically used to extract spatial features from embryo images. These extracted features are then input into a classifier for classification to obtain embryo blastocyst formation prediction results. However, due to the limited local receptive field of convolutional neural networks, their ability to capture global structural changes in embryo images is limited, and their ability to capture long-term dynamic developmental features in embryo images is insufficient, resulting in low accuracy in embryo blastocyst formation prediction results. Summary of the Invention

[0004] This invention provides a video processing method, apparatus, electronic device, and storage medium to achieve accurate prediction of embryo blastocyst formation.

[0005] According to one aspect of the present invention, a video processing method is provided, comprising: Acquire a video stream of the target embryo, which includes multiple video frames of the target embryo at different time points; Multiple video frames are processed based on the cyst formation prediction model to obtain the cyst formation prediction results of the target embryo. The cyst formation prediction model includes multiple spatial feature extraction modules, temporal feature extraction modules, and classification modules. Each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain the target features; The classification module classifies the target features to obtain the blastocyst formation prediction results of the target embryo.

[0006] According to another aspect of the present invention, a video processing apparatus is provided, comprising: The video stream acquisition module is used to acquire the video stream of the target embryo, which includes multiple video frames of the target embryo at different time points. The cyst formation prediction result determination module is used to process multiple video frames based on the cyst formation prediction model to obtain the cyst formation prediction result of the target embryo. The cyst formation prediction model includes multiple spatial feature extraction modules, temporal feature extraction modules, and classification modules. Each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain the target features; The classification module classifies the target features to obtain the blastocyst formation prediction results of the target embryo.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the video processing method of any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the video processing method of any embodiment of the present invention.

[0009] The technical solution of this invention acquires a video stream of the target embryo, which includes multiple video frames of the target embryo at different time points, thus providing comprehensive data support for subsequent analysis and processing. Based on a cystic formation prediction model, the multiple video frames are processed to obtain a cystic formation prediction result for the target embryo. The cystic formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module. Each spatial feature extraction module extracts spatial features from a video frame to obtain the corresponding spatial features. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain target features. The classification module classifies the target features to obtain the cystic formation prediction result for the target embryo. This solves the problem of low accuracy in cystic formation prediction results caused by the insufficient ability of convolutional neural networks to capture long-term dynamic developmental features in embryo images in existing technologies. The cystic formation prediction model can fully capture the spatial features and dynamic developmental changes of the target object in multiple video frames, making the cystic formation prediction result more consistent with the actual situation of the target object and improving the accuracy of the cystic formation prediction result.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a video processing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a spatial feature extraction unit provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a cyst formation prediction model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a temporal feature extraction unit provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of another cyst formation prediction model provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a video processing device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Figure 1 This is a flowchart of a video processing method provided in an embodiment of the present invention. This embodiment is applicable to predicting the ability of an embryo to develop into a blastocyst. The method can be executed by a video processing device, which can be implemented in hardware and / or software. This video processing device can be configured in the electronic device provided in this embodiment of the invention. The electronic device can be a server, computer, or mobile terminal, such as a mobile phone or tablet computer. Figure 1 As shown, the method specifically includes the following steps: S110. Acquire the video stream of the target embryo, which includes multiple video frames of the target embryo at different time points.

[0016] The target embryos are in vitro cultured embryos whose ability to develop into blastocysts is to be predicted. It should be noted that the target embryos need to be cultured in an embryo incubator that provides the necessary culture conditions. Embryo incubators include, but are not limited to, time-lapse incubators. The time-lapse incubator is equipped with a time-lapse microscopy system, which allows observation of the target embryo's state. The video stream is an ordered sequence of images representing the morphology of the target embryo at different time points. The video stream includes multiple video frames of the target embryo at different time points, each representing the static morphology of the target embryo at the corresponding time point. Each video frame includes the target embryo region and the background region. The video stream can be acquired through the time-lapse microscopy system of the time-lapse incubator. For example, continuous monitoring of the target embryo using a time-lapse microscopy system yields the video stream of the target embryo. The video stream can also be acquired through a video stream database. For example, the video stream of the target embryo can be obtained by matching its unique identifier information in the video stream database, which can store video streams of different embryos.

[0017] Specifically, the video stream of the target embryo is obtained by matching the unique identifier of the target embryo in the video stream database, thus realizing the acquisition of the target embryo's video stream and providing comprehensive data support for subsequent analysis and processing.

[0018] Based on the above embodiments, after acquiring the video stream of the target embryo, the method further includes: performing data preprocessing operations on multiple video frames, including one or more of target detection, image cropping, image scaling, and size unification. Target detection can be performed on multiple video frames separately according to a target detection model, retaining the portion corresponding to the target embryo and removing the background portion. Size unification can unify the video frames according to a preset image size, which can be 224×224. The image size can be set according to requirements, and this invention is not limited thereto.

[0019] S120. Based on the cyst formation prediction model, multiple video frames are processed to obtain the cyst formation prediction results of the target embryo.

[0020] The blastocyst formation prediction model is a structure that predicts blastocyst formation from a video stream of the target embryo. This model includes, but is not limited to, a neural network model. The blastocyst formation prediction result characterizes the likelihood of the target embryo developing into a blastocyst. Optionally, the blastocyst formation prediction result includes the blastocyst formation probability, which represents the likelihood of the target embryo developing into a blastocyst. The blastocyst formation prediction result can be determined based on multiple video frames. For example, multiple video frames of the target embryo are input into the blastocyst formation prediction model for processing to obtain the blastocyst formation prediction result for the target embryo.

[0021] Specifically, multiple video frames of the target embryo are input into the blastocyst formation prediction model for processing to obtain the blastocyst formation prediction results of the target embryo. This enables accurate prediction of the target embryo's ability to develop into a blastocyst, providing accurate data support for subsequent analysis and processing.

[0022] The blastocyst formation prediction model includes multiple spatial feature extraction modules, temporal feature extraction modules, and classification modules. Each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain the target features. The classification module classifies the target features to obtain the blastocyst formation prediction result of the target embryo.

[0023] The spatial feature extraction module is the structure in the cyst formation prediction model that extracts spatial features from video frames. This module includes, but is not limited to, a spatial attention module. Spatial features represent the morphological structure of the target object in the video frame. Spatial features can be extracted from a video frame using the spatial feature extraction module. It should be noted that one spatial feature extraction module extracts spatial features from one video frame, while different modules extract spatial features from different video frames. The temporal feature extraction module is the structure in the cyst formation prediction model that extracts temporal features from the spatial features corresponding to multiple video frames. This module includes, but is not limited to, a temporal attention module. Target features represent the morphological structure and dynamic developmental patterns of the target object in multiple video frames. Target features can be extracted from the spatial features corresponding to multiple video frames using the temporal feature extraction module. The classification module is the output structure of the cyst formation prediction model. The cyst formation prediction result can also be obtained by classifying the target features using the classification module.

[0024] Specifically, different video frames are input into the spatial feature extraction module for spatial feature extraction, obtaining the spatial features corresponding to each video frame. Multiple spatial features are simultaneously input into the temporal feature extraction module for temporal feature extraction, obtaining target features that characterize the morphological structure and dynamic developmental changes of the target object. The target features are then classified by the classification module to obtain the cystic formation prediction result of the target embryo. The cystic formation prediction model can fully capture the spatial features and dynamic developmental changes of the target object in multiple video frames, making the cystic formation prediction result more consistent with the actual situation of the target object and improving the accuracy of the cystic formation prediction result.

[0025] Optionally, each spatial feature extraction module includes at least one spatial feature extraction unit, and each spatial feature extraction unit includes a second attention layer, a second residual connection layer, and a feedforward neural network layer.

[0026] The spatial feature extraction unit is the structure used for spatial feature extraction within the spatial feature extraction module. The spatial feature extraction unit includes, but is not limited to, an encoder. Optionally, the spatial feature extraction unit includes a transformer encoder. Each spatial feature extraction module may include one spatial feature extraction unit, or multiple spatial feature extraction units. The number of spatial feature extraction units in the spatial feature extraction module is set according to requirements, and this invention does not impose a limitation. Optionally, each spatial feature extraction unit may have the same structure. For example, each spatial feature extraction unit includes a second attention layer, a second residual connection layer, and a feedforward neural network layer. The second attention layer includes, but is not limited to, a multi-head self-attention layer. It should be noted that the spatial feature extraction unit may include two second residual connection layers, wherein one second residual connection layer performs residual connection and normalization processing on the input and output of the second attention layer, and the other second residual connection layer performs residual connection and normalization processing on the input and output of the feedforward neural network layer. For example, see [link to example]. Figure 2 , Figure 2 This is a schematic diagram of a spatial feature extraction unit provided in an embodiment of the present invention. "+" indicates a residual connection.

[0027] Based on the above embodiments, before inputting the video frames into the spatial feature extraction module for spatial feature extraction, the method further includes: sequentially performing image segmentation, image flattening, and linear transformation on the video frames to convert the video frames into the input dimension of the cyst formation prediction model. This provides unified data for subsequent processing.

[0028] Based on the above embodiments, before inputting the video frame into the spatial feature extraction module for spatial feature extraction, the method further includes: spatially encoding the video frame. Spatially encoding the video frame can be performed using a sine / cosine spatial position encoding method.

[0029] Optionally, the classification module includes a global average pooling unit and a classification unit; wherein, the global average pooling unit performs global average pooling on the target features to obtain global features; and the classification unit performs classification on the global features to obtain the cyst formation prediction result.

[0030] The global average pooling unit is the structure in the classification module that aggregates and reduces the dimensionality of the target features. Global features are information obtained by performing global average pooling on the target features. Global features are used to characterize the morphological structure and dynamic developmental patterns of the target embryo in each video frame. The classification unit is the output structure of the cyst formation prediction model. Optionally, the classification unit includes a multilayer perceptron layer and an activation function layer. The activation function in the activation function layer can be a softmax function or a sigmoid function; the activation function in the activation function layer is set according to requirements, and this invention is not limited thereto. The cyst formation prediction result can also be obtained from the global features. For example, the global features can be input into the classification unit for processing to obtain the cyst formation prediction result.

[0031] Specifically, the target features are processed by global average pooling unit to obtain global features; the global classification unit is then used to classify the global features to obtain cyst formation prediction results. This achieves accurate prediction of cyst formation of the target embryo, making the cyst formation prediction results more consistent with the actual situation of the target object and improving the accuracy of the cyst formation prediction results.

[0032] Optionally, the method further includes: generating prompting information based on the blastocyst formation prediction results, wherein the prompting information is used to characterize whether the target embryo has the ability to develop into a blastocyst.

[0033] The prompt information refers to data characterizing whether the target embryo has the ability to develop into a blastocyst. The prompt information can be generated based on the blastocyst formation prediction result. For example, after obtaining the blastocyst formation prediction result, the prompt information generation logic is triggered to generate the prompt information. The prompt information can include whether the target embryo has the ability to develop into a blastocyst, or it can include whether the target embryo does not have the ability to develop into a blastocyst. For example, when the blastocyst formation prediction result is greater than or equal to a preset blastocyst formation prediction threshold, the corresponding prompt information is that the target embryo has the ability to develop into a blastocyst; when the blastocyst formation prediction result is less than the preset blastocyst formation prediction threshold, the corresponding prompt information is that the target embryo does not have the ability to develop into a blastocyst.

[0034] Specifically, after obtaining the cyst formation prediction result, the logic for generating prompt information is triggered to generate prompt information, providing data support for subsequent decision-making.

[0035] The technical solution of this embodiment acquires a video stream of the target embryo, which includes multiple video frames of the target embryo at different time points, thus providing comprehensive data support for subsequent analysis and processing. Based on a cystic formation prediction model, the multiple video frames are processed to obtain a cystic formation prediction result for the target embryo. This cystic formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module. Each spatial feature extraction module extracts spatial features from a video frame to obtain the corresponding spatial features. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain target features. The classification module classifies the target features to obtain the cystic formation prediction result for the target embryo. The cystic formation prediction model can fully capture the spatial features and dynamic developmental changes of the target object in multiple video frames, making the cystic formation prediction result more consistent with the actual situation of the target object and improving the accuracy of the cystic formation prediction result.

[0036] Figure 3 This is a schematic diagram of the structure of a cyst formation prediction model provided in an embodiment of the present invention. Figure 3 As shown, the cyst formation prediction model includes multiple spatial feature extraction modules 210, temporal feature extraction modules 220, and classification modules 230. Each spatial feature extraction module 210 extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module 220 extracts temporal features from the spatial features corresponding to multiple video frames to obtain target features. The classification module 230 classifies the target features to obtain the cyst formation prediction result of the target embryo. The temporal feature extraction module includes multiple temporal feature extraction units connected in sequence. The first temporal feature extraction unit extracts temporal features from the spatial features corresponding to multiple video frames. Subsequent temporal feature extraction units extract temporal features from the intermediate features output by the previous temporal feature extraction unit.

[0037] The temporal feature extraction unit is a component of the temporal feature extraction module. The temporal feature extraction module 220 may include three or six temporal feature extraction units, with the number set according to requirements; this invention does not impose a limitation. It should be noted that the output of the previous temporal feature extraction unit is connected to the input of the current temporal feature extraction unit. The input of the first temporal feature extraction unit consists of spatial features corresponding to multiple video frames. Intermediate features are data obtained after processing by the temporal feature extraction units. It should be noted that the intermediate features output by the last temporal feature extraction unit are the target features. Optionally, the temporal feature extraction unit includes a transformer encoder. Optionally, each temporal feature extraction unit includes a first attention layer, a first residual connection layer, and a multilayer perceptron layer. The first attention layer includes, but is not limited to, a multi-head self-attention layer. It should be noted that the temporal feature extraction unit may include two first residual connection layers. One first residual connection layer performs residual connection and normalization processing on the input and output of the first attention layer, and the other first residual connection layer performs residual connection and normalization processing on the input and output of the multilayer perceptron layer. For example, see [link to example]. Figure 4 , Figure 4 This is a schematic diagram of a temporal feature extraction unit provided in an embodiment of the present invention. "+" indicates a residual connection.

[0038] Specifically, the first temporal feature extraction unit extracts temporal features from the spatial features corresponding to multiple video frames to obtain intermediate features. The intermediate features output by the first temporal feature extraction unit are then input into the second temporal feature extraction unit for processing to obtain the intermediate features output by the second temporal feature extraction unit. This process continues until the target features output by the last temporal feature extraction unit are obtained. Through the processing of multiple sequentially connected temporal feature extraction units, the dynamic developmental patterns of the target embryo in different video frames can be fully captured, providing accurate data support for subsequent classification and thus improving the accuracy of the classification results.

[0039] Optionally, the temporal feature extraction module also includes a splicing unit, which is located before the first temporal feature extraction unit. The splicing unit splices the spatial features corresponding to multiple video frames to obtain spliced ​​spatial features. The first temporal feature extraction unit is also used to extract temporal features from the spliced ​​spatial features. Subsequent temporal feature extraction units are also used to extract temporal features from the intermediate features output by the previous temporal feature extraction unit.

[0040] The stitching unit is the foremost feature processing unit in the temporal feature extraction module, used to stitch together the spatial features corresponding to multiple video frames. For example, it can stitch together the spatial features corresponding to multiple video frames according to their acquisition time order. The stitched spatial features are a feature sequence formed by stitching together multiple spatial features according to the acquisition time order of each video frame. The stitched spatial features can be directly used as input to the first temporal feature extraction unit. The intermediate features output by the first temporal feature extraction unit can also include information obtained by processing the stitched spatial features through the first temporal feature extraction unit.

[0041] Specifically, the stitching unit stitches together the spatial features corresponding to multiple video frames according to the acquisition time sequence of each video frame to obtain stitched spatial features. The stitched spatial features are then input into the first temporal feature extraction unit for temporal feature extraction, resulting in intermediate features output by the first temporal feature extraction unit. These intermediate features are then input into the second temporal feature extraction unit for processing, resulting in intermediate features output by the second temporal feature extraction unit. This process continues until the target features output by the last temporal feature extraction unit are obtained. While preserving the spatial features corresponding to multiple video frames, this method can fully capture the dynamic developmental patterns of the target embryo in different video frames, providing comprehensive and accurate data support for subsequent classification, thereby improving the accuracy of cyst formation prediction results.

[0042] For example, see Figure 5 , Figure 5 This is a schematic diagram of another cyst formation prediction model provided in an embodiment of the present invention. The cyst formation prediction model includes m spatial feature extraction modules, a temporal feature extraction module, and a classification module. Each spatial feature extraction module includes 6 spatial feature extraction units. The temporal feature extraction module and the classification module include a splicing unit and 6 sequentially connected temporal feature extraction units.

[0043] Based on the above embodiments, before the temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames, the method further includes: temporally encoding the spatial features corresponding to the multiple video frames respectively. Temporally encoding any spatial feature can be performed using a sine / cosine temporal encoding method.

[0044] The technical solution of this embodiment provides a cystic embryo formation prediction model, including multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module. Each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain target features. The classification module classifies the target features to obtain the cystic embryo formation prediction result of the target embryo. The temporal feature extraction module includes multiple temporal feature extraction units connected in sequence. The first temporal feature extraction unit extracts temporal features from the spatial features corresponding to multiple video frames. Subsequent temporal feature extraction units extract temporal features from the intermediate features output by the previous temporal feature extraction unit. While retaining the spatial features corresponding to multiple video frames, this model can fully capture the dynamic developmental patterns of the target embryo in different video frames, which is beneficial to improving the accuracy of the cystic embryo formation prediction result.

[0045] Figure 6 This is a schematic diagram of the structure of a video processing device provided in an embodiment of the present invention. Figure 6 As shown, the device includes a video stream acquisition module 310 and a cyst formation prediction result determination module 320.

[0046] The video stream acquisition module 310 is used to acquire the video stream of the target embryo, which includes multiple video frames of the target embryo at different time points. The cystic formation prediction result determination module 320 is used to process the multiple video frames based on the cystic formation prediction model to obtain the cystic formation prediction result of the target embryo. The cystic formation prediction model includes multiple spatial feature extraction modules, temporal feature extraction modules, and classification modules. Each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain the target features. The classification module classifies the target features to obtain the cystic formation prediction result of the target embryo.

[0047] The technical solution of this embodiment uses a video stream acquisition module to acquire a video stream of the target embryo. The video stream includes multiple video frames of the target embryo at different time points, realizing the acquisition of the video stream of the target embryo and providing comprehensive data support for subsequent analysis and processing. A cystic formation prediction result determination module is used to process multiple video frames based on a cystic formation prediction model to obtain the cystic formation prediction result of the target embryo. The cystic formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module. Each spatial feature extraction module extracts spatial features from a video frame to obtain the spatial features corresponding to the video frame. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain the target features. The classification module classifies the target features to obtain the cystic formation prediction result of the target embryo. The cystic formation prediction model can fully capture the spatial features and dynamic developmental changes of the target object in multiple video frames, making the cystic formation prediction result more consistent with the actual situation of the target object and improving the accuracy of the cystic formation prediction result.

[0048] Based on the above embodiments, optionally, the temporal feature extraction module includes multiple temporal feature extraction units connected in sequence; wherein, the first temporal feature extraction unit performs temporal feature extraction on the spatial features corresponding to multiple video frames respectively; and the non-first temporal feature extraction units perform temporal feature extraction on the intermediate features output by the previous temporal feature extraction unit.

[0049] Optionally, each temporal feature extraction unit includes a first attention layer, a first residual connection layer, and a multilayer perceptron layer.

[0050] Optionally, the temporal feature extraction module also includes a splicing unit, which is located before the first temporal feature extraction unit. The splicing unit splices the spatial features corresponding to multiple video frames to obtain spliced ​​spatial features. The first temporal feature extraction unit is also used to extract temporal features from the spliced ​​spatial features. Subsequent temporal feature extraction units are also used to extract temporal features from the intermediate features output by the previous temporal feature extraction unit.

[0051] Optionally, each spatial feature extraction module includes at least one spatial feature extraction unit, and each spatial feature extraction unit includes a second attention layer, a second residual connection layer, and a feedforward neural network layer.

[0052] Optionally, the classification module includes a global average pooling unit and a classification unit; wherein, the global average pooling unit performs global average pooling on the target features to obtain global features; and the classification unit performs classification on the global features to obtain the cyst formation prediction result.

[0053] Optionally, the device also includes a prompting information generation module for: generating prompting information based on the blastocyst formation prediction results, wherein the prompting information is used to characterize whether the target embryo has the ability to develop into a blastocyst.

[0054] The video processing apparatus provided in the embodiments of the present invention can execute the video processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0055] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0056] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0057] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0058] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as video processing methods.

[0059] In some embodiments, the video processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the video processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the video processing method by any other suitable means (e.g., by means of firmware).

[0060] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0061] Computer programs for implementing the video processing method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0062] This invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a video processing method, the method comprising: The process involves acquiring a video stream of the target embryo, which includes multiple video frames of the target embryo at different time points. The multiple video frames are then processed based on a cystic formation prediction model to obtain a cystic formation prediction result for the target embryo. This cystic formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module. Each spatial feature extraction module extracts spatial features from a single video frame to obtain the corresponding spatial features. The temporal feature extraction module extracts temporal features from the spatial features corresponding to multiple video frames to obtain the target features. The classification module then classifies the target features to obtain the cystic formation prediction result for the target embryo.

[0063] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0064] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0065] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0066] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0067] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0068] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of video processing, the method comprising: including: Obtain a video stream of a target embryo, where the video stream includes multiple video frames of the target embryo at different time points; Process the multiple video frames based on a blastocyst formation prediction model to obtain a blastocyst formation prediction result of the target embryo; Wherein, the blastocyst formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module; Each of the spatial feature extraction modules extracts spatial features from a video frame to obtain spatial features corresponding to the video frame; The temporal feature extraction module extracts temporal features from the spatial features corresponding to the multiple video frames respectively to obtain target features; The classification module performs classification processing on the target features to obtain a blastocyst formation prediction result of the target embryo.

2. The method of claim 1, wherein, The temporal feature extraction module includes multiple sequentially connected temporal feature extraction units; Wherein, the first temporal feature extraction unit extracts temporal features from the spatial features corresponding to the multiple video frames respectively; the non-first temporal feature extraction unit extracts temporal features from the intermediate features output by the previous temporal feature extraction unit.

3. The method of claim 2, wherein, Each of the temporal feature extraction units includes a first attention layer, a first residual connection layer, and a multi-layer perceptron layer.

4. The method of claim 2, wherein, The temporal feature extraction module further includes a splicing unit, and the splicing unit is located before the first temporal feature extraction unit; Wherein, the splicing unit splices the spatial features corresponding to the multiple video frames respectively to obtain spliced spatial features; The first temporal feature extraction unit is further configured to: extract temporal features from the spliced spatial features; the non-first temporal feature extraction unit is further configured to: extract temporal features from the intermediate features output by the previous temporal feature extraction unit.

5. The method of claim 1, wherein, Each of the spatial feature extraction modules includes at least one spatial feature extraction unit, and each of the spatial feature extraction units includes a second attention layer, a second residual connection layer, and a feed-forward neural network layer.

6. The method of claim 1, wherein, The classification module includes a global average pooling unit and a classification unit; Wherein, the global average pooling unit performs global average pooling processing on the target features to obtain global features; The classification unit performs classification processing on the global features to obtain the blastocyst formation prediction result.

7. The method of claim 1, wherein, The method further includes: Generate a prompt message based on the blastocyst formation prediction result, where the prompt message is used to indicate whether the target embryo has the ability to develop into a blastocyst.

8. A video processing apparatus, comprising: including: A video stream acquisition module, configured to obtain a video stream of a target embryo, where the video stream includes multiple video frames of the target embryo at different time points; A blastocyst formation prediction result determination module, configured to process the multiple video frames based on a blastocyst formation prediction model to obtain a blastocyst formation prediction result of the target embryo; Wherein, the blastocyst formation prediction model includes multiple spatial feature extraction modules, a temporal feature extraction module, and a classification module; Each of the spatial feature extraction modules extracts spatial features from a video frame to obtain spatial features corresponding to the video frame; The temporal feature extraction module extracts temporal features from the spatial features corresponding to the multiple video frames respectively to obtain target features; The classification module performs classification processing on the target features to obtain the blastocyst formation prediction result of the target embryo.

9. An electronic device, comprising: The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the video processing method according to any one of claims 1-7.

Images