Video classification method and device, computer equipment, storage medium and program product
By obtaining videos to be examined in the area of interest, using the space-time attention model and Hilbert transformation technology, the blood supply state is objectively evaluated, and the accuracy and reliability problems caused by relying on subjective observation in the existing technology are solved, and high-precision blood supply state evaluation is achieved.
Patent Information
- Application Number
- CN202510653525.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-20
- Publication Date
- 2025-08-15
AI Technical Summary
In the prior art, medical staff rely on subjective observation to judge the blood supply status after finger breakage, broken limb replantation and flap transplantation, resulting in poor accuracy and reliability of the judgment results.
By obtaining the video to be examined in the region of interest, using the spatiotemporal attention model to extract the pulse wave signal, combining Hilbert transformation and instantaneous phase analysis, the instantaneous blood flow velocity of the video frame is determined, thereby objectively evaluating the blood supply state.
The contactless judgment of the blood supply status is achieved, subjective factors are eliminated, and the accuracy and reliability of the blood supply status evaluation is improved.
Smart Images
Figure CN120492669A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a video classification method, apparatus, computer equipment, storage medium, and program product. Background Art
[0002] With the development of medical technology, reconstructive surgeries such as replantation of severed fingers and limbs and skin flap transplantation have become common surgical procedures in the medical field.
[0003] Postoperative blood supply status monitoring is key to successful surgery. In related technologies, medical staff rely on observing the epidermis of the affected area, such as color and degree of epidermal swelling, to comprehensively judge the blood supply status.
[0004] However, the relevant technology relies on medical staff to judge the blood supply status of the affected area, which is easily affected by the subjective factors of medical staff, and thus has the problem of poor accuracy and reliability of the judgment results. Summary of the Invention
[0005] Based on this, it is necessary to provide a video classification method, device, computer equipment, storage medium and program product to address the above technical problems.
[0006] In a first aspect, the present application provides a video classification method, comprising:
[0007] Get the video to be inspected of the area of interest;
[0008] Determine the pulse wave signal based on the video to be inspected;
[0009] The reference blood flow velocity of the video to be inspected is determined according to the pulse wave signal, and the category of the video to be inspected is determined according to the reference blood flow velocity; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
[0010] In one embodiment, determining a pulse wave signal based on a video to be detected includes:
[0011] The video to be inspected is input into the spatiotemporal attention model, and the temporal and spatial features of the video to be inspected are extracted through the spatiotemporal attention model. The temporal and spatial features are then fused to obtain the pulse wave signal.
[0012] In one embodiment, determining a reference blood flow velocity of a video to be inspected based on a pulse wave signal includes:
[0013] determining the instantaneous blood flow velocity of each video frame in the video to be inspected according to the pulse wave signal;
[0014] The reference blood flow velocity of the video to be inspected is determined according to the instantaneous blood flow velocity of each video frame.
[0015] In one embodiment, determining the instantaneous blood flow velocity of each video frame in the video to be inspected based on the pulse wave signal includes:
[0016] Determine the instantaneous phase of each pixel in each video frame according to the pulse wave signal;
[0017] The instantaneous blood flow velocity of each video frame is determined according to the instantaneous phase of each pixel point in each video frame.
[0018] In one embodiment, determining the instantaneous phase of each pixel in each video frame based on the pulse wave signal includes:
[0019] Performing Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal;
[0020] The instantaneous phase of each pixel in the video frame is determined based on the analytical signal.
[0021] In one embodiment, determining the instantaneous blood flow velocity of each video frame according to the instantaneous phase of each pixel in each video frame includes:
[0022] Obtain the instantaneous phase difference between corresponding pixels of two temporally adjacent video frames;
[0023] The instantaneous blood flow velocity of the target video frame is determined according to the instantaneous phase difference and the time interval between the two video frames; the target video frame is the earlier video frame in the temporal sequence of the two temporally adjacent video frames.
[0024] In a second aspect, the present application further provides a video classification device, comprising:
[0025] A video acquisition module is used to acquire the video to be inspected of the area of interest;
[0026] A signal determination module, used to determine the pulse wave signal based on the video to be detected;
[0027] The category determination module is used to determine the category of the video to be inspected according to the pulse wave signal; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
[0028] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the steps of the above-mentioned video classification method when executing the computer program.
[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned video classification methods.
[0030] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, implements the steps of any of the above-mentioned video classification methods.
[0031] In the above-mentioned video classification method, apparatus, computer device, storage medium, and program product, a video of a region of interest is obtained to determine a pulse wave signal based on the video, and then a reference blood flow velocity of the video is determined based on the pulse wave signal. The category of the video is then determined based on the reference blood flow velocity. The category of the video is used to characterize the blood supply status of the region of interest. In the above-mentioned method, a programmatic and objective analysis is performed based on the video of the region of interest to obtain a category of the video that can characterize the blood supply status of the region of interest. Through video classification, contactless determination of the blood supply status of the region of interest is achieved, eliminating subjective factors / contact interference and improving the accuracy and reliability of the classification results. Furthermore, by determining the pulse wave signal based on the video of the region of interest and utilizing the correlation between the pulse wave signal, blood flow velocity, and blood supply status, the pulse wave signal can accurately reflect the blood supply status of the region of interest. Therefore, the category of the video characterizing the blood supply status is determined based on the pulse wave signal, simultaneously improving the accuracy of the classification results. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 is a diagram of the internal structure of a computer device in one embodiment;
[0033] Figure 2 1 is a flow chart of a video classification method according to an embodiment;
[0034] Figure 3 FIG1 is a schematic diagram of a flow chart for determining a reference blood flow velocity in one embodiment;
[0035] Figure 4 FIG1 is a schematic diagram of a process for determining instantaneous blood flow velocity in one embodiment;
[0036] Figure 5 FIG1 is a schematic diagram of a process for determining an instantaneous phase in one embodiment;
[0037] Figure 6 FIG1 is a schematic diagram of a process for determining instantaneous blood flow velocity in another embodiment;
[0038] Figure 7 is a flow chart of a video classification method according to another embodiment;
[0039] Figure 8 FIG. 4 is a structural block diagram of a video classification device in one embodiment. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0041] The video classification method provided in the embodiment of the present application can be applied to Figure 1 The computer device shown may be a terminal. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and computer program stored in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication, which may be achieved via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a video classification method. The display unit of the computer device is used to produce a visually visible image and may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.
[0042] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
[0043] In an exemplary embodiment, Figure 2 As shown, a video classification method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate the process, including the following steps:
[0044] S210: Obtain a video of the region of interest to be inspected.
[0045] The region of interest (ROI) is a region of interest where the blood supply status is concerned. For example, the ROI can be a wound area, or an affected area after a repair surgery such as finger or limb replantation or skin flap transplantation.
[0046] Optionally, the computer device can track the area of interest through a camera to collect the video of interest to be inspected, or it can capture some video clips from the long video of the collected area of interest as the video to be inspected, or it can divide the long video into multiple short videos and use each short video as the video to be inspected.
[0047] For example, the computer device can use a camera to capture a candidate video containing the ROI for a certain length of time at a preset frame rate, identify and intercept the ROI in each video frame in the candidate video, and obtain a video to be inspected for the ROI. For example, if the camera captures a 30-second candidate video at a preset frame rate of 30 frames per second, a video to be inspected for the ROI containing 900 frames can be obtained.
[0048] S220: Determine a pulse wave signal based on the video to be inspected.
[0049] The pulse wave signal may be reflected based on the color change of the ROI in the video frame to be inspected.
[0050] It should be noted that the aforementioned region of interest typically includes human skin. Oxyhemoglobin in the capillaries of the skin's surface has a certain ability to absorb light. The periodic beating of the heart causes fluctuations in the oxyhemoglobin content in these vessels, resulting in variations in the amount of light absorbed and reflected by the blood. The human skin's surface contains numerous capillaries, resulting in minute, periodic color variations on the skin's surface in color space. The video under examination records ROI color changes reflecting blood supply status. These ROI color changes can be used to determine heart rate, which in turn reflects the pulse wave. Therefore, the pulse wave is correlated with blood supply status.
[0051] Optionally, the computer device may perform color change analysis on the video to be inspected to obtain a pulse wave signal.
[0052] Exemplarily, the computer device can obtain the red channel mean of each video frame in the video to be tested, determine the pulse wave intensity of each video frame based on the correspondence between the preset red channel mean and the pulse wave intensity, and sort the video frames according to the time sequence to obtain the pulse wave signal.
[0053] Continuing with the above example, the video to be inspected of the ROI v(t) = [v1, v2, …vN], vN represents the video frame, N represents the time series (for the 30-second candidate video in the above example, N = 900), and the computer device can accordingly obtain the pulse wave signal y(t) = [y1, y2, …yN] corresponding to v(t), where yN represents the pulse wave intensity.
[0054] S230 , determining a reference blood flow velocity of the video to be inspected based on the pulse wave signal, and determining a category of the video to be inspected based on the reference blood flow velocity; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
[0055] The blood supply status is used to characterize the degree of vascular obstruction in the region of interest. For example, the blood supply status can be either blocked or unblocked. Blockage can also be classified into mild, moderate, or severe obstruction. The blood supply status is directly related to blood flow velocity. For example, the lower the blood flow velocity, the more likely the blood supply status is to be blocked; the higher the blood flow velocity, the more likely the blood supply status is to be unblocked.
[0056] It should be noted that the reference blood flow velocity of the video to be tested is used to represent the blood flow movement velocity presented by the video to be tested.
[0057] Optionally, after obtaining the reference blood flow velocity, the computer device may determine the blood supply status of the region of interest based on the reference blood flow velocity, and determine the category of the video to be inspected based on the blood supply status of the region of interest.
[0058] Exemplarily, the computer device may compare the obtained reference blood flow velocity with the speed ranges of different preset blood supply states, determine the speed range to which the reference blood flow velocity belongs, and use the preset blood supply state corresponding to the speed range as the category to which the video to be inspected belongs.
[0059] Optionally, after obtaining the pulse wave signal of the video to be inspected, the computer device may determine a reference blood flow velocity for the video to be inspected based on changes in the pulse wave signal. The reference blood flow velocity may then be used to determine the category of the video to be inspected and serve as the blood supply status of the region of interest. The computer device may determine the reference blood flow velocity for the video to be inspected by matching the signal strength range, or may directly input the pulse wave signal of the video to be inspected into a pre-trained velocity classification model and use the classified velocity as the reference blood flow velocity for the video to be inspected.
[0060] In an embodiment of the present application, a video of a region of interest is obtained to determine a pulse wave signal based on the video, and then a reference blood flow velocity of the video is determined based on the pulse wave signal. The category of the video is then determined based on the reference blood flow velocity. The category of the video is used to characterize the blood supply status of the region of interest. In the above method, a programmatic and objective analysis is performed based on the video of the region of interest to obtain a category of the video that characterizes the blood supply status of the region of interest. This video classification method enables contactless determination of the blood supply status of the region of interest, eliminates subjective factors and contact interference, and improves the accuracy and reliability of the classification results. Furthermore, by determining the pulse wave signal based on the video of the region of interest and utilizing the correlation between the pulse wave signal, blood flow velocity, and blood supply status, the pulse wave signal accurately reflects the blood supply status of the region of interest. Therefore, the category of the video characterizing the blood supply status is determined based on the pulse wave signal, thereby simultaneously improving the accuracy of the classification results.
[0061] To improve the accuracy of the obtained pulse wave signal, in one embodiment, the above S220, determining the pulse wave signal based on the video to be detected, includes:
[0062] The video to be inspected is input into the spatiotemporal attention model, and the temporal and spatial features of the video to be inspected are extracted through the spatiotemporal attention model. The temporal and spatial features are then fused to obtain the pulse wave signal.
[0063] The spatiotemporal attention model is a model that combines both a temporal attention mechanism and a spatial attention mechanism. For example, it can be a 3D Convolutional Neural Network (3D CNN) model or a time-series transformer model.
[0064] Optionally, the computer device can input the video to be inspected into its own 3D CNN model and use the temporal attention mechanism to dynamically adjust the attention of video frames at different time points in the video to be inspected, enhancing the feature extraction capability of video frames at key time points to extract the color features, or temporal features, of the video to be inspected that change over time. Furthermore, the computer device can use the spatial attention mechanism to focus on different spatial regions in the video frames to enhance the feature extraction capability of key spatial regions to extract the spatially varying color features, or spatial features, of the video to be inspected, thereby achieving spatiotemporal feature extraction of the video to be inspected. The extracted temporal and spatial features are then fused to obtain the pulse wave signal of the video to be inspected.
[0065] For example, the 3D CNN model may be trained using mean squared error (MSE) as a loss function and pulse wave signals collected by a contact device (such as a photoplethysmography scanner) as a gold standard.
[0066]
[0067] Where N represents the time series, is the i-th value in the time series output by the model (i.e., the pulse wave signal), is the i-th value in the gold standard pulse wave signal.
[0068] In an embodiment of the present application, the video to be inspected is input into the spatiotemporal attention model, the temporal features and spatial features of the video to be inspected are extracted through the spatiotemporal attention model, and the temporal features and spatial features are fused to obtain a pulse wave signal; in the above method, the spatiotemporal attention model is used to extract and fuse the temporal features and spatial features of the video to be inspected, so as to fully utilize the temporal features and spatial features in the video to be inspected to obtain a pulse wave signal, thereby improving the accuracy and reliability of the obtained pulse wave signal, and being more suitable for determining the blood supply status of limbs including small blood vessels.
[0069] The blood supply status of the region of interest can be determined based on the reference blood flow velocity of the video to be inspected, and the reference blood flow velocity of the video to be inspected can be obtained based on the instantaneous blood flow velocity of each video frame in the video to be inspected. Based on this, in one embodiment, Figure 3 As shown, the above S230 determines the reference blood flow velocity of the video to be inspected according to the pulse wave signal, including:
[0070] S310: Determine the instantaneous blood flow velocity of each video frame in the video to be inspected according to the pulse wave signal.
[0071] Optionally, the computer device may obtain phase information of the pulse wave signal to determine the instantaneous blood flow velocity of each video frame in the video to be inspected based on the relationship between the phase information of the pulse wave signal and the instantaneous blood flow velocity.
[0072] S320: Determine a reference blood flow velocity of the video to be inspected according to the instantaneous blood flow velocity of each video frame.
[0073] Optionally, after obtaining the instantaneous blood flow velocity of each video frame in the video to be inspected, the computer device may process the instantaneous blood flow velocity of each video frame to obtain a reference blood flow velocity of the video frame to be inspected.
[0074] For example, the computer device may obtain the average value of all instantaneous blood flow velocities as the reference blood flow velocity for the video to be tested. Before obtaining the average value, the computer device may filter the instantaneous blood flow velocities of each video frame to remove points of velocity abrupt changes. Alternatively, the computer device may concatenate all instantaneous blood flow velocities in chronological order to form a velocity sequence, then input this velocity sequence into a pre-trained network model. The network model then extracts features from the input velocity sequence and uses the output result as the reference blood flow velocity for the video frame to be tested.
[0075] In an optional embodiment, if Figure 4 As shown, the above S310, determining the instantaneous blood flow velocity of each video frame in the video to be inspected according to the pulse wave signal, includes:
[0076] S410: Determine the instantaneous phase of each pixel in each video frame according to the pulse wave signal.
[0077] Optionally, for each video frame in the video to be inspected, the computer device may calculate the instantaneous phase of each pixel in the corresponding video frame using Hilbert transform based on the pulse wave signal.
[0078] S420: Determine the instantaneous blood flow velocity of each video frame according to the instantaneous phase of each pixel point in each video frame.
[0079] Optionally, for each video frame, after the computer device obtains the instantaneous phase of each pixel point in the video frame, it can determine the instantaneous velocity of each pixel point based on the instantaneous phase of each pixel point in the video frame, and then determine the instantaneous blood flow velocity of the video frame according to the instantaneous velocity of each pixel point in the video frame.
[0080] It should be noted that instantaneous blood flow velocity can more accurately reflect the blood supply status, thereby improving classification accuracy, and is also more suitable for determining the blood supply status of limbs including small blood vessels.
[0081] In an embodiment of the present application, the instantaneous blood flow velocity of each video frame in the video to be inspected is determined based on the pulse wave signal, and the reference blood flow velocity of the video to be inspected is determined based on the instantaneous blood flow velocity of each frequency frame; specifically, the instantaneous phase of each pixel point in each frequency frame is determined based on the pulse wave signal, so as to determine the instantaneous blood flow velocity of each video frame based on the instantaneous phase of each pixel point in each video frame; in the above method, the blood flow velocity is determined based on the correlation between the pulse wave signal and the blood flow velocity, so as to determine the category of the video to be inspected that reflects the blood supply status based on the blood flow velocity. The blood flow velocity can accurately reflect the blood supply status, thereby correspondingly improving the accuracy of the category to which the obtained video to be inspected belongs.
[0082] In order to obtain the instantaneous phase of a pixel, in one embodiment, as Figure 5As shown, the instantaneous phase of each pixel in each video frame is determined according to the pulse wave signal in S410, including:
[0083] S510: Perform Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal.
[0084] Among them, the pulse wave signal is a real-valued signal.
[0085] Optionally, the computer device may perform Hilbert transform on the pulse wave signal of the real-valued signal to obtain an analytical signal of the pulse wave signal.
[0086] For example, the pulse wave signal and analytical signals The relationship is as follows:
[0087]
[0088] Where j represents the imaginary unit, yes Hilt transform of .
[0089] S520: Determine the instantaneous phase of each pixel in the video frame according to the analysis signal.
[0090] Optionally, after obtaining the analysis signal of the pulse wave signal, the instantaneous phase of each pixel in the video frame can be obtained based on the relationship between the analysis signal and the instantaneous phase.
[0091] For example, the parsing signal With instantaneous phase The relationship is as follows:
[0092]
[0093] In an embodiment of the present application, a Hilbert transform is performed on the pulse wave signal to obtain an analytical signal of the pulse wave signal, thereby determining the instantaneous phase of each pixel point in the video frame based on the analytical signal; in the above method, the Hilbert transform is used to analyze and process the pulse wave signal to obtain the instantaneous phase of each pixel point, thereby improving the convenience and accuracy of determining the instantaneous phase.
[0094] The instantaneous phase change can be used to determine the instantaneous blood flow velocity. Based on this, in one embodiment, Figure 6 As shown, the above S420, determining the instantaneous blood flow velocity of each video frame according to the instantaneous phase of each pixel point in each video frame, includes:
[0095] S610: Obtain the instantaneous phase difference between corresponding pixels of two temporally adjacent video frames.
[0096] Optionally, after obtaining the instantaneous phase of each pixel point, the computer device can traverse two temporally adjacent video frames in the video to be inspected, and obtain the difference between the instantaneous phases of corresponding pixel points in the two video frames, that is, the phase change at any time.
[0097] For example, for the corresponding pixel points between two temporally adjacent video frames , the computer device can use the instantaneous phase of the pixel in the subsequent video frame Subtract the instantaneous phase of the pixel in the previous video frame , to obtain the instantaneous phase difference .
[0098]
[0099] in, Indicates the time interval between two temporally adjacent video frames.
[0100] It should be noted that the computer device may also use other algorithms to determine the instantaneous phase change, such as optical flow methods or deep learning-based tracking algorithms. Optical flow methods include but are not limited to the Lucas-Kanade optical flow method, the Kanade-Lucas-Tomasi tracking algorithm, or the Horn-Schunck algorithm.
[0101] S620: Determine the instantaneous blood flow velocity of a target video frame according to the instantaneous phase difference and the time interval between two video frames; the target video frame is the earlier video frame in the temporal sequence of the two temporally adjacent video frames.
[0102] Optionally, the target video frame includes multiple pixel points. After obtaining the instantaneous phase difference corresponding to each pixel point, the ratio of the instantaneous phase difference of each pixel point to the time interval between two video frames can be obtained as the instantaneous velocity of the corresponding pixel point, and then the instantaneous blood flow velocity of the target video frame is determined according to the instantaneous velocity of each pixel point.
[0103] For example, the computer device can obtain the instantaneous speed of each pixel in the target video frame. The average value of the target video frame is used as the instantaneous blood flow velocity .
[0104]
[0105] in, Represents pixel points, N represents the number of pixels, and t represents the time.
[0106] In an embodiment of the present application, the instantaneous phase difference between corresponding pixel points between two temporally adjacent video frames is obtained to determine the instantaneous blood flow velocity of the target video frame based on the instantaneous phase difference and the time interval between the two video frames; the target video frame is the video frame that comes earlier in the temporal sequence of the two temporally adjacent video frames; in the above method, the instantaneous blood flow velocity is determined by utilizing the correlation between the instantaneous phase difference and the instantaneous blood flow velocity, thereby improving the accuracy of the obtained instantaneous blood flow velocity.
[0107] To facilitate understanding by those skilled in the art, the video classification method provided by this application is described in detail below. Figure 7 As shown, the method may include:
[0108] S701, obtaining a video of the area of interest to be inspected;
[0109] S702: Input the video to be inspected into the spatiotemporal attention model, extract the temporal and spatial features of the video to be inspected through the spatiotemporal attention model, and fuse the temporal and spatial features to obtain a pulse wave signal;
[0110] S703, performing Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal;
[0111] S704, determining the instantaneous phase of each pixel in the video frame according to the analysis signal;
[0112] S705, obtaining the instantaneous phase difference between corresponding pixels of two temporally adjacent video frames;
[0113] S706: Determine the instantaneous blood flow velocity of a target video frame based on the instantaneous phase difference and the time interval between the two video frames; the target video frame is the earlier video frame in the temporal sequence of the two temporally adjacent video frames;
[0114] S707, determining a reference blood flow velocity of the video to be inspected based on the instantaneous blood flow velocity of each video frame;
[0115] S708. Determine the category of the video to be inspected according to the reference blood flow velocity; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
[0116] It should be noted that for the descriptions in S701-S708 above, reference may be made to the relevant descriptions in the above embodiments, and the effects are similar, so this embodiment will not be repeated here.
[0117] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.
[0118] Based on the same inventive concept, embodiments of the present application also provide a video classification device for implementing the aforementioned video classification method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more video classification device embodiments provided below can be found in the above-mentioned limitations of the video classification method and will not be repeated here.
[0119] In one embodiment, Figure 8 As shown, a video classification device is provided, comprising: a video acquisition module 801, a signal determination module 802 and a category determination module 803, wherein:
[0120] The video acquisition module 801 is used to acquire the video to be inspected of the area of interest;
[0121] The signal determination module 802 is used to determine the pulse wave signal according to the video to be detected;
[0122] The category determination module 803 is used to determine the reference blood flow velocity of the video to be inspected based on the pulse wave signal, and determine the category of the video to be inspected based on the reference blood flow velocity; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
[0123] In one embodiment, the signal determination module 802 includes:
[0124] The video processing submodule is used to input the video to be inspected into the spatiotemporal attention model, extract the time domain features and spatial domain features of the video to be inspected through the spatiotemporal attention model, and fuse the time domain features and spatial domain features to obtain the pulse wave signal.
[0125] In one embodiment, the category determination module 803 includes:
[0126] The instantaneous velocity submodule is used to determine the instantaneous blood flow velocity of each video frame in the video to be inspected based on the pulse wave signal;
[0127] The reference velocity submodule is used to determine the reference blood flow velocity of the video to be inspected based on the instantaneous blood flow velocity of each video frame.
[0128] In one embodiment, the instantaneous speed submodule includes:
[0129] A phase unit, used to determine the instantaneous phase of each pixel in each video frame based on the pulse wave signal;
[0130] The velocity unit is used to determine the instantaneous blood flow velocity of each video frame according to the instantaneous phase of each pixel point in each video frame.
[0131] In one embodiment, the phase unit includes:
[0132] a signal conversion subunit, configured to perform Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal;
[0133] The phase determination subunit is used to determine the instantaneous phase of each pixel in the video frame according to the analysis signal.
[0134] In one embodiment, the speed unit includes:
[0135] The phase change subunit is used to obtain the instantaneous phase difference between corresponding pixels of two temporally adjacent video frames;
[0136] The velocity determination subunit is used to determine the instantaneous blood flow velocity of the target video frame according to the instantaneous phase difference and the time interval between two video frames; the target video frame is the video frame that is earlier in the temporal sequence of the two temporally adjacent video frames.
[0137] Each module in the above-mentioned video classification device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.
[0138] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0139] Obtain a video of the region of interest; determine a pulse wave signal based on the video; determine a reference blood flow velocity of the video based on the pulse wave signal, and determine the category of the video based on the reference blood flow velocity; the category of the video is used to characterize the blood supply status of the region of interest.
[0140] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0141] The video to be inspected is input into the spatiotemporal attention model, and the temporal and spatial features of the video to be inspected are extracted through the spatiotemporal attention model. The temporal and spatial features are then fused to obtain the pulse wave signal.
[0142] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0143] The instantaneous blood flow velocity of each video frame in the video to be inspected is determined according to the pulse wave signal; and the reference blood flow velocity of the video to be inspected is determined according to the instantaneous blood flow velocity of each video frame.
[0144] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0145] The instantaneous phase of each pixel point in each video frame is determined according to the pulse wave signal; and the instantaneous blood flow velocity of each video frame is determined according to the instantaneous phase of each pixel point in each video frame.
[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0147] Performing Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal; determining the instantaneous phase of each pixel in the video frame according to the analytical signal.
[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0149] The instantaneous phase difference of corresponding pixel points between two temporally adjacent video frames is obtained; the instantaneous blood flow velocity of a target video frame is determined based on the instantaneous phase difference and the time interval between the two video frames; the target video frame is the video frame that comes earlier in the temporal sequence of the two temporally adjacent video frames.
[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:
[0151] Obtain a video of the region of interest; determine a pulse wave signal based on the video; determine a reference blood flow velocity of the video based on the pulse wave signal, and determine the category of the video based on the reference blood flow velocity; the category of the video is used to characterize the blood supply status of the region of interest.
[0152] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0153] The video to be inspected is input into the spatiotemporal attention model, and the temporal and spatial features of the video to be inspected are extracted through the spatiotemporal attention model. The temporal and spatial features are then fused to obtain the pulse wave signal.
[0154] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0155] The instantaneous blood flow velocity of each video frame in the video to be inspected is determined according to the pulse wave signal; and the reference blood flow velocity of the video to be inspected is determined according to the instantaneous blood flow velocity of each video frame.
[0156] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0157] The instantaneous phase of each pixel point in each video frame is determined according to the pulse wave signal; and the instantaneous blood flow velocity of each video frame is determined according to the instantaneous phase of each pixel point in each video frame.
[0158] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0159] Performing Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal; determining the instantaneous phase of each pixel in the video frame according to the analytical signal.
[0160] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0161] The instantaneous phase difference of corresponding pixel points between two temporally adjacent video frames is obtained; the instantaneous blood flow velocity of a target video frame is determined based on the instantaneous phase difference and the time interval between the two video frames; the target video frame is the video frame that comes earlier in the temporal sequence of the two temporally adjacent video frames.
[0162] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0163] Obtain a video of the region of interest; determine a pulse wave signal based on the video; determine a reference blood flow velocity of the video based on the pulse wave signal, and determine the category of the video based on the reference blood flow velocity; the category of the video is used to characterize the blood supply status of the region of interest.
[0164] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0165] The video to be inspected is input into the spatiotemporal attention model, and the temporal and spatial features of the video to be inspected are extracted through the spatiotemporal attention model. The temporal and spatial features are then fused to obtain the pulse wave signal.
[0166] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0167] The instantaneous blood flow velocity of each video frame in the video to be inspected is determined according to the pulse wave signal; and the reference blood flow velocity of the video to be inspected is determined according to the instantaneous blood flow velocity of each video frame.
[0168] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0169] The instantaneous phase of each pixel point in each video frame is determined according to the pulse wave signal; and the instantaneous blood flow velocity of each video frame is determined according to the instantaneous phase of each pixel point in each video frame.
[0170] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0171] Performing Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal; determining the instantaneous phase of each pixel in the video frame according to the analytical signal.
[0172] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:
[0173] The instantaneous phase difference of corresponding pixel points between two temporally adjacent video frames is obtained; the instantaneous blood flow velocity of a target video frame is determined based on the instantaneous phase difference and the time interval between the two video frames; the target video frame is the video frame that comes earlier in the temporal sequence of the two temporally adjacent video frames.
[0174] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.
[0175] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0176] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. A video classification method, characterized in that: The method comprises: Get the video to be inspected of the area of interest; determining a pulse wave signal according to the video to be inspected; A reference blood flow velocity of the video to be inspected is determined according to the pulse wave signal, and a category of the video to be inspected is determined according to the reference blood flow velocity; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
2. The method according to claim 1, characterized in that The step of determining the pulse wave signal according to the video to be detected includes: The video to be inspected is input into a spatiotemporal attention model, the temporal features and spatial features of the video to be inspected are extracted through the spatiotemporal attention model, and the temporal features and the spatial features are fused to obtain the pulse wave signal.
3. The method according to claim 1 or 2, characterized in that Determining the reference blood flow velocity of the video to be inspected according to the pulse wave signal includes: determining the instantaneous blood flow velocity of each video frame in the video to be inspected according to the pulse wave signal; A reference blood flow velocity of the video to be inspected is determined according to the instantaneous blood flow velocity of each video frame.
4. The method according to claim 3, characterized in that Determining the instantaneous blood flow velocity of each video frame in the video to be inspected according to the pulse wave signal includes: determining the instantaneous phase of each pixel in each of the video frames according to the pulse wave signal; The instantaneous blood flow velocity of each video frame is determined according to the instantaneous phase of each pixel point in each video frame.
5. The method according to claim 4, characterized in that Determining the instantaneous phase of each pixel in each video frame according to the pulse wave signal includes: performing a Hilbert transform on the pulse wave signal to obtain an analytical signal of the pulse wave signal; The instantaneous phase of each pixel in the video frame is determined according to the analysis signal.
6. The method according to claim 4, characterized in that Determining the instantaneous blood flow velocity of each video frame according to the instantaneous phase of each pixel point in each video frame includes: Obtain the instantaneous phase difference between corresponding pixels of two temporally adjacent video frames; The instantaneous blood flow velocity of a target video frame is determined according to the instantaneous phase difference and the time interval between the two video frames; the target video frame is the earlier video frame in the temporal sequence of the two temporally adjacent video frames.
7. A video classification device, characterized in that: The device comprises: A video acquisition module is used to acquire the video to be inspected of the area of interest; a signal determination module, configured to determine a pulse wave signal based on the video to be detected; The category determination module is used to determine the reference blood flow velocity of the video to be inspected based on the pulse wave signal, and to determine the category of the video to be inspected based on the reference blood flow velocity; the category of the video to be inspected is used to characterize the blood supply status of the region of interest.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.