A variable gravity particle experiment process real-time detection method based on historical information

Abstract

The application provides a variable gravity particle experiment process real-time detection method based on historical information, and relates to the technical field of video data processing.The method provided by the application determines a video feature vector corresponding to a target video segment, then determines a memory feature vector according to historical feature information of a plurality of video segments before the target video segment in continuous video segments, and then determines a process category corresponding to the target video segment, and a starting time and an ending time corresponding to each process category according to the video feature vector corresponding to the target video segment and the memory feature vector.The method can accurately detect and identify different stage processes of a spatial variable gravity particle material experiment in real time during the spatial variable gravity particle material experiment, and improves data processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of video data processing technology, and in particular to a real-time detection method for a variable gravity particle experiment based on historical information. Background Technology

[0002] Variable gravity particle experiments are a crucial method for studying the behavior of particulate materials under different gravity environments. They hold significant scientific and engineering value for a deeper understanding of the fundamental physical mechanisms of particulate media, optimizing material design, and advancing aerospace engineering applications. Under Earth's gravity, the flow, packing, and compression properties of particulate materials are dominated by gravity. However, under microgravity or variable gravity environments, their internal forces, interparticle interactions, and overall evolution patterns may change significantly. This not only relates to propellant management in spacecraft propulsion systems and material control in space manufacturing, but also involves core technological issues such as in-situ resource utilization (ISRU) in deep space exploration missions to the Moon and Mars. Therefore, studying the evolution of particulate materials under variable gravity environments has a profound impact on advancing fundamental physics research, enhancing technological reserves for future deep space exploration missions, and even promoting related industrial applications.

[0003] Because variable gravity particle experiments typically involve complex procedures, long durations, and multi-stage evolution, efficiently and accurately identifying experimental stages and extracting key experimental information has become a core challenge for experimental data analysis and control. Current data processing methods often rely on offline analysis, making it difficult to obtain key state changes during the experiment in a timely manner, leading to delays in experimental adjustments and impacting experimental efficiency and data quality.

[0004] Therefore, there is an urgent need for a real-time monitoring method for variable gravity particle experiments based on historical information. This method would fully utilize historical data during the experiment, accurately identify the state at each stage, improve data processing efficiency, and enable dynamic optimization and control of experimental conditions, ensuring the stability and scientific value of the experiment. The implementation of this method will not only significantly improve the data utilization efficiency of variable gravity particle experiments but also provide more intelligent and automated experimental monitoring and control tools for future space science experiments. Summary of the Invention

[0005] This invention provides a real-time detection method for variable gravity particle experiments based on historical information. This method can accurately detect and identify different stages of the variable gravity particle material experiment in real time, thereby improving data processing efficiency.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0007] Firstly, a method for real-time detection of variable gravity particle experiments based on historical information is provided. The method includes: acquiring a target video segment of the variable gravity particle experiment, the target video segment comprising multiple consecutive frames of images, the target video segment being one of multiple consecutive video segments of the variable gravity particle experiment; determining a first feature vector corresponding to the target video segment; performing position encoding on the first feature vector and the first historical feature information of k video segments arranged sequentially before the target video segment in the historical feature queue, to obtain a second feature vector corresponding to the first feature vector, and second historical feature information corresponding to the first historical feature information of each of the k video segments; and based on the second historical feature information of each of the k video segments... The memory feature vector is determined by the feature information, which includes short-term historical feature information and long-term historical feature information. The short-term historical feature information is determined based on the second historical feature information of the n video segments adjacent to the target video segment in the historical feature queue. The long-term historical feature information is determined based on the second historical feature information of the m video segments other than the n video segments adjacent to the target video segment in the historical feature queue. k, n, and m are positive integers, and m+n=k. The experimental process detection result of the target video segment is determined by the trained real-time process detection unit based on the second feature vector and the memory feature vector of the target video segment. The experimental process detection result includes at least one process category of variable gravity particle experiment, as well as the start time and end time corresponding to each process category.

[0008] In one possible implementation of the first aspect, the method further includes: performing a projection operation on the experimental process detection results of the target video segment based on a multilayer perceptron to obtain a projection feature vector corresponding to the target video segment; determining first historical feature information corresponding to the target video segment based on the projection feature vector and the first feature vector; and updating the historical feature queue based on the first historical feature information corresponding to the target video segment.

[0009] Among them, the projection feature vector corresponding to the target video segment The formula for determining it is:

[0010] ;

[0011] ;

[0012] The experimental process detection results for the target video segment. This is the first feature vector corresponding to the target video segment. This refers to the first historical feature information corresponding to the target video segment.

[0013] In one possible implementation of the first aspect, updating the historical feature queue based on the first historical feature information corresponding to the target video segment includes: when the target video segment is the t-th video segment in a series of consecutive video segments of the variable gravity particle experiment, and t is less than or equal to k, adding the first historical feature information corresponding to the target video segment to the historical feature queue; when the target video segment is the t-th video segment in a series of consecutive video segments of the variable gravity particle experiment, and t is greater than k, deleting the first historical feature information corresponding to the video segment ranked 1st in the historical feature queue, and adding the first historical feature information corresponding to the target video segment to the historical feature queue.

[0014] In one possible implementation of the first aspect, determining the memory feature vector based on the second historical feature information of each of the k video segments includes: concatenating the second historical feature information of the n video segments adjacent to the target video segment to obtain short-term historical feature information; dividing the second historical feature information of the m video segments in the historical feature queue (excluding the n video segments adjacent to the target video segment) into x information groups, each information group including the second historical feature information of y video segments, where x and y are positive integers and x*y=m; compressing and aggregating the second historical feature information of the y video segments included in each information group based on a compression aggregation network to obtain compressed aggregation features corresponding to each information group, wherein the compression aggregation network is a neural network structure based on Taylor expansion and the Kolmogorov-Arnold representation theorem; concatenating the compressed aggregation features corresponding to each information group to obtain long-term historical feature information; and concatenating the short-term historical feature information and the long-term historical feature information to obtain the memory feature vector.

[0015] In one possible implementation of the first aspect, the compression aggregation network is used to: perform feature transformation on the second historical feature information of each information group, which includes y video segments, based on Taylor polynomials, to obtain the transformed feature corresponding to each second historical feature information; and perform aggregation processing on the transformed features corresponding to the second historical feature information included in each information group to obtain the compressed aggregation feature corresponding to each information group.

[0016] In one possible implementation of the first aspect, the trained real-time process detection unit includes multiple cascaded Transformer Decoder units, which are used to output processing results based on the input query vector, key vector, and value vector.

[0017] Experimental detection results of the target video segment The formula for determining it is:

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] in, For activation functions; The processing result output by the Transformer Decoder unit; The weights for projecting the processing results output by the Transformer Decoder unit into the class space, where c is the number of process classes; Here, K is the query vector, V is the key vector, and V is the value vector. This is the second feature vector of the target video segment; This is the memory feature vector of the target video segment.

[0023] In one possible implementation of the first aspect, before determining the experimental process detection result of the target video segment through the trained real-time process detection unit based on the first feature vector and the memory feature vector of the target video segment, the method further includes: acquiring a training sample set of the variable gravity particle experiment, the training sample set including multiple training samples, each training sample including a training video segment and the corresponding experimental process detection result of the training video segment; constructing a target loss function; and iteratively training the real-time process detection unit based on the target loss function to obtain the trained real-time process detection unit.

[0024] In one possible implementation of the first aspect, the target loss function for:

[0025]

[0026] ;

[0027] ;

[0028] ;

[0029] in, The balance factor for category i in each batch. It is the predicted probability that the number of video frames corresponding to time t corresponds to process category i; yes Time belongs to the process category The true label; As a balance factor; It is the total number of process instances in each batch. The mean statistic is based on logarithmic transformation. It is the process category in each batch. The number of process instances.

[0030] The beneficial effects of this invention are as follows: The method provided by this invention determines the video feature vector corresponding to the target video segment, then determines the memory feature vector based on the historical feature information of multiple video segments preceding the target video segment in a series of consecutive video segments, and then determines the process category corresponding to the target video segment based on the video feature vector and the memory feature vector. This enables accurate detection and identification of different stages of the space variable gravity particle material experiment in real time, improving data processing efficiency. Furthermore, since the memory feature vector includes both short-term and long-term historical feature information, it can accurately detect both short-duration and long-duration processes in the space variable gravity particle material experiment. Moreover, the method provided by this invention, based on a neural network structure using Taylor expansion and the Kolmogorov-Arnold representation theorem, determines long-term historical feature information, effectively reducing computational load while preserving historical feature information, thereby saving computational resources and reducing computational costs. Finally, the method provided by this invention, through the design of a target loss function, dynamically adjusts the balance factor for short-duration process categories during the experiment, allowing the real-time process detection unit to focus on short-duration process categories during training, effectively improving the accuracy of process detection.

[0031] Secondly, the present invention provides a real-time detection system for a variable gravity particle experiment based on historical information. The system includes: a video acquisition unit for acquiring a target video segment of the variable gravity particle experiment, the target video segment comprising multiple consecutive frames of images, the target video segment being one of multiple consecutive video segments of the variable gravity particle experiment; a feature extraction unit for determining a first feature vector corresponding to the target video segment; a position encoding unit for performing position encoding operations on the first feature vector and the first historical feature information of k video segments arranged sequentially before the target video segment in a historical feature queue, to obtain a second feature vector corresponding to the first feature vector, and second historical feature information corresponding to the first historical feature information of each of the k video segments; and a vector determination unit for determining the vector based on the k video segments. The second historical feature information of each video segment in the frequency segment determines the memory feature vector, which includes short-term historical feature information and long-term historical feature information. The short-term historical feature information is determined based on the second historical feature information of the n video segments adjacent to the target video segment in the historical feature queue, and the long-term historical feature information is determined based on the second historical feature information of the m video segments other than the n video segments adjacent to the target video segment in the historical feature queue, where k, n, and m are positive integers and m+n=k. The process detection unit is used to determine the experimental process detection result of the target video segment through the trained real-time process detection unit based on the second feature vector and memory feature vector of the target video segment. The experimental process detection result includes at least one process category of variable gravity particle experiment, as well as the start time and end time corresponding to each process category.

[0032] Thirdly, an electronic device is provided, the electronic device including a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any implementation of the first aspect.

[0033] Fourthly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform a method as described in any implementation of the first aspect.

[0034] Fifthly, a computer program product is provided that, when run on a computer, causes the computer to perform the method in any implementation of the first aspect.

[0035] Understandably, the beneficial effects achieved by the system of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to with reference to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0036] Figure 1 A schematic diagram of a real-time detection method for a variable gravity particle experiment based on historical information provided in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0038] Figure 3 A flowchart of a real-time detection method for a variable gravity particle experiment based on historical information provided in an embodiment of the present invention;

[0039] Figure 4 A flowchart of another real-time detection method for variable gravity particle experiments based on historical information provided in this embodiment of the invention;

[0040] Figure 5 A flowchart of another real-time detection method for variable gravity particle experiments based on historical information provided in an embodiment of the present invention;

[0041] Figure 6 A flowchart of another real-time detection method for variable gravity particle experiments based on historical information provided in an embodiment of the present invention;

[0042] Figure 7 A schematic diagram illustrating the process categories and durations of a spatial variable gravity particle material experiment for verifying a dataset, provided in an embodiment of the present invention;

[0043] Figure 8 This is a schematic diagram of a detection system provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the present invention, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. The "or" in the present invention is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A or B can represent: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Furthermore, in the description of the present invention, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items.

[0045] Furthermore, to facilitate a clear description of the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0046] In this embodiment of the invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this embodiment of the invention should not be construed as superior or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0047] Variable gravity particle experiments are a crucial method for studying the behavior of particulate materials under different gravity environments. They hold significant scientific and engineering value for a deeper understanding of the fundamental physical mechanisms of particulate media, optimizing material design, and advancing aerospace engineering applications. Under Earth's gravity, the flow, packing, and compression properties of particulate materials are dominated by gravity. However, under microgravity or variable gravity environments, their internal forces, interparticle interactions, and overall evolution patterns may change significantly. This not only relates to propellant management in spacecraft propulsion systems and material control in space manufacturing, but also involves core technological issues such as in-situ resource utilization (ISRU) in deep space exploration missions to the Moon and Mars. Therefore, studying the evolution of particulate materials under variable gravity environments has a profound impact on advancing fundamental physics research, enhancing technological reserves for future deep space exploration missions, and even promoting related industrial applications.

[0048] Because variable gravity particle experiments typically involve complex procedures, long durations, and multi-stage evolution, efficiently and accurately identifying experimental stages and extracting key experimental information has become a core challenge for experimental data analysis and control. Current data processing methods often rely on offline analysis, making it difficult to obtain key state changes during the experiment in a timely manner, leading to delays in experimental adjustments and impacting experimental efficiency and data quality.

[0049] Therefore, there is an urgent need for a real-time monitoring method for variable gravity particle experiments based on historical information. This method would fully utilize historical data during the experiment, accurately identify the state at each stage, improve data processing efficiency, and enable dynamic optimization and control of experimental conditions, ensuring the stability and scientific value of the experiment. The implementation of this method will not only significantly improve the data utilization efficiency of variable gravity particle experiments but also provide more intelligent and automated experimental monitoring and control tools for future space science experiments.

[0050] In view of this, embodiments of the present invention provide a real-time detection method for a variable gravity particle experiment based on historical information. The method includes: acquiring a target video segment of the variable gravity particle experiment, the target video segment comprising multiple consecutive frames of images, the target video segment being one of multiple consecutive video segments of the variable gravity particle experiment; determining a first feature vector corresponding to the target video segment; performing position encoding on the first feature vector and the first historical feature information of k video segments arranged sequentially before the target video segment in the historical feature queue, to obtain a second feature vector corresponding to the first feature vector, and second historical feature information corresponding to the first historical feature information of each of the k video segments; and encoding the second historical feature information of each of the k video segments according to the second historical feature information of each of the k video segments. Historical feature information is used to determine the memory feature vector, which includes short-term and long-term historical feature information. The short-term historical feature information is determined based on the second historical feature information of the n video segments adjacent to the target video segment in the historical feature queue. The long-term historical feature information is determined based on the second historical feature information of the m video segments other than the n video segments adjacent to the target video segment in the historical feature queue. k, n, and m are positive integers, and m+n=k. The experimental process detection result of the target video segment is determined by a trained real-time process detection unit based on the second feature vector and the memory feature vector of the target video segment. The experimental process detection result includes at least one process category of variable gravity particle experiment, as well as the start and end times corresponding to each process category.

[0051] The method provided by this invention determines the video feature vector corresponding to the target video segment, then determines the memory feature vector based on the historical feature information of multiple video segments preceding the target video segment in a series of consecutive video segments, and then determines the process category corresponding to the target video segment based on the video feature vector and the memory feature vector. This method can accurately detect and identify different stages of the space variable gravity particle material experiment in real time, thereby improving data processing efficiency.

[0052] In some embodiments, the real-time detection method for variable gravity particle experiment process based on historical information provided by the present invention can be executed by a real-time detection system 100 for variable gravity particle experiment process based on historical information (hereinafter referred to as detection system 100).

[0053] For example, see Figure 1 , Figure 1 This is a schematic diagram of a real-time detection method for a variable gravity particle experiment based on historical information provided in an embodiment of the present invention. First, a target video segment (the t-th segment among multiple consecutive video segments sorted by time series) is obtained from multiple consecutive video segments. Then, a feature extractor determines the first feature vector corresponding to the target video segment. Next, the detection system 100 performs position encoding based on the first historical feature information and the first feature vector of k video segments (the (t-1)th to the tkth video segments) included in the historical feature queue, to obtain the second historical feature information corresponding to each first historical feature information and the second feature vector corresponding to each first feature vector. Then, the detection system constructs short-term historical feature information based on the second historical feature information of n video segments adjacent to the target video segment, and constructs long-term historical feature information based on the second historical feature information of m video segments in the historical feature queue other than the n video segments adjacent to the target video segment. Then, a memory feature vector is constructed through the short-term historical feature information and the long-term historical feature information. Finally, the experimental process detection result of the target video segment is determined by the real-time process detection unit based on the memory feature vector and the second feature vector. Furthermore, the method provided in this embodiment of the invention generates first historical feature information of the target video segment based on the experimental process detection results and the first feature vector of the target video segment, and updates the historical feature queue based on the first historical feature information of the target video segment (equivalent to adding the first historical feature information of the t-th video segment and removing the first historical feature information of the tk-th video segment). The updated historical feature queue includes the first historical feature information of k video segments (the t-th to the t-k+1-th video segments).

[0054] As an example, the detection system 100 can be any electronic device 200 with data processing capabilities, such as a general-purpose computer, personal computer, laptop computer, switch, or tablet computer. The specific implementation of the detection system 100 is not limited here.

[0055] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown. The electronic device 200 includes a processor 210, a memory 220, and a communication interface 230.

[0056] Processor 210 may include one or more processing cores. Processor 210 connects to various parts within electronic device 200 using various interfaces and lines, and performs various functions and processes data of electronic device 200 by running or executing instructions, programs, code sets, or instruction sets stored in memory 220, and by calling data stored in memory 220. Optionally, processor 210 may be implemented using at least one of the following hardware forms: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).

[0057] The memory 220 may include random access memory (RAI) or read-only memory (ROI). Optionally, the memory 220 may include non-transitory computer-readable storage ledger. The memory 220 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a program storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as video acquisition, feature extraction, and process detection), and instructions for implementing the various method embodiments described above.

[0058] Communication interface 230 is used to communicate with other devices, equipment or communication networks, such as data storage devices, image processing devices or Ethernet, wireless access network (RAN), wireless local area network (WLAN), etc.

[0059] In terms of physical implementation, the aforementioned devices (such as processor 210, memory 220, and communication interface 230) can each be devices within the same device (such as a laptop computer). Alternatively, at least two of these devices can be located within the same device, i.e., as different devices within the same device, similar to the deployment of devices or components in a distributed system.

[0060] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 200. In other embodiments of the present invention, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] The following description, in conjunction with the accompanying drawings, illustrates a real-time detection method for variable gravity particle experiments based on historical information, provided by an embodiment of the present invention.

[0062] Figure 3 This is a flowchart illustrating a real-time detection method for a variable gravity particle experiment based on historical information, provided as an embodiment of the present invention. Optionally, this method can be... Figure 1 The detection system 100 shown is... Figure 2 The illustrated electronic device 200 performs this operation. The method may include the following steps:

[0063] S1. Obtain the target video clip of the variable gravity particle experiment.

[0064] Specifically, the target video segment comprises multiple consecutive frames of images, and the target video segment is one of multiple consecutive video segments from the variable gravity particle experiment.

[0065] The variable gravity particle experiment comprises multiple process stages, each corresponding to a different process category. For example, the process categories for each stage include: leftward movement of the baffle before vibration, rightward movement of the baffle before vibration, baffle vibration, baffle cessation of vibration, baffle resetting after vibration, leftward movement of the baffle before collapse, variable gravity before collapse, baffle forward pressure before collapse, and collapse. It should be noted that for the variable gravity particle experiment, these multiple process stages are not necessarily consecutive. A static process stage (also known as the background) may exist between two adjacent process stages. In this invention, the process category corresponding to the static process stage is unclassified.

[0066] It should be understood that the above variable gravity particle experiment is only an illustrative example. The method provided by the embodiments of the present invention can also be used for process detection in other types and fields of experiments, and the embodiments of the present invention do not impose any special limitations on this.

[0067] Specifically, multiple consecutive video segments are video segments arranged in chronological order. Each frame of each video segment has a corresponding timestamp, which is used to characterize the time when each image was acquired.

[0068] In one example, the experimental video data of the variable gravity particle experiment includes 20,000 frames of images. The 20,000 frames of images are divided into 5-frame video segments to obtain 4,000 video segments. The target video segment is the 100th video segment among the 4,000 consecutive video segments arranged in chronological order.

[0069] It should be noted that the number of video segments and the number of image frames in each video segment in the above variable gravity particle experiment are merely illustrative examples. The variable gravity particle experiment may also include more or fewer video segments and each video segment may include more or fewer image frames. This embodiment of the invention does not impose any particular limitations on this.

[0070] S2. Determine the first feature vector corresponding to the target video segment.

[0071] In one possible implementation, the detection system 100 initializes the weights of the Vision Transformer (ViT) based on the weights of the pre-trained model, and then uses the ViT as a feature encoder to extract the first feature vector corresponding to the target video segment.

[0072] Specifically, ViT is a neural network model that applies the Transformer architecture to computer vision tasks. ViT divides each frame of the target video segment into multiple image blocks and uses these multiple image blocks as sequence data. It then extracts image features through the Transformer architecture to obtain the first feature vector corresponding to the target video segment.

[0073] The method provided in this embodiment of the invention determines the first feature vector corresponding to the target video segment by using ViT. Compared with the feature extraction method using a CNN architecture feature extraction network in related technologies, the method provided in this embodiment of the invention can effectively improve the accuracy and stability of feature extraction.

[0074] S3. Perform position encoding on the first feature vector and the first historical feature information of the k video segments that are arranged in order and are located before the target video segment in the multiple video segments, including the first feature vector and the historical feature queue, to obtain the second feature vector corresponding to the first feature vector and the second historical feature information corresponding to the first historical feature information of each of the k video segments.

[0075] Specifically, the historical feature queue includes the first historical feature information of the k video segments preceding the target video segment in a sequence of video segments. That is, the length of the historical feature queue is k. In other words, if the target video segment is the t-th video segment in a sequence of consecutive video segments arranged chronologically, and t is greater than k, the historical feature queue includes the first historical feature information corresponding to the tk-th to t-1-th video segments in a sequence. Referring to the example above, when k=80, and the target video segment is the 100th video segment in a sequence of 200 consecutive video segments arranged chronologically, then the historical feature queue includes the first historical feature information corresponding to the 20th to 99th video segments in a sequence.

[0076] Since the first feature vector corresponding to the target video segment does not include temporal position information, it is necessary to perform position encoding on the first feature vector and the first historical feature information of the k video segments that are arranged in order in the historical feature queue before the target video segment, to obtain the second feature vector corresponding to the first feature vector, and the second historical feature information corresponding to the first historical feature information of each of the k video segments.

[0077] The calculation formula is:

[0078] ;

[0079] It is stored in the historical feature queue HFQ. The first historical feature information corresponding to each video segment It is the first feature vector corresponding to the target video segment. It is after position encoding The second historical feature information corresponding to each video segment It is the second feature vector corresponding to the target video segment after position encoding. This refers to the position encoding operation.

[0080] In one example, the position encoding operation is sine-cosine position encoding.

[0081] S4. Determine the memory feature vector based on the second historical feature information of each of the k video segments. The memory feature vector includes short-term historical feature information and long-term historical feature information.

[0082] Specifically, short-term historical feature information is determined based on the second historical feature information of n video segments adjacent to the target video segment in the historical feature queue, and long-term historical feature information is determined based on the second historical feature information of m video segments other than the n video segments adjacent to the target video segment in the historical feature queue, where k, n, and m are positive integers, and m+n=k.

[0083] It should be noted that after the position is encoded Second historical feature information corresponding to each video segment As a feature memory vector, when A large value increases the computational load during model decoding, reducing decoding efficiency. Furthermore, simply increasing the length of historical feature information cannot effectively improve the model's real-time detection accuracy during experiments. Therefore, the method provided in this embodiment of the invention uses the position-encoded... Second historical feature information corresponding to each video segment It is divided into long-term historical characteristic information and short-term historical characteristic information.

[0084] In some embodiments, see Figure 4 The above S4 specifically includes the following steps:

[0085] S41. The second historical feature information of n video segments adjacent to the target video segment is spliced ​​together to obtain short-term historical feature information;

[0086] Specifically, short-term historical characteristics information The formula for determining it is:

[0087] ;

[0088] It refers to the second historical feature information of the n video segments adjacent to the target video segment in the historical feature queue HFQ.

[0089] The method provided in this embodiment of the invention obtains short-term historical feature information by splicing the second historical feature information of n video segments adjacent to the target video segment, which can provide a more accurate short-term historical information dependency for predicting the process category of the target video segment.

[0090] S42. Divide the second historical feature information of m video segments in the historical feature queue, excluding the n video segments adjacent to the target video segment, into x information groups. Each information group includes the second historical feature information of y video segments.

[0091] Where x and y are positive integers, and x*y=m.

[0092] S43. Based on the compression aggregation network, the second historical feature information of the y video segments included in each information group is compressed and aggregated to obtain the compressed aggregated features corresponding to each information group.

[0093] Among them, the compressed aggregation network is a neural network structure based on Taylor expansion and the Kolmogorov-Arnold representation theorem.

[0094] Specifically, for long-term historical feature information, in order to compress the feature length while highlighting more important long-term historical feature information, the method provided in this embodiment of the invention uses a multi-layer compression aggregation network to compress and aggregate the long-term historical feature information. Each information group includes the second historical feature information of y video segments, which is compressed and aggregated using the same compression aggregation network, that is:

[0095] ;

[0096] ;

[0097] in, Each information group includes the second historical feature information of y video segments. This refers to the compressed aggregated feature corresponding to the x-th information group. TaylorKAN represents the compressed aggregated operation.

[0098] In one example, the number of layers in the compressed aggregation network is set to 3.

[0099] In some embodiments, the compressed aggregation network is used for:

[0100] Based on Taylor polynomials, feature transformation is performed on the second historical feature information of each information group, which includes y video segments, to obtain the transformed feature corresponding to each second historical feature information; the transformed features corresponding to the second historical feature information of each information group are aggregated to obtain the compressed aggregated feature corresponding to each information group.

[0101] Specifically, the compressed aggregation network, also known as the TaylorKAN network, is a variant of the Kolmogorov-Arnold (KAN) network structure. The KAN network structure is based on the Kolmogorov-Arnold representation theorem, which states that if... If is any multivariable continuous function defined on a bounded field, then the function It can be represented as a two-level nested addition of a finite number of single-variable continuous functions, that is, any multivariable continuous function can be represented as a combination of a single-variable continuous function and an addition operation.

[0102] Compared to MLP networks in related technologies, which place fixed activation functions on nodes (neurons), KAN networks place learnable activation functions on edges (weights). This design allows each weight parameter in a KAN network to be replaced by a univariate function, which is usually parameterized in the form of spline functions. This provides extremely high flexibility and enables the simulation of complex functions with fewer parameters, enhancing the interpretability of KAN.

[0103] KAN networks typically use B-spline functions as activation functions to adaptively model complex relationships in the data; these activation functions usually include B-spline basis functions.

[0104] The compressed aggregation network provided in this embodiment of the invention is a variant of the KAN network that uses Taylor polynomials instead of B-spline basis functions as activation functions. It uses Taylor polynomials of different orders to perform local approximation and other operations on the input data to generate local high-order nonlinear features.

[0105] Among them, Taylor polynomial The formula is:

[0106] ;

[0107] For the expansion series.

[0108] S44. The compressed and aggregated features corresponding to each information group are concatenated to obtain long-term historical feature information.

[0109] Specifically, long-term historical characteristics information The formula for determining it is:

[0110] ;

[0111] S45. The short-term historical feature information and the long-term historical feature information are concatenated to obtain the memory feature vector.

[0112] Specifically, memorizing feature vectors The formula for determining it is:

[0113] ;

[0114] ;

[0115] ;

[0116] For memory feature vectors The length of this is the length of the historical information input into the real-time process detection unit after training. It is long-term historical feature information after compression and aggregation. Length, It is short-term historical characteristic information Length, This is the length of the historical feature queue (HFQ). x is the number of information groups.

[0117] In one example Set to 50. It is 10. It is 8. It is 5. The answer is 15. This can also be understood as follows: the historical feature queue includes historical feature information corresponding to 50 video segments. Short-term historical feature information is obtained by concatenating the second historical feature information corresponding to the 10 video segments adjacent to the target video segment. Then, the second historical feature information corresponding to the remaining 40 video segments is grouped into 8 information groups. Each information group includes the second historical feature information corresponding to 5 video segments. The 8 information groups are then compressed and aggregated, and finally concatenated to obtain the long-term historical feature information. Then, short-term historical feature information. and long-term historical characteristics information Perform a concatenation operation to obtain the memory feature vector. .

[0118] S5. Based on the second feature vector and memory feature vector of the target video segment, determine the experimental process detection result of the target video segment through the trained real-time process detection unit.

[0119] The experimental process detection results include at least one process category of the variable gravity particle experiment, as well as the start and end times corresponding to each process category.

[0120] Specifically, the trained real-time process detection unit determines the experimental process detection results of the target video segment. This can also be understood as determining the process category corresponding to each frame of the target video segment, as well as the start and end times corresponding to each process category. The process categories include: leftward movement of the baffle before vibration, rightward movement of the baffle before vibration, baffle vibration, baffle cessation of vibration, baffle reset after vibration, leftward movement of the baffle before collapse, variable gravity before collapse, baffle forward pressure before collapse, and collapse. The process category also includes no category (static process stage).

[0121] For example, if the target video segment includes multiple consecutive frames of images taken when the variable gravity particle experiment A is in a static process phase, then the experimental process detection result of the target video segment includes a process category of "no category". The start time corresponding to "no category" is the timestamp of the first frame of the target video segment, and the end time corresponding to "no category" is the timestamp of the last frame of the target video segment.

[0122] In another example, the target video clip includes multiple consecutive frames of images taken during the static phase and the baffle vibration phase of the variable gravity particle experiment A. In this case, the experimental process detection results of the target video clip include process categories of no category and baffle vibration. The start time corresponding to no category is the timestamp of the first frame of the target video clip, the timestamp corresponding to the frame where the baffle begins to vibrate is the start time corresponding to the baffle vibration, and the end time corresponding to the baffle vibration is the timestamp of the last frame of the target video clip.

[0123] Furthermore, the trained real-time process detection unit also obtains the experimental process detection results for the variable gravity particle experiment based on the experimental process detection results of each video segment in multiple consecutive video clips. The experimental process detection results include at least one process category of the variable gravity particle experiment, as well as the start and end times corresponding to each process category.

[0124] In some embodiments, the trained real-time process detection unit includes multiple cascaded Transformer Decoder units, which are used to output processing results based on the input query vector, key vector, and value vector.

[0125] Experimental detection results of the target video segment The formula for determining it is:

[0126] ;

[0127] ;

[0128] ;

[0129] ;

[0130] in, For activation functions; The processing result output by the Transformer Decoder unit; The weights for projecting the processing results output by the Transformer Decoder unit into the class space, where c is the number of process classes; Here, K is the query vector, V is the key vector, and V is the value vector. This is the second feature vector of the target video segment; This is the memory feature vector of the target video segment.

[0131] In one possible implementation, see Figure 5 Prior to S5 above, the method provided in this embodiment of the invention further includes:

[0132] S51. Obtain the training sample set for the variable gravity particle experiment. The training sample set includes multiple training samples, and each training sample includes a training video clip and the corresponding experimental process detection results.

[0133] S52. Construct the target loss function.

[0134] Target loss function for:

[0135]

[0136] ;

[0137] ;

[0138] ;

[0139] in, The balance factor for category i in each batch. It is the predicted probability that the number of video frames corresponding to time t corresponds to process category i; yes Time belongs to the process category The true label; As a balance factor; It is the total number of process instances in each batch. The mean statistic is based on logarithmic transformation. It is the process category in each batch. The number of process instances.

[0140] Since the video process detection data of variable gravity particle experiments suffers from class imbalance, and real-time process detection based on motion detection extensions exacerbates this problem, the method proposed in this embodiment of the invention can alleviate the class imbalance problem by calculating the balance factor using the Focal Loss function based on small batches of samples, thereby improving detection accuracy.

[0141] S53. Based on the target loss function, iteratively train the real-time process detection unit to obtain the trained real-time process detection unit.

[0142] As can be seen from S1-S5 above, the method provided by the present invention determines the video feature vector corresponding to the target video segment, then determines the memory feature vector based on the historical feature information of multiple video segments preceding the target video segment in a series of consecutive video segments, and then determines the process category corresponding to the target video segment based on the video feature vector and the memory feature vector. This method can accurately detect and identify different stages of the space variable gravity particle material experiment in real time during the experiment, thereby improving data processing efficiency.

[0143] In some embodiments, see Figure 6 The method provided in this embodiment of the invention further includes:

[0144] S61. Based on a multilayer perceptron, the experimental detection results of the target video segment are projected to obtain the projection feature vector corresponding to the target video segment.

[0145] S62. Determine the first historical feature information corresponding to the target video segment based on the projection feature vector and the first feature vector corresponding to the target video segment.

[0146] S63. Update the historical feature queue based on the first historical feature information corresponding to the target video segment.

[0147] Among them, the projection feature vector corresponding to the target video segment The formula for determining it is:

[0148] ;

[0149] ;

[0150] The experimental process detection results for the target video segment. This is the first feature vector corresponding to the target video segment. This refers to the first historical feature information corresponding to the target video segment.

[0151] In one possible implementation, S63 specifically includes the following steps:

[0152] If the target video segment is the t-th segment among a series of consecutive video segments of the variable gravity particle experiment, and t is less than or equal to k, the first historical feature information corresponding to the target video segment is added to the historical feature queue; if the target video segment is the t-th segment among a series of consecutive video segments of the variable gravity particle experiment, and t is greater than k, the first historical feature information corresponding to the video segment ranked 1st in the historical feature queue is deleted, and the first historical feature information corresponding to the target video segment is added to the historical feature queue.

[0153] Specifically, when the target video segment is the t-th video segment and t is greater than k, the length of historical features that the historical feature queue HFQ can store is... Therefore, the historical feature queue HFQ stores... to The first historical feature information at time t. When the process category detection of the target video segment t is completed, the historical feature queue HFQ will... The first historical feature corresponding to each video segment The first historical feature information of the target video segment t is deleted and added to the historical feature queue HFQ. The first historical feature information of the target video segment t is obtained by adding the first feature vector of the target video segment t to the projection feature vector obtained by the detection result of the experimental process through an MLP projection layer.

[0154] In addition, for the first video segment in a series of consecutive video segments, the detection system 100 initializes the historical feature queue HFQ, initializes the historical feature queue HFQ based on the feature vector of all zeros, and sets the length of the historical feature queue HFQ. .

[0155] The following example illustrates the beneficial effects of a real-time detection method for variable gravity particle experiments based on historical information provided by an embodiment of the present invention.

[0156] For example, the method provided in this embodiment of the invention verifies the beneficial effects of the method using a verification dataset. The verification dataset consists of experimental videos of the particulate material compartment A in a variable gravity experimental chamber, captured by a panoramic camera located in the space laboratory. It includes 207 video clips, each capturing rich details of the experimental process at a resolution of 1920×1080 and a frame rate of 25 frames per second, with an average video length of 9.4 minutes. A total of 9 different process categories are labeled in the 207 video clips, totaling 939 process instances, with an average duration of approximately 29 seconds per instance. Furthermore, there is no temporal overlap between all instances.

[0157] See Figure 7 , Figure 7 This diagram illustrates the process categories and durations of a spatial variable gravity particulate material experiment using a validation dataset provided in this embodiment of the invention. The validation dataset predominantly consists of long processes, some of which have significantly longer average durations. For example, the average duration of the baffle vibration process can reach 50 seconds. Meanwhile, the durations of some processes also vary significantly. Compared to baffle vibration, the average duration of the leftward shift of the baffle before vibration is only 1.5 seconds.

[0158] In this example, the method provided by the embodiment of the present invention is verified based on the evaluation metrics Mean Average Precision (mAP) and Average Precision (AP). The mAP values ​​with a step size of 0.1 on tIoU=[0.3,0.7] are compared, where tIoU refers to the intersection-union ratio in time.

[0159] Mean precision (mAP) is an evaluation metric for a single process category. It measures the average precision across different recall levels, while mAP is the average AP across all process categories. Specifically, when calculating AP, the prediction results are sorted according to their confidence levels, then the precision at different recall thresholds is calculated, and finally, the mean precision is obtained through integration or interpolation.

[0160] AP is typically calculated using 11-point interpolation, which means averaging the maximum precision values ​​at 11 points where the recall is 0, 0.1, 0.2, ..., 1. mAP, on the other hand, is the average AP across all process categories.

[0161]

[0162] Where C is the number of process categories.

[0163] tIoU measures the degree of overlap between predicted and actual experimental segments. It is the ratio of the area of ​​intersection to the area of ​​union of the predicted and actual experimental segments at time t. A higher tIoU value indicates that the prediction is closer to the actual result. tIoU is calculated as follows:

[0164] ;

[0165] When calculating mAP, different tIoU thresholds are usually set. The prediction is considered correct only when the tIoU between the predicted result and the actual result is greater than or equal to the threshold.

[0166] In addition, to evaluate the real-time process detection algorithm, frames per second (FPS) is used as an evaluation metric. FPS refers to the number of frames processed per second, generally used to measure the efficiency of a model in processing image or video frames. Here, FPS is used to measure the efficiency of the method provided in this invention for process detection. The formula for calculating FPS is:

[0167] ;

[0168] in, It is the total number of frames processed. This is the total time (in seconds) used to process these frames. FPS measures the speed at which the model processes video frames and reflects the algorithm's real-time processing capability.

[0169] Specifically, the embodiments of the present invention are compared with the SimOn model in related technologies. The results of the comparison experiment are shown in Table 1. It should be noted that the SimOn model is a neural network structure that uses two kinds of context information as a complete context for process detection. The SimOn model does not predict the action score of each frame, but directly generates the action category probability, thereby realizing multiple action prediction.

[0170] The experimental results are shown in Table 1. The experiments were conducted using I3D (RGB+Optical Flow) with optical flow branch and ViT (RGB) with only RGB as feature extractors to determine the first feature vector. In addition, the dimension of video features extracted by I3D is 2048, and the dimension of video features extracted by ViT is 1408.

[0171] Table 1

[0172]

[0173] As shown in Table 1, the method provided in this embodiment of the invention uses ViT as the feature encoder, which achieves significant improvement under different tIoU thresholds. The detection accuracy mAP reaches 82.79%, which is about 11.71% higher than that of the SimOn model that uses ViT to extract video features. This indicates that the proposed LSH-RPD method can better predict the process category of the experiment based on the constructed long and short-term historical information, thus proving the effectiveness of the technical solution of this invention.

[0174] Furthermore, to verify the real-time inference speed of the technical solution of this invention, an FPS comparison experiment was conducted on the verification dataset of spatial variable gravity particle materials experiments. The experimental results are shown in Table 2. In all experiments, the video features (first feature vector) extracted by ViT were used. In addition, the parameters and FPS in the table do not include the ViT part.

[0175] Table 2

[0176]

[0177] As can be seen from Table 2, although the method proposed in this embodiment of the invention introduces a compressed aggregation network to compress and aggregate long-term historical feature information compared to the SimOn model, the number of parameters only increases by about 0.13M, and the FPS does not decrease much, reaching 90.00. Therefore, overall, the technical solution of this invention can ensure highly accurate detection results while completing inference at a relatively fast speed, achieving a relative balance between detection accuracy and inference speed.

[0178] Furthermore, to explore the impact of historical feature length on the detection accuracy of the experimental process, the method provided in this embodiment of the invention also conducts an ablation experiment based on historical feature length. The experimental results are shown in Table 3, where... This refers to the length of short-term historical information. This refers to the length of long-term historical information. This refers to the length of the memory vector that is finally input to the real-time process detection unit. All experiments used ViT as the Encoder to extract video features. In addition, a 3-layer compression and aggregation network was used when compressing and aggregating long-term historical information.

[0179] Table 3

[0180]

[0181] As can be seen from Table 3, the SimOn model achieved an overall detection accuracy (mAP) of 71.08% when the short-term historical information length was 7. However, increasing the short-term historical information length to 15 did not significantly improve the overall detection accuracy (mAP) (71.09%), indicating that simply increasing the historical information length cannot significantly improve the model's detection capability.

[0182] Based on the SimOn model, a historical feature queue (HFQ) is constructed, dividing historical feature information into short-term and long-term historical information. Short-term historical information is left unprocessed, while long-term historical information is compressed and aggregated. The length of short-term historical information is 10, and the length of long-term historical information is 40. These are compressed and aggregated to 5 using the TaylorKAN network, ensuring that the memory vector length of the input real-time process detection unit remains at 15. The final model shows a significant improvement in detection accuracy under different tIoU thresholds, achieving an overall detection accuracy (mAP) of 80.66%. Compared to the SimOn model (71.09%) with the same memory feature vector length, the mAP is improved by approximately 9.57%. This demonstrates that the proposed method of constructing both short-term and long-term historical information can effectively improve the model's detection capability without increasing the computational load of the decoder, proving the effectiveness of the technical solution of this invention.

[0183] Furthermore, to verify the effectiveness of each module in the technical solution of this invention, the method provided in this embodiment of the invention also conducts module ablation experiments based on a verification dataset. The experimental results are shown in Table 4. In all experiments, ViT was used as the Encoder to extract video features. Aggregation layers refer to the network used to compress and aggregate long-term historical information under the premise of storing long-term and short-term historical information using a historical feature queue. The loss refers to the proposed balance factor calculated based on mini-batch samples. The Focal Loss function is used, and the length of the memory feature vector input to the trained real-time process detection unit is 15.

[0184] As shown in Table 4, both using MLP and the more interpretable TaylorKAN network to compress and aggregate long-term historical information can improve the model's detection accuracy of the experimental process. Based on the SimOn model, compared with using MLP, the TaylorKAN network compression and aggregation method improves the detection accuracy mAP by about 2.2%, indicating that compared with MLP, the more interpretable TaylorKAN network can more effectively compress and aggregate long-term historical information, obtain local approximate high-order nonlinear features, highlight more important long-term historical features, and thus improve the model's process detection capability.

[0185] It should be noted that although the various processes in the variable gravity particle experiment are relatively balanced (e.g., Figure 7 As shown in the figure, since the method provided by this invention detects the experimental process category in real time based on different video segments, and can only predict based on the current target video segment (current frame) during the detection process, it is impossible to view future video segments (future frames) that are located after the target video segment in time sequence. Therefore, during the detection process, the experimental video data of the variable gravity particle experiment also suffers from class imbalance. To alleviate this problem, the technical solution of this invention uses a balance factor calculated based on a small batch of samples. The proposed Focal Loss function (target loss function) shows that the model's detection accuracy (mAP) reaches 82.79% after using the proposed target loss function. Compared with the method using TaylorKAN to compress and aggregate long-term historical information (80.66%), the mAP is improved by about 2.13%, indicating that the target loss function effectively alleviates the problem of class imbalance in the detection dataset of space science experiment video process, proving the effectiveness of the technical solution of this invention.

[0186] Table 4

[0187]

[0188] The foregoing mainly describes the solutions of the embodiments of the present invention from a methodological perspective. It is understood that, in order to achieve the above-mentioned functions, the detection system 100 includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present invention.

[0189] In this embodiment of the invention, the detection system 100 can be divided into functional units according to the above method example. For example, the detection system 100 can be divided into functional units corresponding to each function, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this embodiment of the invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0190] For example, Figure 8A schematic diagram of the hardware structure of a detection system provided in an embodiment of the present invention is shown. The detection system 100 includes: a video acquisition unit 110, used to acquire a target video segment of a variable gravity particle experiment, the target video segment including multiple consecutive frames of images, the target video segment being one of multiple consecutive video segments of the variable gravity particle experiment; a feature extraction unit 120, used to determine a first feature vector corresponding to the target video segment; a position encoding unit 130, used to perform position encoding operations on the first feature vector and the first historical feature information of k video segments arranged in order in the multiple video segments preceding the target video segment, including the historical feature queue, to obtain a second feature vector corresponding to the first feature vector, and second historical feature information corresponding to the first historical feature information of each of the k video segments; and a vector determination unit 140, used to determine the first feature vector of each of the k video segments according to the first historical feature information of the target video segment. The memory feature vector is determined by two historical feature information, which includes short-term historical feature information and long-term historical feature information. The short-term historical feature information is determined based on the second historical feature information of the n video segments adjacent to the target video segment in the historical feature queue, and the long-term historical feature information is determined based on the second historical feature information of the m video segments other than the n video segments adjacent to the target video segment in the historical feature queue. k, n, and m are positive integers, and m+n=k. The process detection unit 150 is used to determine the experimental process detection result of the target video segment through the trained real-time process detection unit based on the second feature vector and memory feature vector of the target video segment. The experimental process detection result includes at least one process category of variable gravity particle experiment, as well as the start time and end time corresponding to each process category.

[0191] It should be understood that specific descriptions of the above-mentioned optional methods can be found in the foregoing method embodiments, and will not be repeated here. Furthermore, explanations of any of the detection systems 100 provided above, as well as descriptions of their beneficial effects, can be found in the corresponding method embodiments described above, and will not be repeated here.

[0192] This invention also provides a computer-readable storage medium storing at least one computer instruction, which is loaded and executed by a processor to implement the methods of the various embodiments described above. Explanations of the relevant content and descriptions of the beneficial effects of any of the computer-readable storage media provided above can be found in the corresponding embodiments described above, and will not be repeated here.

[0193] This invention also provides a chip. This chip integrates a control circuit for implementing the functions of the detection system 100 described above, and one or more ports. Optionally, the functions supported by this chip are as described above, and will not be repeated here.

[0194] Those skilled in the art will understand that the program for implementing all or part of the steps of the above embodiments, which can be executed by a program instructing related hardware, can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0195] This invention also provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform any of the methods described in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this invention is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD), etc.

[0196] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of the present invention, such as, but not limited to, the aforementioned memory, computer-readable storage medium, and communication chip, are all non-transitory. Those skilled in the art should recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0197] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for real-time detection of variable gravity particle experiments based on historical information, characterized in that, The method includes: Acquire a target video segment of the variable gravity particle experiment, the target video segment comprising multiple consecutive frames of images, and the target video segment being one of multiple consecutive video segments of the variable gravity particle experiment; Determine the first feature vector corresponding to the target video segment; The first feature vector and the first historical feature information of the k video segments that are arranged in order in the plurality of video segments before the target video segment are subjected to position encoding operation to obtain the second feature vector corresponding to the first feature vector, and the second historical feature information corresponding to the first historical feature information of each video segment in the k video segments; A memory feature vector is determined based on the second historical feature information of each of the k video segments. The memory feature vector includes short-term historical feature information and long-term historical feature information. The short-term historical feature information is determined based on the second historical feature information of the n video segments adjacent to the target video segment in the historical feature queue. The long-term historical feature information is determined based on the second historical feature information of the m video segments in the historical feature queue other than the n video segments adjacent to the target video segment. k, n, and m are positive integers, and m + n = k. The experimental process detection result of the target video segment is determined by the second feature vector and memory feature vector of the target video segment through a trained real-time process detection unit. The experimental process detection result includes at least one process category of variable gravity particle experiment, and the start time and end time corresponding to each process category. The trained real-time process detection unit includes multiple Transformer Decoder units connected in series. The step of determining the memory feature vector based on the second historical feature information of each of the k video segments includes: The second historical feature information of n video segments adjacent to the target video segment is spliced ​​together to obtain short-term historical feature information; The second historical feature information of m video segments in the historical feature queue, excluding the n video segments adjacent to the target video segment, is divided into x information groups. Each information group includes the second historical feature information of y video segments, where x and y are positive integers and x*y=m. Based on the compression aggregation network, the second historical feature information of the y video segments included in each information group is compressed and aggregated to obtain the compressed aggregated features corresponding to each information group. The compression aggregation network is a neural network structure based on Taylor expansion and Kolmogorov-Arnold representation theorem. The compressed and aggregated features corresponding to each information group are concatenated to obtain long-term historical feature information. The short-term historical feature information and the long-term historical feature information are concatenated to obtain the memory feature vector.

2. The method according to claim 1, characterized in that, The method further includes: Based on a multilayer perceptron, the experimental process detection results of the target video segment are projected to obtain the projection feature vector corresponding to the target video segment. The first historical feature information corresponding to the target video segment is determined based on the projection feature vector and the first feature vector corresponding to the target video segment; The historical feature queue is updated based on the first historical feature information corresponding to the target video segment; Wherein, the projection feature vector corresponding to the target video segment The formula for determining it is: ； ； The experimental process detection results for the target video segment are as follows. The first feature vector corresponding to the target video segment. This refers to the first historical feature information corresponding to the target video segment.

3. The method according to claim 2, characterized in that, The step of updating the historical feature queue based on the first historical feature information corresponding to the target video segment includes: If the target video segment is the t-th segment among a series of consecutive video segments of the variable gravity particle experiment, and t is less than or equal to k, the first historical feature information corresponding to the target video segment is added to the historical feature queue. If the target video segment is the t-th segment among a series of consecutive video segments of the variable gravity particle experiment, and t is greater than k, then the first historical feature information corresponding to the video segment ranked 1st in the historical feature queue is deleted, and the first historical feature information corresponding to the target video segment is added to the historical feature queue.

4. The method according to claim 1, characterized in that, The compressed aggregation network is used for: Based on Taylor polynomials, feature transformation is performed on the second historical feature information of each information group, which includes y video segments, to obtain the transformed features corresponding to each second historical feature information. The transformed features corresponding to the second historical feature information included in each information group are aggregated to obtain the compressed aggregated features corresponding to each information group.

5. The method according to claim 4, characterized in that, The multiple serially connected TransformerDecoder units are used to output processing results based on the input query vector, key vector, and value vector; The experimental process detection results of the target video segment The formula for determining it is: ； ； ； ； in, For activation functions; The processing result output by the Transformer Decoder unit; The weights for projecting the processing results output by the Transformer Decoder unit into the class space, where c is the number of process classes; Here, K is the query vector, V is the key vector, and V is the value vector. This is the second feature vector of the target video segment; is the memory feature vector of the target video segment.

6. The method according to claim 5, characterized in that, Before determining the experimental process detection result of the target video segment through the real-time process detection unit based on the first feature vector and the memory feature vector of the target video segment, the method further includes: A training sample set for the variable gravity particle experiment is obtained. The training sample set includes multiple training samples, and each training sample includes a training video segment and the corresponding experimental process detection results. Construct the target loss function; Based on the objective loss function, the real-time process detection unit is iteratively trained to obtain the trained real-time process detection unit.

7. The method according to claim 6, characterized in that, The target loss function for: ； ； ； in, The balance factor for category i in each batch. It is the predicted probability that the number of video frames corresponding to time t corresponds to process category i; yes Time belongs to the process category The true label; As a balance factor; It is the total number of process instances in each batch. The mean statistic is based on logarithmic transformation. It is the process category in each batch. The number of process instances.

8. A real-time detection system for variable gravity particle experiments based on historical information, characterized in that, The system includes: The video acquisition unit is used to acquire a target video segment of the variable gravity particle experiment. The target video segment includes multiple consecutive frames of images and is one of multiple consecutive video segments of the variable gravity particle experiment. A feature extraction unit is used to determine the first feature vector corresponding to the target video segment; The position encoding unit is used to perform position encoding operations on the first feature vector and the first historical feature information of the k video segments arranged in order in the plurality of video segments that are located before the target video segment, to obtain the second feature vector corresponding to the first feature vector, and the second historical feature information corresponding to the first historical feature information of each of the k video segments. A vector determination unit is used to determine a memory feature vector based on the second historical feature information of each video segment in k video segments. The memory feature vector includes short-term historical feature information and long-term historical feature information. The short-term historical feature information is determined based on the second historical feature information of n video segments adjacent to the target video segment in the historical feature queue. The long-term historical feature information is determined based on the second historical feature information of m video segments in the historical feature queue other than the n video segments adjacent to the target video segment. k, n, and m are positive integers, and m + n = k. The process detection unit is used to determine the experimental process detection result of the target video segment based on the second feature vector and memory feature vector of the target video segment through a trained real-time process detection unit. The experimental process detection result includes at least one process category of the variable gravity particle experiment, as well as the start time and end time corresponding to each process category. The trained real-time process detection unit includes multiple Transformer Decoder units connected in series. The vector determination unit is specifically used for: The second historical feature information of n video segments adjacent to the target video segment is spliced ​​together to obtain short-term historical feature information; The second historical feature information of m video segments in the historical feature queue, excluding the n video segments adjacent to the target video segment, is divided into x information groups. Each information group includes the second historical feature information of y video segments, where x and y are positive integers and x*y=m. Based on the compression aggregation network, the second historical feature information of the y video segments included in each information group is compressed and aggregated to obtain the compressed aggregated features corresponding to each information group. The compression aggregation network is a neural network structure based on Taylor expansion and Kolmogorov-Arnold representation theorem. The compressed and aggregated features corresponding to each information group are concatenated to obtain long-term historical feature information. The short-term historical feature information and the long-term historical feature information are concatenated to obtain the memory feature vector.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the real-time detection method for variable gravity particle experiments based on historical information as described in any one of claims 1-7.

Images