Picture and video recognition and analysis method based on end-side multi-modal large model

Abstract

The application provides a picture and video recognition and analysis method based on an end-side multimodal large model, relates to the technical field of artificial intelligence, and comprises the following steps: inputting a picture or a video into an end-side multimodal large model, extracting features and generating candidate recognition results by using a Monte Carlo tree search, calculating a confidence score by using a graph neural network to construct a feature semantic graph, and outputting a result with the highest score to a display interface. The application can improve recognition accuracy, reduce the consumption of computing resources, and enhance multimedia content analysis capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a picture and video recognition and analysis method based on an end-side multi-modal large model. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, picture and video recognition and analysis technology based on multi-modal large models has been widely applied in intelligent devices, security monitoring, autonomous driving and other fields. Traditional image and video recognition mainly relies on cloud servers for processing, and data needs to be uploaded to the cloud for complex calculations before the results are returned to the terminal device. With the improvement of end-side computing power, it is possible to deploy multi-modal large models on terminal devices for real-time image and video recognition and analysis, which not only reduces network transmission delay, but also effectively protects user privacy data.

[0003] Current end-side image and video recognition technology mainly uses deep learning methods, such as convolutional neural networks and recurrent neural networks, to extract and analyze image and video data. Common techniques include image classification and object detection based on deep convolutional neural networks, video behavior recognition based on recurrent neural networks, and multi-modal feature fusion based on transformer architecture. These methods have achieved good recognition results in specific scenarios, but still face many challenges in complex and variable real-world application environments.

[0004] Traditional image and video recognition methods usually use fixed feature extraction paths, lack adaptive analysis capabilities for image and video content, and are difficult to effectively handle diverse information in complex scenarios. In the face of complex environments such as light changes, occlusions, and angle changes, recognition accuracy will decrease significantly.

[0005] Existing end-side multi-modal models have high computational complexity when processing high-dimensional feature information, making it difficult to achieve efficient inference on resource-limited terminal devices. This results in either limited model size affecting recognition results or slow running affecting user experience, failing to meet real-time requirements.

[0006] Existing technologies have deficiencies in the comprehensive evaluation of multiple candidate results, often relying on simple confidence thresholds or voting mechanisms for result screening, lacking in-depth understanding and mining of semantic associations between candidate results, leading to false recognition in complex scenarios, especially when considering scene context information. SUMMARY

[0007] The embodiments of the present application provide a picture and video recognition and analysis method based on an end-side multi-modal large model, which can solve the problems in the prior art.

[0008] In a first aspect, the embodiment of the present application provides a picture and video recognition and analysis method based on an end-side multimodal large model, comprising:

[0009] inputting picture or video data to be processed into a multimodal large model pre-deployed on a device side, calculating sampling probabilities of a plurality of sampling paths based on a probability distribution function, and performing Monte Carlo tree search on the picture or video data according to the sampling probabilities, each node of the Monte Carlo search tree containing a state value function and an action policy function, iteratively updating the state value function and the action policy function of the node in an asynchronous parallel manner, selecting a search path with the largest search value for feature extraction, and generating a plurality of candidate recognition results;

[0010] constructing a feature semantic graph based on the plurality of candidate recognition results, iteratively propagating and calculating the feature semantic graph by a graph neural network to obtain node vector representations, calculating confidence scores of each candidate recognition result according to the node vector representations, and selecting a candidate recognition result with the highest confidence score as a final recognition result;

[0011] outputting the final recognition result to a display interface on the device side, and correspondingly presenting image scene information, target detection information and action analysis information in the display interface.

[0012] calculating sampling probabilities of a plurality of sampling paths based on a probability distribution function, and performing Monte Carlo tree search on the picture or video data according to the sampling probabilities comprises:

[0013] dividing the picture or video data into a plurality of image blocks, calculating a prior probability distribution of each image block based on a variational inference method, constructing a probability sampling space according to the prior probability distribution, performing Markov chain Monte Carlo sampling in the probability sampling space to generate a plurality of sampling paths, and calculating a sampling probability of each sampling path;

[0014] constructing a Monte Carlo search tree for the picture or video data based on the sampling probabilities, selecting a search path with the largest search value of the Monte Carlo search tree for feature extraction, and generating a plurality of candidate recognition results.

[0015] constructing a probability sampling space according to the prior probability distribution, performing Markov chain Monte Carlo sampling in the probability sampling space to generate a plurality of sampling paths, and calculating a sampling probability of each sampling path comprises:

[0016] constructing a probability density function for the prior probability distribution, the probability density function being smoothed by a kernel density estimation method for the prior probability distribution; constructing a probability sampling space based on the probability density function, the probability sampling space containing a support set of the probability distribution and a transition probability matrix;

[0017] Constructing a state transition network of a Markov chain in the probability sampling space, the state transition network comprising a state space and a state transition rule;

[0018] Generating a random walk sequence using the state transition rule, performing Monte Carlo sampling based on the random walk sequence to generate a plurality of sampling paths, each sampling path corresponding to a transition trajectory in the state space;

[0019] Calculating a sampling probability of each sampling path, the sampling probability being calculated based on a state transition probability and a stationary distribution; and outputting the plurality of sampling paths and their corresponding sampling probabilities.

[0020] Each node of the Monte Carlo search tree comprises a state value function and an action policy function, and the state value function and the action policy function of the node are iteratively updated in an asynchronous parallel manner, and a search path with the largest search value is selected for feature extraction, comprising:

[0021] Configuring a state value function and an action policy function at each node of the Monte Carlo search tree, wherein the state value function calculates a value evaluation score of the node state using a residual network structure, and the action policy function generates an action selection probability distribution of the node based on a double policy optimization criterion;

[0022] Starting a plurality of parallel work processes, calculating a confidence upper bound value of the node, selecting a child node with the largest confidence upper bound value for expansion, updating the state value function and the action policy function of the node using an asynchronous stochastic gradient descent algorithm, and broadcasting the updated value evaluation score and the action selection probability distribution to other work processes through an asynchronous message passing mechanism;

[0023] Iteratively searching the Monte Carlo search tree based on a preset search depth, and selecting a search path with the largest search value in each iteration, wherein the search value is a weighted value of the value evaluation score and the action selection probability distribution after normalization.

[0024] Configuring a state value function and an action policy function at each node of the Monte Carlo search tree, comprising:

[0025] Constructing a multi-layer residual connection network, setting a jump connection path in the residual connection network, the jump connection path being used for fusing feature representations of different levels; and calculating a value evaluation score of the node state through feature fusion;

[0026] Constructing an action transition probability matrix using the value evaluation score, the action transition probability matrix describing a probability distribution of state transition of the node;

[0027] The two optimization objectives of online policy gradient and offline value estimation are set for the action transition probability matrix, and a double policy iteration method is used to alternately optimize the action selection policy to generate the action selection probability distribution of the node.

[0028] A confidence upper bound value of the node is calculated, and a child node with the largest confidence upper bound value is selected for expansion, and an asynchronous stochastic gradient descent algorithm is used to update the state value function and the action policy function of the node, including:

[0029] State information of the node to be expanded is obtained, and a feature vector in the state information is extracted;

[0030] A child node set is constructed according to the feature vector, and the historical access times and historical cumulative returns of each child node are counted;

[0031] A confidence interval parameter is calculated based on the historical access times and historical cumulative returns, and the confidence interval parameter is input into an adaptive estimator;

[0032] Thompson sampling is performed using the adaptive estimator to obtain a confidence upper bound value of each child node in the current state, and the confidence upper bound value is used to balance exploration and utilization in the node expansion process;

[0033] The child nodes are sorted according to the confidence upper bound values, and a child node with the largest confidence upper bound value is selected as an expansion target node;

[0034] The feature vector of the expansion target node is mapped to a value space and a policy space to construct a distributed computing network;

[0035] Asynchronous stochastic gradient descent operations are performed in the distributed computing network to optimize the state value function and the action policy function in parallel;

[0036] The state representation of the expansion target node is updated based on the optimization results, and the updated state value function and action policy function are output.

[0037] A feature semantic graph is constructed based on the plurality of candidate recognition results, a node vector representation is calculated by iteratively propagating the feature semantic graph through a graph neural network, a confidence score of each candidate recognition result is calculated based on the node vector representation, and a candidate recognition result with the highest confidence score is selected as a final recognition result, including:

[0038] The plurality of candidate recognition results are projected into a semantic feature space to obtain semantic feature representations, a feature semantic graph is constructed in the semantic feature space, each candidate recognition result corresponds to a node in the feature semantic graph, and an edge connection relationship is dynamically constructed based on feature similarity between nodes;

[0039] performing multi-round information propagation on the feature semantic graph, each round of propagation including: calculating importance weights of a node neighborhood, aggregating neighborhood node information based on the importance weights, nonlinearly fusing the aggregated information with center node information, controlling an information update proportion through a gating mechanism, iteratively performing until node states converge, and obtaining a final node vector representation;

[0040] calculating a confidence score of each candidate recognition result according to the node vector representation; and taking the candidate recognition result with the highest confidence score as a final recognition result.

[0041] In a second aspect, the embodiment of the present application provides an electronic device, comprising:

[0042] a processor;

[0043] a memory for storing processor-executable instructions;

[0044] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0045] In a third aspect, the embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions are executed by a processor to implement the method described above.

[0046] The present application has the following beneficial effects:

[0047] The present application can effectively balance the relationship between computing resources and recognition accuracy by deploying a multi-modal large model on a device and using a Monte Carlo tree search-based method to extract features from picture or video data, thereby achieving efficient processing on the device side, reducing dependence on network environment, and improving system response speed and user experience.

[0048] The present application introduces a feature semantic graph structure and a graph neural network to analyze and screen candidate results, fully considers the semantic correlation between candidate recognition results through iterative propagation of information between nodes, and improves the accuracy and reliability of the recognition results, thereby better processing image and video content recognition tasks in complex scenarios.

[0049] The present application optimizes the Monte Carlo tree search process through an asynchronous parallel mode, significantly improves the computing efficiency of the model on the device side, and combines an intelligent screening mechanism of the graph neural network to achieve high-quality recognition analysis under limited computing resources, so that the device can provide real-time and accurate picture and video content understanding services without relying on cloud computing. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1A flowchart of a picture and video recognition and analysis method based on an end-side multimodal large model according to an embodiment of the present application is shown in FIG. 1.

[0051] Figure 2 A flowchart of a MCTS node function configuration residual network state value function value evaluation dual policy optimization action probability distribution according to an embodiment of the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0053] The technical solutions of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and some embodiments may not be described again for the same or similar concepts or processes.

[0054] Figure 1 A flowchart of a picture and video recognition and analysis method based on an end-side multimodal large model according to an embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the method comprises the following steps.

[0055] The picture or video data to be processed is input into a multimodal large model pre-deployed on a device end, the sampling probabilities of a plurality of sampling paths are calculated based on a probability distribution function, and a Monte Carlo tree search is performed on the picture or video data according to the sampling probabilities. Each node of the Monte Carlo search tree contains a state value function and an action policy function, the state value function and the action policy function of the node are iteratively updated in an asynchronous parallel manner, a search path with the largest search value is selected for feature extraction, and a plurality of candidate recognition results are generated.

[0056] A feature semantic graph is constructed based on the plurality of candidate recognition results, a node vector representation is calculated by iteratively propagating and calculating the feature semantic graph through a graph neural network, a confidence score of each candidate recognition result is calculated according to the node vector representation, and a candidate recognition result with the highest confidence score is selected as a final recognition result.

[0057] The final recognition result is output to a display interface of the device end, and image scene information, target detection information, and action analysis information are correspondingly presented in the display interface.

[0058] In an alternative embodiment, the calculating the sampling probability of the plurality of sampling paths based on the probability distribution function, and performing the Monte Carlo tree search on the picture or video data according to the sampling probability comprises:

[0059] dividing the picture or video data into a plurality of image blocks, calculating a prior probability distribution of each image block based on a variational inference method, constructing a probability sampling space according to the prior probability distribution, performing Markov chain Monte Carlo sampling in the probability sampling space to generate a plurality of sampling paths, and calculating a sampling probability of each sampling path;

[0060] constructing a Monte Carlo search tree for the picture or video data based on the sampling probability, selecting a search path with the largest search value of the Monte Carlo search tree for feature extraction, and generating a plurality of candidate recognition results.

[0061] The input picture or video data is taken as a basic processing unit in frames, the original pixel matrix is read, unified into a fixed color space and bit depth, reproducible gamma correction and normalization strategy is adopted, and a standardized tensor for probability modeling is generated. The video data is processed in time sequence, and the frame index and timestamp are retained for subsequent cross-frame consistency constraint. The super high resolution data is scaled to the target processing size in proportion, and the scaling ratio information is recorded for result backfilling.

[0062] Each frame after standardization is divided into a plurality of image blocks, the division of the image blocks is based on fixed block width and block height, and an overlapping sliding window strategy is adopted between the blocks to cover the content at the block boundary, and the overlapping ratio is a decimal value in the horizontal and vertical directions. Each image block is assigned a unique identifier, and the starting coordinates and size of the image block in the original frame are recorded. A mapping table of blocks to frames and blocks to original image coordinates is established for subsequent feature backfilling and candidate result relocation.

[0063] A prior probability distribution is constructed for each image block, and a variational inference method is used to approximately infer the latent generation variable of the image block. The variational approximation adopts a parameterizable and separable form, and the parameters are output by a lightweight encoding network. The encoding network input is the image block pixel, gradient amplitude histogram, texture direction histogram and local contrast statistics, and the encoding network output is the center, scale and shape parameters of the prior. To avoid mathematical expression, the parameter update is described in words: in the training stage, the difference between the true posterior and the variational approximation is minimized, and the smaller the difference is, the closer the prior is to the true distribution; in the inference stage, the encoding network obtained by training is fixed, and the prior parameters are directly generated from the image block features. For the image blocks in the video frame, temporal consistency regularization is introduced, the prior parameters of the blocks at the same spatial position in adjacent frames are weighted and fused, the weight changes with the optical flow confidence, and the optical flow confidence is evaluated by matching consistency and occlusion detection.

[0064] A probability sampling space is constructed for sampling according to the prior probability distribution of each image block, the sampling space is organized in block units, a point in the space corresponds to a potential feature generation path starting from the block, and each step of the path corresponds to a value or transition of the latent variable. To ensure the reachability and coverage of sampling, a set of candidate actions is set for each block, which includes three types of operations: multi-direction perturbation in the latent space, multi-scale pyramid jump in the pixel domain, and channel subset selection in the feature domain. Each type of operation is accompanied by a preference score calculated based on the prior parameters, and the higher the preference score, the more supported it is under the prior distribution. The sampling space normalizes the actions, so that the opportunity of selecting an action is consistent with its preference score.

[0065] Markov Chain Monte Carlo sampling is performed in the probability sampling space to generate multiple sampling paths for each image block, each path is initialized from the vicinity of the prior center of the block, and the selection of the initial point follows the high-density region given by the prior parameters, with a small amount of random disturbance to break the symmetry. In the iterative step of the path, a candidate transition is generated according to the current path position and the set of candidate actions, the selection probability of the candidate transition is consistent with the preference score of the action, and an exploration degree controlled by an adaptive temperature is superimposed, the temperature decreases in the iteration process to realize the transition from extensive exploration to fine convergence.

[0066] To improve the sampling efficiency, a multi-chain parallel and delayed acceptance mechanism is introduced, first, a low-cost approximate evaluation is used to roughly screen the candidate transitions, only the candidates that pass the rough screening enter the high-cost fine evaluation. For each transition, by comparing the relative support of the candidate position and the current path position under the prior and the symmetry of the proposed action, it is determined whether to accept the transition; if accepted, the path advances, if rejected, a stay is recorded at the current position. To avoid the path stagnating in a local area, a restart strategy is set, when the number of consecutive rejections exceeds a threshold, the starting point is resampled from another high-density sub-region of the prior and the subsequent segment of the same path is continued to generate.

[0067] The sampling probability of each sampling path is calculated, which is expressed as the joint probability of being selected and accepted at each step in the path, including the initial point selection probability, the selection probability of each action proposal, and the occurrence probability of each acceptance or rejection. To facilitate implementation, the probability values generated at each stage of the path construction process are recorded, and these records are multiplied in sequence to accumulate the sampling probability of the path at the end of the path. To improve numerical stability, the accumulation method in the logarithmic domain is used in the implementation, and finally converted to normal probability values when sorting and normalization are needed. For parallel multi-chain paths, their sampling probabilities are calculated and retained separately, without mixing, so that subsequent tree construction stages can be processed hierarchically according to the link source.

[0068] Based on the sampling path and corresponding sampling probability generated by all image blocks, a Monte Carlo search tree is constructed, the search tree is organized in the form of a set of root nodes of blocks, each root node represents an image block, the child nodes of the root node correspond to the first-order potential variable value or the first-order action execution result of the block, and the deep nodes correspond to a multi-step potential decision sequence. The mapping mode of each sampling path in the search tree is to insert or update nodes step by step along the path, and statistical information is accumulated at each node, including the number of visits, the sum of the cumulative sampling probability from the path, and the local quality evaluation when the path reaches the node.

[0069] The local quality evaluation is derived from two types of indicators, one is the prior consistency indicator, which is used to measure the fitting degree of the current node state and the prior high support region; the other is the observation consistency indicator, which is used to measure the matching degree between the potential variable corresponding to the node and the observed features of the image block, and the matching degree is output by the pre-trained discriminative scorer. The search value of the node is jointly calculated by the above statistical quantities, and the exploration and utilization are considered in a weighted manner, and the weight is adaptively adjusted at different depths of the tree, with a bias towards exploration at the shallow layer and a bias towards utilization at the deep layer.

[0070] On the search tree, path selection based on search value is performed, for each root node, a path from the root to the leaf is selected, and the child node with the maximum current search value at each layer is selected as the next expansion direction. When reaching the leaf node or reaching the preset depth, the expansion operation is triggered, from the previously generated sampling path pool, those paths with the same prefix as the current path are filtered, the next step of which is not expanded is added as a new child node to the tree, and the corresponding sampling probability and local quality evaluation are written into the node statistics. After each expansion, the search value of all nodes passed through is updated, and the update rule is adjusted according to the growth of the cumulative sampling probability and the latest local quality evaluation, so that the recent high-quality path has stronger influence. The construction cycle of the tree is terminated when the computation budget is exhausted or the search values of all root nodes converge.

[0071] The complete path with the maximum search value on each root node is selected as the basis for feature extraction, and structured features are extracted from the original image block under the setting of the potential variable corresponding to the path, including multi-scale gradient description, direction consistency description, sparse key point response, channel attention weighted semantic embedding, and boundary consistency feature with adjacent blocks. For video data, under the same path setting, cross-frame trajectory features are extracted along the time axis, the trajectory is corrected by optical flow to eliminate the drift caused by motion, and the trajectory consistency score is written into the quality indicator of the path for subsequent candidate result sorting.

[0072] Based on the path-driven features, a plurality of candidate recognition results are generated, and the recognition task can be target category recognition, target position proposal or instance segmentation contour estimation. For the category recognition task, a pre-trained classification head is input with the path features, and a plurality of high-confidence category labels and their confidence scores are output. For the position proposal task, the displacement component of the path in the latent space and the multi-scale jump record in the pixel domain are combined to back-propagate to the original image coordinate system to generate a set of rectangular box proposals, and the proposals are scored using an observation consistency index and a boundary consistency feature.

[0073] For the instance segmentation task, the channel subset selection and direction consistency description of the path are used to drive a mask decoding head to generate candidate masks, and the boundary splicing and stitching are performed through the consistency measure of the overlapping area between blocks to eliminate the fragmentation effect caused by block division. The comprehensive scores of all candidate results are calculated, and the comprehensive scores are composed of three parts: classification confidence, geometric reasonableness of position or mask, and penalty term of prior consistency, and are normalized. A number of candidate results are output from high to low according to the comprehensive scores, and positioning information mapped to the original image or the original video frame coordinate is provided for each candidate.

[0074] The overlapping sliding window of block division can be realized by two layers of loop traversal of row and column indexes, and if there is an incomplete block at the boundary, mirror padding is used to fill it. The encoding network can use a structure containing multiple convolution and normalization layers, and output the vector form of the prior parameters. The data set in the training stage is prepared according to the real task scene, and the loss function contains the difference degree term and the observation consistency term. The optimization adopts a phased strategy, first freezes the discriminant scorer, and then converges the encoding network alone, and then performs joint fine-tuning.

[0075] The optical flow estimation can use a pyramid cascade structure to output the pixel displacement field and its credibility, and the occlusion detection is completed using forward and backward consistency checking. The preference score of the candidate action is composed of three parts in the implementation, which are multiplied or added respectively, and are derived from the prior center proximity, scale adaptation and shape matching degree; the exploration temperature adopts a time decay table, which is realized by querying a preset table according to the iteration step number or by generating an exponential decay function. The rough screening stage of delayed acceptance can use a fast approximate scorer, which inputs low-resolution blocks and low-dimensional features, and outputs a binary decision whether to enter the fine evaluation, and the fine evaluation calls the complete observation consistency scorer again.

[0076] The record structure of a sampling path is a sequential array, containing the action index, proposal probability, acceptance event and local quality score of each step. These entries are read from head to tail at the end of a path to calculate the sampling probability and write into the search tree node. The data structure of a search tree node includes a child node pointer array, visit count, accumulated sampling probability, accumulated quality score and search value. In parallel execution, atomic addition and lock segment strategy are used for node update to avoid data race, or local accumulation per thread and periodic merging are used. The memory management of the tree uses node pool and free list multiplexing to prevent fragmentation and jitter caused by frequent allocation.

[0077] The post-processing of candidate results includes non-maximum suppression and cross-block merging. The overlap metric of position proposals uses an intersection over union threshold to filter out redundant boxes. The mask results are merged in the overlapping region by pixel-level consistency voting, and the boundaries are refined. For video data, the candidate results of adjacent frames are associated in the time domain trajectory, and the short-lived false responses are removed using trajectory consistency score constraints. The final output is returned in a structured list form, and each result contains type, position or mask, comprehensive score, associated block identifier and frame index information.

[0078] In engineering deployment, batch processing and streaming concurrency strategies are used for large-scale image or video processing. Batch processing aggregates multiple blocks or multiple frames into a single forward and scoring call to reduce overhead. Streaming concurrency decouples through queuing, allowing variational prior generation, sampling path generation, search tree expansion and feature extraction to work in a pipelined manner in independent execution units. To ensure repeatability, all random processes are driven by a seedable pseudo-random number engine, and the seeding strategy combines the hash of the input data and the timestamp. To facilitate debugging and evaluation, the entire process outputs configurable intermediate products at key nodes, including prior parameter heat maps, sampling path coverage maps, search tree statistical summaries and candidate result confidence distributions, all in the form of structured files and visual images.

[0079] An application of a recognition task is illustrated. The input is a video segment containing multiple moving targets. According to the above process, each frame is divided into blocks and the prior is generated. In relatively clear texture blocks, the prior presents a relatively concentrated high support area, and the paths generated by sampling appear more frequently in these areas. The search tree quickly converges to a small number of high search value paths in these blocks, and the trajectory features extracted based on these paths remain stable in the time dimension. The final generated candidate position proposals are distributed around the real targets in space, and a small number of high confidence target boxes are output after non-maximum suppression.

[0080] On blocks with weak texture or occlusion, the prior is more dispersed, the sampling path explores more widely, the search tree keeps sufficient coverage by increasing the exploration weight, and the consistency constraint of adjacent frames is used to suppress noise. The output candidate result has low confidence but is still recorded completely for subsequent fusion. The entire process meets the implementation requirements of probability distribution function-based sampling and Monte Carlo tree search on picture or video data. The disclosed structure, data organization, and operation steps can be directly used by those skilled in the art to construct a corresponding prototype and put it into application.

[0081] In an optional implementation, a probability sampling space is constructed according to the prior probability distribution, a plurality of sampling paths are generated by performing Markov chain Monte Carlo sampling in the probability sampling space, and the sampling probability of each sampling path is calculated, comprising:

[0082] A probability density function is constructed for the prior probability distribution, the probability density function is smoothed by a kernel density estimation method on the prior probability distribution, a probability sampling space is constructed based on the probability density function, and the probability sampling space contains a support set of the probability distribution and a transition probability matrix;

[0083] A state transition network of a Markov chain is constructed in the probability sampling space, and the state transition network contains a state space and a state transition rule;

[0084] A random walk sequence is generated using the state transition rule, Monte Carlo sampling is performed based on the random walk sequence, and a plurality of sampling paths are generated, each sampling path corresponding to a transition trajectory in the state space;

[0085] The sampling probability of each sampling path is calculated based on the state transition probability and the stationary distribution; and the plurality of sampling paths and their corresponding sampling probabilities are output.

[0086] In processing the prior probability distribution, the original data point set is obtained as input, and these data points reflect the prior knowledge of the phenomenon to be modeled. For the input data points, a smooth probability density function is constructed using a kernel density estimation method. In the implementation process of kernel density estimation, a Gaussian kernel function is selected as the smoothing kernel, which places a bell-shaped curve at each data point. The determination of the bandwidth parameter adopts the least squares cross-validation criterion, and the bandwidth value that minimizes the error is selected by calculating the prediction error under different bandwidths. Specifically, the data set is divided into a training set and a validation set, the kernel density estimation of different bandwidths is constructed on the training set, the prediction error is calculated on the validation set, and the bandwidth corresponding to the minimum error is selected.

[0087] After the probability density function is constructed, the support set of the probability distribution is determined. The boundary of the support set is determined by analyzing the effective region of the probability density function. The density threshold is set to one thousandth of the total density peak value, and the region with a density value lower than the threshold is excluded from the support set. For multi-dimensional cases, the boundary range is determined in each dimension respectively, forming a multi-dimensional rectangular region as the support set. Discretization processing is performed within the support set, and a uniform grid division method is used to convert the continuous space into a discrete state space. The selection of grid spacing needs to balance the calculation accuracy and the calculation complexity, and is usually set to one thousandth to one percent of the range of the support set.

[0088] The construction of the transition probability matrix is based on the transition possibility between adjacent states. For each state point in the grid, the transition probability between it and all adjacent state points is calculated. The calculation of the transition probability considers two factors: the spatial distance between states and the ratio of target probability density. The spatial distance uses the Euclidean distance metric, and the transition probability between states with closer distance is higher. The probability density ratio reflects the tendency of transition from the current state to the target state, and the state with higher density is more likely to be transferred. By combining these two factors, the sum of the out-degree transition probability of each state is ensured to be one through normalization processing.

[0089] In the construction process of the Markov chain state transition network, the state space is composed of discretized grid points, and each grid point represents a state. The connection relationship between states is determined based on the adjacency rule, and the Von Neumann adjacency or Moore adjacency method is used. Von Neumann adjacency only considers the states directly adjacent to the current state in each coordinate axis direction, while Moore adjacency also includes the diagonal direction adjacent states. The choice of adjacency relationship affects the connectivity and sampling efficiency of the Markov chain, and Moore adjacency provides more transition options but has relatively high computational complexity.

[0090] The design of the state transition rule is based on the density information of the target distribution and the current state information. The transition rule includes two parts: proposal mechanism and acceptance mechanism. The proposal mechanism selects candidate states from the adjacent states of the current state, and the selection method can be uniform random selection or weighted selection based on transition probability. The acceptance mechanism decides whether to accept the proposed state according to the ratio of target probability density. Specifically, the density ratio of the proposed state and the current state is calculated, and when the ratio is greater than or equal to one, it is directly accepted, and when the ratio is less than one, it is randomly decided as the acceptance probability.

[0091] The generation of random walk sequences starts from an initial state, which can be randomly selected from the support set or chosen as a state with high probability density. In each iteration, the next state is generated based on the current state and the state transition rule. To ensure the Markov chain's memoryless property, the selection of the next state only depends on the current state and is independent of the historical state sequence. The iteration process continues until the preset sequence length is reached or the convergence condition is met.

[0092] Monte Carlo sampling generates multiple sampling paths by running the random walk process multiple times independently, each time using a different random seed to ensure independence between paths. The number of sampling paths is determined by the required statistical accuracy, usually hundreds to thousands of paths are needed to obtain stable statistical results. Each path records the complete state transition sequence from the initial state to the final state, and the path length needs to be long enough to ensure that the sampling chain reaches the stationary distribution.

[0093] The quality assessment of sampling paths is performed through multiple statistical indicators. The stationarity test compares the statistical properties of the first half and the second half of the path, including mean, variance, and distribution shape. If the statistical properties of the two parts are significantly different, it indicates that the chain has not yet reached a stationary state, and the path length needs to be increased or the sampling parameters need to be adjusted. The mixing test evaluates the statistical difference between multiple paths generated from different starting points. Good mixing performance is characterized by the convergence of paths from different starting points to similar statistical properties.

[0094] The calculation of sampling probability is based on the probability of each state transition in the path and the probability of each state in the stationary distribution. For a sampling path containing several states, the transition probability between adjacent states is calculated one by one. These transition probabilities can be directly obtained from the pre-constructed transition probability matrix. The stationary distribution probability is calculated by evaluating the density value of each state in the target probability density function. The density value needs to be normalized to ensure the probability property. The total sampling probability of the path is obtained by multiplying all the transition probabilities and the stationary distribution probability. To avoid numerical underflow problems in numerical calculations, the actual calculation uses logarithmic probability form for accumulation.

[0095] The output format of the sampling path includes the state sequence of the path and the corresponding sampling probability. The state sequence is stored in an array, recording the state identifier or state coordinates at each time in the path. The sampling probability is given in floating-point form, reflecting the relative importance of the path among all paths. To facilitate subsequent analysis, the output result also includes statistical summary information of the path, such as path length, state access frequency, and average state value. These statistical information helps to evaluate the sampling quality and perform path screening, ensuring that the paths used for subsequent reasoning have good representativeness and reliability.

[0096] In an alternative embodiment, each node of the Monte Carlo search tree contains a state value function and an action policy function, the state value function and the action policy function of the node are iteratively updated in an asynchronous parallel manner, and a search path with the maximum search value is selected for feature extraction, including:

[0097] A state value function and an action policy function are configured at each node of the Monte Carlo search tree, wherein the state value function calculates a value evaluation score of the node state using a residual network structure, and the action policy function generates an action selection probability distribution of the node based on a double policy optimization criterion.

[0098] A plurality of parallel work processes are started, a confidence upper bound value of the node is calculated, a child node with the maximum confidence upper bound value is selected for expansion, the state value function and the action policy function of the node are updated using an asynchronous stochastic gradient descent algorithm, and the updated value evaluation score and the action selection probability distribution are broadcast to other work processes through an asynchronous message passing mechanism.

[0099] The Monte Carlo search tree is iteratively searched based on a preset search depth, and a search path with the maximum search value is selected in each iteration, wherein the search value is a weighted value of the value evaluation score and the action selection probability distribution after normalization.

[0100] As shown in Figure 2 , the method comprises:

[0101] During the configuration of the nodes of the Monte Carlo search tree, each node is assigned two core function components: a state value function and an action policy function. The state value function is implemented using a residual network structure, which contains multiple residual blocks, each of which is composed of a convolution layer, a batch normalization layer, and an activation function, and the input is directly transmitted to the output end through a skip connection. Specifically, the input state information is feature-extracted by an initial convolution layer, and then deep feature learning is performed through sixteen residual blocks, each of which contains two three-by-three convolution layers with two hundred and fifty-six convolution kernels. The skip connection fuses the input and the convolution output of the block through element-wise addition, effectively alleviating the gradient vanishing problem of deep networks. The final output layer of the network uses a fully connected layer structure to convert feature mapping into a single value evaluation score, which reflects the expected return under the current state.

[0102] The action policy function is designed based on a dual policy optimization criterion, including an exploration policy and an exploitation policy. The exploration policy encourages the algorithm to explore the insufficiently accessed action space by introducing a noise mechanism. The noise is generated using a Dirichlet distribution, and the distribution parameters are adjusted according to the dimensions of the action space. The exploitation policy selects the optimal action based on historical access statistics and value evaluation results. The merits of each action are evaluated by calculating the cumulative reward and the number of accesses. The fusion of the dual policy is achieved through an adaptive weight mechanism. The weight is dynamically adjusted according to the number of accesses. When the number of accesses is small, the weight is biased towards the exploration policy. When the number of accesses is large, the weight is biased towards the exploitation policy. The output of the policy function is a probability distribution. Each action corresponds to a selection probability, and the sum of all probabilities is one.

[0103] The start of the parallel work process uses a multi-thread or multi-process architecture. The number of processes is determined according to the configuration of computing resources, and is usually set to one to two times the number of processor cores. Each work process maintains an independent copy of the search tree and is responsible for the update calculation of a specific subtree. Inter-process communication is achieved through shared memory or message queue mechanisms to ensure coordination and synchronization between different processes. Load balancing of work processes is achieved through a dynamic task allocation algorithm, which monitors the computational load of each process and allocates new computational tasks to processes with lighter loads.

[0104] The calculation of the confidence upper bound value considers both the value evaluation and the exploration potential of the node. During the calculation process, the average value evaluation score of the node is added to the exploration reward to obtain the confidence upper bound value. The exploration reward is calculated based on the number of accesses of the parent node and the number of accesses of the current node. The square root of the access frequency ratio is multiplied by a preset exploration constant to obtain the exploration reward. The value of the exploration constant affects the degree of exploration of the algorithm. A larger constant value encourages more exploration, while a smaller constant value favors the use of known information. The child node with the largest confidence upper bound value is selected for further expansion. If the node is a leaf node, a new child node is created. If there is already a child node, the search continues downward.

[0105] The implementation of the asynchronous stochastic gradient descent algorithm uses a mini-batch gradient update method. Each work process maintains an independent gradient accumulation buffer. When the buffer reaches a preset size, parameter updating is triggered. Gradient calculation is based on the error between the current node's state information and the target value. The gradient of the state value function is calculated using the mean square error loss function, and the gradient of the action policy function is calculated using the cross-entropy loss function. The learning rate is adjusted adaptively based on historical gradient statistics. When the gradient changes dramatically, the learning rate is reduced. When the gradient changes smoothly, the learning rate is increased. The parameter update uses a momentum mechanism to combine historical gradient information and current gradient information for weighted averaging, improving the stability of parameter updating.

[0106] The asynchronous message passing mechanism realizes parameter synchronization in an event-driven manner. When a certain work process completes parameter updating, the updated parameters are encapsulated as a message and broadcast to other processes. The message content includes node identification, updated value evaluation score, action selection probability distribution, and update timestamp. The receiving process determines the age of the message according to the timestamp and only accepts messages that are more updated than the local parameters. To avoid message storm, a flow control mechanism is used to control the message sending frequency, and a minimum update interval is set to prevent frequent small updates from causing network congestion. The message passing adopts a non-blocking manner, and the sending process does not wait for the receiving confirmation to continue the calculation task, thereby improving the overall calculation efficiency.

[0107] The iterative search process is controlled based on a preset search depth, which is usually set to ten to fifty layers and adjusted according to the problem complexity and computing resources. Each iteration starts from the root node and searches downward along the path with the largest search value until it reaches the leaf node or the search depth limit. All nodes passed during the search process are recorded to form a complete search path. When reaching the leaf node, if it is a terminal state, the state value is directly returned, and if it is a non-terminal state, the value estimate is obtained through neural network evaluation.

[0108] The calculation of the search value combines the value evaluation score and the action selection probability distribution after normalization. The normalization of the value evaluation score is achieved by the max-min standardization method, which maps the score to the interval of zero to one. The normalization of the action selection probability distribution ensures that the sum of all action probabilities is one, and the entropy value of the distribution is calculated as a measure of the uncertainty of the strategy. The weighted combination adopts a linear weighting method, with the weight of the value evaluation score set to 0.7 and the weight of the action probability entropy set to 0.3. The weights can be adjusted according to specific application scenarios. A larger search value indicates a higher potential value of the path, and the path with the largest search value is preferentially selected for further exploration.

[0109] The feature extraction process extracts key information from the selected search path for subsequent decision-making. The extracted features include the state representation of each node on the path, the action sequence, the value evaluation trend, and the strategy probability distribution features. The state representation uses vector encoding to convert complex state information into numerical vectors. The action sequence records all actions performed on the path, forming an action trajectory. The value evaluation trend is obtained by calculating the value difference between adjacent nodes, reflecting the change pattern of the value function during the search process. The strategy probability distribution features include the mean, variance, and entropy of the distribution, describing the concentration and uncertainty of the strategy. These extracted features provide important information support for subsequent decision optimization and strategy improvement, helping the algorithm continuously improve the search strategy and value evaluation accuracy.

[0110] In an alternative embodiment, configuring a state value function and an action policy function at each node of the Monte Carlo search tree comprises:

[0111] A multi-layer residual connection network is constructed, in which a skip connection path is arranged for fusing feature representations of different levels; a value evaluation score of a node state is calculated through feature fusion;

[0112] An action transition probability matrix is constructed using the value evaluation score, which describes the probability distribution of node state transition;

[0113] Two optimization objectives of online policy gradient and offline value estimation are set for the action transition probability matrix, and a double policy iteration method is used to alternately optimize the action selection policy to generate an action selection probability distribution of the node.

[0114] A multi-layer residual connection network structure is constructed for each node of the Monte Carlo search tree, which consists of an input layer, multiple residual blocks and an output layer. Each residual block contains two convolutional layers and a batch normalization layer, and a skip connection path is arranged to fuse feature representations of different levels. Specifically, the input layer receives a node state representation, which can be a feature tensor of size 8x8x16 representing the current game state. The network contains 20 residual blocks, and the first convolutional layer of each residual block uses 256 3x3 convolutional kernels, and the second convolutional layer also uses 256 3x3 convolutional kernels. In each residual block, the input features are processed by the first convolutional layer and the batch normalization layer, then processed by the second convolutional layer, and then added to the original input features to form a skip connection structure. This design allows the network to retain the feature information of the early layers during forward propagation, effectively alleviating the gradient vanishing problem in deep network training.

[0115] In order to fuse feature representations of different levels, the present embodiment sets additional skip connection paths after the 5th, 10th and 15th residual blocks. These paths reduce the dimensionality of the output features of the corresponding residual blocks through a 1x1 convolutional layer, and then perform a concatenation operation with the output features of the final residual block. For example, the output features of the 5th residual block are processed by a convolutional layer with 128 1x1 convolutional kernels to obtain a feature map of size 8x8x128; similarly, the outputs of the 10th and 15th residual blocks are also processed in a similar manner. Finally, these three processed feature maps are concatenated with the output features of the 20th residual block (size 8x8x256) to form a fused feature representation of size 8x8x640.

[0116] When calculating the value evaluation score of the node state by feature fusion, the embodiment adopts two branch networks: a value evaluation branch and a policy branch. The value evaluation branch first reduces the dimension of the fused features through a convolutional layer with 128 1x1 convolutional kernels to obtain a feature map with a size of 8x8x128. Then, the feature map is converted into a 128-dimensional vector using global average pooling, and the vector is processed by two fully connected layers. The first fully connected layer contains 64 neurons and uses a ReLU activation function, and the second fully connected layer outputs a scalar value representing the value evaluation score of the current node state. For example, for a medium advantage state, the value evaluation score is 0.65, indicating that the current player has a 65% chance of winning.

[0117] When constructing the action transition probability matrix using the value evaluation score, the policy branch processes the same fused features through a convolutional layer with 256 1x1 convolutional kernels to obtain a feature map with a size of 8x8x256. The feature map is then processed by a convolutional layer with an output channel number equal to the number of actions (for example, for Go, the output channel number is 362, representing 361 board positions plus one Pass action). The resulting feature map is flattened into a vector and normalized by the Softmax function to generate the action transition probability matrix. For example, in a specific state, the probability of placing a piece at a certain key position is 0.15, while the probabilities of other suboptimal positions are 0.08, 0.07, etc., and the sum of the probabilities of all positions is 1.

[0118] To set the online policy gradient and offline value estimation as two optimization objectives, the embodiment adopts a double policy iteration method for alternating optimization. First, the online policy gradient optimization generates training data by self-play. Each game generates a series of state-action pairs and the final game result. For each state st, record the actual selected action at and the final game result z (1 indicates victory and -1 indicates defeat). The difference between the action probability distribution predicted by the policy network π(a|st) and the actual selected action at, and the difference between the predicted state value v(st) and the actual game result z are used to calculate the loss. Specifically, the total loss function includes policy loss (using cross-entropy) and value loss (using mean square error), as well as a regularization term. For example, for a batch size of 256 training samples, the policy loss weight is set to 1.0, the value loss weight is set to 0.5, and the L2 regularization coefficient is set to 0.0001.

[0119] The offline value estimation optimization is trained using data enhanced by Monte Carlo tree search. During self-play, each state st records not only the actual selected action at and the final game result z, but also the strategy πMCTS(a|st) obtained by Monte Carlo tree search. During training, the action probability distribution π(a|st) predicted by the strategy network is compared with the entire strategy distribution πMCTS(a|st) instead of the single action at actually selected, and the KL divergence is used as the loss function. For example, for the same state, the initial strategy network assigns a probability of 0.12 to a certain position, while after 1600 Monte Carlo simulations, the probability of that position becomes 0.25. The network learns a better strategy by minimizing this difference.

[0120] Through the alternating optimization of online policy gradient and offline value estimation, the network parameters are constantly updated to generate more and more accurate node state value evaluation and action selection probability distribution. In practical applications, the prediction accuracy of the strategy network of the method of the embodiment is improved from 35% initially to 67% after 10000 self-play training, and the mean square error of state value estimation is reduced from 0.42 to 0.18, significantly improving the decision quality and efficiency of the Monte Carlo search tree.

[0121] In an optional implementation, a confidence upper bound value of the node is calculated, a child node with the largest confidence upper bound value is selected for expansion, and the state value function and the action policy function of the node are updated using an asynchronous stochastic gradient descent algorithm, including:

[0122] Obtain state information of the node to be expanded, and extract a feature vector from the state information;

[0123] Construct a child node set according to the feature vector, and count a historical access frequency and a historical cumulative return of each child node;

[0124] Calculate a confidence interval parameter based on the historical access frequency and the historical cumulative return, and input the confidence interval parameter into an adaptive estimator;

[0125] Perform Thompson sampling using the adaptive estimator to obtain a confidence upper bound value of each child node in the current state, and the confidence upper bound value is used to balance exploration and exploitation in the node expansion process;

[0126] Sort the child nodes according to the confidence upper bound values, and select a child node with the largest confidence upper bound value as an expansion target node;

[0127] Map the feature vector of the expansion target node to a value space and a policy space, and construct a distributed computing network;

[0128] Asynchronous stochastic gradient descent operation is performed in the distributed computing network, and state value function and action policy function are optimized in parallel;

[0129] The state representation of the extended target node is updated based on the optimization result, and the updated state value function and action policy function are output.

[0130] During the state information acquisition process of the to-be-extended node, the complete description information of the current state is read from the node storage structure. The state information includes the physical configuration of the environment, the historical action sequence, the time step information, and other related context data. For a chess game scenario, the state information includes the chessboard configuration matrix, the current round, the number of steps executed, etc. For a path planning scenario, the state information includes the current position coordinates, the obstacle distribution, the target position, etc. The encoding of the state information adopts a multi-level structure, and different types of information are numerically represented to form a unified state vector format.

[0131] The extraction of the feature vector is realized through multi-dimensional feature engineering. Spatial features are extracted through a convolutional neural network to convert two-dimensional or three-dimensional state representations into fixed-length feature vectors. Time series features are extracted through a recurrent neural network or a long short-term memory network to capture the time dependence in the state evolution process. Statistical features are obtained by calculating the mean, variance, maximum, minimum, and other statistics of the state vector. Semantic features are mapped from discrete state elements to continuous vector representations through a pre-trained embedding model. All feature vectors are combined into the final state feature representation through concatenation or weighted fusion. The dimension of the feature vector is usually set to between one hundred and twenty-eight and five hundred and twelve.

[0132] The construction of the child node set is based on the legal action space under the current state. For each executable action, a corresponding child node is created and the state after executing the action is calculated. The state of the child node is calculated through a state transition function, which determines the next state based on the current state and the executed action. During the construction process, the legality of the action needs to be verified to exclude actions that violate environmental constraints or game rules. The identification of the child node is generated through a state hash value to ensure that nodes with the same state have unique identifiers and avoid duplication.

[0133] The statistics of the historical access count and the historical cumulative return are realized through access counters and return accumulators. Each node maintains independent statistical information, including the total number of accesses, the immediate return obtained from each access, the cumulative return sum, and the average return. The update of the access count uses atomic operations to ensure data consistency during concurrent access. The record of the return information contains detailed timestamps and source information, which facilitates subsequent statistical analysis and anomaly detection. For newly created child nodes, the initial access count is set to zero, the initial cumulative return is set to zero, and the average return is set to a preset default value.

[0134] The calculation of the confidence interval parameter is based on the Bayesian statistical principle. For each sub-node, the parameter estimation of the revenue distribution is calculated according to the historical access times and the cumulative revenue. Assuming that the revenue follows a normal distribution, the distribution parameters are estimated by the sample mean and sample variance. The sample mean is equal to the cumulative revenue divided by the access times, and the sample variance is calculated by the sum of the squares of the revenue deviations divided by the access times minus one. The width of the confidence interval is inversely proportional to the square root of the access times. The more access times, the narrower the confidence interval, reflecting the degree of certainty of the revenue estimate. The confidence level is set to ninety-five percent, and the corresponding critical value is obtained by table lookup or numerical calculation.

[0135] The construction of the adaptive estimator uses the Thompson sampling framework of the multi-armed bandit. The estimator maintains the prior parameters of the revenue distribution of each sub-node, including the prior distribution parameters of the mean and the variance. The prior distribution uses the normal-inverse-gamma distribution, which is the conjugate prior of the normal distribution, facilitating Bayesian updating. The adaptivity of the estimator is reflected in dynamically adjusting the prior parameters according to new observation data, so that the estimation result better reflects the actual revenue distribution. Parameter updating uses the Bayesian rule to combine the prior distribution with the observation likelihood to obtain the posterior distribution.

[0136] In the execution process of Thompson sampling, a sample is randomly drawn from the posterior distribution of each sub-node as the estimated revenue of that node. The sampling process uses standard random number generation algorithms to ensure the randomness and statistical properties of the sample. For normal distribution, Box-Muller transformation or Ziggurat algorithm is used to generate normal random numbers. The estimated revenue value obtained by sampling is used as the confidence upper bound value of the sub-node in the current state, reflecting the expected revenue considering uncertainty. The calculation of the confidence upper bound value also considers the exploration reward term, which is proportional to the logarithm of the node access times, encouraging the algorithm to explore less visited nodes.

[0137] The sorting of sub-nodes is based on the confidence upper bound value in descending order. The sorting algorithm uses quicksort or mergesort to ensure that the time complexity of sorting is linear logarithmic level. During the sorting process, if the confidence upper bound values of two nodes are equal, a secondary sorting criterion is used to distinguish them, such as the creation time of the node, the lexicographic order of the node identifier, etc. After sorting, the first node is selected as the expansion target node, which has the largest confidence upper bound value, representing the current most potential exploration direction.

[0138] The mapping of the feature vector to the value space and the policy space is achieved through neural network transformations. The value space mapping uses a fully connected network to map the feature vector to a scalar value output. The network structure includes three to five hidden layers, each containing two hundred fifty-six to five hundred twelve neurons, with ReLU or its variants as the activation function. The policy space mapping also uses a fully connected network, but with an output dimension equal to the size of the action space, and the output layer uses a softmax activation function to ensure that the output is a probability distribution. The two mapping networks can share the feature extraction layers in the early stages, reducing the number of parameters and improving training efficiency.

[0139] The construction of the distributed computing network uses a parameter server architecture or a decentralized ring communication architecture. In the parameter server architecture, a dedicated parameter server node is set up to store and update global parameters, and worker nodes obtain the latest parameters from the parameter server and upload gradient updates. In the decentralized architecture, each computing node maintains a copy of the parameters, and parameter synchronization is achieved through communication between neighboring nodes. The fault tolerance of the network is ensured through redundancy design and fault detection mechanisms, so that when a node fails, other nodes can continue to work normally.

[0140] The implementation of asynchronous stochastic gradient descent uses an asynchronous parallel strategy, where each worker process independently calculates the gradient and asynchronously updates the parameters without waiting for other processes to complete the calculation. Gradient calculation is based on small batch samples, and the batch size is determined according to memory capacity and computing power, usually set between thirty-two and two hundred fifty-six. The learning rate uses an adaptive adjustment strategy, such as the Adam or RMSprop algorithm, which dynamically adjusts the learning rate of each parameter based on historical statistical information of the gradient. To avoid the problem of gradient explosion, gradient clipping technology is used to limit the length of the gradient to a pre-set threshold.

[0141] The parallel optimization of state value function and action policy function uses a multi-task learning framework, where the two functions share the underlying feature extraction network but use different output heads and loss functions. The optimization goal of the state value function is to minimize the mean square error between the predicted value and the actual return, and the optimization goal of the action policy function is to maximize the policy gradient or minimize the cross-entropy loss. The loss functions of the two tasks are combined through weighted summation, and the weight parameters are adjusted according to the importance of the task and the convergence speed. During optimization, an alternating update strategy is used, where one task is randomly selected for parameter update in each iteration to balance the learning progress of the two tasks.

[0142] The state representation update of the extended target node is based on the optimized function parameters. During the update process, the new state value function is applied to the feature vector of the node, and the updated value estimate is calculated. After the action policy function is applied to the feature vector, the updated action selection probability distribution is obtained. The statistical information of the node also needs to be updated accordingly, including the average value, access count, last update time, etc. The updated state value function and action policy function are stored in a serialized format for subsequent retrieval and use. At the same time, the update information is transmitted to the relevant parent and child nodes through the message passing mechanism to maintain the consistency of the search tree structure.

[0143] In an optional implementation, a feature semantic graph is constructed based on the plurality of candidate recognition results, a node vector representation is calculated by iteratively propagating the feature semantic graph through a graph neural network, a confidence score of each candidate recognition result is calculated based on the node vector representation, and a candidate recognition result with the highest confidence score is selected as a final recognition result.

[0144] The plurality of candidate recognition results are projected into a semantic feature space to obtain semantic feature representations, and a feature semantic graph is constructed in the semantic feature space, wherein each candidate recognition result corresponds to a node in the feature semantic graph, and an edge connection relationship is dynamically constructed based on feature similarity between nodes;

[0145] A plurality of rounds of information propagation are performed on the feature semantic graph, and each round of propagation includes: calculating importance weights of a neighborhood of a node, aggregating neighborhood node information based on the importance weights, nonlinearly fusing the aggregated information with center node information, controlling an information update ratio through a gating mechanism, iteratively performing until node states converge, and obtaining a final node vector representation;

[0146] A confidence score of each candidate recognition result is calculated based on the node vector representation, and a candidate recognition result with the highest confidence score is selected as a final recognition result.

[0147] During the projection of the candidate recognition results into the semantic feature space, a pre-trained language model is used to encode each candidate result. For text type candidate results, the text is converted into a word vector sequence through a word embedding layer, and then a context-related semantic representation is extracted through a bidirectional long short-term memory network or a transformer encoder. The output of the encoder is aggregated into a fixed-dimensional vector representation through average pooling or attention pooling mechanism, and the vector dimension is set to five hundred and twelve or seven hundred and sixty-eight. For image type candidate results, a convolutional neural network is used to extract visual features, and the network structure includes multiple convolutional blocks and pooling layers, and finally a feature vector is obtained through global average pooling. The construction of the semantic feature space maps high-dimensional features to a low-dimensional space through principal component analysis or t-SNE dimension reduction technology, and maintains the relative distance relationship between the features.

[0148] The construction of the feature semantic graph is based on the semantic similarity relationship between the candidate recognition results. In the graph structure, each candidate recognition result is a node in the graph, and the initial feature vector of the node is set as the projection representation in the semantic feature space. The edge connection relationship between nodes is determined by calculating the cosine similarity between the feature vectors. If the similarity between a pair of nodes exceeds a predetermined threshold, an edge connection is established between them. The threshold is determined by a cross-validation method, and is usually set to between 0.5 and 0.8. For node pairs with high similarity, the weight of the edge is set to the similarity value. For node pairs with low similarity, no edge connection is established to reduce the complexity of the graph. The adjacency matrix of the graph is stored in a sparse matrix format to improve storage efficiency and computing performance.

[0149] The construction of the dynamic edge connection relationship adopts an adaptive threshold adjustment mechanism. In the early stage of graph construction, a lower similarity threshold is used to ensure the connectivity of the graph. As the information propagation proceeds, the threshold is gradually increased to filter weakly related connections. The adjustment of the threshold is based on the connectivity indicators and the degree distribution of the nodes. When the graph has isolated nodes or connected components, the threshold is lowered to increase the connections. When the graph is too dense to affect the computing efficiency, the threshold is increased to reduce the connections. The update of the edge weight adopts an exponential decay mechanism, and the newly calculated similarity is weighted and averaged with the historical weight. The decay factor is set to 0.9.

[0150] In the multi-round iteration process of information propagation, each round of propagation includes four steps: neighborhood importance calculation, information aggregation, nonlinear fusion, and gated update. The calculation of the neighborhood importance weight is based on the attention mechanism. For each neighbor node of the center node, the attention score of its feature vector and the center node feature vector is calculated. The attention score is calculated by a multi-head attention mechanism, which includes linear transformations of query vectors, key vectors, and value vectors. The query vector is generated from the center node feature, and the key vector and the value vector are generated from the neighbor node feature. The multi-head attention uses eight attention heads, each with a dimension of sixty-four. Different heads focus on different aspects of the features.

[0151] The aggregation process of the neighborhood node information is realized by weighted summation. For each neighbor of the center node, the feature vector of the neighbor node is multiplied by the corresponding importance weight, and then all the weighted feature vectors are summed to obtain the aggregated information. In the aggregation process, the influence of the edge weight is considered, and the attention weight is multiplied by the edge weight as the final aggregation weight. In order to maintain numerical stability, the aggregation weight is normalized to ensure that the weight sum is one. The dimension of the aggregated information is consistent with that of the node feature vector, which facilitates subsequent fusion operations.

[0152] The nonlinear fusion process combines the aggregated neighborhood information with the original information of the center node. The fusion operation is implemented through a multi-layer perceptron, which includes two hidden layers and an output layer. The number of neurons in the hidden layer is set to twice the feature dimension, and the activation function uses ReLU or GELU function. The input layer receives the concatenated feature vector, which includes the center node features and the aggregated neighborhood features. The parameters of the fusion network are trained through the backpropagation algorithm, and the loss function uses mean square error or cross-entropy loss. The fused feature vector serves as the intermediate representation of the node and is used for subsequent gated updates.

[0153] The implementation of the gating mechanism uses the structure of the gated recurrent unit, which includes two components: the update gate and the reset gate. The update gate controls the degree of influence of new information on the node state, and the reset gate controls the degree of preservation of historical information. The calculation of the gate is implemented through a sigmoid activation function, with the input being the current node feature and the concatenated feature vector after fusion. The output value of the update gate is between zero and one, with a larger value indicating more use of new information and a smaller value indicating more preservation of historical information. The role of the reset gate is to selectively forget historical information when calculating the candidate update, avoiding information redundancy and gradient vanishing problems.

[0154] The convergence judgment of the node state is based on the change amplitude of the node feature vector between consecutive two iterations. For each node, the Euclidean distance between the current round and the previous round feature vector is calculated, and when the distance of all nodes is less than the preset threshold, it is considered to have reached convergence. The convergence threshold is usually set to between zero point zero one and zero point zero zero one, and the specific value is determined according to the application accuracy requirement. To avoid infinite iteration, the maximum number of iterations is limited, usually between ten and fifty rounds. The convergence check is performed at the end of each round of propagation, and the propagation process is stopped when the convergence condition is reached or the maximum number of iterations is reached.

[0155] The final node vector representation is obtained through the node feature vector after multiple rounds of information propagation. The converged node vector contains information about all related nodes in the graph, representing the relative position and importance of the candidate recognition result in the entire candidate set. The quality of the node vector is evaluated through visual analysis and clustering analysis. High-quality node vectors should make semantically similar candidate results closer in vector space, and semantically different candidate results farther apart.

[0156] The confidence score is calculated based on the node vector representation and a pre-trained confidence evaluation network. The confidence evaluation network uses a multi-layer perceptron structure, with the input being the node vector representation and the output being a confidence score between zero and one. The network contains three hidden layers, each containing two hundred fifty-six neurons, with the activation function using ReLU. The output layer uses a sigmoid activation function to ensure that the output is within a reasonable range. The training data for the network comes from historical recognition tasks, containing the feature representation of the candidate result and the corresponding true confidence label.

[0157] The calculation of the confidence score also takes into account the topological location information of the candidate results in the graph, nodes with higher centrality usually have higher confidence because they are connected to more relevant candidate results. The calculation of centrality uses a weighted combination of degree centrality and betweenness centrality, degree centrality reflects the number of direct connections of a node, and betweenness centrality reflects the role of a node as a bridge in the graph. The weights are determined by experimental verification, usually the degree centrality weight is 0.6 and the betweenness centrality weight is 0.4.

[0158] The selection of the final recognition result is based on the ranking of the comprehensive confidence scores of the candidate results, and the candidate result with the highest confidence score is selected as the final recognition result output. In the case of equal scores, a secondary selection criterion is used, such as the original score, generation time or lexicographic order of the candidate result. The selection process also includes a reasonableness check to verify whether the final result meets the basic constraint conditions of the task, such as format requirements, length limits, etc. After the check passes, the final recognition result and its corresponding confidence score are output, providing a reference basis for subsequent application and analysis.

[0159] The second aspect of the embodiment of the present application provides an electronic device, comprising:

[0160] a processor;

[0161] a memory for storing processor-executable instructions;

[0162] The processor is configured to call the instructions stored in the memory to execute the method described above.

[0163] The third aspect of the embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the method described above.

[0164] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon for executing various aspects of the present application.

[0165] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A picture and video recognition and analysis method based on end-side multi-modal large model, characterized in that, The method comprises the following steps: inputting picture or video data to be processed into a pre-deployed multimodal large model on the device side, calculating the sampling probability of multiple sampling paths based on a probability distribution function, and performing Monte Carlo tree search on the picture or video data according to the sampling probability, which comprises the following steps: dividing the picture or video data into multiple image blocks, calculating the prior probability distribution of each image block based on a variational inference method, constructing a probability sampling space according to the prior probability distribution, performing Markov chain Monte Carlo sampling in the probability sampling space to generate multiple sampling paths, and calculating the sampling probability of each sampling path; constructing a Monte Carlo search tree for the picture or video data based on the sampling probability, selecting a search path with the maximum search value of the Monte Carlo search tree for feature extraction, and generating multiple candidate recognition results; Each node of the Monte Carlo search tree contains a state value function and an action policy function, and the state value function and the action policy function of the node are iteratively updated in an asynchronous parallel manner, a search path with the maximum search value is selected for feature extraction, and multiple candidate recognition results are generated; Based on the multiple candidate recognition results, a feature semantic graph is constructed, a node vector representation is calculated by iteratively propagating the feature semantic graph through a graph neural network, a confidence score of each candidate recognition result is calculated based on the node vector representation, and a candidate recognition result with the highest confidence score is selected as the final recognition result; The final recognition result is output to the display interface of the device side, and image scene information, target detection information and action analysis information are correspondingly presented in the display interface.

2. The method of claim 1, wherein, According to the prior probability distribution, a probability sampling space is constructed, Markov chain Monte Carlo sampling is performed in the probability sampling space to generate multiple sampling paths, and the sampling probability of each sampling path is calculated, which comprises the following steps: A probability density function is constructed for the prior probability distribution, and the probability density function is smoothed by a kernel density estimation method; a probability sampling space is constructed based on the probability density function, and the probability sampling space contains a support set of the probability distribution and a transition probability matrix; A state transition network of the Markov chain is constructed in the probability sampling space, and the state transition network contains a state space and a state transition rule; A random walk sequence is generated using the state transition rule, Monte Carlo sampling is performed based on the random walk sequence, and multiple sampling paths are generated, each of which corresponds to a transition trajectory in the state space; The sampling probability of each sampling path is calculated based on the state transition probability and the stationary distribution; and the multiple sampling paths and their corresponding sampling probabilities are output.

3. The method of claim 1, wherein, Each node of the Monte Carlo search tree contains a state value function and an action policy function, and the state value function and the action policy function of the node are iteratively updated in an asynchronous parallel manner, a search path with the maximum search value is selected for feature extraction, which comprises the following steps: The state value function and the action policy function are configured at each node of the Monte Carlo search tree, wherein the state value function adopts a residual network structure to calculate a value evaluation score of a node state, and the action policy function generates an action selection probability distribution of the node based on a double policy optimization criterion; A plurality of parallel working processes are started to calculate a confidence upper bound value of the node, a child node with the largest confidence upper bound value is selected for expansion, an asynchronous stochastic gradient descent algorithm is used to update the state value function and the action policy function of the node, and the updated value evaluation score and the action selection probability distribution are broadcast to other working processes through an asynchronous message passing mechanism; The Monte Carlo search tree is iteratively searched based on a preset search depth, and a search path with the largest search value is selected in each iteration, wherein the search value is a weighted value of the value evaluation score and the action selection probability distribution after normalization.

4. The method of claim 3, wherein, The state value function and the action policy function are configured at each node of the Monte Carlo search tree, wherein the state value function adopts a residual network structure to calculate a value evaluation score of a node state, and the action policy function generates an action selection probability distribution of the node based on a double policy optimization criterion; A multi-layer residual connection network is constructed, a jump connection path is arranged in the residual connection network, the jump connection path is used to fuse feature representations of different levels, and a value evaluation score of a node state is calculated through feature fusion; An action transition probability matrix is constructed using the value evaluation score, and the action transition probability matrix describes a probability distribution of node state transition; Two optimization objectives of online policy gradient and offline value estimation are set for the action transition probability matrix, a double policy iteration method is used to alternately optimize an action selection policy, and an action selection probability distribution of the node is generated.

5. The method of claim 3, wherein, The confidence upper bound value of the node is calculated, the child node with the largest confidence upper bound value is selected for expansion, and the state value function and the action policy function of the node are updated using an asynchronous stochastic gradient descent algorithm, which includes: State information of a node to be expanded is obtained, and a feature vector in the state information is extracted; A child node set is constructed according to the feature vector, and the historical access frequency and the historical cumulative income of each child node are counted; A confidence interval parameter is calculated based on the historical access frequency and the historical cumulative income, and the confidence interval parameter is input into an adaptive estimator; Thompson sampling is performed using the adaptive estimator to obtain a confidence upper bound value of each child node in the current state, and the confidence upper bound value is used to balance exploration and utilization in the node expansion process; The child nodes are sorted according to the confidence upper bound values, and a child node with the largest confidence upper bound value is selected as an expansion target node; The feature vector of the expansion target node is mapped to a value space and a policy space, and a distributed computing network is constructed; An asynchronous stochastic gradient descent operation is performed in the distributed computing network to optimize the state value function and the action policy function in parallel; The state representation of the expansion target node is updated based on the optimization result, and the updated state value function and the action policy function are output.

6. The method of claim 1, wherein, constructing a feature semantic graph based on the plurality of candidate recognition results, performing iterative propagation calculation on the feature semantic graph by a graph neural network to obtain node vector representation, calculating confidence scores of each candidate recognition result according to the node vector representation, and selecting a candidate recognition result with the highest confidence score as a final recognition result, comprising: projecting the plurality of candidate recognition results to a semantic feature space to obtain semantic feature representation, constructing a feature semantic graph in the semantic feature space, wherein each candidate recognition result corresponds to a node in the feature semantic graph, and a connection relationship between nodes is dynamically constructed based on feature similarity between nodes; performing multiple rounds of information propagation on the feature semantic graph, each round of propagation comprising: calculating importance weights of a node neighborhood, aggregating neighborhood node information based on the importance weights, nonlinearly fusing the aggregated information with center node information, controlling information update proportion through a gating mechanism, iteratively performing until node state converges, and obtaining final node vector representation; calculating confidence scores of each candidate recognition result according to the node vector representation; and selecting a candidate recognition result with the highest confidence score as a final recognition result.

7. An electronic device, comprising: comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any one of claims 1 to 6.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 6.

Images