335 results about "Multi layered perceptron" patented technology

A method and apparatus for reconstructing a three-dimensional (3D) scene from a two-dimensional (2D) input image of the scene using a fully-differentiable transformer-based encoder-decode. A 2D input image encoded into a set of image features using a pre-trained vision transformer model, wherein the vision transformer model is pre-trained with multi-view RGB image supervision and point cloud supervision. The set of image features is projected onto a 3D triplane representation using a transformer decoder to obtain output triplane tokens. A triplane representation is created from the tokens and queried. 3D point features of color and density for volumetric rendering re predicted using a multi-layer perceptron. The geometry of the generated 3D asset is represented with a surface mesh including vertices and triangular faces. A texture map by is created with a multichannel image in UV space. Multiple views of the 3D scene are simultaneously generated based on the surface mesh.

The invention discloses a double-flow time convolution enhanced interactive bearing life prediction method, which comprises the following steps of: acquiring original bearing vibration signal data, and processing and reconstructing the data; constructing a time flow feature extraction module by fusing the local time sequence modeling capability of the TCN and the multi-scale expansion attention, dynamically adjusting the expansion rate through the spectrum entropy, and capturing multi-scale local features by combining sparse multi-head attention; a spatial stream feature extraction module is constructed by fusing frequency sensing position coding and a layered sparse attention mechanism; designing a bidirectional cross-layer attention collaboration mechanism, performing feature interaction on the time flow feature extraction module and the spatial flow feature extraction module, enhancing the degradation characterization capability through hierarchical feature alignment and dynamic weight adjustment, and constructing a double-flow feature extraction layer; inputting the reconstructed signal into a double-flow feature extraction layer to extract features; inputting the extracted features into a multi-layer sensor to carry out RUL prediction; local-global feature complementation is realized; and prediction stability is improved.

The invention discloses a structure perception graph neural network physical field prediction method based on a geometric gating attention mechanism, and belongs to the technical field of artificial intelligence and computational physics crossing. The method comprises the following steps: extracting geometric features for standardization processing and organizing in a graph data structure; inputting the edge-level geometric features into a geometric gating function, generating a dynamic adjustment factor through a multi-layer perceptron, obtaining an attention weight after geometric modulation, and performing normalization processing to obtain output features of nodes; splicing the output features of the plurality of attention heads as final output features of the model; training the graph neural network model to obtain a physical field prediction model; and inputting the geometric features of the three-dimensional structure model to be predicted into the physical field prediction model. According to the method, the precision and physical consistency of complex physical field prediction are improved, and the problems of boundary blur, field intensity sudden change and the like caused by neglecting geometric dynamic change in a traditional static coding method are reduced.

The invention relates to the technical field of intelligent medical systems, and discloses a cerebral apoplexy onset risk assessment and reminding method and system.The method comprises the steps that continuous medical structured detection data are collected, and the data comprise carotid artery blood flow parameters, brain oxygen saturation, heart rate variability and metabolic indexes; inputting a bidirectional LSTM, a differential convolutional network, a wavelet residual network and a multi-layer perceptron to extract nonlinear features; constructing a neural function coupling structure diagram of four nodes of cerebral blood supply, oxygen supply, autonomous regulation and metabolic steady state; calculating inter-node time sequence offset correlation and a stable factor to obtain a coupling anomaly coefficient; and driving the embedded network by using a graph structure and a node feature input mechanism, and outputting a risk state assessment result. According to the method, the neural function coupling structure diagram is constructed and the mechanism is introduced to drive the embedded network, so that high-precision identification of the multi-system collaborative abnormal state and dynamic evaluation of the stroke risk level are realized.

The invention discloses a drug resistance prediction method and system based on comparative learning and multi-modal fusion, and the method comprises the steps: firstly generating a molecular map and a molecular fingerprint based on the SMILES of a target drug, and extracting the molecular features of the drug through a comparative learning model constructed through combining a map attention network and a map convolution network; and then, acquiring protein expression, gene expression and metabolic expression data from the target tissue cells, extracting modal features through a deep convolutional network, a Transform encoder and a multi-dimensional attention network, and realizing adaptive fusion of the multi-modal features through a heterogeneous interactive attention mechanism. And finally, jointly inputting the fused multi-modal features and drug molecular features into a multi-layer sensor to realize high-precision prediction of the drug resistance of cells to drugs. By introducing a contrast learning and multi-modal feature fusion mechanism, the characterization capability and prediction precision of the model are effectively improved, and efficient and reliable support can be provided for drug screening and clinical decision making.

The invention relates to the technical field of electronic detection, and discloses an electronic component dispensing detection method. The method comprises the following steps: acquiring dispensing image data of the surface of the electronic component, and generating a standardized dispensing data matrix containing glue position, thickness and uniformity characteristics through standardized preprocessing; constructing a dispensing correction matrix based on an adaptive window frame, and performing spatial reference dynamic correction on the standardized data matrix to obtain a spatial correction dispensing data matrix; inputting the data into a multi-layer sensor fusion network for feature fusion, and outputting a multi-source fusion dispensing data set; constructing a multi-dimensional abnormal feature incidence matrix based on the data set, and identifying abnormal dispensing data nodes by using a dynamic threshold detection algorithm; performing parameter optimization iteration on the multi-source fusion data set by using a gradient descent optimization algorithm to generate an optimized dispensing parameter set; and finally, constructing a three-dimensional visual dispensing quality model, and establishing a dynamic mapping relationship between model parameters and glue physical characteristics. The method can more comprehensively detect the glue quality.

The invention discloses an oil mist particle trajectory control method and system in oil mist separation, and relates to the technical field of intelligent control, and the method comprises the steps: collecting the three-dimensional position and velocity vector of oil mist particles in real time, and carrying out the three-dimensional modeling of an oil mist trajectory; taking the deviation between the actually measured track and the ideal track as a training sample, inputting the training sample into a multi-layer perceptron, and outputting a current error vector and a compensation increment; constructing a multi-target evaluation function based on the compensated position target and the safe energy consumption factor; optimizing the control parameters by adopting a gray wolf optimization algorithm with improved congestion degree; and issuing the optimized parameters to an execution system to automatically adjust corresponding values. By means of the method, high-precision online control over oil mist particle movement is achieved; the separation efficiency and the energy consumption are effectively balanced; the system has self-adaptive closed-loop capacity, parameters can be quickly adjusted when working conditions change, and separation stability and equipment safety are guaranteed; manual intervention is reduced through automatic execution, and operation reliability and production efficiency are improved.

The invention relates to the technical field of bioinformatics and computational biology, and provides an anti-cancer drug target screening method and system based on hypergraph-knowledge graph dual-channel, a hypergraph-knowledge graph dual-channel architecture is constructed through a feature extraction module, a self-adaptive multi-modal fusion module dynamically integrates multi-source heterogeneous data, and the anti-cancer drug target screening method and system based on the hypergraph-knowledge graph dual-channel architecture are obtained. The method comprises the following steps: adaptively fusing high-order interaction features and semantic association features of a gene, generating a high-quality negative sample by using a generative adversarial network to optimize a training process, finally optimizing feature representation through a discriminator, and calculating a synthetic lethal probability score between any gene pair. According to the application, firstly, the extracted multi-dimensional features are normalized through the feature extraction module, then the importance of different features is dynamically weighted based on the attention mechanism, finally, the synthesis lethal relationship prediction probability of the gene pair is output through the multi-layer sensor, and under the theoretical framework of the gene synthesis lethal effect, the synthesis lethal effect of the gene pair is predicted. And an innovative solution is provided for anti-cancer drug target screening.

The invention discloses a composite insulator defect detection method and system based on a multi-source signal and VMD decomposition, and the method is characterized in that the method belongs to the technical field of power system and equipment monitoring, and comprises the steps: synchronously collecting an ultrasonic echo signal, a mechanical vibration signal and an infrared thermal image signal of a composite insulator, and carrying out the timestamp alignment; performing CEEMDAN denoising and VMD decomposition on the ultrasonic echo signal in sequence to extract time-frequency domain features, and generating an ultrasonic feature vector; performing frequency domain transformation on the mechanical vibration signal to extract fundamental frequency and harmonic characteristics, and generating a vibration characteristic vector; performing surface temperature abnormal feature extraction on the infrared thermal image signal to generate an infrared auxiliary feature vector; the ultrasonic feature vector, the vibration feature vector and the infrared auxiliary feature vector are spliced into a fusion feature vector, and after dynamic weight distribution, the fusion feature vector is input into a multi-layer perceptron model for defect classification; and analyzing and positioning the internal defect position of the composite insulator by combining the ultrasonic signal propagation path time delay and the vibration mode, and generating a three-dimensional visual report.

The invention discloses a one-dimensional plasma simulation method based on deep learning optimization and related equipment, and the method comprises the steps: firstly constructing a one-dimensional basic model containing specific parameters, and then constructing and training a neural network of a multi-layer sensor (MLP) structure for predicting a potential function; then, a particle simulation method (DL-PIC) based on deep learning optimization is executed. Under the condition that 1000 grid points are included, compared with a traditional particle simulation method (PIC), the simulation time of the DL-PIC neural network is shortened by about 33.37%. According to the invention, complex physical phenomena such as reflection oscillation caused by heat effect and space charges can be effectively processed, and the method has good expansibility. The method is not only suitable for one-dimensional simulation, but also can be popularized to higher-dimension and electromagnetic particle simulation. Therefore, the invention provides an efficient and accurate solution for plasma simulation research, and can be widely applied to the technical field of plasma simulation.

A machine-learned architecture for determining whether an object is relevant to a vehicle's action planning may comprise a convolutional neural network, graph neural network, and / or multi-layer perceptron that may determine a relevance score associated with an object that indicates indicating whether an object is likely to impact operation(s) of a vehicle. In some examples, the machine-learned architecture may use scene information and / or an object track to determine the relevance score.

Described herein are systems, methods, and instrumentalities associated with using a multi-layer perceptron (MLP) neural network to process medical images of an anatomical structure. The processing may include padding an input image in accordance with the training of the MLP neural network, splitting the input image (e.g., the padded input image) into patches of a same size, and processing the patches through the MLP neural network over one or more iterations. During an iteration of the processing, the patches may be processed separately and re-combined into an intermediate image before the intermediate image is shifted to concatenate portions of the image that are derived from different patches. This way, global features of the anatomical structure may be learned and used to improve the quality of the image generated by the MLP neural network, without incurring significant computation or memory costs.

The invention discloses a multivariable time sequence prediction method and system based on an implicit neural network. The method comprises the following steps: 1) collecting data and preprocessing the data; 2) performing window division on the standardized or normalized multivariable time sequence and determining the length of a to-be-predicted window; 3) performing variable correlation coding on the input window to obtain a variable-level feature vector; 4) the implicit neural network based on time attention predicts target parameters by using the variable features in the step 3), and implicit neural representation of the target sequence is modeled through the parameters; 5) taking the output of the implicit nerve representation and the original input window as the input of the multi-head attention predictor, and obtaining a prediction result through cross-sequence cross attention calculation performed in the implicit space and multi-layer perceptron conversion output dimension; and 6) training and optimizing model parameters, calculating a mean square error of a prediction result and a real result, taking the mean square error as a loss function, carrying out back propagation to optimize trainable parameters of the variable correlation coding module, the implicit neural network module multi-head attention predictor and the multi-layer perceptron, and then repeating the steps 3) to 6) to obtain the multi-head attention predictor. Until the preset number of iterations is reached or the error of the model on the verification set meets the requirement of early stop; and 7) performing prediction by using a model of training convergence, and performing reverse normalization on a prediction result to obtain a final prediction result. The method has good generalization, and meanwhile, the interpretability of the attention mechanism is remarkably improved by generating the hidden space characteristics of the trend component and the season component.

The invention discloses a model training method and device and electronic equipment. The method comprises the steps that a training data set is acquired, the training data set is used for training a pre-training model, and the training data set comprises image data and corresponding text data; the training data set is sequentially input into the multiple multi-layer perceptron networks in the pre-training model for training, multiple target multi-layer perceptron networks are obtained, and the output result of the previous multi-layer perceptron network serves as the input of the next multi-layer perceptron network; fusing the plurality of target multi-layer perceptron networks to obtain a fused network; and training the fusion network according to the training data set to obtain a target fusion network, and determining a target pre-training model according to the target fusion network. According to the method and the device, the technical problem of poor accuracy due to the fact that a visual encoder model ignores many detail information in the encoding process in the prior art is solved.

The invention provides an atrial fibrillation postoperative recurrence prediction method fusing electrocardiosignals and clinical characteristics. The method comprises the following steps: acquiring data of a patient before an ablation operation, and carrying out resampling, denoising and normalization preprocessing and data segment segmentation on an electrocardiosignal; extracting spatio-temporal features by using a deep network containing a residual convolutional block and a long and short term memory module; screening high-discrimination clinical baseline features through statistical analysis and a machine learning model; extracting time-frequency domain and nonlinear features of short-time heart rate variability; designing a cross-modal attention fusion module to carry out feature adaptive weighted fusion; and outputting a recurrence probability through a multi-layer perceptron based on the fusion features. The method improves the prediction precision through feature complementarity, facilitates the recognition of high-recurrence-risk patients, is suitable for sinus heart rhythm signals or atrial flutter and atrial fibrillation signals, and has a certain application value in the field of cardiovascular precision medical treatment. The method can be popularized to all prediction researches based on the electrophysiological signals.

The invention relates to the field of noninvasive prenatal diagnosis, and particularly discloses a preeclampsia noninvasive screening method based on deep sequencing 8bp oligonucleotide double-fragment characteristics, which comprises the following steps: collecting preeclampsia and healthy pregnant woman peripheral blood samples, and extracting free DNA for high-throughput sequencing; the method comprises the following steps: extracting core 8-mer sequences' GTGCGCCC 'and' GATGGGGT 'in a long fragment of 150-200bp through bioinformatics analysis; an integrated support vector machine, K-nearest neighbor, extreme gradient lifting, a random forest and a multi-layer perceptron are combined with a logistic regression element classifier to construct a stacking model, the frequency of a core sequence is normalized, machine learning analysis is carried out, and the preeclampsia risk is predicted. According to the invention, two 8bp oligonucleotide characteristic fragments are specifically screened, and a deep learning architecture of multi-model fusion is combined, so that the limitations of low specificity and invasive detection of a traditional screening method are effectively broken through.

The invention provides a DDQN unmanned aerial vehicle interruption scene path planning method based on an attention mechanism, and relates to the technical field of path planning. The method comprises the following steps: constructing a comprehensive objective function and a distribution network diagram according to node position information, energy consumption of a unit distance of the unmanned aerial vehicle and a comprehensive interruption risk between nodes; a graph attention network and a multi-layer perceptron are adopted to carry out feature extraction on the distribution network graph and the current state, current graph embedding features and current state features are obtained, the extracted features are spliced and input into a main network, and a predicted Q value, a reward value and an optimal action are obtained; the optimal action is sent to the unmanned aerial vehicle, the next state fed back by the unmanned aerial vehicle is received, the main network is circularly utilized to continuously obtain the optimal action, and finally an optimal path planning strategy is formed; and the path planning success rate and adaptability of the unmanned aerial vehicle under the interference of the multi-source dynamic environmental factors are remarkably improved.

The invention discloses a frequency domain-spatial domain multi-scale feature fusion complex marine environment ship fine-grained identification method, which comprises the following steps: acquiring a ship image to be identified and preprocessing the ship image to obtain a preprocessed ship image; constructing a frequency domain-spatial domain multi-scale feature fusion ship identification model; inputting the preprocessed ship image into a frequency domain-spatial domain multi-scale feature fusion ship identification model for processing, and outputting a ship fine-grained identification result; wherein the frequency domain-spatial domain multi-scale feature fusion ship identification model is obtained by extracting low-frequency features and high-frequency features of ship images through multistage wavelet transform, generating regional feature distribution and aggregation weights by using a learnable multilayer perceptron, optimizing extraction of ship fine-grained features, and training and verifying through a public data set. The public data set is ship image data and public data used for ship target fine granularity identification. According to the invention, the accuracy and robustness of ship identification are improved.

The invention relates to the technical field of artificial intelligence and audio processing, can be applied to the field of intelligent medical treatment and financial science and technology, and discloses an audio watermark processing method and device, equipment and a medium, and the method comprises the steps: carrying out the coding of continuously inputted audio blocks, and obtaining an audio feature vector; acquiring a digital watermark, and performing time modulation on the digital watermark according to the audio lengths of the blocks by using a multi-layer perceptron to obtain a watermark implicit vector; embedding the watermark implicit vector into the audio feature vector to obtain a target audio vector; performing quantization processing on the target audio vector to generate a target audio spectrum embedded with a digital watermark; and decoding the target audio frequency spectrum to obtain a target audio. According to the method, the continuously input audio blocks are coded, the method can adapt to a streaming scene, the audio data are processed segment by segment, the watermark embedding requirement of the continuous audio stream is met, the audio can be compressed, the data volume is effectively reduced, the transmission time consumption is reduced, and the low-delay communication requirement is met.

The invention relates to the technical field of new energy power generation prediction, and particularly provides an ultra-short-term solar irradiance prediction method and device based on a satellite cloud atlas, and the method comprises the steps: obtaining the prediction time period of a prediction result and an error evaluation index of a parameter type relative to each pre-trained multi-layer perceptron model; selecting a pre-trained multi-layer perceptron model with the minimum error evaluation index to predict the prediction result; wherein the parameter types comprise global level irradiance, direct normal irradiance and diffusion level irradiance. According to the technical scheme provided by the invention, model optimization selection is carried out for different irradiance types, and the adaptability and accuracy of prediction are greatly improved.

The invention discloses an AI generated picture blind box detection method and device based on semantic consistency difference. The method comprises the steps that a corresponding descriptive text is generated for a to-be-detected picture based on a pre-training multi-modal large model BLIP of freezing parameters; and embedding an image-text pair composed of a to-be-detected image and a descriptive text corresponding to the to-be-detected image into the joint representation space by using a multi-modal model CLIP, and extracting image-text features. A detection model is constructed through two-stage rumination training: in the first stage, image-text features are spliced and then input into a first multilayer perceptron MLP1, and AI generation confidence is output; and in the second stage, the confidence coefficient is used as a regulation coefficient, the image-text feature cosine similarity is adaptively adjusted in combination with a loss function proposed by the scheme, iterative optimization is carried out in a submerged space, and regulated joint features are generated and input into a second multilayer perceptron MLP2 for final judgment. The method does not need to predict a picture source model, has high universality and detection precision, and is suitable for AI generated image recognition in a complex scene.

The invention relates to the technical field of medical image segmentation, and discloses a meibomian gland segmentation method based on a meibomian gland infrared image, and the method comprises the steps: inputting a to-be-segmented meibomian gland image into an encoder of a trained infrared image segmentation model, enabling the to-be-segmented meibomian gland image to pass through M encoding modules in sequence, obtaining encoding feature maps extracted by the M encoding modules, and obtaining a meibomian gland segmentation model; a global feature aggregation module is input, a deep feature map and a shallow feature map are extracted, after an attention feature map is obtained through a multi-head cross attention mechanism, the attention feature map and the deep feature map are added, and a deep attention feature map is obtained; performing layer normalization and multi-layer perceptron processing on the deep attention feature map in sequence, adding the processed deep attention feature map with the deep attention feature map to obtain a global feature map, and inputting the global feature map into a decoder; in each decoding module, performing jump connection on the input feature map of the module and the encoding feature map output by the corresponding encoding module, and outputting a decoding feature map; and carrying out convolution on the decoding feature map output by the last decoding module to obtain a segmentation result map.

The invention discloses a VisionTransform image tampering positioning method and a VisionTransform image tampering positioning system based on multi-modal prompt guidance, and relates to the cross technical field of computer vision, image tampering positioning and natural language processing. Generating a text prompt related to a to-be-detected image by using an LLaMA model, and respectively obtaining feature representations of the image and the text through a visual feature extractor and a text feature extractor; then, deep fusion and alignment of cross-modal features are realized through a multi-modal interaction prompt module; and finally, outputting an accurate tampering region positioning result in combination with the spatial feature pyramid network and the multi-layer sensor. According to the method, deep alignment of visual features and text semantics is realized through a cross-modal self-attention and cross-attention mechanism, and semantic association understanding of a model on a tampered region is remarkably improved; and meanwhile, in combination with a spatial feature pyramid network and a lightweight SegFormer decoder, the capturing capability of a multi-scale tampered region is effectively enhanced, and the performance is more excellent especially in a small tampering and large-region forging scene.

The invention discloses an underground pipe network dynamic risk detection and early warning method, which comprises the following steps that firstly, a multi-source heterogeneous database is constructed, various data are integrated to form a core database, and the construction step comprises the step of collecting various data; and then data standardization processing is carried out, and various coding and normalization operations are carried out. Then constructing a spatial-temporal characteristic matrix; then, a hybrid model containing a multilayer perceptron and a convolutional neural network is constructed for deep learning training, and a model input layer contains multiple feature tensors and has related strategies; and then the model score is mapped to a geographic information system to generate a risk thermodynamic diagram, a threshold value is set to trigger an early warning signal, linkage with an emergency platform is carried out, and three-dimensional visual display is carried out. Cross-regional cooperative training and risk trend analysis can be carried out in the model, and a digital twin platform can be embedded to realize dynamic simulation detection. According to the method, a comprehensive data basis is constructed, a complex relationship is mined, model performance is improved, and decision support is provided for pipe network risk management and control and the like.

The determination of the probability of gas hydrate formation using artificial neural network (ANN) and multilayer perceptrons (MLPs) models. ANN generally refers to a network of interconnected neurons (also referred to as “nodes”) that model the neurons in a human brain. An MLP refers to a feed-forward network having a specific arrangement of neurons and includes an input layer, one or more hidden layers, and an output layer. Input data such as temperature, pressure, gas mixture composition, and indicators of gas hydrate formation may be obtained and preprocessed for use in training and testing. The ANN and MLPs may be trained using a training set of the input to output a probability of gas hydrate formation. The trained ANN and MLP models may then be used to determine a gas hydrate formation probability for new data associated with a pipeline transporting a gas mixture.

The invention belongs to the technical field of multimedia analysis, and discloses a multi-modal media tampering detection method, system and device based on multi-view comparative learning and a medium, and the method comprises the steps: obtaining a training data set which comprises a training image-training text pair and a corresponding tampering category label; a cross encoder is introduced on the basis of a vision-language model, a plurality of multi-layer sensor head structures are arranged, and three kinds of comparative learning of noise enhancement, prototype-based and multi-label tampering classification are designed to obtain an initial multi-view comparative learning framework; training the initial multi-view comparative learning framework based on the training data set to obtain a trained multi-view comparative learning framework; and based on the trained multi-view contrast learning framework, executing a tampering detection task of the to-be-detected image-text to the data. According to the technical scheme, the accuracy and robustness of multi-label classification can be improved.

The invention relates to the technical field of kiln combustion control, and discloses a dynamic tar blending combustion proportion optimization control method and system. Comprising the following steps: collecting kiln system data to obtain a standardized working condition data set; inputting the multi-layer perceptron model to obtain a combustion stability score; when the score is lower than a stable threshold value, triggering risk assessment: extracting kiln load micro fluctuation characteristics from the data set, classifying shutdown risk levels by using a support vector machine algorithm model, and determining a high-risk early warning signal; based on the signal correlation current working condition, a historical optimal blending combustion proportion in a corresponding historical adjustment record is called, a deviation value is calculated in combination with the current blending combustion proportion, and a preliminary proportion adjustment suggestion value is obtained; and iterative correction is started, real-time fuel characteristic change and a combustion state prediction result are fused for step-by-step adjustment, a correction proportion is input into a multi-layer sensor to calculate a stability score, and an optimization control scheme is output after the stability score reaches the standard. According to the method, the tar blending combustion proportion is dynamically optimized, the combustion stability of the kiln is effectively improved, and the shutdown risk is reduced.

The invention discloses a forest point cloud branch and leaf separation method fusing double attention and edge perception, and relates to the field of forestry environment monitoring, the method is based on a forest point cloud branch and leaf separation network CLEANet, a classical encoder-decoder architecture is adopted, an encoder layer is composed of a down-sampling module and a channel-local point attention CLPA module, and the channel-local point attention CLPA module is composed of a down-sampling module and a channel-local point attention CLPA module. The decoder layer realizes feature recovery through combination of up-sampling, an edge perception module EAM and a multi-layer perceptron MLP, the CLPA module adaptively strengthens geometric detail and semantic feature expression through a double-attention mechanism and effectively captures wood and leaf component differences, the EAM module enhances perception of a network to a local geometric structure through a neighborhood feature propagation and fusion mechanism, and the local geometric structure is effectively captured. And characteristic mutation of the wood and the leaf at the boundary is captured. The method integrates a channel-local point attention mechanism and edge perception, has excellent robustness, good generalization ability and wide practical application potential, and provides powerful technical support for forest resource investigation and ecological environment monitoring.

The invention provides a live pig weight measurement method based on depth estimation and multi-modal feature fusion, which comprises the following steps: firstly, initializing and reading camera setting, capturing an RGB image, then converting the RGB image into a depth image sequence through a depth estimation algorithm, carrying out feature extraction on the depth image by using a vit model, and carrying out feature extraction on the depth image; and carrying out feature fusion on the extracted features and rgb features extracted in depth estimation, and predicting the weight of the pig through a multi-layer perceptron regression model. According to the method, a monocular RGB image is utilized, three-dimensional space structure information of a target is reconstructed through a depth estimation neural network, an advanced monocular depth estimation network is adopted, correlation and influence between different images are captured through a Transform encoder, and the network gradient propagation efficiency is improved in combination with residual connection. Therefore, the robustness of the posture change of the pig, the shielding area and the background complexity is enhanced, the labor cost is greatly reduced through automatic measurement, and the weight measurement precision and efficiency are improved.

The invention relates to a multi-modal 3D target detection method based on progressive deep fusion. Comprising the following steps: acquiring point cloud data and image information, and processing to obtain structured point cloud voxel features and standardized image information; dividing the point cloud into regular cylinder grids, extracting in-column point-level features by using a multi-layer sensor to generate column-level features, mapping the column-level features to a bird's-eye view feature map, generating a three-dimensional anchor frame through a region candidate network to predict a target position and attitude offset, and outputting a spatial ROI (Region of Interest); based on a YOL3D single-stage target detection method, multi-scale features are extracted by adopting a shared backbone and multi-scale fusion processing, multi-head collaborative regression target three-dimensional parameters of depth detection are utilized, and external parameter transformation is performed to generate a spatial 3D-ROI unified with laser radar coordinates; cross-modal geometric alignment is achieved through TAM, feature complementary fusion is completed through RAM, finally, feature cubes are fused, and categories, three-dimensional positions and postures of multiple targets are output. According to the invention, the detection precision of small targets, shielded targets and long-distance targets is improved.