959 results about "Environmental perception" patented technology

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention relates to the technical field of industrial robots, in particular to an industrial robot walking control system based on obstacle recognition. Comprising an environment sensing unit; the obstacle analysis and decision-making unit is used for processing the multi-dimensional data output by the environment sensing unit based on a deep reinforcement learning framework, accurately identifying static obstacle and dynamic obstacle types, motion trails and interaction influences, constructing a two-dimensional decision-making model of static obstacle avoidance and dynamic obstacle avoidance, and carrying out obstacle avoidance and obstacle avoidance on the basis of the two-dimensional decision-making model. A differential obstacle avoidance strategy is triggered; the path planning unit is based on a dynamic game path algorithm under space-time constraint; and an instruction transceiving unit. Through the multi-modal fusion sensing technology and the adaptive parameter adjustment module, space-time alignment and feature fusion of multi-source data such as three-dimensional point cloud, texture features and vibration spectrum are realized, a high-dimensional environment state model is constructed, and the problem of insufficient data fusion depth in the prior art is effectively solved.

The invention discloses a mechanical arm motion control method based on multi-agent cooperation, and the method comprises the steps: firstly, receiving an RGB image through a sub-task generation agent, and generating a structured sub-task sequence according to a natural language task instruction of the RGB image; secondly, performing joint modeling on a task text and a scene image through a 3D sensing intelligent body, positioning specific coordinates of a target object in a three-dimensional space, reasoning dynamic characteristics of a current environment based on historical state information of a robot by combining an environment sensor, and generating an environment sensing vector; and finally, the action generation agent performs fusion modeling according to the subtask text, the subtask target coordinates, the current state of the robot and the environment perception vector, generates a continuous action vector, drives a mechanical arm to complete each subtask action, and constructs closed-loop feedback by a controller and a discriminator to realize task execution state judgment and automatic circulation. The precise action control instruction can be effectively generated, and the task execution fineness of the mechanical arm is remarkably improved.

The invention discloses an article recognition system based on computer vision. The article recognition system comprises a multi-modal data acquisition module, a multi-modal data processing module and a computer vision processing module, wherein the multi-modal data acquisition module is used for acquiring multi-modal data through a multi-modal sensor array; the data preprocessing module is used for standardizing a multi-modal data format and generating a time-space aligned multi-modal tensor; the feature extraction module is used for respectively extracting modal specific features from texture, spectrum and geometric dimensions by adopting ResNet50, 3D-CNN and PointNet + +; the multi-modal fusion module is used for constructing cross-modal joint representation; the adaptive sensing module is used for modeling illumination invariance and scene dynamics based on self-supervised comparative learning and a 3D-STMN space-time memory network, predicting a shielded target trajectory by using Kalman filtering in combination with the shielding sensing propagation module, and generating an environment sensing parameter set; and the recognition engine module is used for integrating YOLOv8 detection, Mask R-CNN segmentation and multi-modal decision tree classification, outputting a target bounding box, a category and confidence in combination with the depth data, and generating three-dimensional space coordinates combined with the depth data.

The invention relates to the technical field of voice processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a voice enhancement method, device and equipment based on noise perception and a medium. Environment feature information is extracted and input into an audio enhancement model to generate an enhanced audio signal; obtaining a reference audio sample, extracting a personalized feature vector, and carrying out personalized processing on the enhanced audio signal; and collecting playing feedback data, determining a playing time domain adjustment parameter and a playing frequency domain adjustment parameter, adjusting the personalized enhanced audio signal, and generating an optimized audio signal. According to the method, dynamic adjustment is realized in combination with the feedback parameters in the playing process by fusing the environmental perception information and the personalized speaker characteristics, clear and natural optimized audio output with personalized styles can be generated in a complex environment, and the voice interaction quality and adaptability are improved.

The invention discloses a control method and equipment for an intelligent robot with a body and a storage medium, and belongs to the technical field of robots. The method comprises the steps of receiving a task instruction and environment perception data, performing fusion processing on the task instruction and the environment perception data, generating task semantic information associated with the task instruction, analyzing the task semantic information through an implicit planner, generating an implicit action mark, and based on the implicit action mark, generating an implicit action. And decomposing a to-be-executed task corresponding to the task instruction into a plurality of task sub-targets layer by layer, generating an action instruction sequence based on the task sub-targets, and executing a control action corresponding to the action instruction sequence. According to the method, based on the constraint of the implicit action mark and the action instruction sequence, the robot with the body can flexibly respond to environment parameter changes such as object position deviation or new task requirements in the task execution process, the coupling degree of the action instruction of the robot with the body and the predefined scene is reduced, and high stability of task execution is ensured.

The invention discloses a multi-modal data processing method and system, computer equipment and a readable storage medium, which can realize deep association and complementarity mining of multi-modal information and improve the accuracy and robustness of multi-modal understanding. The method comprises the following steps: an environment sensing module adjusts an environment sensing strategy according to feedback information transmitted by a self-adaptive decision module, and acquires multi-modal data according to the environment sensing strategy; the multi-modal encoding module encodes the multi-modal data into multi-modal feature vectors of the same dimension; a cross-modal fusion module fuses the multi-modal feature vectors to obtain fusion features; the self-adaptive decision-making module selects a decision-making network matched with the task type from a predefined network library according to the task type of the current decision-making task, inputs the fusion features into the decision-making network, and generates feedback information according to the decision-making process of the decision-making network; and the meta-learning controller evaluates the system performance of the current multi-modal data processing system and adjusts system parameters according to an evaluation result.

The invention discloses an inspection robot navigation method, system and device based on Beidou and binocular vision fusion and a storage medium, and relates to the field of robot navigation and obstacle avoidance, and the method comprises the following steps: carrying out multi-modal data fusion processing based on an initial coordinate of an inspection robot and an image captured by a binocular vision system to obtain an environment sensing result; carrying out global planning and local optimization based on an environment perception result, and carrying out cooperation to generate a trajectory planning scheme; based on the environment sensing result and the trajectory planning scheme, intelligent navigation cooperative regulation and control are carried out, and an inspection robot motion control strategy is obtained; by improving the environmental perception accuracy, optimizing the path planning and performing intelligent navigation regulation and control, the navigation problem of the inspection robot in complex environments such as a transformer substation can be effectively solved, the inspection safety and efficiency are improved, and the method has remarkable practical application value.

The invention provides a three-dimensional Gaussian digital human generation system and method and electronic equipment, and relates to the technical field of augmented reality, and the system comprises a data obtaining unit which is used for obtaining visual depth data, dynamic behavior data and environmental perception data, and generating point cloud data and multi-modal data according to the visual depth data, the dynamic behavior data and the environmental perception data; the rendering adjustment unit is used for monitoring the fixation area in real time and determining a rendering strategy; according to the real-time computing power parameter, determining a rendering specification; the driving unit is used for performing feature extraction on the multi-modal data to obtain multi-modal features and generating face driving parameters and limb driving parameters of the digital human model; and the modeling unit is used for generating a basic grid of the digital human model according to the point cloud data and the initial digital human model, and performing expression rendering and action rendering on the basic grid according to a rendering strategy and a rendering specification in combination with the face driving parameters and the limb driving parameters to obtain a rendered digital human model. The fluency, authenticity and naturalness of the display effect are improved.

The invention discloses a welding robot path planning system cooperating with industrial vision and collision detection, and relates to the technical field of robot automatic welding, the welding robot path planning system comprises an upper computer, the upper computer is in communication connection with the following modules: an environment sensing module used for collecting welding environment information in real time, comprising three-dimensional point cloud data of a workpiece and obstacle information; and the data processing and fusion module is used for fusing the visual identification result and the obstacle detection data. By integrating industrial visual recognition and collision detection technologies, the dynamic change of the welding environment can be sensed in real time, an accurate environment model is constructed based on three-dimensional point cloud data, compared with a traditional welding robot with a preset path, the welding track can be dynamically adjusted, welding defects caused by environment interference are avoided, and meanwhile the welding efficiency is improved. In combination with an assembly error compensation mechanism, the deviation between a theoretical model and an actual workpiece can be corrected, the welding position precision is ensured, and the high-precision ship manufacturing requirement is met.

The invention discloses a collision identification method and system for a window cleaning robot, and relates to the technical field of robot environment perception, and the method comprises the steps: carrying out the time difference compensation and space synchronization based on a glass material type and collision detection parameters through a distributed perception collaborative analysis method, and generating a three-dimensional space vibration information matrix; enhancing the three-dimensional space vibration information matrix through a dynamic wavelet packet decomposition algorithm and a sliding window analysis method according to the type of a glass material; according to the enhanced three-dimensional space vibration information matrix, collision coordinates are calculated through a time difference positioning algorithm, the characteristic frequency of the vibration waveform is analyzed, and collision characteristic information is obtained; based on the collision feature information, combining safety parameters of glass material types, adopting a hybrid decision algorithm to analyze the collision feature information, and generating and executing a robot obstacle avoidance instruction; according to the method, the high-resolution three-dimensional vibration field is constructed through space-time collaborative modeling, and the collision coordinate calculation precision and the anti-interference capability are improved.

The invention discloses a flood detention area emergency evacuation early warning method based on environmental perception and intelligent analysis, and relates to the technical field of flood control and disaster mitigation, and the method comprises the steps: collecting the original data of a flood detention area site, and forming a monitoring array; a monitoring array is divided into a plurality of monitoring units according to geographic grids, a rainfall permeation model is established according to rainfall data and soil humidity data, a surface runoff field is constructed in combination with terrain elevation data, and the confluence direction of the runoff field is calibrated through water level data; based on a rainfall penetration model and a surface runoff field, generating a flood detention area inundation evolution sequence, segmenting into an inundation grade map according to a time step length, planning a flood detention area emergency evacuation path, matching with a road network in a geographic information system, generating an optimal evacuation path, and issuing graded early warning information to an early warning area. And sending a withdrawal instruction to residents in the flood detention area through the emergency broadcasting system. According to the invention, by establishing a partition early warning mechanism and a differential early warning strategy, accurate delivery and efficient transmission of early warning information are realized.

The invention discloses an emergency takeover triggering method and system based on environment perception and driver state recognition, and relates to the technical field of emergency takeover, and the method comprises the following steps: obtaining environment perception data and driver state data, and carrying out the risk grade evaluation; judging whether to initiate a takeover request according to the risk level; if a takeover request is sent, instruction consistency detection is carried out based on game modeling, a driver and an automatic driving system are abstracted into two agents, and operation intentions of the two agents are inferred respectively; if the detection results are inconsistent, entering a conflict strategy processing flow, evaluating the advantages and disadvantages of the strategies of the two conflict parties according to the driver state confidence and the environmental risk level, and outputting a game reasoning result; the problems of lack of consistency judgment, high calculation complexity and difficulty in real-time response in the prior art are solved.

The invention provides an acoustic array adaptive calibration correction system applied to an underwater moving target, which relates to the technical field of marine equipment and comprises a multi-source excitation and environment perception module, an intelligent array perception and diagnosis module, an adaptive position inversion and uncertainty quantification module and a closed-loop calibration execution and self-learning optimization module. The multi-source excitation and environment perception module is responsible for providing reference signals required by calibration and establishing a correlation model of environment and formation distortion, and the intelligent array perception and diagnosis module realizes multi-modal data acquisition, array element health state monitoring and formation geometry self-perception. The adaptive position inversion and uncertainty quantization module completes signal processing, position estimation and error quantization propagation, and the closed-loop calibration execution and self-learning optimization module executes a compensation strategy, verifies a calibration effect and continuously optimizes system performance through online learning; the system solves the problem that a traditional method cannot process environment time varying, multi-sensor conflicts and uncertainty quantization.

The invention discloses an information processing method and system based on the Internet of Things, and relates to the technical field of Internet of Things services, and the method comprises the steps: dynamically obtaining the computing power, energy consumption and network state of each node through an environment perception module, and constructing a node capability vector and a link performance vector; the task analysis module is used for carrying out structured modeling on tasks and identifying candidate segmentation points and unloading targets; the cost modeling module establishes a joint cost function of time delay and energy consumption; the strategy generation module generates an optimal unloading strategy based on multi-agent reinforcement learning; and the scheduling communication module executes task unloading and result integration according to the strategy, and feeds back execution data to update the system state. According to the method, intelligent, low-delay and energy-consumption optimization processing of task segmentation unloading is realized, and the method is suitable for a dynamic heterogeneous Internet of Things computing environment.

The invention relates to an intelligent inspection control system and method. The system comprises an environment sensing module, a situation recognition module, a task management and scheduling module, a path planning module, an execution control module and a communication linkage module. The system can collect various environmental parameters of an inspection area in real time, dynamically adjust the priority of an inspection task after identifying an abnormal state, and generate an optimal path to drive a robot to execute response operation; meanwhile, abnormal information, images, videos, sensor snapshots and other contents are uploaded to a background management platform through a communication linkage module, and sound-light alarm equipment, broadcast equipment and other equipment are linked to carry out field response. The control method has task preemption and path reconstruction capabilities, and supports distributed sensing deployment and whole-process execution state monitoring. Compared with the prior art, integrated fusion of environment situation awareness, task dynamic scheduling and response control is achieved, and the adaptive capacity, the response speed and the safety guarantee level of the system to complex scenes are improved.

The invention discloses an active real-time interaction system based on LLM. The active real-time interaction system comprises a self-cognition module, a real-time perception module, an active planning module, a situation decision module, a dynamic execution module, an emotion engine module, an adaptive interaction module and a safety management and control module. The system adopts a multi-dimensional vector to represent role attributes, and constructs a hierarchical memory storage structure to record interaction experience; generating an environment state vector through a multi-modal data processing technology; executing the target task decomposition algorithm to generate a multi-path execution plan; making a real-time decision based on the multi-dimensional decision factor; mapping the abstract decision into an instruction sequence and monitoring an execution process; simulating a system emotional state and influencing decision expression; dynamically adjusting an interaction strategy according to the user characteristics; evaluating decision rationality and starting a corresponding intervention mechanism. All the modules form a closed-loop workflow through a standardized data interface, the problems that a traditional AI system is passive in response, lacks self-cognition, is limited in environmental perception, is single in planning capability, lacks flexibility in decision making and the like are solved, and active service, self-evolution and safety controllability of the system are achieved.

The invention relates to an Internet of Vehicles edge computing multi-target unloading method and system fusing dynamic environment modeling and improved SARSA, and belongs to the field of intelligent traffic and edge computing fusion. The method and the system comprise MEC environment perception and multi-dimensional state construction, dynamic reward feedback oriented to multi-dimensional performance indexes, intelligent decision model construction and learning based on improved SARSA, and antagonism training oriented to real disturbance. A high-fidelity environment model is constructed through a space-time attention mechanism, an SARSA algorithm is improved to realize hierarchical qualification trace attenuation and collaborative Q table updating, a multi-target hierarchical reward engine is combined to implement differential optimization on an emergency task and a conventional task, and an adversarial training mechanism is introduced to improve robustness. The core problems of high mobility, task diversity, resource limitation and the like in the Internet of Vehicles are effectively solved, the comprehensive performance is optimal in multiple dimensions of delay, energy consumption, resource utilization rate and the like, and the actual landing of the edge computing technology of the Internet of Vehicles is promoted.

The invention discloses a scene semantic understanding-oriented surveying and mapping technical scheme intelligent generation method, and particularly relates to the technical field of surveying and mapping intellectualization, and the method comprises the following steps: constructing a task description vector to extract task feature parameters, obtaining dynamic environment perception data to recognize semantic feature tags, and generating structured representation; constructing a task-driven semantic vector field based on the task description vector and the semantic feature tag, distributing gravitation weight, calculating semantic weight entropy, dynamically adjusting the priority, finally determining a main control semantic path according to a main semantic sorting structure, and generating a surveying and mapping technical scheme matched with the current task. According to the method, the task description vector and the semantic vector field are constructed, the standardized expression of a surveying and mapping task target and dynamic association modeling of scene semantics are realized, a semantic weight entropy regulation and control mechanism is introduced, the problems of semantic priority misjudgment and scheme deviation are solved, and the accuracy, the stability and the intelligent level of surveying and mapping technical scheme generation are improved.

The invention discloses a flexible surgical instrument-oriented multi-source sensing data adaptive fusion method and a related device. The method comprises the following steps: acquiring optical, mechanical and pose multi-mode sensing data based on a timestamp synchronization technology in a working environment; clock compensation and space coordinate transformation based on an instrument kinematics model are carried out on the multi-modal data, and space-time registration is completed; inputting the registered data into a neural radiation field model to generate continuous space implicit scene representation; inputting the implicit representation into a pre-trained RWKV fusion network to obtain a fusion feature; and superposing the fusion feature and the original low-frequency component through residual connection, and integrating to generate three-dimensional voxelization environment state data. According to the method, multi-source information high-fidelity fusion is achieved through a single path, data redundancy is remarkably reduced, and the real-time performance and precision of intra-operative environment perception are improved.

The invention relates to the field of coal mine exhaust control, in particular to a coal mine exhaust control method and device based on environmental perception. The method comprises the following steps: acquiring multi-dimensional monitoring signals based on environment sensing sensors at multiple positions of a mine, performing distributed sensing marking, and constructing a global gas concentration sensing graph; the method comprises the following steps: acquiring position information of operating personnel in real time based on a miner positioning sensor, performing short-time movement prediction on the operating personnel, and generating a miner movement prediction path; obtaining a coal mine internal structure chart, and carrying out average oxygen consumption rate budget and instantaneous oxygen supply demand calculation based on the miner movement prediction path to obtain oxygen supply demand indexes of each unit; and according to the oxygen supply demand index of each unit, intelligent exhaust power calculation adjustment is carried out, and a conventional exhaust control strategy is constructed. Ventilation efficiency of the coal mine area is improved, emergency blast treatment is carried out under the risk condition, the risk problem response capacity is greatly improved, and safety of the coal mine area is enhanced.

The invention relates to the technical field of pedestrian environment perception, and discloses a pedestrian floor tile environment perception method based on multi-modal fusion. The method comprises the following steps: acquiring a multi-mode sensing data sequence of pedestrian floor tiles in a preset monitoring area, wherein the multi-mode sensing data sequence comprises pressure distribution, a surface texture image and an environment temperature and humidity data sequence; then, combining a preset environmental parameter dynamic threshold value and target sensing area spatial characteristics, and screening a dynamic sensing target area through positive equilibrium relation matching, the former being pre-configured based on historical sensing data statistical distribution and a preset environmental anomaly judgment rule; the latter is calculated based on floor tile laying density, a pedestrian flow density peak value and an abnormal region thermodynamic diagram distribution range; dividing a plurality of continuous sensing time windows according to the boundary position of the target area and the multi-modal sensing node distribution topology; and finally, according to a window sequence, window sensing operation is completed in a gradual multi-source data fusion mode, and comprehensive sensing of the environment state of the pedestrian floor tiles is realized.

The invention discloses a tunnel structure water leakage identification method for robot dog detection, and the method comprises the steps: firstly, guaranteeing the safety of an inspection robot dog in photographing a picture through constructing an environment perception algorithm based on the fusion of a laser radar point cloud and a camera laser point cloud; then, mobile shooting is carried out on the water leakage condition of the tunnel structure through inspection mobile robot dog equipment, and data preprocessing is carried out on images; and finally, constructing tunnel internal connection leakage water identification data features, and proposing an improved Yolov8 + HorNet + DASI + CBAM model to identify the missing condition of leakage water. According to the invention, the improved Yolov8 + HorNet + DASI + CBAM model is embedded into the inspection robot capable of carrying the camera, so that the problem that inspection workers are difficult to enter an underground structure is solved, leakage water occurring in an underground complex environment is accurately identified, potential safety hazards caused by leakage water occurring in the underground structure are eliminated, and the operation safety of the underground structure is guaranteed.

The invention discloses a multi-modal data fusion-based household aging transformation demand evaluation and scheme generation system. According to the system, technologies such as attitude sensing and image acquisition are comprehensively applied, acceleration and angular velocity data of a user are acquired by means of an attitude sensing module, and behavior image data are acquired by an image acquisition module. Fusion analysis is carried out on the multi-mode signals through edge AI processing equipment, accurate recognition of the behavior state of the old people is achieved, and alarm is triggered when an abnormal posture is detected. Meanwhile, the system utilizes the environment sensing module to assist in behavior judgment and path planning, has the advantages of non-intrusive type, high real-time performance, high accuracy and the like, can effectively improve the life safety and behavior monitoring intelligence level of the old people under an unattended condition, is suitable for a safety monitoring scene in a family old people nursing environment, and has a wide application prospect. Good popularization and application values are realized.

The invention discloses an intelligent fishery environment sensing and self-adaptive intelligent adjustment method and system, and belongs to the technical field of intelligent fishery and intelligent environment regulation and control. The method comprises the following steps: acquiring multi-dimensional data such as water quality, weather and fish school behaviors through an environment sensing module, processing the multi-dimensional data through a data standardization and treatment module, extracting key features by using an environment-behavior feature engineering module, and performing state assessment and trend prediction through a coupling modeling and risk assessment module; a regulation and control strategy is generated and executed in combination with a multi-objective optimization and execution arrangement module, and finally, through feedback and optimization of a closed-loop evaluation and adaptive learning module, accurate sensing, dynamic prediction and intelligent adjustment of a fishery environment and a fish school state are realized, the stability of a breeding environment and the health level of a fish school are effectively improved, and environmental risks are reduced.

The invention discloses an electricity consumption anomaly detection method, system and device based on an environmental perception graph convolutional network and a medium, and belongs to the technical field of smart power grids. The method comprises the steps that historical electricity consumption of a power grid area unit and external environmental factors are acquired to construct a multi-dimensional space-time association graph; a periodic trend component and environment-related residual fluctuation are separated through the multi-dimensional space-time correlation diagram, and a two-channel diagram convolution feature is obtained; in combination with the convolution features of the two-channel graph, feature space decoupling of a periodic channel and a residual channel is realized, and decoupling features are obtained; and extracting a historical residual fluctuation sequence based on the decoupling features, and constructing an environmental state sensitive probability density function. According to the method, the historical electricity consumption and the external environment factors such as temperature, humidity or weather events are closely fused by constructing the multi-dimensional space-time association diagram, so that the space-time dependence of the electricity consumption behavior is effectively captured, and misjudgment caused by ignoring environment dynamics in a traditional method is avoided.

The invention relates to the technical field of underwater desilting robot cruise control, in particular to an underwater desilting robot intelligent cruise control method based on environmental perception, and the method comprises the steps: determining a track deviation distance by measuring the position distance of an underwater desilting robot between each moment and all moments within a preset duration before; and determining a second gain adjustment coefficient based on the variation trend of the track deviation distance at all moments within a preset time period before the current moment, and determining a gain adjustment coefficient in combination with the first gain adjustment coefficient so as to optimize the gain coefficient of the sliding mode controller and control the attitude of the underwater dredging robot. According to the method, the problem of hysteresis course errors of the underwater dredging robot caused by fixed gain in traditional sliding mode control is solved, gain adaptive optimization is realized by dynamically fusing multi-source trend data, and the dredging efficiency of the underwater dredging robot is improved.

The invention discloses a gas recognition method based on multi-source information fusion and environmental perception, and the method comprises the steps: constructing a deep feature learning framework of multi-source fusion through combining the spatial response features, time sequence features and external environmental information of gas; the method comprises the following specific steps: preprocessing collected gas data, and respectively extracting features of an image mode, a sequence mode and an environment mode; fusing the image features and the sequence features through a cross attention fusion module, and capturing the space-time correlation of the data; a cross-modal attention compensation module is introduced, so that main-modal gas data adaptively gathers key information in an auxiliary-modal environment, and effective compensation of environmental factors on gas recognition performance is realized; and finally, gas category prediction is performed through a classification decision head. According to the method, the problems that the detection is easily interfered by environmental factors, the stability is poor or the qualification is inaccurate due to the fact that modeling depends on single modal data in the existing gas identification technology are solved.

The invention discloses an agricultural heterogeneous multi-core SoC (System on Chip) architecture and equipment integrating vision and environmental perception. According to the architecture, a heterogeneous multi-core design is adopted, a universal CPU and an AI acceleration and preprocessing module are integrated on an on-chip system, an efficient task scheduling mechanism is combined, real-time management and control of multi-source heterogeneous tasks such as image processing, radar resolving and path planning are achieved, and the system response speed and the task execution efficiency are improved. In the aspects of communication and storage, NoC and AXI buses are adopted to cooperatively construct a high-speed data path, DDR4 and Flash mixed storage is supported, and the transmission stability of large-bandwidth data is guaranteed. The power supply and clock management module realizes modularized start and stop and energy consumption optimization through a multi-voltage domain and dynamic power consumption control technology, and is suitable for outdoor long-time operation. The IO and peripheral interface module provides abundant interfaces, and system expansibility and deployment flexibility are improved. The overall framework integrates the advantages of high integration level, high adaptability and low power consumption, and a solid calculation basis is provided for agricultural intelligent equipment.

The invention discloses an inpatient falling early warning and intervention method based on environmental perception, and the method comprises the steps: continuously obtaining the video frames of a patient through a camera, and extracting the coordinates and confidence of seventeen key points through a pre-training deep learning posture estimation model; respectively processing the time sequence and the space-time diagram by using a hierarchical time sequence convolutional network branch and a graph neural network branch, and generating a visual behavior feature vector; in the same risk assessment period, bidirectional cross-modal attention fusion is carried out on the physiological feature vector and the visual feature vector collected by the wearable device, and a multi-modal fusion feature vector is output; then combining historical state features and static archive features in a hospital information system, carrying out unified modeling through a Transform encoder, and mapping to fall risk probability distribution; and when the risk is judged to be high, an intervention instruction is immediately issued to an intelligent sickbed or ward falling prevention execution mechanism through an Internet of Things protocol, and a perception-decision-intervention closed loop is completed.