97 results about "Inference engine" patented technology

This invention discloses a method and system for scheduling large-scale model inference requests based on global state awareness, relating to the technical field of large language model inference systems. This method constructs a network and protocol layer, a session scheduling layer, and an inference engine layer within a single process. Requests are received and session objects are created via a network listening thread. Enqueueing and timeout cleanup are completed using a double-buffered queue and a polling thread. Worker threads claim sessions based on a global state-aware scheduling algorithm, complete protocol parsing and token encoding within the same process, and submit them to the inference engine. The inference engine performs continuous batch processing scheduling and KV cache reuse. Finally, the token is decoded, written back to the client, and session resources are released. This invention, through a co-process fusion architecture and a global state-aware scheduling algorithm, enables the service layer to obtain the internal state of the inference engine in real time and optimize scheduling decisions accordingly, eliminating cross-layer boundary overhead and information asymmetry problems.

This invention relates to the field of information review and management technology, and discloses a dynamic approval management system for work order video uploads, comprising: a video upload module, a deep model prediction module, a knowledge graph inference engine, a dynamic rule conflict detection module, a manual review workbench, a blockchain evidence storage module, a two-way anchoring module, and a change warning module; the video upload module is used to receive videos associated with work orders; the deep model prediction module is used to identify preset violations frame by frame in the video and output pre-review tags, confidence levels, and heatmaps of suspicious areas. This dynamic approval management system for work order video uploads, through parallel verification of deep model prediction and knowledge graph inference engines and dynamic conflict detection, can automatically identify suspicious videos whose model output is inconsistent with business rules and generate suspicious tags with causal chains, guiding accurate manual review and significantly reducing the workload of ineffective reviews.

This application relates to the field of computer technology, specifically to a method, apparatus, computer device, and storage medium for running an open-source engine. The method for running the open-source engine is applied to a hardware extension plugin system for an inference engine. The hardware extension system includes at least an inference engine module and a hardware extension plugin module. The method includes: determining whether the current hardware is configured with all target operators required for inference computation of the model to be computed; if missing, determining whether the inference engine interfaced with the current hardware is configured with missing hardware operators; if the inference engine interfaced with the current hardware is not configured with missing hardware operators, calling the missing hardware operators from a pre-configured CPU operator library in the CPU through the hardware extension plugin system; and registering all target operators to the inference engine and / or the current hardware. This provides a secure, reliable method for third-party hardware to access an open-source engine with complete evolvability.

The application discloses an offline evaluation system for enterprise ecological strategy based on a Web front end, comprising a front-end application layer, a resource management layer, a core function layer and a data storage layer; user registration is completed through the resource management layer, and a core shard storage module is pre-cached, and a WebAssembly inference engine module is initialized; user input is received through a user interface, and data input by the user is stored in the core shard storage module in real time, network state detection is performed through the resource management layer, when in an offline state, the WebAssembly inference engine module obtains data from the core shard storage module and performs relevant calculation tasks to determine the risk of the ecological system; the WebAssembly inference engine module transmits the calculation result to the front-end application end and visually presents it to the user. In the application, enterprise ecological strategy evaluation can be performed whether the user is in a network environment or not, the continuity of enterprise decision-making work is ensured, and internal data does not need to be uploaded to a third-party platform, so that the risk of leakage of core business secrets is reduced.

The application discloses a kind of electric vehicle charging pile order exception clearing settlement method and device, comprising: obtaining the real-time order of vehicle networking platform;Based on the inference engine of preset detection real-time order whether it is abnormal order;Response to real-time order is normal order, then directly upload real-time order to vehicle networking platform;Response to real-time order is abnormal order, then based on correction algorithm or preset prediction model carries out numerical correction or data prediction to real-time order, obtains revised abnormal order and uploads to vehicle networking platform.One aspect, according to the mass historical abnormal data obtained by vehicle networking platform, mining clearing screening rule, combined with the experience rule of power grid expert custom clearing screening rule, the inference engine obtained can accurately detect abnormal order, on the other hand, it can be to electric vehicle charging pile abnormal order numerical correction or data prediction, reduce the economic loss caused by order exception.

This invention provides an intelligent safety management method and system for boiler walkway gratings, relating to the field of industrial safety technology. The method includes: S1, acquiring in real-time the obstruction signal sequence generated by an infrared beam sensor deployed at the grating inlet and the weight signal sequence generated by a pressure sensor deployed below the grating, and synchronizing the obstruction and weight signal sequences with timestamps; S2, inputting the synchronized obstruction and weight signal sequences into a pre-constructed lightweight spatiotemporal feature extraction network to extract multi-dimensional temporal features representing personnel flow, object presence, and weight distribution; and inputting these multi-dimensional temporal features into a risk fusion inference engine built based on Dempster-Shafer evidence theory. This system can eliminate perceptual ambiguity, overcome decision-making rigidity, and mine data value in the harsh edge environment of boiler walkways, achieving progressive intelligent safety management from precise risk identification to dynamic and flexible assessment and early hazard insight.

The application provides a large model processing method, an electronic device and a storage medium, and relates to the technical field of models. The method comprises the following steps: acquiring input data; performing an inference process by using an inference engine deployed in the electronic device and the input data, to obtain an inference result; generating an inference quality function according to performance data; when it is detected that the inference quality function has not converged, performing permutation and combination on a network structure in the inference engine based on the inference quality function, and updating network parameters corresponding to the inference engine; performing multiple iterations on the inference engine according to the above steps; when it is detected that the inference quality function has converged, fixing the network structure in the inference engine and the network parameters corresponding to the inference engine, to obtain a target large model. This implementation mode can simplify the entire process of deploying and optimizing the large model on the electronic device side, significantly reduce the cost, reasonably utilize the computing power of the electronic device side, improve the performance of the electronic device, and save the resource overhead of the electronic device.

This invention discloses a method, apparatus, device, and medium for accelerating inference of a deep learning model. The method includes: deploying multiple inference engine modules to an on-board unit; identifying the model to be inferred and splitting it into multiple operators; sequentially traversing each of the split operators; for the current operator, determining candidate inference engine modules from the multiple inference engine modules, and selecting an inference engine module that meets preset inference performance requirements as the target inference engine module corresponding to the current operator, wherein the preset performance requirements include achieving a preset inference speed; after traversing all operators, concatenating the target inference engine modules corresponding to each operator according to the connection relationship of each operator in the model to be inferred, to obtain the target inference engine system corresponding to the model to be inferred. By adopting the above technical solution, the inference speed of the deep learning model is improved.

This invention discloses a non-intrusive adaptive compliant control and end-to-end security remote control system and method for industrial equipment, relating to the fields of industrial automation and the Internet of Things. The system includes a non-intrusive actuator, edge computing, cloud control, and energy management modules. The actuator employs a three-degree-of-freedom adaptive displacement adjustment platform, which is magnetically attached to the equipment panel via a magnetic base, and an encrypted data tunnel is established using a wireless network communication module and intranet penetration technology. The core of the method lies in constructing a multimodal state space involving visual and tactile perception and time delay. A Deep-PESO PID inference engine utilizes the Gaussian probability distribution parameters of the PID gain output by Meta-RL, and generates composite control commands through probability domain evolution sampling and LSTM time delay prediction. This invention also integrates physical circuit breaking based on IMU attitude reconstruction, digital twin auditing, and autonomous homing charging mechanisms, achieving zero-threshold rapid deployment, high-precision compliant control, end-to-end security protection, and long-term maintenance-free operation of industrial equipment.

The application provides a method and system for deploying a large language model on a graphics card with limited computing power, relating to the technical field of software engineering, and aiming at the characteristics of the target graphics card hardware architecture lacking a dedicated matrix acceleration unit, reconstructing key operators required by the large language model, including: adapting the core computing process of the key operator to be executed by the general computing unit of the graphics card; optimizing the key operator to make it suitable for a computing scenario with an input dimension of 1xN; quantizing the model weights in the key operator by 4-bit integers and using 16-bit floating-point numbers for activation in the forward calculation process; constructing a lightweight inference engine; loading the target large language model and executing the inference task on the target graphics card by using the lightweight inference engine. The application realizes system-level improvement in three dimensions of hardware compatibility, resource utilization, and inference efficiency, and provides a feasible technical path for deploying advanced large models on a wide range of existing and low-power graphics cards.

An image is received / divided into a first / second slices, at a first convolutional layer in a CNN. A first / second tensor data for a second / third convolutional layer are generated for the first slice respectively using a kernel. The first tensor data is divided into a first / second portions. The second portion is needed by the second convolutional layer of the second slice to complement partial tensor data generated by the second slice. The second tensor data is divided into a first / second portions where the second portion is needed by the third convolutional layer of the second slice to complement partial tensor data generated by the second slice. The first / second / third convolutional layers of the first slice are processed using a subset of processing tiles. The second portion of the first tensor data and the second portion of the second tensor data generated by the first slice are stored in a memory component.

This invention discloses a multi-agent collaboration method and system for domestically developed operating systems, belonging to the field of intelligent technology for domestically developed desktop operating systems. It aims to solve the core problems of existing solutions, such as lack of collaboration standards, black-box decision-making, and difficulty in reusing experience. The method establishes a collaboration system by building an MCP protocol stack and a native DBus mapping, generating a structured inference chain through the System-2 inference engine, and achieving a closed loop through dynamic task distribution, three-level memory reuse, and visual auditing. The system includes a main agent, a cluster of sub-agents, and multiple core modules to collaboratively support the implementation of the method. This solution enables plug-and-play intelligent agents, full traceability of decision-making, and improves the execution efficiency of similar tasks by more than 30%. It is deeply adapted to multiple versions of openKylin and domestically developed hardware, meeting the security compliance and efficient collaboration needs of key scenarios such as government affairs and finance.

This application provides a method and related apparatus for dynamically constructing a surgical operation knowledge ontology based on surgical stage layering and a two-layer constraint graph neural network. The method includes: S1. Implementing a three-layer surgical stage layering: stage layer, operation layer, and action layer; S2. Extracting cross-layer operation relationships to obtain a set of cross-layer operation relationships; S3. Performing knowledge inference using a two-layer constraint graph neural network, with a hybrid score Score(r) = λ_rule·Score_rule(r) + λ_gnn·Score_gnn(r); S4. Performing ontology consistency verification using an OWL inference engine; S5. Incrementally expanding the surgical operation knowledge ontology; S6. Performing a three-dimensional quality score and outputting a knowledge ontology quality report. This application also provides related apparatus corresponding to the method, including devices, electronic devices, computer-readable storage media, and computer program products.

A man-machine collaborative quality detection and early warning system for intelligent factories, comprising an edge visual acquisition terminal, a cognitive anchor generation module and a hybrid intelligent inference engine, the hybrid intelligent inference engine extracts a high-dimensional feature vector of a workpiece image using a residual network and calculates a Mahalanobis distance. When the distance is in a fuzzy decision interval, a man-machine collaboration request is triggered, and a feature heat map is displayed on the terminal. The cognitive anchor generation module receives defect key pixel points labeled by an operator, maps them to a high-dimensional feature space to construct a feature repulsion sphere, and adds the feature repulsion sphere as a regularization term to the loss function of the residual network for online fine-tuning. The scheme corrects the feature boundary in a targeted manner without changing the weight distribution of the model main body, overcomes long-tail defect missed defects, and improves the feature boundary stability of the detection model under continuous production conditions.

This invention relates to the field of financial risk early warning technology, specifically a method for dynamic prediction of repayment ability and overdue early warning based on multi-source time-series data. The method includes: acquiring multi-source time-series data of a target object over a continuous time period; generating a clean time-series data stream through time axis normalization and data quality restoration; constructing a feature set of income stability, expenditure volatility, and debt pressure based on this data stream, forming a multi-dimensional risk feature space; inputting this data into a probabilistic graphical model based on state transitions, outputting a probability distribution of repayment ability states; and calling an inversion inference engine to trace the key characteristic variables and change trajectories of abnormal states through state inversion technology, comparing them with preset risk thresholds to determine the risk critical point and level. This method achieves dynamic prediction of repayment ability and overdue early warning, improving the accuracy and targeting of risk identification.

The application discloses a multi-competition face searching system and method based on GPU parallel acceleration and Milvus, and relates to the technical field of face recognition and vector retrieval. The system is divided into an access layer, a service scheduling layer, a GPU inference engine layer and a three-level storage layer; GPU batch parallel face detection and 512-dimensional feature extraction are realized by relying on CUDA; video frame feature batch deduplication is completed by using high-speed operation of the GPU; Milvus adopts competition_id as a partition key to realize multi-competition data partition isolation, and is matched with Milvus-GPU to accelerate partition vector retrieval; the service scheduling layer dynamically divides GPU computing power priority, and preferentially guarantees real-time retrieval resources; the system configures hierarchical API keys and IP white lists to realize safety management and control, and is compatible with watermark storage and extra_data parameter transparent transmission functions. The application overcomes the defects of low efficiency of traditional CPU serial processing, large video storage redundancy and high full-database retrieval delay, has the advantages of fast processing speed, high hardware utilization rate, small storage cost and strong safety, and is suitable for competition personnel verification, park visitor retrieval, security blacklist control and other scenes.

This invention belongs to the field of artificial intelligence technology, specifically a reinforcement learning fault-tolerant training system oriented towards human feedback. The system comprises a decoupled recovery engine, a mirror checkpoint engine, and a generation phase checkpoint engine; it supports component-level hot recovery to shorten restart latency after a failure; it utilizes data redundancy between the training and inference engines based on the RLHF paradigm to achieve low-overhead mirror checkpointing; and it sets semantic checkpoints in the generation phase to limit recomputation to sub-iteration ranges, thereby reducing recovery overhead and improving resource utilization. Experimental results show that this invention can reduce interruption time caused by cluster failures, training failures, and suspensions in reinforcement learning jobs based on human feedback, improving cluster training efficiency, and is suitable for large-scale RLHF training under high-performance computing resources.

The present invention provides a method for implementing inference acceleration for machine-learning and large-language models based on an encapsulation engine, comprising: receiving a Boolean parameter enable_cache in an inference engine; on the basis of an incoming enable_cache value, performing conditional determination logic to determine whether to execute a cache query; creating a Triton model repository under a specified file path, and establishing an independent folder for each model; in a config.pbtxt file for each model, defining version information, input and output tensor names, data types, and dimensions of the model; and using a command line tool to start a Triton Inference Server, and specifying a model repository path. The present invention demonstrates significant advantages in aspects such as improving inference speed, optimizing resource usage, enhancing user experience, reducing costs, and increasing scalability, providing strong support for practical applications of large models.

This invention relates to the field of multimodal interaction technology and discloses a multimodal perception-based intelligent interactive feedback system. The system includes generating a multimodal data slice sequence with precisely aligned timestamps by synchronously recording and time-stamping speech, facial images, and vital signs; extracting text, facial expressions, gaze patterns, and physiological feature sequences through parsing, and jointly analyzing these sequences using an inference engine to obtain state results; and querying a knowledge graph to generate a composite feedback strategy containing content identifiers, rhythm parameters, and channel instructions. This method improves the reliability of state inference through precise temporal alignment of multimodal data; and through the collaborative generation of the composite strategy, it achieves adaptive and unified control of feedback content, rhythm, and channels, thereby enhancing the naturalness of the interaction and the overall system response performance.

This invention relates to the field of visual monitoring technology for IoT devices, and discloses an AI-based image detection method, system, and storage medium for IoT. The method synchronously receives image and environmental parameter data streams, performs temporal slicing on the image stream, and extracts deep visual features from the current slice. The environmental data stream is decomposed into waveforms, and corresponding environmental state segments are extracted based on image timestamps. A dynamic difference spectrum is generated by calculating the differences between current and past image features, and visual features are coupled and encoded with environmental segments to obtain enhanced visual representations. Using both as inputs, a cascaded diagnostic inference engine is driven to perform anomaly localization, pattern matching, and defect inference, ultimately outputting detection conclusions. This method can capture gradual changes in device status and achieve more accurate fault diagnosis under complex environmental fluctuations.

A distributed environmental control system comprises control nodes that communicate via wireless mesh networks to predict user movement and automate loads without centralized coordination. Each node executes a local inference engine (Markov model, HMM / DBN, or reinforcement learning policy) to predict occupancy patterns, broadcasts predictions to neighbors, and makes autonomous actuation decisions based on local and shared data. When multiple nodes control loads in a shared zone, a Runtime Actuation Authority mechanism designates a single decision-maker based on most recent manual interaction, with authority epochs and deterministic total order resolution ensuring distributed consistency. The system prevents feedback loops via causal action identifiers, logical timestamps, and recent action caches. Sensor data is transmitted as compressed waveform encodings. A deterministic policy layer enforces safety and comfort constraints. The reinforcement learning embodiment learns household preferences from user feedback, adapting actuation behavior to optimize satisfaction, comfort, and energy efficiency while maintaining safety guarantees.

This invention relates to a causal consistency long-chain reasoning method based on structured semantic parsing, comprising a semantic structured representation module for decomposing complex tasks into structured semantic units with causal relationships; a causal consistency reasoning module, which mainly constructs a multi-stage causal reasoning graph to logically model and divide the task execution path into stages; and a path evaluation and error correction module, which performs confidence evaluation and logical verification of intermediate states during the reasoning process. This invention constructs a causal consistency long-chain reasoning mechanism based on structured semantic parsing, achieving interpretability of reasoning steps, verifiability of process logic, and error-correcting capability of the reasoning path through structured modeling of the reasoning process.

The application discloses a classroom sitting posture detection and identity recognition method and system based on multi-modal fusion and feature engineering and belongs to the technical field of artificial intelligence and educational informatization. In order to solve the problem of identity tracking interruption caused by shielding, bowing and the like and the problem of single sitting posture evaluation dimension in a complex classroom scene, the application constructs a complete monitoring link from multi-path video acquisition, AI inference to Web visualized interaction based on a Sophon BM1684X edge computing device. The method core comprises the following steps: a three-level dynamic identity fusion strategy of "face first - ReID relay - temporary ID bottom-up" is adopted to realize continuous identity tracking in a shielding scene; based on 11 upper body key points, a hierarchical 19-dimensional geometric feature vector normalized by shoulder width is constructed, and a light-weight classification model is combined to realize multi-dimensional sitting posture discrimination; a class level and a student level double queue structure is adopted, a Redis cache and an MQTT asynchronous uploading mechanism are combined, and closed-loop data management without data loss in a disconnected network is realized. The system supports multi-path video stream concurrent processing on a single processor through FP16 quantization and a SAIL inference engine. Experiments show that the identity recognition accuracy is 99.2%, and the sitting posture judgment accuracy is 97.2%.

The application relates to the field of building engineering safety monitoring data processing and discloses a building engineering safety monitoring method and system based on artificial intelligence, which comprises the following steps: receiving an original video code stream and writing the original video code stream into a ring buffer; establishing a bypass listening flow process to analyze data packet header information, extract motion vector data, and calculate direction angle data; calculating a vector field direction discrete value; performing logical gate control decision; when a modulus sum exceeds a threshold value and the direction discrete value represents consistent distribution, shielding triggering is performed; only when discrete distribution occurs, an awakening interrupt is generated and a time stamp is recorded; according to the time stamp backtracking, an original video code stream segment is locked and delivered to a deep neural network inference engine; and the application realizes negative logic filtering of high-energy-consumption computing tasks by distinguishing wind-induced global disturbance from independent target motion from a mathematical statistical level through a vector field space consistency verification mechanism.

This application discloses a method, apparatus, device, and storage medium for generating SCADA configuration screens based on generative AI, relating to the field of industrial control technology. The method includes: collecting natural language design requirements using an intent understanding module and the perception layer in an AI Agent cognitive architecture; obtaining state acquisition data and integrating it before inputting it to the cognitive layer; the inference engine and large language model in the cognitive layer decompose the configuration design task of the integrated data, generating a configuration design operation instruction set and inputting it to the execution layer; the execution layer processes the operation instruction set to obtain each component to be laid out in the configuration screen and its configuration data; the layout engine determines the component layout result in the configuration canvas; the variable binding engine binds and maps each component to the real-time data points of the SCADA system; and the target configuration screen is generated by combining the Shell command security module and the security layered architecture to improve the efficiency of generating SCADA configuration screens.

Method and computing device for inferring via a neural network environmental data of an area of a building based on visible and thermal images of the area. A predictive model generated by a neural network training engine is stored by the computing device. The computing device determines a visible image of an area based on data received from at least one visible imaging camera. The computing device determines a thermal image of the area based on data received from at least one thermal imaging device. The computing device executes a neural network inference engine, using the predictive model for inferring environmental data based on the visible image and the thermal image. The inferred environmental data comprise geometric characteristic(s) of the area, an occupancy of the area, a human activity in the area, temperature value(s) for the area, and luminosity value(s) for the area.

This invention belongs to the field of smart agriculture and knowledge graph technology, specifically disclosing a knowledge graph-based platform for optimizing the environment and making feeding decisions for mud crab farming. The platform includes an underwater video acquisition device, a water quality sensor array, a behavioral feature extraction unit, a dynamic knowledge graph construction unit, a spatiotemporal alignment processing unit, a health status inference engine, and a feeding decision execution unit. Through the fusion of visual and sensor data, it aligns mud crab behavior with water quality parameters in the spatiotemporal dimension, combines the dynamic knowledge graph to infer its health status, and generates precise feeding instructions accordingly. By adopting the above technical solution, this invention can achieve early warning of mud crab health status and individualized feeding decisions, improving resource utilization efficiency and the level of intelligent farming.

A method includes in a first iteration receiving a machine learning (ML) network model comprising a plurality of ML operations in high-level code; generating an internal representation (IR) for the ML network model, the IR mapped to one or more components in a multi-processing tile device; generating primitive functions based on the IR; generating an allocation list based on the primitive functions; determining when a tensor data within the allocation list is no longer needed; and inserting a deallocation function associated with the tensor data to the primitive functions to form an updated primitive functions, the inserting frees up a memory space associated with the tensor data when the tensor is no longer needed; and in a second iteration generating a compilation of the updated primitive functions to map the IR to the one or more components in the multi-processing device, wherein the compilation is a low-level instructions.

The application provides an image style migration method and system based on WebGPU technology, which comprises the following steps: S1, constructing an image style migration model; S2, training the image style migration model; S3, exporting the trained and optimized image style migration model into an ONNX format, and then converting the image style migration model in the ONNX format into an intermediate format suitable for a Web-side inference engine, while generating necessary metadata and model atlas; S4, integrating a large language model, and automatically generating or optimizing style migration parameters and prompt words according to a style description or specified requirements input by a user; S5, integrating and deploying a complete image style migration model, and integrating and aligning the complete image style migration model with a prompt word optimization model to obtain a final image style migration model and convert the final image style migration model into a browser-side model suitable for WebGPU technology. The application realizes real-time migration of multi-style high-resolution images, and greatly improves the style migration speed and visual expressiveness.

An image processing system includes an image sensor, an object detector, an artificial intelligence (AI) inference engine and a mode controller. The image sensor is configured to capture an image. The object detector is configured to determine whether an object exists in the image, and utilize a bounding box to frame the object to obtain a region of interest (ROI) image after determining that the object exists in the image. The artificial intelligence (AI) inference engine is configured to recognize the ROI image to obtain a recognition result. The mode controller is configured to compare an ROI size of the ROI image with a lower limit of a ROI size threshold, and switches the image processing system to a crop mode or a binning mode.