833 results about "Invalid Data" patented technology

The invention relates to the technical field of data processing, in particular to an industrial environment data acquisition device and method, and the device comprises a sensing layer, an edge calculation layer, a cloud platform layer, and an application layer. Compared with a traditional scheme in which a single sensor is adopted to work independently, software synchronization is relied on, and signal conditioning parameters are fixed, the technical scheme of the invention realizes significant breakthrough through collaborative design of a multi-mode sensor array and an intelligent signal conditioning system, microsecond-level clock synchronization is realized by adopting a hardware-level PTP protocol, and the signal conditioning precision is improved. The program control amplifier is combined with the self-adaptive anti-aliasing filter, so that the conditioning circuit can be self-adaptive to the working condition change of equipment; meanwhile, nanosecond-level real-time processing of signals is achieved through an FPGA + ARM heterogeneous processor architecture, abnormal high-speed sampling is dynamically triggered in cooperation with an ARM, compared with a traditional scheme, the real-time performance is improved by three orders of magnitude, meanwhile, invalid data transmission is reduced by 80%, and a high-timeliness and low-redundancy intelligent monitoring solution is provided for early failure early warning of industrial equipment.

The invention relates to the technical field of vehicle network control, in particular to an efficient vehicle network control system and method based on CAN-FD, and the system comprises a data transmission optimization module, a bandwidth management module, a signal scheduling control module, an encryption security control module and an anomaly detection module. In the invention, the bit rate of a data frame, the length of a data packet and the proportion of filling bytes are optimized, the transmission efficiency is improved, the bandwidth occupied by invalid data is reduced, a priority screening mechanism is called, the stable transmission of high-priority signals is ensured, the bandwidth allocation is adjusted based on the flow state and node frequency monitoring, the burst flow congestion is relieved, and the resource allocation is balanced; in combination with arbitration priority and bus conflict state, non-emergency signal priority is optimized, key data blocking risk is reduced, encryption and integrity verification are adopted, data security is ensured, tampering and leakage are prevented, data abnormity score and detection are introduced, abnormal data are screened, identification capability is improved, and network potential safety hazard is reduced.

The invention relates to the field of energy-saving computing, and discloses a low-power-consumption control method and system of a controller, which are used for realizing prediction-sleep-wake-up-response full-link hardware in the controller. Time sequence state data of a controlled object is collected in real time, and a predictive sleep window value is dynamically generated by utilizing a predictive cooperative computing unit, so that intelligent scheduling of sleep and awakening is realized, and the power consumption of a controller is effectively reduced. Meanwhile, in combination with the technologies of threshold comparison, DMA data transmission optimization and the like, the data processing efficiency is improved, and invalid data transmission is reduced. And on the awakening mechanism, an asynchronous awakening circuit and an interrupt request mechanism are adopted to ensure that power supply is quickly recovered and control response is executed when an effective instruction is received, so that the real-time performance of the system is guaranteed. According to the invention, a prediction algorithm, hardware acceleration and an intelligent scheduling technology are fused, the performance of the controller is ensured, the power consumption is obviously reduced, and the endurance time of equipment is prolonged.

The embodiment of the invention provides a memory recovery method and device, electronic equipment and a storage medium, and relates to the technical field of data storage. Under the condition that both the file system and the storage device support the perceptual garbage collection function, the device state of the storage device is obtained in response to the starting of the perceptual garbage collection function; according to the equipment state, determining whether an execution condition for sensing a garbage collection function is met or not; and if the execution condition of the garbage collection sensing function is met, the file system is linked to perform memory collection on the storage device, so that the file system can know the data blocks moved by the storage device, the file system is prevented from moving the data blocks moved by the storage device again, and the efficiency of the file system is improved. In addition, invalid blocks in the file system and the storage device can be synchronized, the situation that invalid data blocks in the file system are moved when the storage device executes memory recovery is avoided, and therefore the service life expenditure of the storage device and the write-in amplification value of the storage device are reduced.

The invention discloses a multi-stage equipment energy efficiency optimization control system and method for an efficient machine room, and relates to the technical field of equipment optimizing.The method comprises the steps that environment parameter data and design parameter data of a refrigerating machine room are obtained, an environment correction factor is analyzed, and the cold load rate is dynamically corrected based on the environment correction factor; on the basis of preset delay time, phase-space reconstruction is carried out on the historical corrected cold load rate sequence, the final embedded dimension is analyzed, and a hybrid prediction model is constructed on the basis of the final embedded dimension; matching optimal operation parameters based on physical resonance characteristics of equipment in combination with comprehensive energy efficiency indexes of the machine room; and performing data structure compression on the optimal operation parameters and the comprehensive energy efficiency indexes of the corresponding machine room, performing screening analysis on data points, and updating a parameter library. And invalid data interference is reduced, so that the system can realize self-adaptive optimization in long-term operation, and the data storage and calculation efficiency is improved.

The invention discloses a real-time interaction and intelligent management method for a multi-dimensional data space, and relates to the technical field of data interaction management, and the method comprises the following steps: I, collecting multi-source data, including structured data, unstructured data and spatio-temporal data, in real time through a diversified collection terminal, and dynamically adjusting the sampling frequency based on a data entropy value, carrying out cross-device space-time synchronization calibration and edge end lightweight processing on the acquired data; iI, preprocessing the data flow in real time, fusing the multi-modal data, and the like. According to the method, the sampling frequency is dynamically adjusted based on the data entropy value, and in combination with edge end lightweight processing, invalid data transmission is reduced, and the key data acquisition efficiency is improved; hot data is stored in a time sequence database and a memory database, real-time monitoring of an emergency scene and second-level response of emergency command are supported, dynamic capacity expansion and shrinkage of computing resources are matched, edge-cloud end collaboratively distributes tasks, network delay is reduced, and system stability and response speed in a high-concurrency scene are guaranteed.

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a data migration method, device, equipment and medium based on task offload, which comprises the following steps: scanning a source end directory hierarchy and a target end directory hierarchy, and generating source end file metadata and target end file metadata; files are classified into source-end unique files, target-end unique files and two-end shared files based on file paths, and the files are transmitted to a comparison processing module; generating a source end newly-added migration task, a difference coverage migration task and a target end deletion migration task; and transmitting the migration task to a migration processing module and executing the migration task to generate a migration result. According to the method, source-end unique files, difference files and target-end redundant files are processed in a classified mode, full-amount migration is disassembled into incremental tasks to be executed in a split mode, and the invalid data transmission amount is reduced; a difference coverage task is generated based on double comparison of metadata and content, repeated coverage of homonymous files is avoided, and redundant data is automatically cleaned.

The invention discloses a power plant data management method and system based on an SIS system, and relates to the technical field of data management, and the method comprises the steps: obtaining power plant operation data, and carrying out the unified formatting processing of the power plant operation data; preprocessing the formatted data, and removing noise and invalid data; performing quality verification on the preprocessed data, and checking data integrity and consistency; classifying, marking and storing the data subjected to quality verification into a database; and extracting and analyzing data from the database according to operation requirements of the power plant, and generating a decision report. According to the power plant data management method based on the SIS system, through unified formatting and time alignment processing, data format difference and time disorder between different devices are eliminated, and consistency and accuracy of data are ensured. And noise elimination, interpolation processing and anomaly detection are adopted, so that the data quality is guaranteed, and the reliability of an analysis result is improved. And aiming at data with different access frequencies.

The invention discloses a storage space recovery method, electronic equipment, a storage medium and a program product, and relates to the technical field of computers, the influence of various factors on storage space recovery is comprehensively considered, and the recovery score of a data block is determined specifically according to the invalid data rate, the data heat degree and the wear degree; according to the method and the device, the data blocks are selected to be subjected to storage space recovery, the recovery priorities of the data blocks are determined according to the recovery scores, the data blocks are subjected to storage space recovery according to the recovery priorities, and the data blocks with the maximum recovery benefits are preferentially processed, so that the reasonability of the data blocks subjected to storage space recovery is improved; the effective data migration rate of the recovered data blocks can be reduced, so that the recovery efficiency of the storage space is improved, the influence on the performance of the storage system can be reduced by considering the heat data, in addition, the wear degree of the data blocks is considered, and the service life of the storage system is also prolonged to a certain extent.

A hotspot data elimination and maintenance method in a distributed cache system comprises the steps that the access frequency, the time locality score, the service weight and the real-time data popularity value of data are acquired, and the dynamic priority score of the data is generated; determining failure data based on the dynamic priority score; dividing the data into hot spot area data, temperature spot area data and cold spot area data, when the cache space is insufficient, preferentially eliminating the data with the lowest dynamic priority score and the retention benefit value lower than a preset threshold value in the cold spot area data, and if the cache space is still insufficient, performing data elimination on the temperature spot area data according to the latest access time of the temperature spot area data; packaging the cold point area elimination data, the warm point area elimination data and the failure data into a final transaction message; and broadcasting the final transaction message through a preset protocol, and performing distributed consistency maintenance operation on the final transaction message through the receiving node. According to the invention, intelligent and efficient cache data elimination and consistency maintenance management can be realized.

The invention discloses an MQTT message transmission optimization method and system. The method comprises the steps of analyzing a theme, extracting a device type, a data feature and a geographic position triple, calculating a hash value, and mapping a device to a specified Broker fragment cluster node to generate a fragment mapping table; processing the equipment data in the edge domain in the fragment mapping table through an edge calculation layer, removing invalid data according to a preset rule, merging the equipment data in the same fragment node, and embedding a fragment node ID for an aggregation message generated after merging; identifying a fragment node ID and routing to a target fragment node, positioning a corresponding shared memory pool, and writing the message into the shared memory pool; and responding to a direct access request of the client, so that the client directly accesses the data from the shared memory pool through the user mode network stack. According to the method, the problem of uneven load is effectively solved, multiple times of state switching in the data transmission process is avoided, and the MQTT message transmission efficiency is improved while the transmission cost is reduced.

Methods, systems, and computer-readable storage media for receiving a real data table, providing a synthetic structured table based on the real data table, providing a sampled data table comprising a sub-set of real data of the real data table, transmitting a prompt to a LLM system, the prompt being generated based on the real data table and the synthetic structured data table, receiving synthetic unstructured data from the LLM system, providing an aggregate synthetic table that includes at least a portion of the synthetic unstructured data, and training a ML model using the aggregate synthetic table.

The invention relates to the technical field of tunnel engineering monitoring, in particular to a tunnel health monitoring method and system based on edge computing, and the method comprises the steps: a data collection module obtains the original state data of a tunnel through a sensor deployment network, carries out the primary processing of the original state data based on edge computing, and obtains the actual state data; during the period, the data acquisition module performs data filtering on the original state data based on a state data threshold value, rejects invalid data in the original state data and marks abnormal data in the valid data according to an anomaly detection algorithm, and the remaining unmarked original state data is the actual state data of the tunnel; and the monitoring cloud platform obtains the actual state data every a preset period, and carries out data summarization, data storage and data analysis processing on all the actual state data. The system has the beneficial effects that part of calculation tasks are issued to the data acquisition module, so that the processing burden of the monitoring cloud platform is reduced, and the real-time performance and effectiveness of monitoring and early warning are improved.

The invention provides a cache cleaning method and device based on large model reasoning. The method comprises the following steps: monitoring the cache capacity of a hash table cache pool; when the current cache capacity of the hash table cache pool is greater than or equal to a preset capacity threshold value, obtaining access information corresponding to one or more cache entries, and screening a first to-be-cleaned cache entry through a preset cache cleaning algorithm according to the access information; and / or obtaining importance information corresponding to one or more cache entries, and screening a second to-be-cleaned cache entry meeting an importance cleaning rule according to the importance information; and according to the first to-be-cleaned cache entry and / or the second to-be-cleaned cache entry, cleaning the cache in the hash table cache pool, and updating the cache capacity of the hash table cache pool. According to the method and the device, the cleaned cache items are identified and determined according to the access information and / or the context importance information of the cache items, so that the cache redundancy and the accumulation of invalid data are reduced, and the cache management efficiency is improved.

The invention discloses an unmanned aerial vehicle cluster authentication method based on a certificateless homomorphic network coding signature, and belongs to the field of cryptography and network security. In an unmanned aerial vehicle network system comprising a KGC, a source node, an intermediate node and an authentication server CS, the following steps are executed: the KGC generates system parameters and a master key; the source node registers to the KGC and requests part of private keys; the source node carries out vector splitting and expansion on the authentication message, randomly selects a message identifier, executes a signature algorithm to respectively calculate the signature of each vector, and sends the vectors and the signatures to the intermediate node; and the intermediate node executes a verification algorithm to judge the validity of the currently received data packet, discards the invalid data packet, calculates to obtain a signature of the combined vector, and forwards the signature to the adjacent intermediate node. When the CS obtains a sufficient number of valid data packets, the original authentication message can be recovered and an authentication success message is sent to the source node unmanned aerial vehicle. According to the invention, the transmission performance and reliability of the unmanned aerial vehicle network are improved.

The invention relates to the technical field of logistics management, and discloses an operation and control analysis method, device and equipment based on logistics transportation, and a medium, and the method comprises the steps: carrying out the preprocessing of to-be-controlled waybill data, removing invalid data, and screening an effective delivery record as a judgment basis; establishing a multi-layer affiliation relationship according to the judgment basis and marking label information; inputting the waybill data, the corresponding multi-layer affiliation relationship and the label information into an abnormal path identification model, and outputting an abnormal type corresponding to the waybill data, wherein the abnormal type at least comprises trunk line empty running, advanced departure and non-network area recovery type anomalies; and generating an abnormal path trend according to the waybill data and the corresponding abnormal type, and generating delivery optimization information according to the abnormal path trend, the delivery vehicle corresponding to the waybill data and the main delivery area so as to perform delivery optimization on the waybill data within the set time and distance range. The method is suitable for intelligent management of logistics enterprises on delivery path optimization, transport capacity scheduling and operation efficiency improvement.

The invention discloses an intelligent cold data migration method and system, and relates to the technical field of data migration, and the method comprises the steps: predicting the state space of a data block through a time sequence analysis algorithm, automatically classifying the cold and hot degrees of the data block based on a clustering algorithm, and optimizing the migration data size in combination with a data deduplication technology after recognizing repeated data blocks in clustering. And carrying out association analysis on the data blocks needing to be migrated, carrying out secondary optimization on the migrated data volume, and carrying out multi-level migration processing on the data blocks needing to be migrated by utilizing an edge-cloud collaborative grading method in combination with a data block optimization result and a cold and hot degree clustering result. After the migration system automatically classifies the cold and hot degrees of the data blocks, repeated data blocks can be further identified in the clustering process, and by combining a data deduplication technology, invalid data migration is reduced, the storage utilization rate is improved, the distribution mode of storage nodes is dynamically adjusted, so that data storage resources are more balanced, and the problem that single-point storage pressure is too high is avoided; and the overall storage efficiency of the system is improved.

The invention provides a multi-source log fusion driven database monitoring method and system, and relates to the technical field of computers.The method comprises the steps that format analysis processing is conducted on multi-source heterogeneous logs of a to-be-monitored database on the basis of a dynamic analysis template library, and initial database logs are obtained; performing data preprocessing on the initial database log based on a rule engine to obtain a database log; and performing streaming processing on the database log to obtain a log event stream, and sending the log event stream to the server side, so that the server side detects the log event stream based on the log anomaly detection model and the associated alarm analysis engine, and generates a monitoring analysis result of the to-be-monitored database. By means of the mode, the technical bottleneck that heterogeneous database logs are difficult to analyze is broken through, the problem that the terminal is poor in database log format compatibility can be solved, invalid data transmission between the terminal and the server side is effectively reduced, and the database monitoring effect is improved.

The application discloses a safety production monitoring data management method and device, equipment and a storage medium, and comprises the following steps: acquiring real-time production data in the production process of an enterprise in real time and storing the real-time production data into a preset database; extracting multi-source time sequence data from the preset database to generate an original data set; screening out data that does not conform to a preset rule in the original data set to generate a primary selected data set; according to the strong time correlation characteristics of the dangerous chemical production data in the primary selected data set, eliminating invalid data through a preset validity rule to generate an effective data set; periodically extracting data from the effective data set, and displaying the management result of the time sequence data in a trend chart through a preset visual display model. The application can improve the accuracy of analysis results when analyzing safety production monitoring data.

The invention provides a data write-back method and device, equipment and a medium. The data write-back method and device are used for quickly and efficiently performing data write-back. In the method, an identifier of a cache page where data is written, a bitmap identifier and a writing timestamp are recorded in data information of a cache region, and the bitmap identifier is generated according to whether the data is written into each sub-storage position of the cache page after the data is written into the cache page; for any data information in the cache region, judging whether other data information of which the identifier and the bitmap identifier of the cache page are the same as those of the cache page and the bitmap identifier of the data information exists in the cache region or not; if yes, determining the data information reserved in the cache region according to the write-in timestamp of the data information and the write-in timestamps of other data information; the corresponding data is written back into the storage device based on the data information reserved in the cache region, so that invalid data can be prevented from entering a write-back process, and the write-back efficiency is quickly improved.

The invention discloses a control method and system of storage equipment, electronic equipment and a storage medium, and the method applied to the storage equipment comprises the following steps: receiving a read command or a write command sent by a host end; wherein the read command and the write command carry a discard command; and discarding invalid data corresponding to the discarding command in the storage device, and executing the read command or the write command. According to the technical scheme, the discarding command is carried in the read command and the write command, so that when the host side sends the read command or the write command, the storage device can quickly and effectively discard invalid data according to the discarding command carried in the read command and the write command, and the storage performance of the storage device is improved.

The invention provides a garbage collection method and device for a storage device, a medium and computer equipment. The garbage collection method comprises the steps that a write-in command issued by a host and a GC operation command triggered by a system are cached through a cache actuator; the GC operation command is evaluated through a dynamic priority scoring model according to the invalid data proportion and the block wear proportion, the number of high-priority GC tasks can be adjusted through a dynamic adjustment coefficient of the dynamic priority scoring model, and bandwidth occupation of the GC operation command is restrained in a high-load scene. And the maximum bandwidth occupation of the GC operation command is kept in a low-load scene. The real-time bandwidth load of the I / O channel is monitored and processed, the execution number and the bandwidth occupancy rate of the high-priority GC tasks are cooperatively adjusted, the processing number of the high-priority tasks is considered, and the high load occupied by the resources of the GC tasks is effectively limited.

Provided are a handover method and a communication device, which can initiate handover of a user plane bearer of a terminal device according to a notification message for indicating that the terminal device is to be handed over to a target cell, in other words, the handover of the user plane bearer of the terminal device can be triggered by the notification message. Compared with the mode that switching of the user plane bearer of the terminal equipment is initiated after the indication that the terminal equipment successfully accesses the target cell is received, switching of the user plane bearer of the terminal equipment can be executed earlier. And after the switching of the user plane bearer of the terminal equipment is executed, the service data of the terminal equipment can be directly sent to the target access network equipment without being forwarded by the source access network equipment. Therefore, the data in the data forwarding process from the source access network equipment to the target access network equipment can be reduced, the invalid data is reduced, and the service experience is improved.

The invention provides an information compression and pruning RAG database construction method. The method comprises the implementation steps that node objects and a hierarchical clustering tree are constructed; compressing the node object text information in the clustering tree; and pruning the clustering tree after the partial node object text information is compressed based on a QR-like decomposition method. According to the method, the sentence unit text information in the sub-nodes in the clustering tree is compressed, the embedded vectors of the compressed sub-nodes are updated, and the most valuable external information can be screened and stored according to the actual requirements of the large language model, so that invalid data storage in an RAG database is reduced, and the reliability of the RAG database is improved. According to the method, text information stored in the RAG database is guaranteed to have gain all the time, meanwhile, pruning is carried out on a clustering tree obtained after partial node object text information is compressed based on a QR-like decomposition method, redundant child node objects with high semantic similarity are effectively recognized and deleted, and the storage overhead of the node objects in the RAG database is reduced.

The invention relates to the technical field of computer vision and edge computing collaborative optimization, in particular to a continuous domain adaptation method based on cloud collaboration, which realizes real-time performance optimization of a lightweight model of end equipment through an uncertain guided sample screening mechanism (UGS) and an undetectable jointly guided dynamic visual cue update strategy (VUPLU). Specifically, invalid data transmission is reduced through UGS, distribution difference between data of a source domain and data of a target domain is reduced through cross-domain alignment, input features are reconstructed through visual prompt prompt, soft supervision signals of a teacher model and VUPLU dynamic parameter updating are combined, environment interference is effectively removed, the model generalization ability is improved, and the method has the advantages of being high in practicability and easy to popularize. The problems of false label noise, low communication efficiency, unstable parameter updating and the like in a traditional method are solved, and the end-cloud cooperative reasoning precision and universality are remarkably improved.

The invention discloses a driving style scoring method and device, electronic equipment and a storage medium, and is applied to the field of big data. The method comprises the following steps: firstly, acquiring historical driving data, then preprocessing the historical driving data to remove invalid data to obtain first data, and performing feature extraction on the first data to obtain an initial data set for distinguishing a positive sample and a negative sample; and finally, training a driving style scoring model by taking the initial data set as input data, and scoring the driving style of the user by using the trained driving style scoring model. According to the method, the risk degree of the driving data is rapidly measured, and then the driving style score of the driver is efficiently and accurately determined.

The invention discloses a data forwarding processing method and system for a campus network, and the method comprises the steps: controlling a plurality of independent cameras to generate a multicast video stream carrying a permission identifier according to an access control terminal triggering event; the method comprises the following steps: synchronously creating (S, G) table entries with permission marks and never expired (*, G) table entries on a path multicast router according to a first video stream data packet received by a multicast source side designated router source DR; according to an IGMP join message which is sent by a receiver and carries a permission identifier, through a matching result of an RP comparison (*, G) table entry output interface permission and a multicast data permission identifier, whether a data stream is forwarded along a shared tree (RPT) is decided; and according to a matching result of the permission identifier of the new receiver and the pruned (S, G) flow permission, activating a pruning state on the multicast path and recovering data flow forwarding. According to the embodiment of the invention, occupation of invalid data streams on campus network bandwidth can be reduced, and the overall utilization rate of network resources is improved.

The invention discloses a low-power FIFO (First In First Out) module, which comprises an FIFO memory and an FIFO controller, the FIFO memory comprises a writing unit, a memory unit and a multiplexer, the FIFO controller comprises a writing pointer register, a reading pointer register, a first adder and a second adder, the FIFO controller further comprises a mux selection register and a third adder, the third adder enables a mux selection pointer signal output by the mux selection register to increase by 1 only when valid data is cached in the storage unit or new valid data is to be cached while the storage unit outputs the last valid data; and when the read signal is high, the multiplexer selects the data on the corresponding address from the storage unit according to the mux selection pointer signal and outputs the data. According to the invention, the problem of invalid flipping caused by invalid data output is solved, so that the power consumption is reduced.

The invention discloses a garbage collection optimization method and device, equipment and a medium. The invention relates to the technical field of solid state disks, which comprises the following steps that: a global invalid data bit chart is established in a host memory, each table item in the global invalid data bit chart corresponds to a physical block, and a bitmap bit in each table item corresponds to a physical page in the physical block; bit values corresponding to the bitmap bits comprise a first bit value and a second bit value; responding to a data updating event through an access acceleration engine so as to dynamically update the bit value of the corresponding physical page in the global invalid data bit graph; and when garbage collection is triggered, screening the physical page of which the bit value is the second bit value in a to-be-collected source physical block according to the global invalid data bit chart and the access acceleration engine to perform validity verification and data migration, and skipping the physical page of which the bit value is the first bit value. According to the invention, the garbage recycling efficiency is improved.

The invention discloses a ClickHouse-based fused CDN (Content Delivery Network) business data processing method, device and equipment, and the method comprises the steps: obtaining business data of a corresponding fused CDN manufacturer through calling a data interface of each fused CDN manufacturer in real time; performing preprocessing including data format standardization, field expansion and invalid data filtering on the service data by using Spark, packaging the preprocessed service data and sending the packaged service data to a Kafka cluster; the method comprises the following steps: creating a distributed data table based on a ReplacingMergeTree engine in a ClickHouse database, consuming service data in a Kafka cluster in real time by utilizing Spark, and writing the service data into the distributed data table of the ClickHouse database in batches; and retrieving target business data from the ClickHouse database according to a data query request of a user, and converting the target business data into a JSON format to serve as a query result to be returned. The query performance, the storage efficiency, the expandability and the write-in performance can be improved.