112 results about "Distributed collection" patented technology

A distributed collection of web-crawlers to gather information over a large portion of the cyberspace. These crawlers share the overall crawling through a cyberspace partition scheme. They also collaborate with each other through load balancing to maximally utilize the computing resources of each of the crawlers. The invention takes advantage of the hierarchical nature of the cyberspace namespace and uses the syntactic components of the URL structure as the main vehicle for dividing and assigning crawling workload to individual crawler. The partition scheme is completely distributed in which each crawler makes the partitioning decision based on its own crawling status and a globally replicated partition tree data structure.

A distributed query system includes a distributed collection of dynamically created compiled queries. As each client submits a query, a parameterized query skeleton is identified, which identifies the general form of the query, and the parameters associated with the particular query. If a compiled form of the skeletal query is available within the distributed system, it is executed with the parameters of the current query. If the compiled form of the skeletal query is not available within the distributed system, a compiled form is created, and the location of this compiled skeletal query is stored for subsequent access by this client, or other clients. The executable compiled skeletal queries may be stored at each client system, in a commonly available server storage system, or within one or more database servers. A routing system may be provided to efficiently route parameterized queries to the appropriate location(s) of the compiled skeletal query.

There is provided a device and a method for collecting information from a customer concerning a point of interest. The device comprises a memory including a unique location identifier associated with a point of interest and a processor, the processor configured to store in the memory a received unique customer identifier and received information concerning the point of interest from a customer. The device can transmit this stored data to a center via a network, where the center can aggregate the data. There is also provided a system for distributed collection of information from customers concerning points of interest, utilizing a plurality of devices and a center for aggregating data. The center can apply data analysis and mining to the aggregate data, allowing rapid and real-time reaction to feedback from customers, efficient turnaround for resolving customer service issues, and enabling efficient research of customer preferences and trends.

The invention relates to a distributed extendable resource monitoring system and method based on a cloud platform. The distributed extendable resource monitoring system comprises a monitoring object database, a collection management node, collection servers, a management node database and a performance database. The monitoring object database is used for storing configuration information of a target monitoring object; the collection management node is used for distributing collection tasks to the collection servers according to the target monitoring equipment information assembly collection tasks and maintaining states and performance of all the collection servers; the collection servers are used for monitoring dispatch of the collection tasks, managing collection plug-ins and storing collection result data; the management node database is used for storing login information and state information of the collection servers; the performance database is used for storing the data collected by the collection servers. The accurate real-time monitoring data are provided for a user, and the user can fast perceive whether system resources can support a current service requirement or have over redundancy by analyzing the monitoring data, so that the purposes of stable service operation and maximized utilization of the resources are achieved.

The invention provides a distributed collection method oriented to user generated content. The method includes the steps that first, page types are divided according to collection amount of collection pages and collection difficulty, and on the basis of the page types, collection tasks are built and added into a collection queue, wherein the collection tasks comprise composite collection tasks, and the composite collection tasks are collection tasks formed in the mode that a plurality of same type collection pages are classified into same collection tasks; second, the collection tasks are taken out of the collection queue concurrently, the collection tasks are executes, and acquired information is fed back. The invention further provides a corresponding distributed collection system which comprises a master control node and a plurality of sub-nodes, wherein the master control node is used for constructing and maintaining the queue of the collection tasks, and the sub-nodes are used for executing the collection tasks concurrently. According to the distributed collection method and system, collection speed is high, and the UGC news collection instantaneity is remarkably improved; the distributed collection method and system is applicable to collection of the pages in different types, and diversified collection tasks are executed; monitoring measures of collected objects can be avoided.

The invention discloses a log analysis system and method based on distributed collection. The system comprises a log file collecting application module, a log receiving application cluster module, a distributed type log storage module and a Zookeeper registration center cluster module. In addition, the invention further discloses a log analysis method based on distributed collection. By means of the system and method, the horizontal expansion capacity of a log processing layer and a log storage layer is improved, cross-service calling logs are grouped by generating request identification (Tracing ID) for each request according to a thread isolation principle, and data log circulation and abnormity details in the service calling process are clearly positioned.

The invention discloses a distributed collection and storage method of a journal. The method adopts transmission control protocol (TCP) / internet protocol (IP) to interchange data, a sending terminal serializes the data, and a receiving terminal deserializes the data. In order to maintain long connection, a client terminal sends heartbeat packet at regular intervals to determine whether a journal server terminal is on line, and when the journal server terminal corresponds the heartbeat packet when the journal server terminal is on line. A first speed regulator controls a connected FD (file descriptor) integrally and storages the processed data in a data-caching pool. A second speed regulator reads a certain amount of data from the data-caching pool and sends the data to a remote terminal. After the remote terminal of a computer receives the data, the remote terminal writes the data in a data base. By means of the distributed collection and storage method of the journal, the transmission efficiency of the client terminal and the journal server are improved, the handling capacity operation of the journal server and the data base is improved, and the instantaneity and the usability of the system are improved.

The invention discloses a task parallel processing method for an electricity utilization information collection system. Distributed collection communication units are established, parallel collection threads of the collection communication units are correspondingly distributed on terminals of various cities to execute collection tasks, and collected tasks are decomposed according to different types of the terminals; operating states of all the collection threads of current collection communication units are inquired through management of the threads, and when vacant collection threads exist, tasks needing to be executed in a task queue are fetched; parallel processing is carried out on the collection tasks of the cities; data items collected through multiple collection threads are stored at the same time, and the threads are partitioned through a data caching area and are simultaneously read and written. Through the distributed parallel processing method, timeliness of task data of different users is guaranteed, and execution efficiency of the collection system is improved.

The invention discloses a log processing system. The log processing system comprises a data source layer, a data integration layer and a data calculation processing layer, Wherein the data source layer is used for providing a data source object; Wherein the data source object comprises but is not limited to an operating system, a database, an application system and network equipment; The data integration layer is used for obtaining a log from the data source object in a distributed collection mode and carrying out preprocessing operation on the log; Wherein the preprocessing operation comprises at least one of normalization processing, classification processing, filtering processing and merging processing on the logs; And the data calculation processing layer is used for analyzing the logsthrough an association analysis engine and carrying out distributed storage on the logs. By adopting the embodiment of the invention, the operation and maintenance problem event can be effectively and intelligently pre-warned, and rich visual charts and instrument panels are integrated, so that the data is more vivid, and the association of log data is easy to find.

The invention discloses a network security monitoring system applied to a power monitoring system, and belongs to the technical field of network security monitoring systems. The network security monitoring system comprises a dispatch control system, a substation monitoring system, a power plant monitoring system, an automatic power distribution system and a load control system, output ends of thedispatch control system, the substation monitoring system, the power plant monitoring system, the automatic power distribution system and the load control system are electrically connected with an input end of a distributed collection module, the output end of the distributed collection module is electrically connected with the input end of a network security management platform, real-time monitoring and effective management of network space security are achieved through the independent collection of network security events of various monitoring points and the distributed processing of data, real-time monitoring and online management can be performed on the security risks and the security events of the power monitoring system to avoid the problem that the whole system cannot run due to a single point fault, the subsequent expansion and maintenance are facilitated, and the security coefficient is high.

A system and method for distributed collection, organization and sharing over the web of digital images captured and displayed by camera equipped computing devices. In one aspect, digital images are collected by a single camera device and are organized into one or more albums and sub-albums. Albums are related to an occasion. Sub-albums relate to moments within an occasion. A user can invite friends to contribute photos and are provided the ability to upload images to an album server. Before being uploaded, photos are tagged and stored in a database. In another aspect, digital images are collected from multiple camera devices capturing digital images of multiple moments occurring within a single occasion. The method then organizes those images so that images correlating to each moment are grouped together. In both instances, the method further provides for pre-loaded or user-generated moments to aid in capturing of the digital images.

Methods and systems crowdsourcing collection and distribution of photographs. A network of computing devices communicates with a content storage and client computing devices operated by users. The network includes application instance computing devices that register users, receive photographs from users (after a description of a common theme has been published), and store photographs in the content storage. Identifications of multiple external and independently operated social networks are received for each photograph, and each photograph is posted on the social networks identified for the photograph. After these posts, a new registration request is received from a new user. After the new user is registered, a new photograph is received from the new user and stored in the content storage. The new photograph is associated with new identifications of social networks, and posted on the social networks identified in the new identifications to thereby crowdsource collection of photographs across the social networks.

The invention relates to a power supply system distributed controlling and managing subsystem computer, comprising a power supply system managing computer which is arranged in a machine box, an alternating current primary power distribution controlling device, a direct current primary power distribution controlling device and a bus, the alternating current primary power distribution controlling device and the direct current primary power distribution controlling device realize distributed collection of the power supply system, the collection is transmitted to the power supply system managing computer by the bus to be calculated in centralization, and a generated control order is transmitted to the alternating current primary power distribution controlling device and the direct current primary power distribution controlling device by the bus to realize the control. The power supply system distributed controlling and managing subsystem computer realizes distributed control and management of the power supply system, realizes distributed power distribution digital management, increases the service efficiency of the power supply, realizes automated management of a load, reduces the burden of a pilot and increases the maintainability and reliability of an airplane.

The application raises a cylindrical k-space collecting method and system for three-dimensional dynamic magnetic resonance imaging. The method comprises the steps of establishing a cylindrical k-space collecting model, determining the collecting track of the echo signal in said k-space collecting model, wherein the collecting track is to conduct echo collection in radial direction passing through the circle center in any plane parallel to the undersurface of said cylinder, conduct Descartes distributed collection upwardly on the cylinder axis, and the radial azimuth and the axial position of the echo signal are calculated according to the two-dimensional golden section proportion; designing the scanning time series according to said collecting tracking, and calculating the encoding gradient of magnetic field to be applied required by the magnetic resonance imaging system; setting up the magnetic resonance imaging system according to aforesaid scanning time series, and collecting corresponding k-space data according to aforesaid collecting track. The method can realize the homogeneous and consecutive collection of three-dimensional k-space data, so as to obtain three-dimensional dynamic magnetic resonance imaging in high time resolution.

The invention provides a distributed data collection system and a data collection method based on the same. High-speed synchronous signal collection among sensors and flexible networking are realized via built-in Ethernet exchange and Ethernet transmission in the intelligent sensor node side and Ethernet based time synchronization. The data collection method overcomes disadvantages of present network and methods, advantages of easy wiring, convenient installation and maintenance and low cost of a distributed collection manner are maintained, signals among the different sensors can be collected synchronously, and further distributed collection can be applied to occasions which need multi-channel cooperative work.

The invention discloses a memory database-based simulation grid node quick migration method, which comprises the steps of utilizing a memory database to establish a complete state mirror image set of various simulation nodes in a real-time manner, carrying out distributed collection and management on state mirror images, completing fault simulation node detection, pause and running control of the simulation process, fault node or process migration, fault node state mirror image injection and continuous running control of the simulation process through simulation running management, state monitoring, node migration and recovery and other mechanisms in a simulation grid, and further realize the quick migration of fault nodes in the simulation grid.

The invention discloses an unmanned automatic gas extraction system. A multi-pipeline acquisition module mounting area is divided into a plurality of groups, and each group comprises a plurality of gas extraction pipelines which are adjacently arranged; an extraction pipe of each gas extraction pipeline extends into a gas extraction drill hole of a coal bed; each gas extraction pipeline in each group is connected in parallel to a corresponding multi-way connecting disc; the multi-way connecting discs in each group are connected to an extraction negative pressure main pipe through the extraction negative pressure branch pipes arranged in parallel. Through the design of the multi-pipeline collection module, distributed collection can be carried out on the interior of a mining area, independent grouping control can be carried out, the extraction efficiency is improved, and energy is reduced; due to the design of a gas filtering module, impurities in extracted gas can be filtered and removed, and the purity degree of the gas content is guaranteed; a gas leakage monitoring module and an alarm module are used in cooperation, monitoring can be conducted in the gas collection process, serious safety accidents caused by the fact that leakage is not known by people are avoided, and the safety in the working process is improved.

The application puts forward a transient electromagnetic multichannel coverage observing device and method. The method comprises the following steps: emission currents are injected underground via emitting electrodes at a plurality of emitting points; a plurality of receivers that are interconnected are used for orderly receiving underground induction signals, caused by the emission currents, of all emitting points; according to transient electromagnetic observation data obtained by distributed collection stations, wave filed conversion operation and data extension operation are performed, and virtual seismic data of a measuring line at any underground depth is obtained; according to the virtual seismic data of the measuring line at any underground depth, an underground virtual seismic cross-section diagram is drawn; according to change of lineups on the virtual seismic cross-section diagram, an undulation condition of an underground lithologic layer interface can be determined; in a condition of large electromagnetic interference, dense population and plenty of buildings, underground coal-bearing structures of urban areas can be effectively detected step by step, various kinds of detection results supplement each other, and therefore the underground coal-bearing structures of the urban areas can be effectively detected.

The invention relates to a Spark-based application software running log collection and service processing system and a method thereof, and provides log services for different resources at different levels, wherein the log services include a log collection service of a log data resource layer, a log data storage service of a log data service layer, and a user acquiring log data service of a log data application layer. After collecting the original data through the log collection service of the log data resource layer, the original data is pushed to the log data service layer for preprocessing the log data, the data is then stored by the log data storage service, and finally the log data service is provided to the user at the log data application layer. The invention may realize the collection of log data by using a distributed collection strategy, defines a multi-level data storage structure to store the log data, and provides a log data query service to the user, so that the user obtains useful application software to run the log data, and the efficiency of fault diagnosis is improved by the obtained log.

The invention discloses an ammonia nitrogen / nitrite content prediction system and method for multi-source information fusion in aquaculture environment. The system comprises a distributed collection node cluster, a wireless transmission node cluster, an intelligent gateway, a cloud end big data analysis server and a remote management end, wherein the distributed collection node cluster is connected with the wireless transmission node cluster; the wireless transmission node cluster is connected with the intelligent gateway; and the intelligent gateway is connected with the cloud end big data analysis server and the remote management end. Readily available data such as pH, water temperature and dissolved oxygen can be acquired in a distributed manner through a wireless sensor network under the condition that ammonia nitrogen / nitrite data can not be detected, real-time on-line prediction of ammonia nitrogen / nitrite data can be realized, decision-making support is provided for the precisecontrol over aquaculture water ammonia nitrogen / nitrite content, the biological toxicity to aquaculture objects is reduced, and the healthy development of aquaculture is promoted.

The invention discloses a MapReduce-based FTP distributed collection method, which comprises the following steps of (S1) preparing information and log file paths of multiple FTP servers in advance and storing configuration information into an HDFS of Hadoop as data input of MapReduce; (S2) setting an input directory of the MapReduce and a task number of the MapReduce; (S3) distributing different log records into different HDFS cluster nodes for processing by using the MapReduce; and (S4) reading the information of the FTP servers by each HDFS cluster node, connecting the FTP servers by using an account password, spreading the log file paths prepared in advance and writing files into the HDFS through IO stream, thereby achieving the condition that multiple HDFS cluster nodes simultaneously collect the log information of the multiple FTP servers. According to the MapReduce-based FTP distributed collection method, the collection speed can be improved and the maintenance work can be simplified.

The invention discloses a self-propelled fallen jujube sweeping and picking machine. The machine comprises a rack provided with walking wheels. Two horizontally distributed collection roller brushes are installed on the lower portion of the front end of the rack. The two roller brushes are distributed in a V shape. A sweeping roller is transversely arranged on the portion, behind the roller brushes, below the rack. A jujube shovel is obliquely hinged to the portion, behind the sweeping roller, below the rack. A contour wheel is arranged on the bottom face of the front end of the jujube shovel. The front end of the jujube shovel extends below a roll brush of the sweeping roller. An impurity removing conveying belt is obliquely arranged on the portion, below the rack, at the rear end of the jujube shovel. An inclined lifting belt is arranged in the middle of the rack. A plurality of lifting hoppers are horizontally hinged to the lifting belt at intervals. Impurity falling holes are evenly formed in the lifting hoppers at intervals. The length of the lifting hoppers is matched with the width of the impurity removing conveying belt, the lower lifting hoppers are located below the tail end of the impurity removing conveying belt, and the portion, behind the lifting belt, on the rack is sequentially provided with a jujube impurity removing grid screen and a jujube collecting box from top to bottom. The roller brushes, the conveying belt and the lifting belt are all driven by an engine installed on the rack to rotate.

The invention relates to a method for collecting light energy, in particular to a method for collecting light energy by a diffuse reflection system. Its purpose is to realize the collection and utilization of large-area light energy without enlarging the area of ​​the light energy conversion material. We fixedly install the light energy collector (with light energy conversion material on the surface) in one place, and install multi-faceted reflectors scattered around the light energy collector. The reflectors generally use concave mirrors to adjust the reflected light pair The quasi-light energy collector increases the light receiving intensity of the light energy conversion material and realizes the large-area collection and utilization of light energy. The invention is beneficial to the collection and utilization of light energy in a large area, and the method can be applied to the design of household solar power generators and building lighting, and can be applied in super large solar power stations.

This invention provides a device and method of multi-channel transient electromagnetic method three-dimensional detection. The device comprises an emitting electrode pair and a distributed collection station which is arranged at a position with a first preset distance to the emitting electrode pair. The distributed collection station comprises multiple electrode pairs which are arranged on multiple measurement lines, and each of the measurement lines is provided with a receiving electrode pair or multiple receiving electrode pairs. The method comprises a step of carrying out equivalent conversion of the transient electromagnetic observation data obtained by the distributed collection station into ground virtual seismic field data, a step of obtaining the virtual seismic data of any point in a ground through the ground virtual seismic field data obtained through conversion, and a step of determining the boundary surface of a ground target body to a surrounding rock according to the seismic data of any point in the ground. The practicability is strong, the data collection precision is high, a fine interpretation cross section can be obtained, and the boundary surface of the ground target body and the surrounding rock is intepreted visually.

The invention relates to the application of distributed power and in particular to a distributed power grid which is a supplement to national power grid by utilizing distributed collection of distributed power, new energy resources and regenerative resources. The distributed power grid includes a plurality of distributed power supply apparatuses. The plurality of distributed power supply apparatuses generate electricity and then adopt parallel integrated DC boost grid-connection so as to form a DC transmission and distribution network. The DC transmission and distribution network directly transmits electricity to terminal users while independent from the AC power distribution network, and / or transmits electricity to terminal users in same lines after connecting to the AC distribution network. The distributed power grid has various forms, flexible combination, can materialize distributed arrangement, and is more conducive to healthy development of ecology. The distributed power grid has small capacity, often an independent small-sized-modular power supply of kilowatts or megawatts, is compatible with the environment, and can be arranged near loads.

A method and system for answering set-expression cardinality queries while lowering data communication costs by utilizing a coordinator site to provide global knowledge of the distribution of certain frequently occurring stream elements to significantly reduce the transmission of element state information to the central site and, optionally, capturing the semantics of the input set expression in a Boolean logic formula and using models of the formula to determine whether an element state change at a remote site can affect the set expression result.

The invention provides a network management system and a method for collecting performance. The method comprises the steps as follows: automatically connecting a master control unit when a collection unit is started, obtaining self-managed network element information according to network element distribution information which is stored in the master control unit in advance, and building a connection with the self-managed network element; issuing a collection command to the managed network element by the collection unit according to a timing period set in advance and storing the performance data returned by the network element. In the invention, the performance data of the network element by the distributed collection unit is collected, the resource is effectively utilized, the performance collection efficiency when the network management system manages a plurality of network elements is improved, loss of performance data, caused by excessive network elements in the network management is avoided. Simultaneously, the master control unit carries out unified management to configuration of all the collection units and the network elements, so as to avoid the effect of the network management on the other businesses, and improve the reliability of the network management system and the integrity of the performance data.

The invention discloses a virtual machine credible evidence collection method and a virtual machine credible evidence collection device based on a cloud computing platform network. The method comprises the steps as follows: providing a collection item and task scheduling and rule configuration interface for a user based on a browser, saving configuration information to a database, and starting a task instruction; parsing out a task according to the task instruction, scheduling and distributing collection items, sending a collection instruction to a virtual machine, and preprocessing responded data; calculating and matching the cached data and rule configuration information, filtering and merging duplicate data and redundant data respectively, and matching alarm rules; persistently saving the cached data, the filtered and merged data and the alarm data after alarm rule matching; and getting the saved data based on defined query conditions, and visually displaying the data. The method and the device are based on a variety of data communication protocols, can be used to get all kinds of information about a virtual machine under the condition that a virtual network is isolated or not isolated, and have certain application value.

The invention relates to a distributed collection processing system and method of internet metadata. The distributed collection processing system comprises a configuration management graphical user interface module, a metadata collecting assembly module and a metadata output module, wherein the configuration management graphical user interface module is used for collecting distributed assembly parameter configuration, collecting metadata and searching input logs; the metadata collecting assembly module is used for reading metadata collection configuration of each service application system, periodically establishing and executing a collection mission dispatching program in a distributed manner, and generating metadata extensible markup language files required by each service application system; and the metadata output module is in charge of outputting the metadata extensible markup language files to each service application system. Each service application system disclosed by the invention has the advantages that universal and open distributed assemblies and services are provided for the internet metadata, so that a uniform and standard data source is provided for further realizing in-station accurate search of an internet television and vertical search of content aggregation type applications.