3015 results about "Big data processing" patented technology

Here, we introduce Z-webs, including Z-factors and Z-nodes, for the understanding of relationships between objects, subjects, abstract ideas, concepts, or the like, including face, car, images, people, emotions, mood, text, natural language, voice, music, video, locations, formulas, facts, historical data, landmarks, personalities, ownership, family, friends, love, happiness, social behavior, voting behavior, and the like, to be used for many applications in our life, including on the search engine, analytics, Big Data processing, natural language processing, economy forecasting, face recognition, dealing with reliability and certainty, medical diagnosis, pattern recognition, object recognition, biometrics, security analysis, risk analysis, fraud detection, satellite image analysis, machine generated data analysis, machine learning, training samples, extracting data or patterns (from the video, images, and the like), editing video or images, and the like. Z-factors include reliability factor, confidence factor, expertise factor, bias factor, and the like, which is associated with each Z-node in the Z-web.

The invention discloses a remote sensing monitoring system for automotive exhaust emission of an urban road network. The system is mainly composed of a remote measurement device layer, a site selection and position arrangement layer, and a data processing layer. Through mobile, horizontal and vertical exhaust remote measurement devices, real-time data of the automotive exhaust emission in running is obtained; by adopting an advanced site selection and position arrangement method, the remote measurement devices are scientifically networked; and in combination with external data of weather, traffic, geographic information and the like, the real-time remote measurement data of the automotive exhaust emission is subjected to intelligent analysis and data mining by adopting big data processing and analysis technologies such as deep learning and the like, and key indexes and statistical data with optimal identification performance are obtained, so that effective support is provided for government departments to make related decisions.

The present invention relates to an apparatus for providing energy information, including: an energy information input unit configured to receive energy information for each device extracted from combined energy information according to a classification of the device using collected energy-related information; an energy information processing unit configured to generate energy use information of the device by using the energy information for each device; and an energy information transmitting unit configured to transmit the generated energy use information to a user terminal According to the present invention, it is possible to improve data stability by processing and managing cloud data by an energy information provider, and continuously improve performance through an upgrade of software and a database. Furthermore, it is possible to eliminate burden of initial investment and maintenance costs for all other devices and equipment to a user in addition to an energy monitoring device, and it is also possible to reduce the burden of learning expert knowledge on big data processing and information interpretation and of having separate permanent managing personnel by allowing cloud-based energy information management by professionals.

The invention discloses a matrix calculation method of distributed large-scale matrix multiplication based on Spark. The method comprises the following steps of adopting a system framework based on a distributed computation execution engine Spark and single-machine BLAS (Basic Linear Algebra Subprograms); defining an operation interface related to a packaging matrix in a distributed system, reading a matrix file from a distributed file system, and selecting a proper scheme to execute the distributed multiplication for the quantity of resource of a distributed computing environment and the scale of a to-be-processed matrix; if the scales of the two matrixes are small, gathering to the locality to carry out single-machine multiplication; if the scale of one matrix is smaller, broadcasting the matrix execution multiplication; and if the scales of the two matrixes are large, adopting block-based distributed matrix multiplication. As for the latter two conditions, the invention respectively provides two efficient solutions, so that the problems of low performance and the poor expansibility of the existing big data processing platform about the distributed matrix operation are solved.

The invention discloses a cloud good health service platform of elderly people and big data processing method. The platform comprises an access module, a big data module, an expert system module and an information pushing module. The big data module is used to store and check personal information, general information and safety and health record data of a wearer to process big data analyzing to obtain a standard data reference supported by samples measured data, and store health measurement results, health care plans and/or treatment regimes. The expert module is used to obtain the health measurement results, health care plans and/or treatment schemes according to the personal information, general information and safety and health record data of the wearer. The information pushing module sends the health measurement results, health care plans and/or treatment generator Transmitter optical subassembly to an intelligent terminal. The health service platform for cloud good rehabilitation allows the accuracy, scientificity and universality of health assessment, diagnostic report, health care plans and treatment schemes to be more applicable by big data analysis.

The invention discloses a mixed type processing system and method oriented to industry big data diversity application. The system comprises a distributed data collection subsystem used for collecting data from an external system, a storing and parallel calculating subsystem used for storing and calculating the collected data, and an integrated resource and system management platform used for managing the stored and calculated data. The storing and parallel calculating subsystem comprises a big data storing subsystem and a big data processing subsystem, wherein the big data processing subsystem comprises a memory calculating engine which is used for providing distributed memory abstract in a shared-nothing cluster and conducting parallel streamlined and thread lightweight processing on the collected data. The mixed type processing system and method can meet the requirements of industry big data diversity service application, the big data processing performance can be improved by ten or more times through acceleration of the memory calculating engine, and the usability, reliability and expandability of the system can be ensured through the integrated management platform.

The invention provides a traffic signal control system and method based on road condition traffic big data. The method comprises the following steps: a client of a vehicle sending client information to a cloud platform; carrying out massive traffic data information cloud storage; dividing the data into real-time data and historical data according to time sequence; establishing a corresponding static database and a dynamic database according to the type of the data; generating a corresponding optimization control scheme and a coordination control scheme; determining road conditions of a control area according to the schemes, and setting corresponding configuration schemes; selecting the corresponding optimization control scheme; adjusting signal period, signal phase sequence and relative phase difference; and driving a signal lamp to execute an instruction transmitted based on the client. By utilizing the cloud storage and based on big data processing, trend is obtained, and an optimization scheme is made according to change rules; the optimal operation phase of the traffic signals is controlled, so that people are allowed to go out conveniently; and by comparing the collected real-time data with the historical data, the coordination control scheme is generated, so that the operation phase sequence, time (period) and phase sequence of the signal lamps can be adjusted in time, and emergency traffic conditions can be coped with.

An apparatus and a method to apply 5G communication systems to IoT networks is provided. The apparatus includes technologies, such as a sensor network, machine type communication (MTC), and machine-to-machine (M2M) communication may be implemented by beamforming, multiple-input multiple-output (MIMO), and array antennas. Application of a cloud radio access network (RAN) as the above-described big data processing technology may also be considered to be as an example of convergence between the 5G technology and the IoT technology. The disclosure relates to a method and an apparatus for controlling network access in a next generation mobile communication system.

The invention discloses a vehicle security risk warning and accident reconstruction system and method of big data processing. The system comprises a vehicle-mounted video terminal and a cloud big data processing platform, all kinds of parameters, video images, voice information and the like of vehicle operation are analyzed by the big data processing platform in the background to provide effective information for accident authentication, driving behaviors of drivers, vehicle conditions and the like, and provide a novel method for confirmation of responsibility, vehicle improvement, traffic management and the like. The system is a research and development system of a key common technology based on cloud computing, collected videos are encoded in an H.264 manner, stored locally and are circularly covered, and after an abnormal driving behavior is detected, video data within 10-30 seconds before and after the time point are transmitted to a service system for accident analysis and reconstruction. The system solves the vehicle driving security risk problem and provides a vehicle security public service platform.

The invention relates to a big data-based social insurance index warehouse building system and method and belongs to the technical field of big data application. The invention aims to build an accurate and regularly-updated multi-dimensional statistical index base based on the organization of cubic structure storage by using the big data processing technology and distributed storage technology. The building system consists of a data source module, a business data collection module, a data pretreatment module, an index calculating module, an index metadata storage module, a data storage module and a data query module. The building method is based on the building system. The system and the method uses social insurance business data as the data source and builds an accurate and regularly and incrementally-updated index data warehouse by using the big data processing technology and distributed storage technology, and allows rapid index multi-dimensional analysis and query.

The invention discloses a new-energy electric automobile overall performance improving system and method based on big data. The system comprises a local system, a big data processing platform, and a remote virtual system. The local system is used for driving and controlling a local vehicle, and uploading relevant parameters of the local vehicle to the big data processing platform via a monitoring network. The big data processing platform is used for performing data classification, statistic, analysis, storage, and excavation on the relevant parameters of the local vehicle. The remote virtual system is used for establishing a virtual model of a vehicle, a motor, a battery, and a motor controller, using the various parameters processed by the big data processing platform as an input of the virtual model, optimizing the virtual model by using a vehicle optimum parameter as an output target, and downloading various parameters generated by the virtual model to the local system. The local system can be optimized according to the downloaded various parameters, and returns the downloaded various parameters to a manufacturer in order to provide data reference for the production process of the manufacturer and achieve improvement in the new-energy electric automobile overall performance.

The invention relates to the technical field of intelligent driving, in particular to an anti-fatigue safe driving method for vehicles based on big data. A driver identity identification anti-theft function, a driver biological monitoring technology, a vehicle driving state monitoring technology, a big data synchronization processing technology, an early warning reminding function and an assisteddriving function are integrated together. The driver and vehicle driving state monitoring can be simultaneously performed, and whether the driver is tired or asleep or not is comprehensively judged based on the state of the driver and the vehicle. In the case of fatigue driving or sleeping driving of the driver, after comprehensive comparison judgment of the big data processing, the early warningreminding function is opened and the intelligent assisted driving function is turned on to prevent human error from causing a fatal vehicle crash. All the driving data can be saved to the cloud in synchronization, and when the driver drives other vehicles, the previous driving data of the driver is downloaded to the current vehicle for synchronization.

The invention provides a job scheduling management system and method. The job scheduling management system structurally comprises an FTP (File Transfer Protocol) server, a client, a control node and a plurality of processing nodes. The job scheduling management method includes that a task node sends task running state information including middle progress state, error or exception of a task to the control node, and the control node executes exception processing or stopping according to an exception or error condition; the control node starts a heartbeat test to test whether a node is alive or not when the control node do not receive heartbeat information of the task node, and the control node reschedules all uncompleted tasks when the node crashes. Compared with the prior art, the job scheduling management system and method improves the level stack of large data processing software, enables Hadoop to break resources telescopic bottlenecks and business expansion limit of a single data center, promotes the integration of multiple data centers, and further expands computing capabilities and improves the resource utilization rate.

The invention relates to a computer HDFS field and discloses an improved method aimed at small files of an HDFS. Partial authority of a Namenode is delegated to a Datanode which caches element data information of part of small files and the Datanode is used for processing the writing and reading request of most small files. Burdens of the Namenode are reduced to the utmost. The new processing method is provided for solving the problem that the efficiency in processing the small files by the HDFS is low. The improved method aimed at the small files of the HDFS can effectively solve the problem that burdens of a single node, namely the Datanode are excessively large and pressure of the small files is distributed to the Datanode. Thus, large file and small file processing efficiency and performance of the integral large data processing colony are quite ideal.

The invention provides a caching optimizing method of internal storage calculation. The method includes the steps that monitoring codes are inserted into a Spark source program, and dynamic semantic analysis is performed on an application program to construct a DAG; out-degrees of all vertexes in the DAG are calculated, RDDs of the vertexes of which the out-degrees are larger than one are screened, and the screened RDDs are RDDs needing to be cached to an internal storage; according to a greedy algorithm, the execution sequence of Action is adjusted so that the access sequence of RDD data calculation can be optimized; the weights of the RDDs are calculated, and the replaced RDDs in the internal storage are determined according to an internal storage replacement algorithm; it is determined how to process the replaced RDDs according to a multi-level caching algorithm. By the utilization of the caching optimizing method of internal storage calculation, a programmer does not need to examine and weigh internal storage using and display the RDDs of the appointed loading internal storage in the process of programming, programming loads of the programmer are reduced, meanwhile, the utilization rate of the internal storage is improved, and then the speed of processing big data is increased.

The present invention provides a method for processing video image information of mass data and a database system. According to the present invention, a big data processing technology is utilized and processing, storage, switching and analysis application of the mass video image information data. According to the present invention, for the mass video image information, an event is used as a core and structural and/or unstructured data based on event contents is formed by defining the event contents of a video image so as to establish video image content information flow on the basis of video streaming. Therefore, the database system disclosed by the present invention constructs a distributed storage platform and a distributed parallel analysis processing mechanism for the video image information and the structural and/or unstructured data based on the event contents so as to implement real-time flow processing on the video image content information flow. According to the present invention, data sharing and exchange of the video image information are realized, a data isolated island is overcome, and deepening application of data is strengthened.

The invention discloses a medical system big data processing device and method, and relates to a medical information management technology. Main content is achieving establishing and sharing of medical big data through a website platform of a modern network system, the content comprises online medical big data resource query, online registration, online sharing of medical resource big data of various medical institutions, online medical service data processing, real-time medical insurance settlement of medical prescriptions and the like. According to medical system big data processing device and method, online, real-time and whole-society medical resource sharing is achieved, and the whole-society medical service and medical resource utilization efficiency are greatly improved.

The invention discloses a security and protection monitoring big data processing method and system based on cloud computing. The method comprises the following steps: obtaining security and protection monitoring data of an Internet of Things node; dividing the obtained security and protection monitoring data into first security and protection monitoring data and second security and protection monitoring data according to a set security threshold; sending the first security and protection monitoring data to a block chain platform for data processing; and sending the second security and protection monitoring data to a cloud platform for data processing. According to the security and protection monitoring big data processing method and system, the second security and protection monitoring data with relatively low security requirements are sent to the cloud platform for data processing, thereby ensuring the security and protection monitoring big data processing efficiency; meanwhile the first security and protection monitoring data with relatively high security requirements are sent to the block chain platform for data processing, and DDOS attack is defended by the features of openness, transparency and non-tempering of the block chain technology, thereby eliminating the potential safety hazards caused by the DDOS attack to the cloud platform and the distributed Internet of Things nodes, and being securer. The security and protection monitoring big data processing method and system can be widely used in the field of security and protection.

The invention relates to the technical field of big data processing, and particularly relates to a block chain based large scale data sharing method. The method is based on a block chain technology, and pieces of block data are connected in series according to the occurrence sequence to form the whole large scale data; corresponding functions are realized through a data access module, a payment module and a data evaluation module; the data access module is provided for a data source reporting party to report a data source and earning value pricing of a response, records the reported information and is used for receiving earnings; the payment module issues relevant tokens, a data demanding party pays tokens according to data utilization and acquisition conditions, the data source reportingparty acquires the tokens, and an intermediate party acquires service tokens; the tokens can be converted into actual earnings; and the data evaluation module is provided for the data source reportingparty to define an earning mode of the data, and a block chain data sharing network performs pricing on the data earnings according to the set method and lists pricing lists of various types of data.The method solves problems of slow response speed, IO bottleneck and the like in the traditional data centralization process, can promote utilization of various types of data and improves the utilization value.

The invention relates to a database technology, in particular to a data desensitization method and related equipment. The method comprises the steps of sensitive field positioning, sensitive field generation rule base establishing, desensitized data generating and the like. Accordingly, massive high-simulation data desensitization processing can be achieved, the purposes of optimizing the processing process and improving the data processing efficiency and simulation degree are achieved, and reliable protection on sensitive and private data is achieved while big data processing and utilizing are achieved.

The invention discloses an execution strategy generation method and device under cross-IDC (Internet Data Center) big data processing architecture, and relates to the field of big data processing. According to the method, current available computing resources and internet transmission resources reported by each IDC cluster are received; on the basis of a task classification model base, optimum execution strategies of tasks are determined according to the types of the tasks; the optimum execution strategies of the tasks comprise the demanded computing resources and consumed time of the tasks; the current optimum execution strategies of the tasks are obtained through calculating according to every two of [cluster available computing resources and task data distribution] matrixes in an available resource pool and [demanded computing resources and consumed time of each task and cluster available internet resources] matrixes. Compared with the existing scheme, the technical scheme provided by the invention has the advantages that the computing and internet resources can be utilized in a high utilization rate on a big data processing platform built cross-IDC, the service quality demands of different services can be satisfied, and the flexible service extension capability is realized.

The invention relates to a mass online historical data-based power system transient stability evaluation method. The method comprises the steps of selecting mass online historical data to generate emulated data; based on the emulated data, extracting statistic features and generating an initial sample set; extending an instable sample and compressing a stable sample in the extended-boundary manner so as to form a calculation sample; based on the calculation sample, conducting the classification model training and the parameter optimization based on the SVM algorithm to finally form a classification model, and evaluating the transient stability of a power system. According to the technical scheme of the invention, a reusable theoretical solution is established from an original online calculation result to a main problem involved in forming an actual available SVM model. The above technical scheme reflects some characteristics of the mass data processing process and provides a novel method for the mass data technical packet of the power system.