104 results about "Data scraping" patented technology

The invention discloses a website AI intelligent risk assessment real-time evasion system and an evasion method thereof. The system comprises a VPN connection module to ensure that data capture is safe and hidden; the data capturing module collects various types of information of a target website in real time; the risk feature extraction module performs multi-dimensional feature extraction; the AI risk analysis and evaluation model identifies risks from multiple aspects of images, scripts and the like and grades the risks; the risk avoiding and warning module executes a hierarchical response strategy according to a result; the feedback and model optimization module collects a feedback and sample optimization model. According to the method, website risk avoidance is realized through the steps of VPN connection, data acquisition, feature extraction, risk assessment, strategy execution, model optimization and the like. According to the method, VPN and artificial intelligence technologies are combined, website risks can be comprehensively evaluated in real time, access strategies are dynamically adjusted, complex network attacks are effectively coped, and user network access safety is guaranteed.

The invention discloses an anti-phishing webpage detection system and method. The anti-phishing webpage detection system comprises a webpage data capture module, a risk analysis module, a malicious webpage judgment module and a response decision module, relates to the technical field of anti-phishing webpage detection. According to the method, multi-dimensional data, including URL information, domain name information and webpage content information, of a webpage are extracted through the webpage data capturing module, a comprehensive original data basis is provided, and accurate support is provided for follow-up risk analysis; according to the method, comprehensive risk analysis is carried out by utilizing various webpage features including URL features, domain name features and content features, and potential phishing webpages can be effectively identified by calculating risk scores of the URL, the domain name and the webpage content; according to the method, the web pages are classified according to the comprehensive risk scores, the high-risk web pages are intercepted, the emergency response logs are generated, the low-risk web pages are continuously monitored, and safe browsing of a user is ensured.

The invention discloses an intelligent public opinion research and judgment method and system for public security actual combat, and the method achieves the high-throughput data capture and real-time public opinion inflow through multi-source crawler, breakpoint resume, incremental collection and asynchronous concurrent processing, and greatly reduces the workload of manual screening. A unified, quantifiable and structured public opinion risk grading standard is established, and risk grading standard unification is realized by constructing three types of structured public opinion dictionaries of event subjects, event properties and sensitivity grades; the grading rule is quantified and objective and adapts to the grading system of the public security actual combat, and the subjectivity and inconsistency of research and judgment are remarkably reduced; and the recessive public opinion recognition capability is improved through the deep semantic model. According to the method, the BERT model, keyword vector fusion and the police affair large model are combined, so that accurate analysis of complex semantics such as metaphor, black talk and irony mood is realized; high-robustness semantic matching and duplicate removal are carried out; through accurate extraction of risk factors and event identification, the problem of high missed judgment rate in the prior art is improved.

The invention discloses a manipulator target grabbing path intelligent planning method based on visual depth perception. The manipulator target grabbing path intelligent planning method comprises the following steps that grabbing target point cloud data are obtained and preprocessed to generate a structured point cloud data set; executing SuperPoint division and adjacency relation construction, and generating a SuperPoint graph structure; constructing a plurality of groups of action guide graphs and embedding the action guide graphs into an SPGNet attention mechanism to form an improved SPGNet structure; inputting structured point cloud data, extracting node features and capturing response, and generating a capturing candidate region and a sorting result; determining grabbing point coordinates and grabbing posture parameters according to the sorting result; path search and obstacle prediction are executed in combination with the tail end pose of the mechanical arm, and grabbing path trajectory data are generated; new point clouds are collected in the grabbing process, a structure topology self-adaptive updating unit is called to reconstruct a graph structure, and a final control instruction is output. According to the invention, high-precision grabbing and path self-adaptive control of the manipulator in a complex environment are realized, and the dynamic operation stability is effectively improved.

The invention provides a manipulator commodity grabbing system and method based on visual recognition, and relates to the technical field of equipment control, and the method comprises the steps: collecting the image information of a commodity in a working area based on a multi-modal visual collection device, analyzing the commodity image information, and obtaining the depth information of a commodity image; the intelligent detection module is used for intelligently analyzing the depth information of the commodity image, extracting image features of the commodity image and determining attribute information of the commodity according to the image features; the optimal grabbing box parameter acquisition module is used for analyzing the attribute information of the commodity to generate optimal grabbing box parameters; the spatial pose acquisition module is used for determining spatial pose data of the manipulator during commodity grabbing according to the optimal grabbing frame parameters; and the grabbing control module is used for generating a grabbing control instruction of the manipulator according to the spatial pose data, and controlling the manipulator to grab the commodity according to the grabbing control instruction. And the grabbing success rate and the operation efficiency in a complex scene are remarkably improved.

The invention belongs to the technical field of data safety, and particularly relates to a visual safety brain modeling and early warning platform of an expressway toll collection system, which comprises a data capturing module used for acquiring real-time interaction data and a toll station three-dimensional model of the expressway toll collection system, and constructing an expressway toll station panoramic three-dimensional model in a three-dimensional environment; the dynamic display module comprises a first floating information board, a second floating information board, a third floating information board, a fourth floating information board and a fifth floating information board; the early warning module is used for analyzing the real-time interaction data acquired by the data capturing module to find abnormal data and tracing an abnormal event; and the adjusting module provides a dynamic adjusting mechanism according to the matching result. According to the method, the distribution strategy is adjusted in real time to cope with changes of emergency situations and tasks, the empty window period occurring in the adjustment process is avoided as much as possible, and network security management vulnerabilities of the highway toll collection system can be prevented.

The application provides a mechanical hand commodity grabbing system and method based on visual recognition, and relates to the technical field of equipment control. The system comprises the following steps: collecting image information of commodities in a working area based on a multi-modal visual collection device, analyzing the commodity image information, and obtaining depth information of the commodity image; an intelligent detection module is used for intelligently analyzing the depth information of the commodity image, extracting image features of the commodity image, and determining attribute information of the commodity according to the image features; an optimal grabbing frame parameter acquisition module is used for analyzing the attribute information of the commodity to generate optimal grabbing frame parameters; a spatial pose acquisition module is used for determining spatial pose data of the mechanical hand in grabbing the commodity according to the optimal grabbing frame parameters; and a grabbing control module is used for generating grabbing control instructions of the mechanical hand according to the spatial pose data, and controlling the mechanical hand to grab the commodity according to the grabbing control instructions. The grabbing success rate and work efficiency in a complex scene are significantly improved.

The present invention discloses a real-time website AI risk assessment and avoidance system and its avoidance method. The system includes a VPN connection module to ensure secure and concealed data capture; a data capture module to collect multiple types of information from the target website in real time; a risk feature extraction module to perform multi-dimensional feature extraction; an AI risk analysis and assessment model to identify and rate risks from multiple aspects such as images and scripts; a risk avoidance and warning module to implement a graded response strategy based on the results; and a feedback and model optimization module to collect feedback and sample optimization models. The method implements website risk avoidance through the steps of VPN connection, data collection, feature extraction, risk assessment, strategy execution, and model optimization. Combining VPN with artificial intelligence technology, the present invention can comprehensively and real-timely assess website risks, dynamically adjust access policies, effectively respond to complex network attacks, and ensure user network access security.

The invention provides an image data processing method and device, equipment, a storage medium and a program product, and the method comprises the steps: responding to a first display interrupt signal arriving according to a preset period and a data capture request, determining a state of a coding operation of first image data triggered based on a second display interrupt signal arriving according to a preset period, the second display interrupt signal arriving earlier than the first display interrupt signal, and the first image data being image data of a first frame stored in a frame buffer queue before the second display interrupt signal arrives; triggering a third display interrupt signal in response to determining that the encoding operation of the state representation for the first image data is completed; and based on the third display interrupt signal, obtaining second image data currently stored in the first frame from the frame buffer queue, and processing the second image data. According to the method, the display interruption can be dynamically adjusted, so that the coding efficiency can be improved, and the frame loss ratio can be reduced.

The invention provides an ISTIO-based content anti-capture method, and relates to the technical field of data security, and the method comprises the steps: building an ANP model, carrying out the dynamic analysis on the basis of the relation of a request state, a duration and an associated request, carrying out the dynamic processing and real-time control through the early warning stage and the blocking recognition stage, and carrying out the real-time analysis of the state of the request, the duration and the associated request. On the basis of supporting blocked dynamic release, fine-grained connection control and release are added, data security is powerfully guaranteed, the security and performance of the system are effectively improved, the workload of personnel intervention is reduced, abnormal requests and data capturing behaviors are dynamically recognized, real-time blocking is carried out, and the data capturing efficiency is improved. And meanwhile, a dynamic release mechanism of blocking behaviors is supported, so that the normal access experience of a user is ensured while the network security is ensured.

The present invention provides a multi-channel data fusion method and system, which relates to data processing technologies. The method is based on a data acquisition unit to capture data from data sources corresponding to each user terminal, obtain acquisition data of multiple dimension features corresponding to each user terminal, construct an initial data column corresponding to each user terminal, perform arrangement and update on the initial data column according to the acquisition data corresponding to each dimension feature to obtain a fusion data column, acquire fusion metrics of multiple dimension features corresponding to the user terminal in the fusion data column, calculate user portrait data corresponding to each user terminal according to the fusion metrics, generate push data corresponding to each user terminal according to the user portrait data, and send the push data to the corresponding user terminal.

The application provides an automobile IBC product offline appearance identification detection system and method, and relates to the technical field of automobile part detection. The system comprises: a feeding vision module for identifying the placement posture of the IBC product to be detected on the conveying line and outputting posture data; a mechanical hand module for grabbing the product according to the posture data and transferring it to a detection station; a rotary table module arranged at the detection station for fixing and rotating the product; a vision detection module for scanning the surface of the product during rotation, acquiring three-dimensional image data, and performing defect detection and classification based on the three-dimensional image data; a data tracing module in communication with a central control system and an enterprise MES system for binding the unique identification of the product and the detection data; and the central control system for coordinating the operation of each module and controlling the mechanical hand module to perform a sorting and stacking operation corresponding to the result according to the result of defect detection and classification. Full automation and full coverage of the detection process are realized.

The present invention belongs to the technical field of data tracking, and provides a system and method for tracking a data crawling path based on object storage, comprising: screening out a data text candidate set based on subject keyword comparison, processing to obtain a retrieval range value based on the data text's domain of origin and the number of texts, judging whether it is necessary to reduce the index range, and if so, reducing the index range by combining a Boolean operator, thereby improving index matching efficiency; utilizing natural language processing technology to obtain semantic vectors of the data text and semantic vectors of the data retrieval text, and processing to obtain similarity; combining feature data of the data text in the corresponding new data text candidate set, and performing correlation analysis to obtain a correlation value, and combining the similarity between the data text and the data retrieval text, processing to obtain a matching degree, and recommending corresponding data text to a user based on the matching degree, thereby improving the crawling accuracy of stored data.

1. Name of the Design Product: Graphical User Interface of Teacher Portrait for Electronic Devices. 2. Use of the Design Product: For electronic devices. 3. Design Key Points of the Design Product: Lie in the graphical user interface in the electronic device. 4. Picture or Photograph that Best Illustrates the Design Key Points: Front View. 5. The electronic device is of conventional design, and other views are omitted. 6. Use of the Graphical User Interface: This design product is used to generate a teacher portrait through data scraping, integration, and analysis, and can intuitively display various information of teachers. 7. Human-Computer Interaction Method of the Graphical User Interface: The front view is used for teacher information display, including basic information, teacher honors, and teacher awards, etc.; Click the "Rank Details" button in the upper left corner of the interface in the front view to jump to the interface change state diagram 1, and the interface change state diagram 1 shows the chart information of the teacher rank; Click or hover the mouse pointer over the year node in the chart in the interface change state diagram 1 to jump to the interface change state diagram 2, and the interface change state diagram 2 shows the corresponding rank information.

The invention discloses a perimeter intrusion monitoring method and system for a tunnel portal high-speed rail line, and relates to the technical field of intelligent monitoring, and the method comprises the steps: obtaining a line tunnel portal sequence driving data set according to a high-speed rail line advancing tunnel portal sequence of a sorted target high-speed rail line, and carrying out the intrusion risk recognition, obtaining a tunnel portal sequence intrusion risk coefficient set; meanwhile, interference analysis is carried out on a high-speed rail line advancing tunnel portal sequence, a tunnel portal sequence interference factor set is obtained, a tunnel portal sequence intrusion risk coefficient set is corrected, and a preset video data capture frequency is determined based on the corrected tunnel portal sequence intrusion risk correction coefficient set; and then differentiated monitoring analysis is carried out on the high-speed rail line tunnel portal, and tunnel portal perimeter intrusion information is determined for early warning management and control. The technical effects that differentiated intelligent monitoring and analysis of the tunnel portal are achieved, the accuracy of intelligent monitoring and analysis and the processing efficiency of intrusion monitoring data are improved, and then the timeliness of early warning of the perimeter intrusion of the tunnel portal is ensured are achieved.

The invention relates to the technical field of digital employee data processing, in particular to a social security digital employee data processing and model building method based on cloud edge collaboration, and the method specifically comprises the following steps: collecting multi-source social security data, and capturing a policy text; constructing a training sample based on the collected multi-source social security data and policy text, and dividing the training sample into a training set, a verification set and a test set in proportion; constructing a social security digital employee question and answer model by adopting a neural network architecture based on an encoder and a decoder, inputting the training set into the model, and processing the training set by the encoder and the decoder to generate an answer sequence; training the model, performing multiple iteration training on the model, ending the training of the model until a preset iteration stopping condition is met, and selecting the model with the best performance through the verification set to be the optimal model; and inputting the test set into the optimal model to generate an answer sequence. According to the method, the social security digital employee question and answer model is constructed, so that the processing efficiency can be improved, the data processing accuracy is enhanced, and the operation cost is reduced.

The invention discloses a big data network security adaptive defense system based on deep learning, and relates to the technical field of network security, the system comprises a monitoring center, the monitoring center is in communication connection with a deep learning module, a data capture module, a data processing module, a data analysis module and an adaptive defense module; a deep learning model is utilized to quickly identify risks of various network security events possibly occurring in real-time network interaction information, so as to obtain defense modes for each network security event, and meanwhile, the defense intensity of the corresponding defense modes is configured according to risk coefficients of various network security events, so that the defense efficiency is improved. Therefore, the limitation of a single defense mode is avoided, excessive defense is avoided, meanwhile, the adaptive configuration defense system can improve the network security to the maximum extent, and the network security defense efficiency is improved.

The invention discloses a multi-protocol data capturing method and system, and relates to the technical field of data capturing. The multi-protocol data capturing method is applied to the multi-protocol data capturing system, and the multi-protocol data capturing system comprises a client and a server which is in communication connection with the client through multiple network protocols. The multi-protocol data capturing method comprises the following steps: a client sends a data capturing request to a server through one of a plurality of network protocols; the server calls a corresponding processor to perform data capture based on the request type carried in the data capture request; the server converts the grabbed data into a unified intermediate format; and the server converts the data in the unified intermediate format to the client through one of the plurality of network protocols and sends the data to the client. According to the multi-protocol data capturing method and system disclosed by the invention, cross-language and cross-protocol data capturing service can be realized.

The invention discloses an automatic processing system and method for electric power information communication scheduling operation data. The system comprises a man-machine interaction module, a data capture module, a report processing module, a file management module and a data verification module. The man-machine interaction module is used for realizing data interaction between the system and an operator; the data capturing module is used for automatically extracting work order data through a cross-communication scheduling core business system; the report processing module is used for realizing monthly report updating, daily report generation and end-of-month data archiving; the file management module is used for managing template files, result files and log files related to the system; and the data verification module is used for verifying the accuracy of the data processing result. The method disclosed by the invention is high in data processing efficiency, good in integrity and relatively high in accuracy.

The application discloses a kind of methods for assisting large model networking query, the method is by receiving and encapsulating as standardization demand to parse user query request, matching dynamic residential proxy IP pool, IP dynamic rotation and load balancing mechanism are started, generate standardization collection task, realize multi-platform data scraping through multi-tool cooperation, anti-crawling avoidance and high concurrency scheduling, clean, double check and structure transformation are carried out to original data, add standardization field annotation, push structured data to large model and monitor data satisfaction degree;The application is through dynamic residential proxy IP pool and IP rotation mechanism, avoid regional restriction and anti-crawling interception, with headless browser, API identification technology, adapt to dynamic and waterfall page, improve collection integrity and efficiency, after cleaning, check and structure transformation, combined with automatic completion query condition and daily optimization mechanism, the accuracy, timeliness and stability of large model real-time query response are greatly improved.

The invention provides an online banking service processing method and system, and relates to the field of data processing, and the method comprises the steps: obtaining a to-be-handled online banking service, and positioning a target optimal shield device in a plurality of preset optimal shield devices based on a bank identifier of the to-be-handled online banking service; inputting an optimal shield password in an online banking login page corresponding to the to-be-handled online banking service through a simulation key-in module, and obtaining a target digital certificate in target optimal shield equipment in response to the fact that the input optimal shield password is verified to be correct; inputting a login password through a simulation key-in module, and entering a service operation interface in response to the fact that the target digital certificate is valid and the input login password is verified to be correct; entering a specific function interface from the service operation interface based on the to-be-handled online banking service and preset online banking interface characteristics, and performing data capture on the specific function interface to obtain original service data; and obtaining target business data according to the original business data. And the whole-process automatic processing of the online banking business is realized.

The invention relates to the technical field of big data intelligent decision, and provides a love and marriage individual positioning method and system based on multi-dimensional data fusion, and the system comprises a multi-modal input module, a dynamic data collection middle platform, a data fusion processing module, an SWOT analysis matrix generator and a visual feedback module. According to the method, the data capture and analysis capability of a big data model is utilized, multi-dimensional dynamic weight fusion is carried out on regional economic indexes, environmental factors and personalized basic data of users, and the users are helped to objectively know the love and marriage environment where the users are located and carry out relative positioning through quantitative evaluation and SWOT analysis; and meanwhile, the advantages, disadvantages, opportunities and threat items of the user in the love and marriage environment are presented by virtue of visual analysis, and corresponding strategies are provided, so that the love and marriage anxiety of the user can be effectively relieved.

This application discloses a method, apparatus, device, and storage medium for constructing a dataset based on 3D games. The method first obtains a compiled game data scraping script for the target 3D game; runs the target 3D game and obtains game screen data based on the game data scraping script, wherein the game screen data includes image data and non-image data; then cleans the game screen data to obtain sample data to be labeled; finally, a data labeling system is constructed, and the sample data is labeled using the data labeling system. It can be seen that this application achieves the acquisition of synthetic data from 3D games, which is more efficient than data acquisition in the real world. Furthermore, the synthetic game data can extract corresponding semantic information from the game, which can greatly simplify the data labeling process, enabling data obtained from games to be used for model training.

The application provides a BOP data acquisition EWI generation method, medium and system, and belongs to the technical field of production business data.The application firstly constructs a multidimensional BOP data load matrix, accurately calculates the contribution value of each parameter through a thermodynamic entropy equation, then establishes a standard EWI basic matrix, and optimizes the BOP data acquisition strategy and calculates the data integrity score by using the minimum spanning tree algorithm.In the data processing stage, the EWI mergable index is accurately generated by using the Lagrange multiplier method, the transverse error is calculated by using the Euclidean distance function to analyze the spatial relationship between parameters, the longitudinal error is calculated by using the Fourier transform to monitor the time change trend of parameters, the two kinds of errors are comprehensively formed into a hybrid error evaluation result, and finally, the comprehensive index accurately reflecting the production quality state is output in combination with the EWI mergable index.

The invention relates to an analysis method and device for a chip. The method comprises the steps that data capture is conducted on latest results of a target chip by different analysis tools to obtain target data, the number of the analysis tools is at least two, and analysis conducted on the target chip by the analysis tools is different; taking the target data corresponding to each analysis tool as an analysis basis and sending the analysis basis to other analysis tools; performing anomaly evaluation based on the target data to obtain an evaluation result; under the condition that the evaluation result is abnormal, the abnormal evaluation result serves as an analysis basis for optimization and is sent to a first tool in the analysis tools, and all the analysis tools are controlled to end analysis until an end condition is met; the end condition is that the evaluation result is normal. The accuracy and speed of chip analysis are remarkably improved, multi-dimensional linkage analysis is achieved, and the method is suitable for multiple research and development stages of chip development and wide in application range.

The invention discloses an information extraction method and device, equipment and a medium, and relates to the technical field of electronic commerce. According to the scheme, the target sites are automatically collected, and the types of the target sites are determined by using a rule-based hierarchical feature detection technology, so that the retrieval times can be reduced, the flow consumption can be reduced, and the recognition accuracy can be improved; the links of the pages in the corresponding sites are obtained according to the types of the target sites, efficient collection of the links of the commodity detail pages is achieved, manual intervention of personnel is reduced, and the grabbing efficiency and accuracy in large-scale sites are greatly improved; meanwhile, the pages corresponding to all the links are clustered and classified and recognized, automation of page classification and recognition is achieved, and finally the target information is recognized based on the target page. According to the scheme, through automatic and intelligent technical means, the data capturing efficiency and quality in large-scale e-commerce sites are greatly improved, the requirement for real-time data acquisition can be met, and the method is particularly suitable for large-scale data capturing.

The invention discloses an abstract content generation method and device based on improved Bert, and the method comprises the steps: firstly setting a seed URL, selecting a webpage link, capturing webpage data through employing a crawler frame, carrying out the duplication removal of the webpage data, carrying out the classified storage of webpage contents through employing an efficient storage mechanism, and carrying out the generation of abstract contents based on the classified storage of the webpage contents. The method comprises the following steps of: performing multi-level filtering and identification on advertisement contents by using an HTML (Hypertext Markup Language) structure, URL (Uniform Resource Locator) and href identification and a neural network model, outputting an advertisement dichotomy result, performing image-text preprocessing by using an improved Bert model based on a dynamic self-adaptive decision boundary, and generating a webpage abstract by combining semantic information of texts and images through a theme-enhanced multi-modal fusion method. According to the method, data capture and storage are carried out in combination with a crawler framework, advertisement multi-level filtering is carried out based on rules and a neural network model, and a dynamic self-adaptive decision boundary image-text preprocessing and theme enhancement multi-modal fusion method of an improved Bert model is adopted, so that efficient webpage data automatic processing is realized.

The invention discloses a mechanical arm grabbing anti-sliding control method, and relates to the technical field of intelligent sorting and recycling, and the mechanical arm grabbing anti-sliding control method comprises the following steps: collecting visual information of an object in real time, and determining grabbing information of a clamping jaw on a mechanical arm based on the visual information; wherein the visual information comprises displacement data, three-dimensional shape data and anti-skid data, and the grabbing information comprises a grabbing position, grabbing force, a clamping jaw opening and closing angle and a mechanical arm tail end attitude angle. Object characteristics are accurately described through displacement data, three-dimensional shape data and anti-slip data, adaptive grabbing information is output in combination with a machine learning model, and the slip risk is reduced from the source; a secondary grabbing mechanism with the preset sliding path matched with the real-time sliding path is designed according to the sliding risk, the multiple preset sliding paths are dynamically simulated, the optimal preset sliding path is dynamically matched in combination with the real-time track, rescue grabbing of the sliding object can be achieved, and the grabbing success rate can be increased.

This application discloses an adaptive data crawling method, device, storage medium, and apparatus, relating to the field of Internet technology. The method includes: configuring request parameters based on an interface request initiated by a target user, a preset configuration file, and a preset custom mapping rule table to obtain data configuration information; performing pre-crawling verification on the data configuration information to obtain pre-verification results; and matching a target-level thread pool from a preset-level thread pool based on the pre-verification results and a preset sliding window algorithm, and activating threads in the target-level thread pool to complete data crawling. Compared to traditional data crawling methods that require manual intervention for system updates and maintenance, affecting the accuracy and success rate of data crawling, this application can automatically map request parameters, notify the custom mapping rule table to assemble complex parameters and perform pre-verification of parameters, and filter thread priorities to complete data crawling, thereby improving the accuracy and success rate of data crawling.

This invention relates to a method, apparatus, equipment, and medium for automatic configuration and cost calculation of miscellaneous charges. The method includes constructing a multi-dimensional standardized template library and completing import verification to form a set of callable pricing rules; upon receiving a calculation request, retrieving and preprocessing order-related data across systems; filtering candidate rules through a rule matching engine and determining the optimal billing basis by weighting and sorting according to a preset priority system; adapting the billing mode and formula based on this basis, initiating batch parallel calculations and handling anomalies synchronously; and supporting multi-format export and cross-system synchronization after integrating the results. This invention includes modules for template library construction and verification, data retrieval and preprocessing, rule matching and sorting, cost calculation and anomaly handling, and result integration and synchronization. It achieves automatic configuration and efficient calculation of miscellaneous charges, improves data accuracy and process closure, adapts to the needs of multiple scenarios throughout the warehousing and logistics process, and significantly reduces labor costs.