Building industry supplier portrait construction system

Abstract

The invention discloses a building industry supplier portrait construction system, and belongs to the technical field of big data analysis, and the system comprises a data collection and preprocessing module which is used for achieving a data collection and preprocessing task; the multi-dimensional cross analysis module is used for realizing a data analysis task; the supplier portrait construction module is used for realizing a portrait construction task; wherein the data acquisition and preprocessing task, the data analysis task and the portrait construction task are all completed in an offline environment by means of a local server or a distributed computing cluster; and uniformly storing the processed data and the constructed supplier portraits in a local database or a data warehouse for enterprises to call and view. By adopting the technical scheme of the invention, the requirement of constructing an accurate, efficient and safe supplier portrait in the building industry can be met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data analysis, in particular to a construction industry supplier portrait construction system. BACKGROUND

[0002] With the vigorous development of the construction industry and the accelerated promotion of digital transformation, the importance of construction industry supply chain management is increasingly prominent. In the era of big data, data elements and AI technology have become the core driver of new quality productivity. Under this background, constructing accurate supplier portraits is of great significance to improving the efficiency and scientific nature of construction enterprises in supplier evaluation, selection, management, etc. In the past, construction enterprises relied on traditional manual experience judgment in supplier management, lacking comprehensive and in-depth analysis of suppliers. Today, the construction industry has gradually begun to introduce data analysis technology, trying to improve the significant defects in traditional enterprise supplier management, promoting the fragmentation of supplier relationship dimensions to modelization, the change from manual response to dynamic response in supplier information change, and the change from traditional single-dimensional data analysis to model correlation analysis. For example, some large construction enterprises have begun to try to use big data technology to analyze the performance, qualifications, etc. of suppliers, and some enterprises have constructed a relatively simple supplier information database. Overall, these early explorations have many limitations, and a mature, comprehensive and efficient supplier portrait construction system based on offline data analysis has not yet been formed.

[0003] In the field of supplier profiling based on data analysis in the construction industry, there are currently two representative technologies: advanced large model technology relying on real-time online data processing and traditional simple data processing method. The first is the AI large model released by a certain construction enterprise, which relies on matrix model technology such as supply level, performance evaluation, customer insight, and dynamic risk to build a multi-dimensional supplier feature portrait system including performance, behavior, and ability. The working principle is to analyze and process a large amount of real-time online data based on enterprise data to generate a supplier portrait. During data processing, a stable data transmission network is needed to transmit data from various places such as construction sites to data processing equipment, which relies on powerful real-time computing capabilities to analyze, integrate, and model these data to form a multi-dimensional supplier portrait. The second is the traditional simple data processing method, where some construction enterprises use a combination of simple database storage and conventional data analysis software to build a supplier portrait. The product structure is relatively simple, with a database used to store supplier-related data and conventional data analysis software used to process these data. The working principle is to input limited structured data into the database and use conventional data analysis software to perform simple statistical analysis on these structured data, such as summing and averaging the performance data of suppliers, to build a supplier portrait.

[0004] The AI large model released by a certain construction enterprise is based on enterprise data and intelligently builds a multi-dimensional supplier feature portrait system including performance, behavior, and ability, relying on matrix model technology such as supply level, performance evaluation, customer insight, and dynamic risk for innovative applications in supply and demand insight in the construction industry. However, this technology relies on real-time online data processing, requiring high stability of data transmission networks and immediate computing capabilities of data processing equipment. In some poorly networked construction sites or when data volume is too large for existing online processing capabilities, the efficiency of this model will be greatly reduced, and even data processing delays or interruptions may occur. The data processing cost of this model is relatively high, and for some small and medium-sized construction enterprises, it is difficult to bear the cost of hardware equipment required for continuous operation of the model. A large amount of real-time online data transmission faces a higher risk of leakage, and once the data is leaked, it will cause serious losses to the enterprise and suppliers. Some construction enterprises use a combination of simple database storage and conventional data analysis software to build a supplier portrait, which processes limited structured data and cannot effectively utilize a large amount of unstructured data, resulting in a single-dimensional supplier portrait with incomplete information that cannot meet the needs of precise assessment in the complex and changing construction market.

[0005] First, advanced large model technology that relies on real-time online data processing has high requirements for network and computing power, and is heavily dependent on real-time online data processing, which requires high stability of data transmission network and instant computing power of data processing equipment. In addition, in the construction project site, the network condition is often complex and changeable, and in some remote areas or areas with poor network coverage, network signal instability or even interruption occurs from time to time. Once the network has problems, data cannot be transmitted in real time, and the large model cannot run normally, resulting in that the supplier portrait cannot be updated or generated in time. At the same time, when the data volume is large enough to exceed the processing capacity of the existing online system, the lack of instant computing power of the data processing equipment will greatly reduce the efficiency of the large model, resulting in data processing delay, interruption and other problems. For example, in some large construction projects, the amount of supplier-related data generated every day can reach several GB or even more, and the existing data processing equipment is difficult to complete the instant processing of such a large amount of data in a short time. Second, the cost of data processing is high. Running the large model requires high-performance data processing equipment and stable and high-speed data transmission network, which means high hardware equipment procurement cost, network bandwidth rental fee and equipment maintenance fee for enterprises. For some small and medium-sized construction enterprises, due to limited funds, they are difficult to bear the cost of running the model, thereby limiting the wide application of the technology in the industry. Third, the risk of data security is high: a large amount of real-time online data transmission faces a higher risk of leakage. During data transmission, network hackers, malicious software, etc. may attack the data transmission channel and steal sensitive supplier and enterprise data. Once the data is leaked, it will not only cause the leakage of business secrets, economic losses and other problems for enterprises, but also cause serious damage to the business reputation and interests of suppliers. For example, if the sensitive information such as the supplier's pricing data and the enterprise's project bidding plan is leaked, it may be used by competitors, resulting in the enterprise being in a disadvantaged position in market competition.

[0006] The traditional simple data processing method has two problems. First, the data is not fully utilized, and only limited structured data is processed. For a large amount of unstructured data existing in the construction industry, such as supplier enterprise introduction documents, project experience description texts, and customer evaluations, it is difficult to effectively utilize them. Unstructured data often contains rich information and can reflect the actual situation of the supplier from multiple dimensions. For example, the supplier's enterprise introduction document may contain information about its unique technical advantages and corporate culture, and customer evaluations may contain real feedback on the supplier's service quality and product quality. Due to the inability to process these unstructured data, the constructed supplier portrait is single-dimensional and not comprehensive, making it difficult to meet the needs of precise evaluation in the complex and changing construction market. Second, the accuracy of the portrait is insufficient. Since only limited structured data can be processed, and the analysis method is mainly simple statistical analysis, it is difficult to conduct in-depth mining and multi-dimensional cross-analysis of supplier data. In the construction market, the performance of a supplier is influenced by a variety of factors, and simple statistical analysis cannot fully reflect the relationship between these factors. For example, a supplier's performance may be affected by multiple factors such as product quality, delivery timeliness, and after-sales service. The traditional simple data processing method cannot accurately analyze the specific impact of these factors on performance, resulting in a supplier portrait that cannot accurately reflect the supplier's true ability and performance, and cannot provide reliable decision-making basis for the enterprise in supplier evaluation, selection, and management.

[0007] In summary, the current construction industry has obvious shortcomings in the technology of constructing a supplier portrait based on data analysis. The advanced large model technology that relies on real-time online data processing is prone to interruption or delay in data transmission at construction sites with poor network conditions, which seriously affects the progress of supplier portrait construction. It also requires high real-time computing power, resulting in high costs for hardware device upgrades and network bandwidth rentals, which small and medium-sized construction enterprises cannot afford. At the same time, real-time online transmission of a large amount of sensitive data poses a high risk of leakage. The simple database storage combined with conventional data analysis software used by some enterprises can only process structured data and cannot utilize unstructured data such as supplier online reputation evaluations and network forum discussions. The constructed portrait is single-dimensional and cannot accurately evaluate the supplier's true situation. Therefore, the existing technology has defects such as poor data processing environment adaptability, high cost, and inability to effectively utilize unstructured data. SUMMARY

[0008] The present application provides a construction industry supplier portrait construction system to solve the problems of poor data processing environment adaptability, high cost, and inability to effectively utilize unstructured data in the prior art.

[0009] To solve the above technical problems, the present application provides the following technical solutions: In one aspect, the present application provides a construction industry supplier portrait construction system, comprising: A data acquisition and preprocessing module is used to realize data acquisition and preprocessing tasks, including: collecting supplier data and preprocessing the collected data; A multi-dimensional cross analysis module is used to realize data analysis tasks, including: multi-dimensional cross analysis of the preprocessed supplier data, mining hidden information behind the data, and obtaining analysis results of the supplier data; A supplier portrait construction module is used to realize portrait construction tasks, including: constructing a supplier portrait from multiple dimensions according to the analysis results of the supplier data and combining the actual business needs of construction industry supplier management; Wherein, the data acquisition and preprocessing tasks, data analysis tasks and portrait construction tasks are completed in an offline environment relying on a local server or a distributed computing cluster; the processed data and the constructed supplier portrait are stored in a local database or data warehouse for enterprise to call and view.

[0010] Further, the supplier data includes structured data and unstructured data; The collected supplier data includes: For structured data, a preset data acquisition tool is used to design a database interface program to realize seamless connection with the enterprise internal business system database; according to a preset time period, the structured data of the supplier is extracted to a data analysis temporary storage area regularly; For unstructured data, a customized web crawler program is developed based on a preset technical framework, text data about the supplier is grabbed from the supplier-related website by setting the crawling rules; at the same time, an intelligent analysis model interface is developed to realize intelligent analysis of the unstructured data submitted by the supplier, extract the information therein, and store the grabbed text data and the extracted information in a distributed file system.

[0011] Further, the preprocessing of the collected data includes: For structured data, a data cleaning algorithm based on rule matching and statistical methods is used; by setting date format rules, the error data in the date field is automatically detected and corrected; by using a preset statistical tool, the outliers in the numerical data are identified and corrected; at the same time, by means of a data format conversion function library, the standardization processing of different data types is realized, and the data format is unified; For unstructured data, a preset text processing algorithm is used for deep processing; with the help of jieba library, Chinese word segmentation is performed, key information is extracted from the text by combining with a keyword extraction algorithm, and unstructured data is converted into structured data form, so as to realize the structured conversion of data.

[0012] Further, the system further comprises a task arrangement and scheduling module for realizing task arrangement and scheduling tasks.

[0013] Further, the task arrangement task is specifically: Based on the directed acyclic graph technology, the data acquisition and preprocessing task, the data analysis task and the portrait construction task are disassembled into a plurality of mutually related subtasks, and the dependency relationship between the subtasks is clear; with the help of a preset task arrangement tool, the task flow is intuitively defined in a visual manner, and the task parameters are set, including the frequency of data acquisition and the selection of data processing algorithm, so as to realize the planning of the task flow and the configuration of the parameters.

[0014] Further, the scheduling task is specifically: For the data acquisition task, a timing scheduling mechanism is set according to the actual update frequency of the data; for the data preprocessing task and the data analysis task, the execution is automatically triggered after the data acquisition task is completed; for the portrait construction task, the update is performed at a set time node after the data analysis task is completed; A preset scheduling system is used to monitor the task execution state in real time, and once the task execution fails, the retry is automatically performed, and the operation and maintenance personnel are timely notified through email or SMS, so as to ensure that all tasks are successfully completed according to the plan.

[0015] Further, the system further comprises a data resource pool for realizing centralized management and sharing of data, improving the reusability of data, and facilitating access and calling of data by different tasks; The data resource pool adopts a distributed storage architecture to realize centralized storage and management of collected raw data, preprocessed data, analysis result data and constructed supplier portrait data; The data resource pool is divided into four functionally clear data storage areas, including a raw data area, a temporary processing area, an intermediate result area and a final achievement area; wherein the raw data area is used to store structured and unstructured raw data without any processing; the temporary processing area is used to temporarily store intermediate data in the data preprocessing process; the intermediate result area stores various intermediate result data generated in the data analysis process; and the final achievement area is used to store the constructed supplier portrait data.

[0016] Further, the system further comprises a data security module for realizing data encryption and access control; The data encryption specifically comprises: sensitive data stored in the data resource pool is subjected to encryption processing; before data is written into the storage system, the data is subjected to encryption processing by using a symmetric encryption algorithm, an encryption key is generated, and the key is stored in a key management system; when data is read, the key is acquired from the key management system, and the encrypted data is decrypted, so that the confidentiality of the data in the storage and transmission process is ensured, and data leakage is prevented. The access control specifically comprises: an access control mechanism is established, and different users are assigned with differentiated permissions; the data administrator is assigned with all operation permissions of the data; the data analyst can only access the data related to the analysis task of the data analyst; and the enterprise decision maker can only view the supplier portrait data built and has no right to modify the data.

[0017] Further, in the offline environment, the data collection and preprocessing module collects data according to a set scheduling strategy, and stores the collected data in the raw data area of the data resource pool; after the data collection is completed, the data preprocessing task is automatically triggered to preprocess the raw data, and the processed data is stored in the temporary processing area and the intermediate result area; subsequently, the multi-dimensional cross analysis module uses the pre-developed analysis model to perform multi-dimensional cross analysis based on the data in the intermediate result area, mines the hidden information behind the data, and stores the analysis result in the intermediate result area again.

[0018] Further, the supplier portrait construction module is specifically used for: According to the multi-dimensional cross analysis result, a multi-dimensional supplier portrait is constructed from multiple dimensions of basic information, performance capability, qualification level, product quality, service level and market reputation, and is stored in the final achievement area; the basic information, performance capability, qualification level, product quality, service level and market reputation of the supplier are intuitively presented; and a weight distribution method is used to comprehensively evaluate each index of the supplier, so that an intuitive, comprehensive and quantitative supplier portrait is formed.

[0019] In another aspect, the present application also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to realize the above-mentioned system.

[0020] In another aspect, the present application also provides a computer readable storage medium, which stores at least one instruction, which is loaded and executed by the processor to realize the above-mentioned system.

[0021] The technical solution provided by the present application has at least the following beneficial effects: The present application has significant beneficial effects in data processing efficiency, cost control, image accuracy and data security, and provides strong technical support for the optimization of building industry supply chain management. Firstly, the data processing efficiency and stability are improved. The present application adopts an offline data analysis mode, which eliminates the dependence on real-time network environment. Compared with the prior art which depends on real-time online data processing, the present application avoids the problems of data transmission delay or analysis work stagnation caused by network fluctuations and interruptions, greatly improving the stability of data processing. The data cleaning and conversion link lays a good foundation for subsequent analysis, optimizes the data analysis process, and makes the entire data processing process more efficient and smooth. For example, when processing massive supplier data, the traditional method needs to spend several hours for data arrangement and preprocessing, while the present application can shorten this time to within half an hour, greatly improving the data processing efficiency. Secondly, the hardware and network investment is reduced. There is no need to continuously maintain a high-speed and stable online data transmission network and powerful real-time computing hardware devices, so that the cost of hardware device upgrade and network bandwidth rental of building enterprises is significantly reduced.

[0022] The present application constructs a more accurate and comprehensive supplier portrait. The present application not only collects traditional structured data, but also widely collects unstructured data, realizing the fusion of multi-source data. This comprehensive data collection method provides a rich data basis for constructing an accurate supplier portrait. Through the use of various advanced data analysis algorithms, structured and unstructured data are deeply mined. In the aspect of structured data analysis, each index is accurately calculated to provide quantitative basis for the supplier portrait; in the aspect of unstructured data analysis, the market reputation and industry influence of the supplier are deeply mined. The comprehensive analysis and feature fusion link integrates the analysis results of different dimensions to construct a more comprehensive portrait that can accurately reflect the overall strength and characteristics of the supplier.

[0023] The present application constructs a more accurate and comprehensive supplier portrait, and also enhances data security, reduces the need for real-time online data transmission, and reduces the risk of data leakage caused by network transmission. Data is stored and processed in an offline state, and enterprises can adopt more stringent internal security measures to ensure data security. The data collection and processing process is more controllable, and enterprises can better develop and implement data security management systems. Avoid excessive collection of sensitive information to ensure data security from the source. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is an architecture diagram of a construction industry supplier portrait construction system provided by an embodiment of the present application; Figure 2 is a data collection flowchart provided by an embodiment of the present application; Figure 3 is a data resource pool schematic diagram provided by an embodiment of the present application; Figure 4 is a data encryption flowchart provided by an embodiment of the present application; Figure 5 is a multi-dimensional cross-analysis index card (part) schematic diagram provided by an embodiment of the present application; Figure 6 is a block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the present application clearer, the following will further describe the embodiments of the present application in combination with the drawings.

[0027] First, it should be noted that in the embodiments of the present application, the words such as "exemplarily", "for example" are used to represent as an example, illustration or explanation. Any embodiment or design scheme described as "exemplary" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "exemplarily" is intended to present the concept in a specific way. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0028] First embodiment

[0029] The present embodiment provides a construction industry supplier portrait construction system as shown in Figure 1 The present embodiment provides a construction industry supplier portrait construction system as shown in

[0030] The system uses offline data analysis means to deeply mine and process related data of suppliers in the construction industry, constructs a precise and comprehensive supplier portrait, and meets the business needs of the construction industry in supplier evaluation, selection, management, etc. Through offline data analysis, the adaptability of the data processing environment is improved, the cost is reduced, and the use of unstructured data is realized, realizing efficient, low-cost and safe construction of a comprehensive and accurate supplier portrait in the construction industry. Thus, the demand for constructing a precise, efficient and safe supplier portrait in the construction industry is met.

[0031] The construction industry supplier portrait construction system mainly includes the following modules: I. Data development 1. Data acquisition module As shown in Figure 2 , advanced data acquisition tools are introduced, and for structured data, a special database interface program is designed to achieve seamless connection with enterprise internal procurement management systems, project management systems and other business system databases. According to the preset time period, the basic information of the supplier (such as enterprise name, registered address, legal representative, etc.), performance data (past project size, type, completion quality, etc.), qualification certificates (construction qualification, quality certification, etc.) and other structured data are extracted to the data analysis temporary storage area in a timely manner, ensuring the timeliness and accuracy of the data.

[0032] For unstructured data, a customized web crawler program is developed based on a mature technical framework. By setting detailed crawling rules, text data is accurately scraped from supplier websites, industry forums, third-party evaluation platforms and other websites. At the same time, an intelligent analysis model interface is developed to intelligently analyze non-structured information such as PDF, Word and other formats of enterprise introduction documents and project reports submitted by suppliers, efficiently extract key information, and store it uniformly in a distributed file system, laying a foundation for subsequent data processing.

[0033] 2. Data preprocessing module

[0034] For structured data, a data cleaning algorithm based on rule matching and statistical methods is used. By setting date format rules, automatically detect and correct errors in date fields; use statistical tools such as box plots to accurately identify outliers in numerical data and make reasonable corrections to ensure data accuracy and completeness. At the same time, with the help of data format conversion function library, standardize processing of different data types (such as date, numerical value, string, etc.), unify data format, and improve data standardization to facilitate subsequent analysis.

[0035] For unstructured data, professional text processing algorithms are used for in-depth processing. With the help of the jieba library, Chinese word segmentation is performed, and key information is extracted from the text by combining keyword extraction algorithms. Unstructured data is converted into structured data form, realizing the structured conversion of data and creating favorable conditions for data fusion analysis.

[0036] Based on the above, the embodiment widely collects multi-channel unstructured data of suppliers in the construction industry, such as supplier product evaluation, market reputation, industry dynamic discussion, and other information. The data sources are rich, which is in sharp contrast to the prior art which only relies on traditional structured data. In the data processing stage, the collected structured data and unstructured data are fused and analyzed. Through data cleaning, conversion and other preprocessing steps, a data foundation is provided for realizing accurate portrait construction. Based on this, the embodiment can construct a comprehensive and accurate supplier portrait, solving the problem of single portrait dimension in the prior art.

[0037] II. Data resource pool

[0038] A unified data resource pool is constructed, as shown in Figure 3 The data resource pool of the embodiment adopts a distributed storage architecture to realize centralized storage and management of the collected raw data, preprocessed data, analysis result data and constructed supplier portrait data. The data resource pool is divided into four functionally clear data storage areas: the raw data area is used to store unprocessed structured and unstructured raw data; the temporary processing area is used to temporarily store intermediate data in the data preprocessing process; the intermediate result area stores various intermediate result data generated in the data analysis process; and the final achievement area is used to store the constructed supplier portrait data. Through the data resource pool, efficient centralized management and sharing of data can be realized, the reusability of data is significantly improved, and fast access and calling of data by different tasks are facilitated.

[0039] III. Data security

[0040] 1. Data encryption

[0041] As shown in Figure 4 Sensitive data (such as supplier financial data and customer information) stored in the data resource pool is strictly encrypted. Before data is written into the storage system, symmetric encryption algorithm is used to encrypt the data, generate an encryption key, and store the key in a secure and reliable key management system. When data is read, the key is securely obtained from the key management system to decrypt the encrypted data, ensuring the confidentiality of the data during storage and transmission, and preventing data leakage risks.

[0042] 2. Access control

[0043] Establish a comprehensive and rigorous access control mechanism, employing a Role-Based Access Control (RBAC) model to assign differentiated permissions to different user roles (such as data administrators, data analysts, and enterprise decision-makers). Data administrators are granted full access to data, including CRUD operations, and are responsible for the overall management and maintenance of the data. Data analysts can only access data relevant to their specific analytical tasks, allowing them to focus on their work. Enterprise decision-makers can only view pre-built supplier profile data for decision-making reference and have no right to modify the data. This granular access control effectively limits the scope of user operations on the data, preventing unauthorized access and tampering, and ensuring data security and integrity.

[0044] IV. Multidimensional Cross-analysis

[0045] like Figure 5 As shown, after data collection and preprocessing are completed, based on the data in the intermediate results area, multidimensional cross-analysis is performed using pre-developed clustering analysis, association rule mining and other analysis models to deeply explore the hidden information behind the data, and the analysis results are stored in the intermediate results area again.

[0046] V. Supplier Profile Construction

[0047] Based on multidimensional cross-analysis results and closely integrated with the actual business needs of supplier management in the construction industry, a comprehensive and intuitive supplier profile is constructed from multiple dimensions. Through diverse formats such as visualization charts and tagging systems, key information about suppliers, including basic information, performance capabilities, qualification levels, product quality, service levels, and market reputation, is presented intuitively. For example, various supplier indicators are assigned corresponding tags, such as "high-performing supplier" and "high-quality service supplier," and a scientific weighting method is used to comprehensively evaluate these indicators, forming an intuitive, comprehensive, and quantifiable supplier profile. This transforms complex data into easily understandable information, providing construction companies with clear and explicit decision-making basis in key aspects such as supplier evaluation, selection, and management, helping companies achieve scientific decision-making and effectively improve supply chain management.

[0048] Based on the above, this embodiment uses statistical analysis methods to calculate supplier financial indicators and project execution capability indicators, and utilizes association rule algorithms to uncover potential relationships between different structured data. These algorithms delve into the supplier's market reputation and industry influence, enriching the dimensions of the supplier profile. A comprehensive evaluation model is constructed, assigning appropriate weights to different types of data analysis results, and weighting the analysis results from each dimension to obtain the supplier's overall score. This achieves effective integration of multi-source data and various analysis results.

[0049] VI. Task Arrangement and Scheduling

[0050] 1. Task orchestration

[0051] Based on DAG (Directed Acyclic Graph) technology, the core steps of data collection, preprocessing, analysis, and portrait construction are broken down into multiple interrelated sub-tasks, and the strict dependency between each sub-task is clearly defined. For example, the data preprocessing task must be started after the data collection task is completed and the data is successfully stored in the temporary storage area; while the supplier portrait construction task depends on the completion of the multi-dimensional cross-analysis task and outputs the analysis results. With the help of professional task orchestration tools, the task flow is defined in a visual way, and the task parameters are flexibly set, including the frequency of data collection, the selection of data processing algorithms, etc., to realize clear planning of the task flow and convenient configuration of the parameters.

[0052] 2. Scheduling tasks

[0053] A scientific and reasonable task scheduling strategy is formulated. For data collection tasks, a timed scheduling mechanism is set according to the actual update frequency of the data, such as structured data collection once a day at 3 am, and unstructured data collection once a week at dawn. Data preprocessing and analysis tasks are automatically triggered after data collection is completed to ensure the timeliness of data processing. The supplier portrait construction task is updated every Friday afternoon after the data analysis task is completed to ensure the timeliness of the portrait. Real-time monitoring of task execution status is achieved using professional scheduling systems. Once the task execution fails, the system automatically retries and notifies the operation and maintenance personnel in time through email or SMS to ensure that all tasks are completed successfully according to the plan.

[0054] Among them, the embodiment adopts an offline data analysis method, and a working system independent of real-time network transmission is constructed. All data collection, preprocessing, analysis, and portrait construction processes are carried out in an offline environment and do not depend on continuous and stable network connection. Even in a building project site with poor or no network, the construction of the supplier portrait can be smoothly promoted. The powerful computing power of local high-performance servers or distributed computing clusters is relied on to complete data processing tasks. The processed data and constructed supplier portraits are stored in local databases or data warehouses, and enterprises can call and view them at any time according to actual needs without relying on real-time network transmission of data. This offline data processing and storage method effectively reduces the dependence on real-time network environment, significantly improves the stability and efficiency of data processing, and solves the problem of unstable data processing caused by network problems in the prior art. At the same time, local storage avoids the security risks of data in the transmission process, greatly reduces the investment cost of hardware devices and network bandwidth, and achieves the purpose of stable construction of the supplier portrait in a complex network environment. An economical, efficient, and secure data processing and portrait construction scheme is provided for enterprises.

[0055] Specifically, in the offline environment, the data collection tool collects data from various data sources according to the set scheduling strategy, and stores the data in the raw data area of the data resource pool. After the data collection is completed, the data preprocessing task is automatically triggered to perform comprehensive cleaning and conversion on the raw data, and the processed data is stored in the temporary processing area and the intermediate result area. Subsequently, the data analysis task uses various analysis models such as clustering analysis and association rule mining to perform multi-dimensional cross-analysis based on the data in the intermediate result area, deeply mines the hidden information behind the data, and stores the analysis results in the intermediate result area again.

[0056] Finally, the supplier portrait construction task constructs a comprehensive and accurate multi-dimensional supplier portrait based on the analysis results and other related data in the data resource pool from multiple dimensions such as basic information, performance capability, qualification level, product quality, service level, and market reputation, and stores the portrait in the final result area. The entire process is orderly managed through task scheduling and scheduling tasks, and the data is efficiently stored and shared through the data resource pool, while the data security is comprehensively guaranteed through data security measures, thereby realizing the construction of a precise and comprehensive supplier portrait in the construction industry and providing scientific and reliable decision-making basis for supplier management of construction enterprises.

[0057] Through the above technical solutions, the construction industry supplier portrait construction system of the embodiment realizes the formation of a comprehensive and three-dimensional multi-dimensional supplier portrait through offline data analysis and integration, effectively solves the problems existing in the background technology, and promotes the digitalization and intelligent development of the construction industry supply chain management.

[0058] In summary, the construction industry supplier portrait construction system of the embodiment reduces the dependence on real-time network environment through innovative data processing mode, improves the stability and efficiency of data processing, optimizes the data processing process, reduces the hardware device and network bandwidth investment cost, and uses advanced algorithms to comprehensively analyze structured data and unstructured data, thereby constructing a more comprehensive and accurate portrait that can reflect the actual situation of the supplier, helping construction enterprises make more scientific and reasonable decisions in supplier evaluation, selection, and management, and promoting the digitalization and intelligent development of the construction industry supply chain management.

[0059] Second embodiment

[0060] The embodiment provides an electronic device, such as Figure 6As shown, the electronic device includes a processor and a memory; wherein the processor and the memory can be connected through a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device can also include a transceiver, and the processor and the transceiver can be connected through a communication bus, and the transceiver is used to communicate with other devices.

[0061] In the following, the embodiments of the present application will be described in detail in conjunction with Figure 6 The various components of the electronic device will be described in detail: The processor is the control center of the electronic device, and the electronic device can include multiple processors, each of which can be a single-CPU or a multi-CPU. The processor here can be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPU), which can also be other general-purpose processors, application specific integrated circuits (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor can perform various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0062] In a specific implementation, as an embodiment, the processor can include one or more CPUs, such as the CPU0 and CPU1 shown in Figure 6 of course, this is only an exemplary description.

[0063] The memory is used to store software programs for implementing the solutions of the present application, and is controlled by the processor to execute, and the specific implementation can refer to the above-mentioned method embodiments, which will not be described here.

[0064] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may be integrated with the processor or exist independently, and may be accessed through the interface circuit of the electronic device ( Figure 6 (Not shown in the image) is coupled to the processor; however, this embodiment of the invention does not impose specific limitations on this.

[0065] The transceiver may include a receiver and a transmitter. Figure 6 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function. The transceiver can be integrated with the processor or exist independently, and can be connected through the interface circuit of the electronic device (…). Figure 6 (Not shown in the image) is coupled to the processor, and this embodiment of the invention does not specifically limit this.

[0066] In addition, it should be noted that, Figure 6 The structure of the electronic device shown is not intended to limit the device. Actual devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects achieved by this electronic device when performing the method of the first embodiment described above can be referenced to the technical effects described in the first embodiment; therefore, they will not be repeated here.

[0067] Third Embodiment

[0068] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0069] Moreover, it should be noted that the present application can be provided as a method, an apparatus, or a computer program product. Therefore, the embodiments of the present application can take the form of an entirely or partially hardware embodiment, an entirely or partially software embodiment, or an embodiment combining software and hardware aspects. Furthermore, when implemented in software, the embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, a computer diskette, an optical storage medium, a magnetic storage medium, and a semiconductor memory device). The computer program product includes one or more computer instructions that when loaded and executed by a computer, cause the computer to carry out the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, such as from a website, a computer, a server, or a data center to another website, computer, server, or data center through a wired (for example, infrared, wireless, microwave, or the like) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, data center, or the like, including one or more collections of available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0070] The embodiments of the present application are described with reference to flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate a device that implements the flowcharts and / or block diagrams. Figure 1 The flowcharts and / or block diagrams Figure 1 The apparatus that implements the functions specified in one or more flows and / or blocks.

[0071] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product that includes instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 The flowcharts and / or block diagrams Figure 1the functions specified in the individual block or blocks. Such computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable devices provide processes for implementing the functions specified in the flowchart block(s) or block(s). Figure 1 the functions specified in the individual block or blocks. Such computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable devices provide processes for implementing the functions specified in the flowchart block(s) or block(s). Figure 1 the functions specified in the individual block or blocks. Such computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer-implemented process, thus the instructions executed on the computer or other programmable devices provide processes for implementing the functions specified in the flowchart block(s) or block(s).

[0072] It should also be noted that, in the present document, the terms such as first and second, etc. are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "including a…", does not exclude the presence of other identical elements in the process, method, article or terminal device including the element. In addition, the term "and / or" is merely a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B together, and the existence of B alone, where A and B can be singular or plural. In addition, the character " / " in the present document generally represents an "or" relationship between the preceding and following associated objects, but it can also represent an "and / or" relationship, which can be understood in the context before and after. "At least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0073] In addition, it can be understood that in various embodiments of the present application, the size of the sequence number of the above processes does not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0074] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or in a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0075] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of functional modules / units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms. The units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present, or two or more units can be integrated in one unit.

[0076] If the method is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0077] Finally, it should be noted that the above description is only the preferred embodiment of the application, it should be pointed out that although the preferred embodiment of the application has been described, for those skilled in the art, once the basic creative concept of the application is known, several improvements and refinements can be made without departing from the principles of the application, and these improvements and refinements should also be considered as the protection scope of the application. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the application.

Claims

1. A system for constructing supplier profiles in the construction industry, characterized in that, include: The data acquisition and preprocessing module is used to perform data acquisition and preprocessing tasks, including: acquiring supplier data and preprocessing the acquired data; The multidimensional cross-analysis module is used to perform data analysis tasks, including: performing multidimensional cross-analysis on preprocessed supplier data, uncovering hidden information behind the data, and obtaining the analysis results of the supplier data; The supplier profile building module is used to perform profile building tasks, including: building supplier profiles from multiple dimensions based on the analysis results of supplier data and the actual business needs of supplier management in the construction industry; Among them, data collection and preprocessing tasks, data analysis tasks, and profile building tasks are all completed in an offline environment, relying on local servers or distributed computing clusters; the processed data and the built supplier profiles are uniformly stored in a local database or data warehouse for enterprises to access and view.

2. The system for constructing supplier profiles in the construction industry as described in claim 1, characterized in that, The supplier data includes structured data and unstructured data; The collected supplier data includes: For structured data, pre-set data acquisition tools are used, and database interface programs are designed to achieve seamless integration with the enterprise's internal business system database; according to a pre-set time cycle, the supplier's structured data is periodically extracted to the temporary data analysis storage area; For unstructured data, a customized web crawler program is developed based on a pre-defined technical framework. By setting crawling rules, text data related to suppliers is crawled from their websites. At the same time, an intelligent analysis model interface is developed to intelligently parse the unstructured data submitted by suppliers, extract the information, and store the crawled text data and the extracted information in a unified distributed file system.

3. The system for constructing supplier profiles in the construction industry as described in claim 2, characterized in that, The preprocessing of the collected data includes: For structured data, a data cleaning algorithm combining rule matching and statistical methods is adopted; by setting date format rules, erroneous data in date fields are automatically detected and corrected; by using preset statistical tools, outliers in numerical data are identified and corrected; at the same time, a data format conversion function library is used to achieve standardized processing of different data types and unify data formats. For unstructured data, a pre-defined text processing algorithm is used for in-depth processing; the jieba library is used for Chinese word segmentation, and a keyword extraction algorithm is used to extract key information from the text, and the unstructured data is transformed into structured data, thus achieving the structured transformation of the data.

4. The system for constructing supplier profiles in the construction industry as described in claim 1, characterized in that, The system also includes a task orchestration and scheduling module, which is used to orchestrate and schedule tasks.

5. The system for constructing supplier profiles in the construction industry as described in claim 4, characterized in that, The specific tasks to be orchestrated are as follows: Based on the directed acyclic graph (DAG) technique, the data collection and preprocessing tasks, data analysis tasks, and profile building tasks are decomposed into multiple interrelated sub-tasks, and the dependencies between each sub-task are clearly defined. With the help of a pre-set task orchestration tool, task processes can be defined intuitively in a visual way, and task parameters can be set, including the frequency of data collection and the selection of data processing algorithms, so as to realize the planning of task processes and the configuration of parameters.

6. The system for constructing supplier profiles in the construction industry as described in claim 4, characterized in that, The scheduling task is specifically as follows: For data acquisition tasks, a timed scheduling mechanism is set according to the actual update frequency of the data; for data preprocessing and data analysis tasks, execution is automatically triggered after the data acquisition task is completed; for profile building tasks, updates are performed at set time nodes after the data analysis task is completed. The system uses a pre-set scheduling system to monitor the task execution status in real time. If a task fails, it will automatically retry and notify the operations and maintenance personnel in a timely manner via email or SMS to ensure that all tasks are completed smoothly as planned.

7. The system for constructing supplier profiles in the construction industry as described in claim 1, characterized in that, The system also includes a data resource pool, which is used to realize centralized management and sharing of data, improve data reusability, and facilitate access to and call of data by different tasks. The data resource pool adopts a distributed storage architecture to achieve centralized storage and management of the collected raw data, preprocessed data, analysis results data, and constructed supplier profile data; The data resource pool is divided into four functionally defined data storage areas: raw data area, temporary processing area, intermediate results area, and final results area. The raw data area is used to store unprocessed structured and unstructured raw data. The temporary processing area is used to temporarily store intermediate data during data preprocessing. The intermediate results area stores various intermediate results data generated during data analysis. The final results area is used to store the completed supplier profile data.

8. The system for constructing supplier profiles in the construction industry as described in claim 7, characterized in that, The system also includes a data security module for implementing data encryption and access control; The data encryption specifically involves: encrypting sensitive data stored in the data resource pool; encrypting the data using a symmetric encryption algorithm before writing it into the storage system, generating an encryption key, and storing the key in the key management system; and decrypting the encrypted data by obtaining the key through the key management system when reading the data, thereby ensuring the confidentiality of the data during storage and transmission and preventing data leakage. The access control specifically involves: establishing an access control mechanism to assign differentiated permissions to different user roles; Data administrators are granted full access to the data; data analysts can only access data relevant to their own analytical tasks; and corporate decision-makers can only view pre-built supplier profile data and have no right to modify it.

9. The system for constructing supplier profiles in the construction industry as described in claim 1, characterized in that, In an offline environment, the data acquisition and preprocessing module acquires data according to the set scheduling strategy and stores the acquired data in the raw data area of ​​the data resource pool. After the data acquisition is completed, the data preprocessing task is automatically triggered to preprocess the raw data. The processed data is then stored in the temporary processing area and the intermediate result area, respectively. Subsequently, the multidimensional cross-analysis module uses a pre-developed analysis model to perform multidimensional cross-analysis based on the data in the intermediate results area, uncovering hidden information behind the data, and then stores the analysis results back into the intermediate results area.

10. The system for constructing supplier profiles in the construction industry as described in claim 1, characterized in that, The supplier profiling module is specifically used for: Based on the results of multidimensional cross-analysis and combined with the actual business needs of supplier management in the construction industry, a multidimensional supplier profile is constructed from multiple dimensions, including basic information, performance capabilities, qualification level, product quality, service level, and market reputation, and stored in the final results area. The basic information, performance capabilities, qualification level, product quality, service level, and market reputation of suppliers are presented intuitively. Furthermore, a weighting method is used to comprehensively evaluate the various indicators of suppliers, forming an intuitive, comprehensive, and quantitative supplier profile.

Images