Public resource transaction AI intelligent witness terminal and method based on multi-mode perception

Abstract

The invention relates to the technical field of AI intelligent witnessing terminals, in particular to a public resource transaction AI intelligent witnessing terminal and method based on multi-modal perception, and the terminal comprises an end-side AI intelligent witnessing terminal and a cloud platform, and the end-side AI intelligent witnessing terminal and the cloud platform are connected through an encrypted communication link to realize data interaction and cooperative work. According to the invention, through end-side real-time early warning and cloud deep analysis, abnormal behaviors and potential risks in the transaction process can be found in time, the change from manual watching to intelligent witness is realized, and the supervision efficiency is greatly improved; compared with a traditional manual inspection mode, the intelligent witness terminal can monitor multiple transaction items in real time at the same time, problems can be found in the first time, early warning can be given out, and the supervision efficiency is improved by several times.

Description

Technical Field

[0001] This invention relates to the field of AI intelligent witnessing terminal technology, specifically to an AI intelligent witnessing terminal and method for public resource transactions based on multimodal perception. Background Technology

[0002] Currently, in specialized regulatory scenarios such as public resource transactions, finance, and healthcare, traditional electronic monitoring systems generally suffer from problems such as limited monitoring dimensions, weak real-time analysis capabilities, poor system compatibility, and low data security. For example, during the bidding process in public resource transactions, traditional video surveillance or screen recording methods struggle to simultaneously capture the details of bidding experts' screen operations, actions, and voice communications, resulting in blind spots and a lack of real-time identification and early warning capabilities for abnormal behaviors (such as unauthorized absences, fatigued operation, sensitive remarks, and delays). Furthermore, most existing systems are incompatible with Windows and domestically developed IT systems, hindering rapid deployment and cross-platform operation.

[0003] On the other hand, existing regulatory solutions largely rely on manual post-event review of massive amounts of audio and video data, which is inefficient and unable to achieve real-time intervention. Although some intelligent monitoring systems have attempted to introduce facial recognition or voice detection technologies, their functions are relatively limited, failing to achieve simultaneous collection and fusion analysis of multimodal data, and lacking a deep verification mechanism for edge-cloud collaboration, making them prone to false alarms or missed detections. Summary of the Invention

[0004] The purpose of this invention is to provide an AI-powered intelligent witnessing terminal and method for public resource transactions based on multimodal perception, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The AI-powered smart witnessing terminal for public resource transactions based on multimodal perception includes:

[0007] The edge AI smart witnessing terminal and the cloud platform are connected via an encrypted communication link to achieve data interaction and collaborative work.

[0008] The edge-side AI intelligent witnessing terminal includes:

[0009] The intelligent sensing module is used to synchronously collect multimodal data from the public resource trading site. The multimodal data includes video image data, voice data, and trading terminal screen operation data. The intelligent sensing module consists of an intelligent camera array, a directional microphone array, a face recognition device, and a screen capture device.

[0010] The data processing and early warning module is equipped with a high-performance edge computing chip for real-time processing and analysis of multimodal data collected by the intelligent sensing module. The data processing and early warning module has built-in AI algorithms, including an abnormal behavior recognition algorithm and a biased speech detection algorithm. The data processing and early warning module can trigger an early warning mechanism when an anomaly is detected.

[0011] The hardware module includes an interface unit, a communication unit, and a security encryption unit. The interface unit is used to enable external device connection and data interaction, the communication unit is used to establish a network connection with the cloud platform, and the security encryption unit is used to encrypt the data.

[0012] Preferably, the cloud platform includes:

[0013] The data storage and management module adopts distributed storage technology to store multimodal data and early warning information uploaded by the edge AI intelligent witness terminal, and establishes a data index and classification system;

[0014] The deep analysis module is used to perform in-depth mining and analysis of stored multimodal data, build a risk behavior feature library and personnel profile model, and output heat maps of high-frequency violation areas and personnel risk indices;

[0015] The intelligent decision-making module is used to generate intelligent regulatory decision-making suggestions based on the analysis results of the deep analysis module;

[0016] The interface service module is used to interface with external systems to achieve data sharing and interaction.

[0017] The witnessing method of the public resource transaction AI-powered intelligent witnessing terminal based on multimodal perception includes the following steps:

[0018] S1: The AI-powered smart witness terminal on the edge initializes itself, completes device self-test, parameter configuration and network connection, registers with the cloud platform and obtains a unique device ID;

[0019] S2: Personnel involved in public resource transactions complete identity authentication through the facial recognition device of the AI ​​smart witness terminal on the edge. After successful authentication, the current transaction project is bound, and the terminal starts multimodal data collection.

[0020] S3: The intelligent sensing module of the edge AI intelligent witness terminal continuously collects multimodal data, which includes video image data, voice data and transaction terminal screen operation data;

[0021] S4: The data processing and early warning module of the edge AI intelligent witness terminal performs real-time analysis on the collected multimodal data. If an anomaly is detected, an early warning mechanism is triggered and the early warning information and related evidence data are encrypted and uploaded to the cloud platform.

[0022] S5: The cloud platform receives the warning information and related evidence data, and stores and classifies them through the data storage and management module;

[0023] S6: The cloud platform's deep analysis module performs in-depth mining and analysis of the stored data, and verifies the authenticity of anomalies by combining the preset risk behavior feature library and personnel profile model;

[0024] S7: The cloud platform's intelligent decision-making module generates intelligent regulatory decision-making suggestions based on in-depth analysis results and pushes them to the regulatory personnel's terminals;

[0025] S8: Regulatory personnel take regulatory intervention measures based on the intelligent regulatory decision-making suggestions, and feed back the processing results to the end-side terminal and cloud platform to achieve system optimization.

[0026] Preferably, in step S1, the initialization further includes parameter configuration, which includes the acquisition frequency configuration of multimodal data and the early warning threshold configuration.

[0027] Preferably, in step S2, the identity authentication is achieved by comparing the face image collected by the face recognition device with information from a preset expert database.

[0028] Preferably, in step S6, the deep analysis module performs multimodal fusion analysis on video image data, voice data, and screen operation data using a pre-trained large model, and verifies the authenticity of the anomaly by combining the current transaction project information and the historical violation case library, so as to eliminate false alarms.

[0029] Preferably, in step S5, the cloud platform further stores the confirmed abnormal events and related evidence data through a blockchain evidence storage platform to ensure that the data is tamper-proof.

[0030] Preferably, in step S7, the intelligent supervision decision suggestion is pushed to at least one of the following terminals of the supervisor: PC, large screen, and mobile device.

[0031] Preferably, in step S8, the system optimization includes the edge AI intelligent witness terminal adjusting the early warning threshold and data collection strategy according to the feedback results, and the cloud platform optimizing the risk behavior feature database, personnel profile model and intelligent decision-making algorithm according to the feedback results.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] This invention, through real-time edge-side early warning and in-depth cloud-based analysis, can promptly detect abnormal behaviors and potential risks during transactions, achieving a shift from "manual monitoring" to "intelligent witnessing" and significantly improving regulatory efficiency. Compared to traditional manual inspection methods, the intelligent witnessing terminal can simultaneously monitor multiple transaction projects in real time and can detect problems and issue early warnings immediately, increasing regulatory efficiency several times over.

[0034] This invention utilizes advanced AI technology to perform multi-dimensional and in-depth analysis of transaction data, constructing a risk behavior feature database and personnel profiling models. This enables accurate identification of abnormal behavior and potential risks, effectively avoiding misjudgments and omissions caused by human factors. Through learning and training on a large amount of historical transaction data, the intelligent witnessing terminal achieves an abnormal behavior identification accuracy rate of over 95%, significantly improving the accuracy and reliability of supervision.

[0035] This invention enables intelligent witnessing terminals to comprehensively and in real-time perceive and monitor the entire transaction process from start to finish, achieving full-process supervision of public resource transactions. Whether it's the behavior of the bidding experts, the operations of the tendering party, or the activities of the agency, all are under the supervision of the intelligent witnessing terminal, ensuring that every aspect of the transaction is standardized, fair, and transparent.

[0036] This invention, through data analysis and intelligent decision-making on a cloud platform, can provide regulatory authorities with scientific decision-making basis, helping them to rationally allocate regulatory resources and improve the targeting and effectiveness of supervision. For example, based on heat maps of high-frequency violation areas and personnel risk indices, regulatory authorities can focus on supervising certain areas and personnel, avoiding waste of regulatory resources and achieving optimized allocation of regulatory resources.

[0037] The AI-powered intelligent witnessing terminal for public resource transactions of this invention effectively curbs human intervention and opaque operations in the public resource transaction process, creating a more fair, just, and transparent market environment and promoting fair competition among market participants. By promptly detecting and handling violations, it maintains market order, protects the rights and interests of legitimate market participants, and promotes the healthy development of the public resource transaction market. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0039] Figure 2 This is a schematic diagram of the system architecture of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Please see Figure 1-2 The AI-powered intelligent witnessing terminal for public resource transactions based on multimodal perception includes:

[0042] The edge AI smart witnessing terminal and the cloud platform are connected through an encrypted communication link to achieve data interaction and collaborative work.

[0043] The edge AI smart witness terminal includes:

[0044] The intelligent sensing module is used to synchronously collect multimodal data from the public resource trading site. The multimodal data includes video image data, voice data, and trading terminal screen operation data. The intelligent sensing module consists of an intelligent camera array, a directional microphone array, a face recognition device, and a screen capture device.

[0045] The intelligent camera array in the intelligent sensing module is a 1080P high-definition camera that supports face and body behavior tracking; the directional microphone array has noise reduction processing function to improve the signal-to-noise ratio of voice acquisition.

[0046] The screen capture device connects to the trading terminal via an HDMI-IN interface and captures screen operation images at a preset frequency.

[0047] The data processing and early warning module is equipped with a high-performance edge computing chip, which is used to process and analyze the multimodal data collected by the intelligent sensing module in real time. The data processing and early warning module has built-in AI algorithms, including abnormal behavior recognition algorithms and biased speech detection algorithms. The data processing and early warning module can trigger an early warning mechanism when an anomaly is detected.

[0048] The abnormal behavior recognition algorithm in the data processing and early warning module is used to identify at least one of the following behaviors: unauthorized departure, use of mobile phone, turning around to communicate, prolonged stillness, and fatigue operation.

[0049] Human body key point feature extraction:

[0050] The OpenPose model was used to extract the coordinates of 18 key nodes in the human body and construct a behavioral feature vector.

[0051] formula:

[0052]

[0053] in, This is a behavioral feature vector containing the horizontal and vertical coordinates of 18 keypoints. ;

[0054] Number the key points (1-18, such as head, shoulders, hands, etc.);

[0055] Abnormal behavior determination (taking "unauthorized departure" as an example):

[0056] By calculating the distance between key head points and the seating area, and combining this with a time threshold, the behavior of leaving the seat is determined.

[0057] formula:

[0058] ;

[0059] like And duration Determine if abnormal;

[0060] Key points of the head With the center of the seat The Euclidean distance;

[0061] The distance threshold is preset based on the seating layout at the trading venue, such as 1.5 meters.

[0062] A time threshold (e.g., 60 seconds; exceeding this time is considered unauthorized absence).

[0063] Multi-behavior fusion determination:

[0064] For various behaviors such as "using a mobile phone" and "turning around to communicate", a weighted summation is used to calculate a comprehensive abnormal score, and an alert is issued if the score exceeds the threshold.

[0065] formula: ;like This triggers an alert;

[0066] The overall abnormal score;

[0067] For the first The weight of each behavior (e.g., "using a mobile phone" has a weight of 0.3, "turning around to communicate" has a weight of 0.2);

[0068] The confidence level for identifying the k-th behavior (0-1, output by the AI ​​model);

[0069] Set a score threshold (e.g., 0.6, calibrated based on historical violation data).

[0070] The biased speech detection algorithm is used to identify at least one type of speech information, including sensitive words such as "bid rigging" and "favoring a certain unit," as well as non-bid evaluation content communication and abnormal volume.

[0071] Formula for biased speech detection algorithm (based on text semantic analysis)

[0072] Biased speech detection uses speech-to-text (ASR) to obtain text, then calculates the similarity between the text and sensitive semantics to identify inappropriate speech.

[0073] Text semantic vector transformation:

[0074] The BERT pre-trained model is used to convert text into semantic vectors to capture contextual information.

[0075] formula:

[0076] For text The semantic vector (768 dimensions);

[0077] The text after speech-to-text conversion (e.g., "This project takes care of Company A");

[0078] Sensitive semantic similarity calculation:

[0079] Calculate the cosine similarity between the text vector and a preset sensitive word vector library to determine whether it is biased speech.

[0080] formula: ;like This is considered abnormal.

[0081] Text vectors and sensitive word vectors Cosine similarity;

[0082] , Let L2 be the norm of the vector;

[0083] The similarity threshold (e.g., 0.7, which can be adjusted through training with historical cases);

[0084] The abnormal volume detection method combines the decibel value of the voice signal to identify abnormal communication with excessively high volume.

[0085] formula: ;like The volume was determined to be abnormal.

[0086] The decibel value of the speech signal;

[0087] For voice signal power;

[0088] For reference sound pressure level;

[0089] The volume threshold is set (e.g., 70 decibels, preset according to the trading environment).

[0090] The hardware module includes an interface unit, a communication unit, and a security encryption unit. The interface unit is used to enable external device connection and data interaction, the communication unit is used to establish a network connection with the cloud platform, and the security encryption unit is used to encrypt the data.

[0091] The external systems that the cloud platform's interface service module interfaces with include electronic service systems, electronic transaction systems, and administrative supervision systems;

[0092] The data storage and management module also includes a data backup mechanism to regularly back up the stored multimodal data.

[0093] The cloud platform's deep analytics module is also used to analyze the historical irregularities of agencies in trading venues to provide early warnings of potential collusion risks; the cloud platform includes:

[0094] The data storage and management module adopts distributed storage technology to store multimodal data and early warning information uploaded by the AI ​​intelligent witness terminal on the storage side, and establishes a data index and classification system;

[0095] The deep analysis module is used to perform in-depth mining and analysis of stored multimodal data, build a risk behavior feature library and personnel profile model, and output heat maps of high-frequency violation areas and personnel risk indices;

[0096] The intelligent decision-making module is used to generate intelligent regulatory decision-making suggestions based on the analysis results of the deep analysis module;

[0097] The intelligent decision-making module generates intelligent regulatory decision-making recommendations, including at least one of the following: suspension of transactions for high-risk projects, restriction of transactions for frequently violating bidding experts or agencies, and training and education recommendations.

[0098] The interface service module is used to interface with external systems to achieve data sharing and interaction.

[0099] The edge AI smart witness terminal includes:

[0100] The intelligent sensing module is used to synchronously collect multimodal data from the public resource trading site. The multimodal data includes video image data, voice data, and trading terminal screen operation data. The intelligent sensing module consists of an intelligent camera array, a directional microphone array, a face recognition device, and a screen capture device.

[0101] The data processing and early warning module is equipped with a high-performance edge computing chip, which is used to process and analyze the multimodal data collected by the intelligent sensing module in real time. The data processing and early warning module has built-in AI algorithms, including abnormal behavior recognition algorithms and biased speech detection algorithms. The data processing and early warning module can trigger an early warning mechanism when an anomaly is detected.

[0102] The early warning mechanism includes at least one of the following: audible and visual alarms, system pop-up warnings, and SMS notifications. When the early warning mechanism is triggered, multimodal data segments of a preset duration before and after the anomaly are simultaneously captured as evidence and stored locally and uploaded to the cloud platform.

[0103] The hardware module includes an interface unit, a communication unit, and a security encryption unit. The interface unit is used to enable external device connection and data interaction, the communication unit is used to establish a network connection with the cloud platform, and the security encryption unit is used to encrypt the data.

[0104] The hardware module's interface units include a DC5525 power interface, a Type-C data interface conforming to the USB 2.0 protocol, an RJ-45 Gigabit Ethernet interface, an HDMI-IN / OUT audio / video interface, and a USB 2.0 / 3.0 peripheral expansion interface;

[0105] The communication unit supports at least one of the following network access methods: Gigabit Ethernet, Wi-Fi 6, and 5G.

[0106] The security encryption unit uses the national cryptographic SM2 / SM4 algorithm to achieve end-to-end data encryption.

[0107] like Figure 2 As shown, the witnessing method of the public resource transaction AI intelligent witnessing terminal based on multimodal perception includes the following steps:

[0108] S1: The AI-powered smart witness terminal on the edge initializes itself, completes device self-test, parameter configuration and network connection, registers with the cloud platform and obtains a unique device ID;

[0109] Initialization also includes parameter configuration, which includes the configuration of the acquisition frequency of multimodal data and the configuration of the warning threshold. The acquisition frequency of screen operation data is 1 frame per second.

[0110] S2: Personnel involved in public resource transactions complete identity authentication through the facial recognition device of the AI-powered smart witness terminal. After successful authentication, the terminal is bound to the current transaction project and initiates multimodal data collection.

[0111] Identity authentication is achieved by comparing the facial image captured by the facial recognition device with information from a preset expert database. Authentication is considered successful when the comparison pass rate is not less than 95%.

[0112] S3: The intelligent sensing module of the edge AI smart witness terminal continuously collects multimodal data, which includes video image data, voice data, and transaction terminal screen operation data;

[0113] S4: The data processing and early warning module of the edge AI intelligent witness terminal performs real-time analysis on the collected multimodal data. If an anomaly is detected, an early warning mechanism is triggered and the early warning information and related evidence data are encrypted and uploaded to the cloud platform.

[0114] S5: The cloud platform receives early warning information and related evidence data, and stores and classifies them through the data storage and management module;

[0115] The cloud platform also uses a blockchain-based evidence storage platform to store confirmed abnormal events and related evidence data to ensure that the data is tamper-proof.

[0116] S6: The cloud platform's deep analysis module performs in-depth mining and analysis of the stored data, and verifies the authenticity of anomalies by combining the preset risk behavior feature library and personnel profile model;

[0117] The deep analysis module performs multimodal fusion analysis on video image data, voice data and screen operation data through a pre-trained large model, and verifies the authenticity of anomalies by combining current transaction project information and historical violation case database to eliminate false alarms;

[0118] Multimodal fusion analysis algorithm formula (edge-cloud collaborative verification):

[0119] The cloud-based deep analysis module integrates video, audio, and screen operation data to verify the authenticity of anomalies and eliminate false alarms;

[0120] Multimodal feature weight fusion is used to sum the anomaly confidence scores of the three modalities by weighted summation, resulting in a fused anomaly score.

[0121] formula:

[0122] ;

[0123] in, For multimodal fusion anomaly scores;

[0124] , , Weights for video, audio, and screen operation data;

[0125] , , Confidence level for anomaly identification in each modality;

[0126] Anomaly verification based on historical data

[0127] By incorporating the number of personnel's historical violations and the project's risk level, the integrated score is revised to improve the accuracy of the judgment.

[0128] formula:

[0129] ; Confirm the anomaly;

[0130] in, This is the final anomaly verification score;

[0131] This refers to the number of historical violations committed by the current personnel.

[0132] The current project risk level;

[0133] , This is a correction factor;

[0134] This is the final verification threshold;

[0135] S7: The cloud platform's intelligent decision-making module generates intelligent regulatory decision-making suggestions based on in-depth analysis results and pushes them to the regulatory personnel's terminals;

[0136] Intelligent regulatory decision-making suggestions are pushed to at least one of the following terminals for regulatory personnel: PC, large screen, and mobile device;

[0137] S8: Regulatory personnel take regulatory intervention measures based on the intelligent regulatory decision-making suggestions, and feed back the processing results to the end-side terminal and cloud platform to achieve system optimization;

[0138] System optimization includes adjusting warning thresholds and data collection strategies on the edge AI intelligent witness terminal based on feedback results, and optimizing the risk behavior feature database, personnel profile model and intelligent decision-making algorithm on the cloud platform based on feedback results.

[0139] The following uses a decentralized bidding scenario for public resource transactions as an example to illustrate the implementation process of this invention in detail:

[0140] (1) System deployment and initialization

[0141] One AI smart witnessing terminal is deployed at each bid evaluation workstation. The terminal connects to the bid evaluation computer via an HDMI-IN interface, to the workstation monitor via an HDMI-OUT interface, to the power adapter via a DC5525 interface, and to the transaction center's local area network via an RJ-45 interface. The terminal automatically starts up after being powered on and registers with the cloud-based smart witnessing platform via the network. The platform assigns a unique device ID and binds it to the project information (project number, bid evaluation period) of the current workstation.

[0142] (2) Binding of the identity of the evaluation experts

[0143] After the evaluation experts enter their workstations, the terminal activates its built-in camera to capture facial images and simultaneously reads the experts' ID card information. The system compares the collected information with the facial and identity data in the public resource transaction expert database. When the comparison pass rate is ≥95%, the identity binding is completed, and the terminal automatically associates with the current evaluation project and enters the data collection ready state.

[0144] (3) Multimodal data acquisition

[0145] Screen data: The terminal captures the screen of the evaluation computer through the HDMI-IN interface and generates an operation log at a frequency of 1 frame / second to record the expert's operations such as viewing the bid documents and entering scores;

[0146] Video data: A 1080P high-definition camera captures real-time frontal video of the expert, and AI algorithms track the position of the face to ensure that the image is always focused on the expert's face and upper body;

[0147] Voice data: A directional microphone array is used to collect the voices of experts discussing the topic. A noise reduction algorithm is used to filter out environmental noise (such as air conditioning noise and distant conversations) to improve voice clarity.

[0148] (4) Real-time analysis and early warning

[0149] The terminal's built-in lightweight AI model analyzes video and audio streams in real time:

[0150] Video analysis: Identify abnormal behaviors such as "unauthorized departure (expert leaving the workstation for more than 5 minutes)," "using a mobile phone (hand touching pocket / taking out mobile phone)," and "turning to communicate (body turning to other workstations at an angle greater than 45°)."

[0151] Voice analysis: Detects sensitive words such as "bid rigging", "favoring a certain unit", and "giving high scores", and also identifies abnormal volume (such as sudden loud arguments);

[0152] Warning Execution: Upon detecting an anomaly, the terminal immediately issues a buzzer alarm (audio warning) and a flashing red indicator light (visual warning), and sends a pop-up warning to the computer of the witness at the trading center. At the same time, it captures video and audio clips 10 seconds before and after the anomaly, encrypts them, and uploads them to the cloud.

[0153] (5) Deep cloud processing and evidence storage

[0154] After receiving the data uploaded by the terminal, the cloud platform performs the following operations:

[0155] Multimodal fusion analysis: By calling a pre-trained large model and combining it with information on the bidding project (such as bidding type and participating units), expert historical behavior (such as past violation records), and violation case database, the authenticity of anomalies is verified (such as eliminating the misjudgment of experts "turning around to retrieve documents").

[0156] Blockchain-based evidence storage: After confirming the anomaly, the warning event (anomaly type, time of occurrence, and personnel involved) and key evidence (video and audio clips) are written into the blockchain-based evidence storage platform to ensure that the data is tamper-proof.

[0157] Multi-device notification: Send warning notifications to regulatory personnel via SMS and mobile app, along with links to abnormal evidence, and support real-time viewing.

[0158] (6) Supervision and retrospection

[0159] Real-time monitoring: Supervisors can view the real-time status (normal / warning) and warning list (sorted by urgency) of all bidding stations through the visual monitoring platform, and can click to view evidence of abnormalities;

[0160] Data backtracking: The platform supports querying historical data by project number, expert name, anomaly type (such as "sensitive remarks" or "unauthorized absence"), and time period (such as "October 20, 2024, 9:00-12:00"), and exporting evidence such as operation logs and video clips, providing a complete chain of evidence for dispute resolution and regulatory auditing.

[0161] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A public resource transaction AI-powered intelligent witnessing terminal based on multimodal perception, characterized in that: include: The edge AI smart witnessing terminal and the cloud platform are connected via an encrypted communication link to achieve data interaction and collaborative work. The edge-side AI intelligent witnessing terminal includes: The intelligent sensing module is used to synchronously collect multimodal data from the public resource trading site. The multimodal data includes video image data, voice data, and trading terminal screen operation data. The intelligent sensing module consists of an intelligent camera array, a directional microphone array, a face recognition device, and a screen capture device. The data processing and early warning module is equipped with a high-performance edge computing chip for real-time processing and analysis of multimodal data collected by the intelligent sensing module. The data processing and early warning module has built-in AI algorithms, including an abnormal behavior recognition algorithm and a biased speech detection algorithm. The data processing and early warning module can trigger an early warning mechanism when an anomaly is detected. The hardware module includes an interface unit, a communication unit, and a security encryption unit. The interface unit is used to enable external device connection and data interaction, the communication unit is used to establish a network connection with the cloud platform, and the security encryption unit is used to encrypt the data.

2. The AI-powered intelligent witnessing terminal for public resource transactions based on multimodal perception as described in claim 1, characterized in that, The cloud platform includes: The data storage and management module adopts distributed storage technology to store multimodal data and early warning information uploaded by the edge AI intelligent witness terminal, and establishes a data index and classification system; The deep analysis module is used to perform in-depth mining and analysis of stored multimodal data, build a risk behavior feature library and personnel profile model, and output heat maps of high-frequency violation areas and personnel risk indices; The intelligent decision-making module is used to generate intelligent regulatory decision-making suggestions based on the analysis results of the deep analysis module; The interface service module is used to interface with external systems to achieve data sharing and interaction.

3. The witnessing method of the public resource transaction AI intelligent witnessing terminal based on multimodal perception according to any one of claims 1-2, characterized in that, Includes the following steps: S1: The AI-powered smart witness terminal on the edge initializes itself, completes device self-test, parameter configuration and network connection, registers with the cloud platform and obtains a unique device ID; S2: Personnel involved in public resource transactions complete identity authentication through the facial recognition device of the AI ​​smart witness terminal on the edge. After successful authentication, the current transaction project is bound, and the terminal starts multimodal data collection. S3: The intelligent sensing module of the edge AI intelligent witness terminal continuously collects multimodal data, which includes video image data, voice data and transaction terminal screen operation data; S4: The data processing and early warning module of the edge AI intelligent witness terminal performs real-time analysis on the collected multimodal data. If an anomaly is detected, an early warning mechanism is triggered and the early warning information and related evidence data are encrypted and uploaded to the cloud platform. S5: The cloud platform receives the warning information and related evidence data, and stores and classifies them through the data storage and management module; S6: The cloud platform's deep analysis module performs in-depth mining and analysis of the stored data, and verifies the authenticity of anomalies by combining the preset risk behavior feature library and personnel profile model; S7: The cloud platform's intelligent decision-making module generates intelligent regulatory decision-making suggestions based on in-depth analysis results and pushes them to the regulatory personnel's terminals; S8: Regulatory personnel take regulatory intervention measures based on the intelligent regulatory decision-making suggestions, and feed back the processing results to the end-side terminal and cloud platform to achieve system optimization.

4. The witnessing method of the public resource transaction AI intelligent witnessing terminal based on multimodal perception according to claim 3, characterized in that, In step S1, the initialization also includes parameter configuration, which includes the acquisition frequency configuration of multimodal data and the early warning threshold configuration.

5. The witnessing method of the AI-powered intelligent witnessing terminal for public resource transactions based on multimodal perception as described in claim 3, characterized in that, In step S2, the identity authentication is achieved by comparing the facial image collected by the facial recognition device with information from a preset expert database.

6. The witnessing method of the AI-powered intelligent witnessing terminal for public resource transactions based on multimodal perception as described in claim 3, characterized in that, In step S6, the deep analysis module performs multimodal fusion analysis on video image data, voice data and screen operation data through a pre-trained large model, and verifies the authenticity of the anomaly by combining the current transaction project information and the historical violation case library to eliminate false alarms.

7. The witnessing method of the public resource transaction AI intelligent witnessing terminal based on multimodal perception according to claim 3, characterized in that, In step S5, the cloud platform also stores the confirmed abnormal events and related evidence data through a blockchain evidence storage platform to ensure that the data is tamper-proof.

8. The witnessing method of the AI-powered intelligent witnessing terminal for public resource transactions based on multimodal perception as described in claim 3, characterized in that, In step S7, the intelligent supervision decision suggestions are pushed to at least one of the following terminals for supervisors: PC, large screen, and mobile device.

9. The witnessing method of the public resource transaction AI intelligent witnessing terminal based on multimodal perception according to claim 3, characterized in that, In step S8, the system optimization includes the edge AI intelligent witness terminal adjusting the early warning threshold and data collection strategy according to the feedback results, and the cloud platform optimizing the risk behavior feature library, personnel profile model and intelligent decision-making algorithm according to the feedback results.

Images