Paperless conference data real-time acquisition and analysis system

Abstract

The invention discloses a paperless conference data real-time acquisition and analysis system, and relates to the field of data analysis. According to the paperless conference data real-time acquisition and analysis system, environment information labeling is carried out on a generated conference environment picture set, picture labeling detection is carried out on labeled conference environment pictures, and whether current conference environment video data needs to be locally stored or not is judged according to a picture labeling detection result; and cloud storage is performed on the conference environment video data if the judgment result is no, and local storage is performed on the conference environment video data if the judgment result is yes, so that respective storage of the video information acquired by the vehicle recorder in real time is realized, and the storage pressure of a local storage system is reduced; and meanwhile, important environment video information existing in the conference operation environment is locally stored through annotation and annotation detection of the conference environment pictures, so that video query is facilitated.

Description

Technical Field

[0001] The invention relates to data analysis technology, and in particular to a paperless conference data real-time collection and analysis system. Background Art

[0002] The paperless conference system is an intelligent conference interactive system based on LAN, private network or mobile Internet. It uses communication technology, audio technology, video technology and software technology to realize paperless conference through electronic exchange of files. Its characteristics are: networked file transmission, electronic file display, intelligent file editing, and controllable file input and output. The core functions are mainly to realize conference sign-in, name guidance, file distribution and upload, file synchronization demonstration, voting, handwritten annotations, call service, voice subtitle display, remote video conference and other functions. Windows, IOS, Android cross-system platform interactive application has been realized. Paperless conference terminals are diversified: there are LCD lifting conference terminals, flip LCD terminals, flat-panel paperless, desktop conference terminals, etc. Ultra-thin LCD lifting paperless conference terminals have become mainstream and high-end. The scene application is mainly for public security, procuratorial, judicial, government, finance, education, enterprises and institutions.

[0003] However, the existing conference recorders currently store the video data recorded by the conference recorders in the local storage system during use. This will result in a large amount of video in the local storage system, requiring a long query time when searching for videos. At the same time, a large amount of local storage data will cause a heavy burden on the storage system. Summary of the invention

[0004] The purpose of the present invention is to provide a paperless conference data real-time collection and analysis system to solve the above-mentioned deficiencies in the prior art.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a paperless conference data real-time collection and analysis system, comprising:

[0006] A real-time acquisition module for a conference recorder, which is used to collect video data of the conference environment in real time when the conference is running;

[0007] A picture sampling module, the picture sampling module is used to set the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the conference is running;

[0008] An information annotation module, wherein the information annotation module is used to annotate the conference environment picture set sampled by the picture sampling module with environmental information;

[0009] A labeling detection module, wherein the labeling detection module is used to perform image labeling detection on the labeled conference environment image;

[0010] A data judgment module, which is used to judge the storage mode of the current conference environment video data according to the picture annotation detection result in the annotation detection module;

[0011] A local storage system, wherein the local storage system is used for the conference recorder to locally store important conference environment video data;

[0012] A cloud storage system is used to store conference environment video data that is judged to be safe in the cloud.

[0013] Furthermore, the specific working steps of the cloud storage system include:

[0014] A) The conference recorder collects video data, compresses and encodes the video data to generate a video data stream;

[0015] B) using the TCP / IP protocol, uploading the video data stream directly to a cloud data storage server through a network interface, and storing it in the data storage space of the cloud data storage server according to a set file format, and uploading the video data stream to the data storage space of the cloud data storage server after transferring it through a streaming media forwarding server;

[0016] C) the cloud storage management program of the cloud data storage server monitors and counts the space capacity and time used by the video data stream in real time, and executes step D) or D');

[0017] D) determining whether the space used by the video data stream reaches the maximum capacity, if so, executing step E); otherwise, continuing with the determination of this step;

[0018] D') determining whether the time used by the video data stream reaches the maximum storage time, if the determination result is yes, executing step E); otherwise, continuing the determination of this step;

[0019] E) Delete and overwrite the old video data stream according to the first-in-first-out rule.

[0020] Further, the following working steps are included:

[0021] S1, the conference recorder collects video data of the conference environment in real time when the conference is running;

[0022] S2, setting the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the conference is running;

[0023] S3, annotating the conference environment picture set generated in step S2 with environment information;

[0024] S4, performing image annotation detection on the annotated conference environment image;

[0025] S5, judging whether the current conference environment video data needs to be stored locally according to the image annotation detection result in step S4;

[0026] S6, the conference environment video data for which the judgment result in step S5 is no is stored in the cloud, and the conference environment video data for which the judgment result in step S5 is yes is stored locally.

[0027] Furthermore, the specific working steps of step S3 for marking the environment information of the conference environment picture set are:

[0028] A1, set up the target detection model;

[0029] A2, target feature extraction of conference environment images;

[0030] A3, sample simplification based on feature extraction and clustering;

[0031] A4, transmit the simplified conference environment picture to the staff for manual annotation;

[0032] A5, transfer the manually annotated conference environment pictures to the annotated sample set;

[0033] A6, transferring the labeled sample set to the autonomous learning system for image annotation learning;

[0034] A7, the system automatically labels the unlabeled conference environment pictures.

[0035] Furthermore, the specific working steps of the image annotation detection in step S4 are as follows:

[0036] B1, collect image annotations of all conference environment images;

[0037] B2, performing text preprocessing on the image annotation information collected in step B1 to obtain interference-free image annotation text data;

[0038] B3, determine whether it is an abnormal text by using an abnormal text detection algorithm, if the judgment result is no, execute step B4, if the judgment result is yes, execute step B5;

[0039] B4, transmitting the conference environment image to the security database, and returning to step B1;

[0040] B5, transferring the abnormal text to the structured database through the abnormal text entity recognition algorithm;

[0041] B6, transfer the abnormal text to the relational database through the abnormal text relation extraction algorithm, and return to step B1.

[0042] Furthermore, the specific working steps of determining whether the current conference environment video data needs to be stored locally in step S5 are:

[0043] C1, each module in the system is initialized and the superior and subordinate relationships are bound;

[0044] C2, the image annotation detection result input module records the image annotation detection result, slices the image annotation detection result, and writes the sliced ​​data into the queue of the noise reduction integration module;

[0045] C3, after the noise reduction integration module calls the SDK of the noise reduction model to perform data noise reduction, it reorganizes the slices into sentences and writes them into the startup instruction recognition module for processing;

[0046] C4, start the command recognition module to call the SDK of the command recognition model to perform judgment and noise reduction, and write the command recognition result into the state management module;

[0047] C5, the status management module updates the current node status according to the instruction recognition result. When the node is in the normal login state, it will send a request to the timing module to refresh the login status, and send a start judgment recognition request to the IO module;

[0048] C6, the TCP connection node of the computing node receives the judgment and identification request, decodes the message through data programming means, and creates a judgment and processing ASR task according to the request parameters;

[0049] C7, the ASR task adds the judgment and recognition task to the server through the add task interface provided by the server. The server shards the judgment data and delegates the task to the thread pool for calculation;

[0050] C8, after the thread pool is executed, it will determine whether the text data is stored through the interface provided by the data channel. The data buffer obtains the memory cache area corresponding to the ID through the unique ID of the edge node. If the remaining space in the cache area is not enough to store the next data, execute step 4. Otherwise, the data can be stored in the cache area, and the key value of the record is returned to the ASR task. This key value is a unique message ID composed of the edge node ID and the timestamp, and execute step 5;

[0051] C9: The data channel obtains a new memory cache area and synchronizes the data block index of the cache area that was previously full of data as metadata to the central leader node. After the synchronization is completed, the data block is asynchronously dropped to the disk through the disk thread pool to prevent memory data loss. After the disk is successfully dropped, the block ID is returned;

[0052] C10, the ASR task assembles the corresponding message according to the key value and sends it to the TCP connection node. The TCP connection node serializes the data and returns it to the edge node, and determines that the recognition request is completed.

[0053] Furthermore, the specific steps of starting instruction identification in step C4 are:

[0054] D1, first, the local model detects each sub-data spectrum in the image annotation detection result;

[0055] D2, if the posterior score of a sub-spectrum graph exceeds the threshold A1, it is considered as a local trigger;

[0056] D3, the system records the start time T1 and end time T2 of consecutive local triggers, where the start time T1 is the left edge of the window corresponding to the first local trigger, and the end time T2 is the right edge of the window corresponding to the last local trigger;

[0057] D4, when continuous local triggering occurs, it indicates that the wake-up word exists in the image annotation from T1 to T2;

[0058] D5, at the same time, the maximum a posteriori score among these consecutive local triggers is used as the score F1 output by M1. If no consecutive local trigger occurs, F1 = 0;

[0059] D6, according to T1 and T2, cut out sub-spectrograms from the original input spectrogram, and these sub-spectrograms are called candidate spectrograms;

[0060] D7, due to the influence of speech rate changes, the length of the candidate spectrogram is also changing. The video flow rate is normalized by using the spectrogram interpolation method to obtain a fixed-length spectrogram;

[0061] D8, the global model M2 detects the fixed-length spectrogram and obtains the posterior score F2;

[0062] D9, when no continuous local triggering occurs in the first stage, F2 = 0;

[0063] D10, combining the local and global models to detect the start command, the score F is the average of F1 and F2. If F is greater than the threshold, the start command is triggered.

[0064] Compared with the prior art, the present invention provides a paperless conference data real-time collection and analysis system, which annotates the generated conference environment picture set with environmental information, performs picture annotation detection on the annotated conference environment pictures, and judges whether the current conference environment video data needs to be stored locally based on the picture annotation detection result. The conference environment video data with a judgment result of "no" is stored in the cloud, and the conference environment video data with a judgment result of "yes" is stored locally. This arrangement not only realizes the separate storage of video information collected in real time by the vehicle recorder, and reduces the storage pressure of the local storage system, but also stores important environmental video information in the conference operation environment locally through the annotation and annotation detection of the conference environment pictures to facilitate video query. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0066] Figure 1 A schematic diagram of the overall structure provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0068] See also Figure 1 , a paperless conference data real-time collection and analysis system, comprising:

[0069] The real-time acquisition module of the conference recorder is used to collect the video data of the conference environment in real time when the conference is running;

[0070] The picture sampling module is used to set the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the conference is running;

[0071] An information annotation module, which is used to annotate the environment information of the conference environment picture set sampled by the picture sampling module;

[0072] A labeling detection module is used to perform image labeling detection on the labeled conference environment image;

[0073] A data judgment module, which is used to judge the storage mode of the current conference environment video data according to the image annotation detection result in the annotation detection module;

[0074] Local storage system: The local storage system is used for conference recorders to store important conference environment video data locally;

[0075] The cloud storage system is used to store the video data of the conference environment that is judged to be safe in the cloud.

[0076] In this way, the real-time acquisition module of the conference recorder is used to collect the conference environment video data when the meeting is running in real time, and the picture sampling module is used to set the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the meeting is running, and the generated conference environment picture set is annotated with environmental information by the information annotation module, and the annotated conference environment pictures are detected by the annotation detection module. The data judgment module determines whether the current conference environment video data needs to be stored locally based on the picture annotation detection result, and the conference environment video data with a judgment result of "no" is stored in the cloud, and the conference environment video data with a judgment result of "yes" is stored locally. This setting not only realizes the separate storage of video information collected by the vehicle recorder in real time, and reduces the storage pressure of the local storage system, but also stores important environmental video information in the conference operation environment locally through the annotation and annotation detection of the conference environment pictures to facilitate video query.

[0077] The specific working steps of the cloud storage system include:

[0078] A) The conference recorder collects video data, compresses and encodes the video data to generate a video data stream;

[0079] B) Using TCP / IP protocol, the video data stream is directly uploaded to the cloud data storage server through the network interface, and stored in the data storage space of the cloud data storage server according to the set file format, and the video data stream is transferred through the streaming media forwarding server and uploaded to the data storage space of the cloud data storage server;

[0080] C) The cloud storage management program of the cloud data storage server monitors and counts the space capacity and time used by the video data stream in real time, and executes step D) or D');

[0081] D) determining whether the space used by the video data stream reaches the maximum capacity, if so, executing step E); otherwise, continuing with the determination of this step;

[0082] D') Determine whether the time used for the video data stream reaches the maximum storage time. If the determination result is yes, execute step E); otherwise, continue to determine in this step;

[0083] E) Delete and overwrite the old video data stream according to the first-in-first-out rule.

[0084] A paperless conference data real-time collection and analysis system includes the following working steps:

[0085] S1, the conference recorder collects video data of the conference environment in real time when the conference is running;

[0086] S2, setting the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the conference is running;

[0087] S3, annotating the conference environment picture set generated in step S2 with environment information;

[0088] S4, performing image annotation detection on the annotated conference environment image;

[0089] S5, judging whether the current conference environment video data needs to be stored locally according to the image annotation detection result in step S4;

[0090] S6, the conference environment video data for which the judgment result in step S5 is no is stored in the cloud, and the conference environment video data for which the judgment result in step S5 is yes is stored locally.

[0091] In this way, the conference environment video data of the meeting is collected in real time through the conference recorder, and the conference environment video data is sampled at the same sampling time interval to obtain a set of conference environment pictures when the meeting is running, and the generated conference environment picture set is annotated with environmental information. The annotated conference environment pictures are detected by image annotation. According to the image annotation detection results, it is determined whether the current conference environment video data needs to be stored locally. The conference environment video data with a judgment result of "no" is stored in the cloud, and the conference environment video data with a judgment result of "yes" is stored locally. This setting not only realizes the separate storage of video information collected by the vehicle recorder in real time, and reduces the storage pressure of the local storage system, but also stores important environmental video information in the conference operation environment locally through the annotation and annotation detection of the conference environment pictures to facilitate video query.

[0092] The specific working steps of step S3 for marking the environment information of the conference environment picture set are as follows:

[0093] A1, set up the target detection model;

[0094] A2, target feature extraction of conference environment images;

[0095] A3, sample simplification based on feature extraction and clustering;

[0096] A4, transmit the simplified conference environment picture to the staff for manual annotation;

[0097] A5, transfer the manually annotated conference environment pictures to the annotated sample set;

[0098] A6, transferring the labeled sample set to the autonomous learning system for image annotation learning;

[0099] A7, the system automatically labels the unlabeled conference environment pictures.

[0100] The specific working steps of step S4 for image annotation detection are as follows:

[0101] B1, collect image annotations of all conference environment images;

[0102] B2, performing text preprocessing on the image annotation information collected in step B1 to obtain interference-free image annotation text data;

[0103] B3, determine whether it is an abnormal text by using an abnormal text detection algorithm, if the judgment result is no, execute step B4, if the judgment result is yes, execute step B5;

[0104] B4, transmitting the conference environment image to the security database, and returning to step B1;

[0105] B5, transferring the abnormal text to the structured database through the abnormal text entity recognition algorithm;

[0106] B6, transfer the abnormal text to the relational database through the abnormal text relation extraction algorithm, and return to step B1.

[0107] The specific working steps for determining whether the current conference environment video data needs to be stored locally in step S5 are:

[0108] C1, each module in the system is initialized and the superior and subordinate relationships are bound;

[0109] C2, the image annotation detection result input module records the image annotation detection result, slices the image annotation detection result, and writes the sliced ​​data into the queue of the noise reduction integration module;

[0110] C3, after the noise reduction integration module calls the SDK of the noise reduction model to perform data noise reduction, it reorganizes the slices into sentences and writes them into the startup instruction recognition module for processing;

[0111] C4, start the command recognition module to call the SDK of the command recognition model to perform judgment and noise reduction, and write the command recognition result into the state management module;

[0112] C5, the status management module updates the current node status according to the instruction recognition result. When the node is in the normal login state, it will send a request to the timing module to refresh the login status, and send a start judgment recognition request to the IO module;

[0113] C6, the TCP connection node of the computing node receives the judgment and identification request, decodes the message through data programming means, and creates a judgment and processing ASR task according to the request parameters;

[0114] C7, the ASR task adds the judgment and recognition task to the server through the add task interface provided by the server. The server shards the judgment data and delegates the task to the thread pool for calculation;

[0115] C8, after the thread pool is executed, it will determine whether the text data is stored through the interface provided by the data channel. The data buffer obtains the memory cache area corresponding to the ID through the unique ID of the edge node. If the remaining space in the cache area is not enough to store the next data, execute step 4. Otherwise, the data can be stored in the cache area, and the key value of the record is returned to the ASR task. This key value is a unique message ID composed of the edge node ID and the timestamp, and execute step 5;

[0116] C9: The data channel obtains a new memory cache area and synchronizes the data block index of the cache area that was previously full of data as metadata to the central leader node. After the synchronization is completed, the data block is asynchronously dropped to the disk through the disk thread pool to prevent memory data loss. After the disk is successfully dropped, the block ID is returned;

[0117] C10, the ASR task assembles the corresponding message according to the key value and sends it to the TCP connection node. The TCP connection node serializes the data and returns it to the edge node, and determines that the recognition request is completed.

[0118] The specific steps of starting instruction recognition in step C4 are:

[0119] D1, first, the local model detects each sub-data spectrum in the image annotation detection result;

[0120] D2, if the posterior score of a sub-spectrum graph exceeds the threshold A1, it is considered as a local trigger;

[0121] D3, the system records the start time T1 and end time T2 of consecutive local triggers, where the start time T1 is the left edge of the window corresponding to the first local trigger, and the end time T2 is the right edge of the window corresponding to the last local trigger;

[0122] D4, when continuous local triggering occurs, it indicates that the wake-up word exists in the image annotation from T1 to T2;

[0123] D5, at the same time, the maximum a posteriori score among these consecutive local triggers is used as the score F1 output by M1. If no consecutive local trigger occurs, F1 = 0;

[0124] D6, according to T1 and T2, cut out sub-spectrograms from the original input spectrogram, and these sub-spectrograms are called candidate spectrograms;

[0125] D7, due to the influence of speech rate changes, the length of the candidate spectrogram is also changing. The video flow rate is normalized by using the spectrogram interpolation method to obtain a fixed-length spectrogram;

[0126] D8, the global model M2 detects the fixed-length spectrogram and obtains the posterior score F2;

[0127] D9, when no continuous local triggering occurs in the first stage, F2 = 0;

[0128] D10, combining the local and global models to detect the start command, the score F is the average of F1 and F2. If F is greater than the threshold, the start command is triggered.

[0129] Working principle: When in use, the real-time acquisition module of the conference recorder is used to collect the conference environment video data when the meeting is running in real time. The picture sampling module is used to set the same sampling time interval to sample the conference environment video data to obtain the conference environment picture set when the meeting is running. The generated conference environment picture set is annotated with environmental information through the information annotation module. The annotated conference environment pictures are annotated with the annotation detection module. The data judgment module is used to judge whether the current conference environment video data needs to be stored locally based on the picture annotation detection results. The conference environment video data with a judgment result of "no" is stored in the cloud, and the conference environment video data with a judgment result of "yes" is stored locally. This setting not only realizes the separate storage of video information collected by the vehicle recorder in real time, and reduces the storage pressure of the local storage system, but also stores important environmental video information in the conference operation environment locally through the annotation and annotation detection of the conference environment pictures for video query.

[0130] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A paperless conference data real-time collection and analysis system, characterized in that: include: A real-time acquisition module for a conference recorder, which is used to collect video data of the conference environment in real time when the conference is running; A picture sampling module, the picture sampling module is used to set the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the conference is running; An information annotation module, wherein the information annotation module is used to annotate the conference environment picture set sampled by the picture sampling module with environmental information; A labeling detection module, wherein the labeling detection module is used to perform image labeling detection on the labeled conference environment image; A data judgment module, which is used to judge the storage mode of the current conference environment video data according to the picture annotation detection result in the annotation detection module; A local storage system, wherein the local storage system is used for the conference recorder to locally store important conference environment video data; A cloud storage system is used to store conference environment video data that is judged to be safe in the cloud.

2. A paperless conference data real-time collection and analysis system according to claim 1, characterized in that: The specific working steps of the cloud storage system include: A) The conference recorder collects video data, compresses and encodes the video data to generate a video data stream; B) using the TCP / IP protocol, uploading the video data stream directly to a cloud data storage server through a network interface, and storing it in the data storage space of the cloud data storage server according to a set file format, and uploading the video data stream to the data storage space of the cloud data storage server after transferring it through a streaming media forwarding server; C) the cloud storage management program of the cloud data storage server monitors and counts the space capacity and time used by the video data stream in real time, and executes step D) or D'); D) determining whether the space used by the video data stream reaches the maximum capacity, if so, executing step E); otherwise, continuing with the determination of this step; D') determining whether the time used by the video data stream reaches the maximum storage time, if the determination result is yes, executing step E); otherwise, continuing the determination of this step; E) Delete and overwrite the old video data stream according to the first-in-first-out rule.

3. A paperless conference data real-time collection and analysis system according to claim 2, characterized in that: The following steps are included: S1, the conference recorder collects video data of the conference environment in real time when the conference is running; S2, setting the same sampling time interval to sample the conference environment video data to obtain a conference environment picture set when the conference is running; S3, annotating the conference environment picture set generated in step S2 with environment information; S4, performing image annotation detection on the annotated conference environment image; S5, judging whether the current conference environment video data needs to be stored locally according to the image annotation detection result in step S4; S6, the conference environment video data for which the judgment result in step S5 is no is stored in the cloud, and the conference environment video data for which the judgment result in step S5 is yes is stored locally.

4. A paperless conference data real-time collection and analysis system according to claim 3, characterized in that: The specific working steps of step S3 for marking the environment information of the conference environment picture set are: A1, set up the target detection model; A2, target feature extraction of conference environment images; A3, sample simplification based on feature extraction and clustering; A4, transmit the simplified conference environment picture to the staff for manual annotation; A5, transfer the manually annotated conference environment pictures to the annotated sample set; A6, transferring the labeled sample set to the autonomous learning system for image annotation learning; A7, the system automatically labels the unlabeled conference environment pictures.

5. A paperless conference data real-time collection and analysis system according to claim 4, characterized in that: The specific working steps of the image annotation detection in step S4 are as follows: B1, collect image annotations of all conference environment images; B2, performing text preprocessing on the image annotation information collected in step B1 to obtain interference-free image annotation text data; B3, determine whether it is an abnormal text by using an abnormal text detection algorithm, if the judgment result is no, execute step B4, if the judgment result is yes, execute step B5; B4, transmitting the conference environment image to the security database, and returning to step B1; B5, transferring the abnormal text to the structured database through the abnormal text entity recognition algorithm; B6, transfer the abnormal text to the relational database through the abnormal text relation extraction algorithm, and return to step B1.

6. A paperless conference data real-time collection and analysis system according to claim 5, characterized in that: The specific working steps of determining whether the current conference environment video data needs to be stored locally in step S5 are: C1, each module in the system is initialized and the superior and subordinate relationships are bound; C2, the image annotation detection result input module records the image annotation detection result, slices the image annotation detection result, and writes the sliced ​​data into the queue of the noise reduction integration module; C3, after the noise reduction integration module calls the SDK of the noise reduction model to perform data noise reduction, it reorganizes the slices into sentences and writes them into the startup instruction recognition module for processing; C4, start the command recognition module to call the SDK of the command recognition model to perform judgment and noise reduction, and write the command recognition result into the state management module; C5, the status management module updates the current node status according to the instruction recognition result. When the node is in the normal login state, it will send a request to the timing module to refresh the login status, and send a start judgment recognition request to the IO module; C6, the TCP connection node of the computing node receives the judgment and identification request, decodes the message through data programming means, and creates a judgment and processing ASR task according to the request parameters; C7, the ASR task adds the judgment and recognition task to the server through the add task interface provided by the server. The server shards the judgment data and delegates the task to the thread pool for calculation; C8, after the thread pool is executed, it will determine whether the text data is stored through the interface provided by the data channel. The data buffer obtains the memory cache area corresponding to the ID through the unique ID of the edge node. If the remaining space in the cache area is not enough to store the next data, execute step 4. Otherwise, the data can be stored in the cache area, and the key value of the record is returned to the ASR task. This key value is a unique message ID composed of the edge node ID and the timestamp, and execute step 5; C9: The data channel obtains a new memory cache area and synchronizes the data block index of the cache area that was previously full of data as metadata to the central leader node. After the synchronization is completed, the data block is asynchronously dropped to the disk through the disk thread pool to prevent memory data loss. After the disk is successfully dropped, the block ID is returned; C10, the ASR task assembles the corresponding message according to the key value and sends it to the TCP connection node. The TCP connection node serializes the data and returns it to the edge node, and determines that the recognition request is completed.

7. A paperless conference data real-time collection and analysis system according to claim 6, characterized in that: The specific steps of starting instruction recognition in step C4 are: D1, first, the local model detects each sub-data spectrum in the image annotation detection result; D2, if the posterior score of a sub-spectrum graph exceeds the threshold A1, it is considered as a local trigger; D3, the system records the start time T1 and end time T2 of consecutive local triggers, where the start time T1 is the left edge of the window corresponding to the first local trigger, and the end time T2 is the right edge of the window corresponding to the last local trigger; D4, when continuous local triggering occurs, it indicates that the wake-up word exists in the image annotation from T1 to T2; D5, at the same time, the maximum a posteriori score among these consecutive local triggers is used as the score F1 output by M1. If no consecutive local trigger occurs, F1 = 0; D6, according to T1 and T2, cut out sub-spectrograms from the original input spectrogram, and these sub-spectrograms are called candidate spectrograms; D7, due to the influence of speech rate changes, the length of the candidate spectrogram is also changing. The video flow rate is normalized by using the spectrogram interpolation method to obtain a fixed-length spectrogram; D8, the global model M2 detects the fixed-length spectrogram and obtains the posterior score F2; D9, when no continuous local triggering occurs in the first stage, F2 = 0; D10, combining the local and global models to detect the start command, the score F is the average of F1 and F2. If F is greater than the threshold, the start command is triggered.

Images