Feature analysis complementary system of multi-modal data

Abstract

The application relates to a feature analysis complementary system of multi-modal data applied to the field of electric data processing, wherein a fusion relay station is arranged to perform feature analysis on multi-modal data of multiple edge perception devices under the same scene; on one hand, after fusion features are sent to a control center, the data transmission load can be effectively reduced, and the problem of excessive information amount received by the control center and low processing efficiency can be solved; on the other hand, by comparing feature information of the multi-modal data, data with inconsistent feature information can be obtained in time, and by replacing a data link, abnormal factors of the data can be effectively judged, and edge perception device fault conditions or data link fault conditions can be known in time, so that early warning and corresponding subsequent operations can be performed in time, and the safety risks caused by the fault factors to the fusion relay station and the control center can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to a feature analysis complementary system, in particular to a feature analysis complementary system for multi-modal data applied in the field of electrical data processing. BACKGROUND

[0002] Multi-modal data fusion involves synthesizing different information obtained by different sensors to obtain a more comprehensive and accurate scene representation. Each sensor data has its unique advantages and limitations, for example, microwave data can provide stable target information in complex weather and low light environment, while infrared data can show the temperature distribution of objects and provide deep information about the state of objects, and is not affected by light conditions.

[0003] Chinese patent CN120030499B discloses a multi-modal data complementary, error correction and fault tolerance mechanism fusion method, which fully utilizes the complementarity between multi-modal data through the driving of deep learning, solves the problems of information loss, registration error and fault tolerance, significantly improves the precision and robustness of microwave and infrared data fusion, and is suitable for target detection and identification in complex data fusion tasks.

[0004] For example, Chinese patent CN113407759B discloses a multi-modal entity alignment method based on adaptive feature fusion, which calculates the similarity score of entity-picture to filter noise pictures, and obtains more accurate entity visual feature representation based on similarity; an adaptive feature fusion mechanism is designed to fuse the structural features and visual features of entities with variable attention, fully utilizing the complementarity of multi-modal information, and improving the alignment effect.

[0005] Existing multi-modal data is prone to abnormality due to edge device failure, transmission network failure and other factors during transmission, but it is difficult to diagnose the cause of data abnormality in the prior art in a timely manner, thereby affecting the operation safety of the multi-modal data fusion process and related terminal equipment. SUMMARY

[0006] In view of the above prior art, the technical problem to be solved by the present application is that it is difficult to determine the cause of multi-modal data abnormality in the existing multi-modal data fusion process in a timely manner.

[0007] To solve the above problems, the application provides a multi-modal data feature analysis complementary system, comprising a control center, a plurality of fusion relay stations connected to the control center, a plurality of edge perception devices distributed in the same scene connected to the fusion relay stations, the plurality of edge perception devices being respectively used for collecting different types of data and respectively transmitting the data to the fusion relay stations through different data links, the fusion relay station comprising a data receiving module, a feature analysis module, an uploading module, a data filtering module, a link switching module, a judgment processing module and a multi-source early warning module, the data receiving module being used for receiving the data of the plurality of edge perception devices, the uploading module being used for transmitting the feature information of the multi-modal data to the control center, the link switching module being used for replacing the data link of the edge perception device for data transmission, the data filtering module being used for filtering out the data with inconsistent feature information expression, and the judgment processing module being used for judging the faults of the plurality of edge perception devices and the plurality of data links when the feature information of the multi-modal data is inconsistent.

[0008] The method for using the multi-modal data feature analysis complementary system comprises the following steps:

[0009] S1, the plurality of edge perception devices transmit the collected data to the fusion relay station through different data links, so that the fusion relay station receives different modal data in the same scene, and the fusion relay station extracts and analyzes the features of the plurality of different modal data;

[0010] S2, when the feature information of the different modal data is consistent, the fusion relay station fuses the feature information of all modal data and transmits the same to the control center;

[0011] S3, when the feature information of a modal data is inconsistent with the feature information of other modal data, the inconsistent feature information is filtered out first, the remaining consistent feature information is fused, and the same is transmitted to the control center, and the following operations are performed:

[0012] S31, the plurality of modal data with consistent features are all recorded as normal data, the corresponding edge perception devices are recorded as device A1, device A2 and device An respectively, the modal data with inconsistent features is recorded as abnormal data, the edge perception device corresponding to the abnormal data is recorded as device B, and the data link corresponding to the abnormal data is link N;

[0013] S32, the data links of all edge perception devices are replaced with each other, so that the data link of the device B after replacement is the initial data link of the device A1, and the initial link N of the device B is used as the data link of the device A2 after replacement;

[0014] S33, the fusion relay station continues to analyze the features of the plurality of different modal data received;

[0015] When the data feature expression of device B is still inconsistent with the data feature expression of other modalities, but the data feature expression transmitted by link N is consistent with the data feature expression of other modalities, it is determined that device B is malfunctioning, and device warning is performed;

[0016] When the data feature expression of device B is consistent with the data feature expression of other modalities, but the data feature expression transmitted by link N is inconsistent with the data feature expression of other modalities, it is determined that link N is malfunctioning, and link warning is performed.

[0017] As a further supplement to the present application, the fusion relay station is connected with a verification end, and the verification end is fixedly connected with a screen end at one end facing the fusion relay station, and the verification end comprises a data detection module, a safety feedback module, a task execution module and a display module, the task execution module is connected with a code library, and the display module is connected with the screen end.

[0018] As a further supplement to the present application, the fusion relay station is fixedly connected with an image collector at one end facing the verification end, the image collector visually monitors the screen end, and the fusion relay station further comprises an interface monitoring module connected with the image collector.

[0019] As a further supplement to the present application, after device warning is performed, the following step S4 is performed:

[0020] S41, the fusion relay station continues to receive the data collected by device B and transfers it to the verification end, and then disconnects the network connection with the verification end;

[0021] S42, the data detection module in the verification end performs safety detection on the data, and displays the detection result on the screen end, at the same time, the task execution module executes the task code in the code library in a timely manner, and displays the task code execution process on the screen end;

[0022] S43, the image collector on the fusion relay station visually monitors the screen end, and the interface monitoring module determines whether the data collected by device B has a security risk and whether the verification end is attacked by risky data according to the monitoring data.

[0023] As a further supplement to the present application, after link warning is performed, the following step S5 is performed:

[0024] S51, keep the allocation of the data link in step S32, the data collected by device A2 is transmitted through another data link to the fusion relay station while being transmitted through link N, and the fusion relay station transfers the data transmitted through link N to the verification end;

[0025] S52, steps S42 and S43 are performed in sequence.

[0026] As a further supplement to the present application, the side end of the fusion relay station is provided with an interface, the inside of the interface is provided with an inner joint, the inside of the image collector is fixedly connected with a driver, the output end of the driver movably extends into the interface and is fixedly connected with one end of the inner joint, and the inspection end is connected with an outer joint through a transmission cable, the outer joint is inserted into the interface and contacts the inner joint.

[0027] As a further supplement to the present application, the fusion relay station further comprises a wired disconnection module connected with the driver.

[0028] In summary, the fusion relay station is provided, the multi-modal data of multiple edge perception devices in the same scene is subjected to feature analysis, on the one hand, after the fusion feature is sent to the control center, the data transmission load can be effectively reduced, and the problem of large information amount received by the control center and low processing efficiency can be solved, on the other hand, by comparing the feature information of the multi-modal data, the data with inconsistent feature information can be obtained in time, and by replacing the data link, the abnormal factors of the data can be effectively judged, the fault condition of the edge perception device or the fault condition of the data link can be known in time, so that the corresponding subsequent operation can be carried out in time, and the safety risk caused by the fault factors to the fusion relay station and the control center can be reduced.

[0029] By providing the inspection end corresponding to the fusion relay station, when the equipment fails or the link fails, the abnormal data is transferred to the inspection end and isolated from the fusion relay station, the inspection end detects and displays the abnormal data, and at the same time, the safety condition of the abnormal data and the running condition of the inspection end are effectively judged through the timing display of the task code execution process in the inspection end, and then the type of equipment failure and the type of link failure are further obtained. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The system of the first embodiment of the present application Figure 1 ;

[0031] Figure 2 The system of the first embodiment of the present application Figure 2 ;

[0032] Figure 3 The system schematic diagram when the data link is replaced in the first embodiment of the present application

[0033] Figure 4 The flowchart of the first embodiment of the present application

[0034] Figure 5 The system of the second embodiment of the present application Figure 1 ;

[0035] Figure 6Fig. 2 is a structural schematic diagram of the fusion of the relay station and the inspection end in the second embodiment of the present application;

[0036] Figure 7 Fig. 3 is a system of the second embodiment of the present application Figure 2 ;

[0037] Figure 8 Fig. 4 is a structural schematic diagram of the fusion of the relay station and the inspection end in the third embodiment of the present application when connected;

[0038] Figure 9 Fig. 5 is a structural schematic diagram of the fusion of the relay station and the inspection end in the third embodiment of the present application when disconnected.

[0039] Explanation of the figure:

[0040] 1 screen end, 2 image collector, 3 external connector, 4 internal connector, 5 driver, 6 interface. DETAILED DESCRIPTION

[0041] The three embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0042] First embodiment:

[0043] The present application provides a feature analysis complementary system of multi-modal data, please refer to Figure 1 and Figure 2 , including a control center, the control center is connected with a plurality of fusion relay stations, the fusion relay station is connected with a plurality of edge perception devices distributed in the same scene, a plurality of edge perception devices are respectively used for collecting different types of data and transmitting data to the fusion relay station by using different data links respectively, the fusion relay station includes a data receiving module, a feature analysis module, an uploading module, a data filtering module, a link switching module, a judgment processing module and a multi-source early warning module, the data receiving module is used for receiving data of a plurality of edge perception devices, the feature analysis module is used for extracting and analyzing the received data to obtain feature information, the uploading module is used for transmitting the feature information of multi-modal data to the control center, the link switching module is used for replacing the data link of the edge perception device for data transmission, the data filtering module is used for filtering the data with inconsistent feature information expression, and the judgment processing module is used for fault judgment of a plurality of edge perception devices and a plurality of data links when the multi-modal data feature information expression is inconsistent.

[0044] Please refer to Figure 4 , the method for using the feature analysis complementary system of multi-modal data includes the following steps:

[0045] S1. Multiple edge sensing devices send the collected data to the fusion relay station through different data links, so that the fusion relay station receives data of different modalities in the same scene, and performs feature extraction and analysis on multiple different modalities of data;

[0046] S2. When the feature information of different modal data is consistent, the fusion relay station fuses the feature information of all modal data and sends them together to the control center.

[0047] S3. When the feature information of a certain modality is inconsistent with the feature information of other modalities, first filter out the inconsistent feature information, merge the remaining consistent feature information, and send them together to the control center, then perform the following operations:

[0048] S31. Multiple modal data with consistent feature expressions are all recorded as normal data, and their corresponding edge sensing devices are recorded as device A1, device A2... and device An, respectively. Modal data with inconsistent feature expressions are recorded as abnormal data, the edge sensing device corresponding to abnormal data is recorded as device B, and the data link corresponding to abnormal data is recorded as link N.

[0049] S32, such as Figure 3 As shown, the data links of all edge sensing devices are interchanged, so that the data link of device B after the interchange becomes the initial data link of device A1, and the initial link N of device B becomes the data link of device A2 after the interchange.

[0050] S33. The fusion relay station continues to perform feature analysis on the received data from multiple different modalities;

[0051] When the data feature expression of device B is still inconsistent with the data feature expression of other modes, but the data feature expression transmitted using link N is consistent with the data feature expression of other modes, it is determined that device B has a fault and a device warning is issued.

[0052] When the data feature expression of device B is consistent with the data feature expression of other modes, but the data feature expression of data transmitted using link N is inconsistent with the data feature expression of other modes, it is determined that link N has failed and a link warning is issued.

[0053] Since multiple edge sensing devices are in the same scene, although the types of data they collect are different (such as image data, audio data, point cloud data, etc.), the event results they express are consistent. The following explains the above usage process for a specific implementation scenario:

[0054] In the security monitoring scene, for the same area, multiple edge perception devices are set, including night vision white light cameras, infrared thermal imaging cameras, sound sensors, millimeter wave radars, etc. The night vision white light camera collects image data, the sound sensor collects environmental audio data, and the millimeter wave radar collects point cloud data. The above different modal data is transmitted to the fusion relay station through different data links for feature extraction and analysis, i.e. step S1;

[0055] When a person appears in the above area, the night vision white light camera collects the person's image, the infrared thermal imaging camera collects the human body's thermal radiation information, the sound sensor monitors the human body's moving sound, and the millimeter wave radar collects the human body's moving speed, position and other information. Under normal circumstances, the feature information expressed in multiple modal data is consistent, i.e. there is an unexpected person in the area. At this time, the feature information of all modal data can be fused and sent to the control center together, i.e. step S2. Compared with directly sending all original modal data, the application can effectively reduce the data transmission load and reduce the problem of excessive information amount received by the control center and low processing efficiency by fusing the features before sending;

[0056] When the feature information of a certain modal data in step S3 is inconsistent with the feature information of other modal data, for example: the millimeter wave radar, the night vision white light camera and the infrared thermal imaging camera all collect corresponding unexpected person information, but the audio information does not appear corresponding sound signal; or the night vision white light camera, the sound sensor and the infrared thermal imaging camera all monitor the corresponding unexpected person information, but the millimeter wave radar does not collect the object moving information. In the above cases, since the feature information expressed by most modal data is consistent, in order to timely deliver correct and important monitoring results, the consistent feature information should be fused and uploaded to the control center, effectively ensuring the timeliness and accuracy of perception monitoring, and then the inconsistent data is verified and judged, i.e. steps S31 to S33.

[0057] The factors that cause single data inconsistency include the following two: one is that the corresponding edge perception device has a fault (including normal damage of the device, malicious intrusion of the device, etc.) leading to abnormal data, and the other is that the corresponding data link has a fault (including unstable data link network card, data link network attack, etc.) leading to abnormal data. Therefore, through the operation of steps S31 to S33, the fault direction can be initially judged.

[0058] The subsequent operation of the use method of steps S1 to S33 includes the following:

[0059] Subsequent operation one, when it is determined that device B fails, on the one hand, the device warning in step S33 is performed, and on the other hand, the data transmission of device B can be disconnected, that is, the fusion relay station no longer receives the data of device B, and the analysis and fusion of other modal data can be performed before personnel maintenance, and in the case of partial data loss, the monitoring effect of the required scene is also achieved to a certain extent, that is, the complementary value of multi-modal data is realized.

[0060] Subsequent operation two, when it is determined that link N fails, on the one hand, the link warning in step S33 is performed, and on the other hand, since link N fails, the connection of link N is disconnected, that is, the use of link N is suspended, and the allocation of the remaining data links in step S32 is maintained, and then device A2 is transmitted to the fusion relay station through another data link, that is, device A2 shares a data link with a certain edge perception device, so that the fusion relay station can continue to perform feature analysis and fusion upload on all modal data.

[0061] In addition, when there are multiple data with inconsistent feature information expressions, for example, when both the millimeter wave radar and the infrared thermal imaging camera monitor the corresponding unexpected person information, but the image data collected by the night vision white light camera and the audio data monitored by the sound sensor do not appear the corresponding person feature information, since the probability of simultaneous failure of multiple edge perception devices or multiple data links is low, the above situation can be determined as environmental factors (such as severe weather such as storm and thunderstorm), at this time, no warning is performed, and the initial data link allocation is maintained.

[0062] The second embodiment:

[0063] The present embodiment is based on the first embodiment, and an inspection end is added, as follows: please refer to Figure 5 and Figure 6 The fusion relay station is connected with an inspection end, the end of the inspection end facing the fusion relay station is fixedly connected with a screen end 1, and the end of the fusion relay station facing the inspection end is fixedly connected with an image collector 2, which visually monitors the screen end 1, please refer to Figure 7 The fusion relay station further includes an interface monitoring module connected with the image collector 2, and the inspection end includes a data detection module, a safety feedback module, a task execution module and a display module, the data detection module is connected with the determination processing module of the fusion relay station, and is used for receiving data and performing safety detection, the safety feedback module is used for displaying the detection result on the screen end 1, the task execution module is connected with a code library, and the display module is connected with the screen end 1.

[0064] Through the above settings, the present embodiment also supplements the following two operations:

[0065] Operation I, when the device warning is carried out, the following steps S4 are carried out:

[0066] S41, the fusion relay station continues to receive the data collected by the device B and transfers it to the inspection end, and then disconnects the network connection with the inspection end;

[0067] S42, the data is detected by the data detection module in the inspection end, and the detection result is displayed on the screen end 1, and the task execution module executes the task code in the code library at regular intervals, and the task code execution process is displayed on the screen end 1;

[0068] S43, the image collector 2 on the fusion relay station visually monitors the screen end 1, and the interface monitoring module determines whether the data collected by the device B has security risks and whether the inspection end is attacked by risky data according to the monitoring data, and the specific determination method is as follows:

[0069] When the image collector 2 monitors that the screen end 1 appears abnormal picture, including the following two cases, case one, the screen end 1 displays "data detection result exists risk", and the task code execution process is normally displayed, which indicates that the data collected by the device B may exist virus, at this time, it is determined that the data collected by the device B has security risks, that is, the device B may be maliciously invaded, and the inspection end is not attacked by risky data; Case two, the screen end 1 does not display the task code execution process at regular intervals, which may be that the virus hidden in the data affects the normal operation of the inspection end, so that the task code is delayed or stopped, at this time, it is determined that the device B may be maliciously invaded, and the inspection end has been attacked by risky data;

[0070] When the image collector 2 monitors that the screen end 1 picture is normal, that is, the screen end 1 displays "data detection result is not found risk", and the task code execution process is normally displayed, which can preliminarily determine that the device B may exist normal damage, then the fusion relay station restores the connection with the inspection end, and continues to transfer the data collected by the device B to the inspection end, and detects again through the inspection end, when the screen end 1 picture is normal for continuous multiple times (such as 5 times), it can be determined that the device B is not maliciously invaded, and there may be normal damage.

[0071] The step S4 is adopted instead of the subsequent operation I in the first embodiment, without affecting the operation of the fusion relay station, the depth analysis of the device B fault condition is realized, and the device B fault type is further obtained.

[0072] Operation II, when the link warning is carried out, the following step S5 is carried out:

[0073] S51, keep the allocation of the data link in step S32, the data collected by the device A2 is transmitted through another data link to the fusion relay station while being transmitted through the link N, and the fusion relay station transfers the data transmitted through the link N to the verification end on the basis of normally performing data fusion uploading;

[0074] S52, sequentially perform steps S42 and S43, similarly, the specific judgment mode is similar to that in operation 1: when the task code execution process is normally displayed on the screen end 1, it is determined that the link N is possibly attacked maliciously, and the verification end is not attacked by the risk data; when the task code execution process is not displayed on the screen end 1, it is determined that the link N is possibly attacked maliciously, and the verification end has been attacked by the risk data; when the screen end 1 is normal, it is determined that the link N network is unstable and causes abnormal data transmission.

[0075] The step S5 is adopted instead of the subsequent operation in the first embodiment, so that the depth analysis of the link N fault condition is realized without affecting the operation of the fusion relay station, and the link N fault type is further obtained.

[0076] The third embodiment:

[0077] In the embodiment, the wireless data transmission between the fusion relay station and the verification end in the second embodiment is replaced by wired data transmission, and the remaining contents remain the same as those in the second embodiment, and the specific implementation is as follows: please refer to Figure 8 and Figure 9 The side end of the fusion relay station is provided with an interface 6, the inside of the interface 6 is provided with an inner connector 4, the inside of the image collector 2 is fixedly connected with a driver 5, the driver 5 can adopt an electric push rod, the output end of the driver 5 is movably extended into the interface 6 and fixedly connected with one end of the inner connector 4, the verification end is connected with an outer connector 3 through a transmission cable, the outer connector 3 is inserted into the interface 6 and contacts with the inner connector 4, and the fusion relay station further comprises a wired disconnection module (not shown in the figure), which is connected with the driver 5.

[0078] Through the above structure, when the operation of disconnecting the network connection with the verification end in step S41 is needed, the driver 5 is started through the wired disconnection module, the driver 5 drives the inner connector 4 to move away from the outer connector 3 and cannot contact and conduct electricity, and the disconnection between the fusion relay station and the verification end is completed, when the connection between the two is needed to be restored, the driver 5 is started again to make the inner connector 4 contact and conduct electricity with the outer connector 3 again, and the connection between the fusion relay station and the verification end is restored;

[0079] In the second embodiment, when the test terminal is attacked by the risk data, there is a risk that the virus program controls the network adapter to automatically reconnect to the original network, resulting in the possibility that the fusion relay station is affected by the virus of the test terminal. Based on the above risk problem, the wired connection and disconnection between the fusion relay station and the test terminal are realized, the wired data transmission between them is realized, compared with the wireless data transmission in the second embodiment, the security of the fusion relay station and the control center is further improved.

[0080] In combination with the current actual demand, the above-mentioned embodiments adopted by the application are not limited to this, various changes made within the knowledge range of those skilled in the art without departing from the concept of the application still fall within the protection scope of the application.

Claims

1. A complementary system for feature analysis of multi-modal data, characterized by: The application relates to a control center connected with a plurality of fusion relay stations, the fusion relay stations are connected with a plurality of edge perception devices distributed in the same scene, the plurality of edge perception devices are respectively used for collecting different types of data and transmitting the data to the fusion relay stations through different data links, the fusion relay station comprises a data receiving module, a feature analysis module, an uploading module, a data filtering module, a link switching module, a judgment processing module and a multi-source early warning module, the data receiving module is used for receiving the data of the plurality of edge perception devices, the uploading module is used for transmitting the feature information of the multi-modal data to the control center, the link switching module is used for replacing the data link of the edge perception device for data transmission, the data filtering module is used for filtering the data with inconsistent feature information expression, and the judgment processing module is used for judging the faults of the plurality of edge perception devices and the plurality of data links when the feature information expression of the multi-modal data is inconsistent. The use method of the multi-modal data feature analysis complementary system comprises the following steps: S1, the plurality of edge perception devices transmit the collected data to the fusion relay station through different data links, so that the fusion relay station receives different modal data in the same scene, and the fusion relay station extracts and analyzes the features of the plurality of different modal data; S2, when the feature information expression of the different modal data is consistent, the fusion relay station fuses the feature information of all the modal data and transmits the feature information to the control center; S3, when the feature information expression of a certain modal data is inconsistent with the feature information expression of other modal data, the feature information with inconsistent expression is filtered out first, the feature information with consistent expression is fused, and the fused feature information is transmitted to the control center, and the following operations are performed: S31, the plurality of modal data with consistent feature expression are all regarded as normal data, the corresponding edge perception devices are respectively regarded as device A1, device A2 and device An, the modal data with inconsistent feature expression is regarded as abnormal data, the edge perception device corresponding to the abnormal data is regarded as device B, and the data link corresponding to the abnormal data is link N; S32, the data links of all the edge perception devices are replaced with each other, so that the data link of the device B after replacement is the initial data link of the device A1, and the initial link N of the device B is used as the data link of the device A2 after replacement; S33, the fusion relay station continues to analyze the features of the received plurality of different modal data; When the data feature expression of the device B is still inconsistent with the feature expression of other modal data, but the data feature expression transmitted through the link N is consistent with the feature expression of other modal data, it is judged that the device B is faulty, and device early warning is performed; When the data feature expression of the device B is consistent with the feature expression of other modal data, but the data feature expression transmitted through the link N is inconsistent with the feature expression of other modal data, it is judged that the link N is faulty, and link early warning is performed.

2. The feature analysis complementary system of multi-modal data according to claim 1, characterized in that: The fusion relay station is connected with an inspection end, one end of the inspection end towards the fusion relay station is provided with a screen end (1), the inspection end comprises a data detection module, a safety feedback module, a task execution module and a display module, the task execution module is connected with a code library, and the display module is connected with the screen end (1).

3. The feature analysis complementary system of multi-modal data of claim 2, wherein: The fusion relay station is fixedly connected with an image collector (2) at one end towards the inspection end, the image collector (2) visually monitors the screen end (1), and the fusion relay station further comprises an interface monitoring module connected with the image collector (2).

4. The feature analysis complementary system of multi-modal data according to claim 3, characterized in that: When the device warning is performed, the following step S4 is performed: S41, the fusion relay station continues to receive the data collected by the device B and transfers the data to the inspection end, and then disconnects the network connection with the inspection end; S42, the data detection module in the inspection end performs safety detection on the data, and the detection result is displayed on the screen end (1), at the same time, the task execution module executes the task code in the code library in a time manner, and the task code execution process is displayed on the screen end (1); S43, the image collector (2) on the fusion relay station visually monitors the screen end (1), and the interface monitoring module determines whether the data collected by the device B has a safety risk and whether the inspection end is attacked by the risk data according to the monitoring data.

5. The feature analysis complementary system of multi-modal data of claim 4, wherein: When the link warning is performed, the following step S5 is performed: S51, the data link allocation in step S32 is maintained, the data collected by the device A2 is transmitted through another data link to the fusion relay station while being transmitted through the link N, and the fusion relay station transfers the data transmitted through the link N to the inspection end; S52, steps S42 and S43 are sequentially performed.

6. The feature analysis complementary system of multi-modal data of claim 2, wherein: The side end of the fusion relay station is provided with an interface (6), the inside of the interface (6) is provided with an inner connector (4), the inside of the image collector (2) is fixedly connected with a driver (5), the output end of the driver (5) movably extends into the interface (6) and is fixedly connected with one end of the inner connector (4), and the inspection end is connected with an outer connector (3) through a transmission cable, the outer connector (3) is inserted into the interface (6) and contacts the inner connector (4).

7. The feature analysis complementary system of multi-modal data of claim 6, wherein: The fusion relay station further comprises a wired disconnection module connected with the driver (5).

Images