Industrial field multimodal embodied data collection and automatic annotation optimization system and method

Abstract

The application discloses an industrial field multi-modal embodied data acquisition and automatic annotation optimization system and method, comprising: acquiring embodied action information and matching with industrial operation semantics; collecting multi-modal data when the trigger condition is met; automatic annotation and correction based on different modal data; consistency verification through industrial physical constraint rules; and adjusting the collection strategy through the end-cloud collaborative mechanism. The application improves the correlation between data acquisition and actual operation, improves the accuracy of automatic annotation, and reduces the burden of the end-side device.

Description

Technical Field

[0001] This invention relates to the field of industrial field data acquisition and intelligent processing technology, specifically to a multimodal embodied data acquisition and automatic annotation optimization system and method based on industrial operation semantics. Background Technology

[0002] Current industrial field data acquisition methods mostly employ timed or continuous acquisition, resulting in high data redundancy, insufficient data during critical operational phases, and low accuracy of automatic annotation. Furthermore, the lack of constraints between multimodal data makes annotation results prone to inconsistencies with actual industrial physical laws.

[0003] Therefore, there is a need for a data acquisition and automatic labeling optimization method that takes industrial operation behavior as the core trigger. Summary of the Invention

[0004] I. Technical Solution This invention provides a method for multimodal embodied data acquisition and automatic annotation optimization in industrial settings, comprising: S1 Get Action Information Acquire embodied action information in the industrial field and match the action information with preset industrial operation semantics to determine the current operation stage.

[0005] S2 semantic-driven triggering acquisition When the embodied action is detected to meet the triggering conditions corresponding to the current operation stage, a data acquisition command is generated.

[0006] S3 Multimodal Data Acquisition When the triggering conditions are met, at least two types of sensor data are collected to form a multimodal dataset.

[0007] S4 Automatic Annotation Processing Automatic annotation of the multimodal data includes: Initial annotation results are generated based on the first modality data; The initial annotation results are corrected by introducing second modality data; Consistency verification of annotation results is performed based on physical constraint rules in industrial operations.

[0008] S5 End-to-End Cloud Collaboration Optimization The annotation results are sent to the cloud for analysis to generate data collection strategy parameters, which are then sent to the edge devices to adjust subsequent data collection behavior.

[0009] II. System Solution The present invention also provides a system comprising: Motion sensing module; Data trigger control module; Multimodal data acquisition module; Automatic annotation processing module; End-to-cloud collaborative processing module.

[0010] The modules work together to implement the above methods and steps. Beneficial effects

[0011] Compared with the prior art, the present invention has the following advantages: By using semantic triggering mechanisms, the relevance between data collection and actual operations can be improved; Improve the accuracy of automatic annotation through a multimodal correction mechanism; Physical constraint verification enhances the consistency of annotation results; By optimizing the edge-cloud collaboration, the processing pressure on the edge is reduced. Attached Figure Description To more clearly illustrate the technical solution of the present invention, the present invention will be briefly described below with reference to the accompanying drawings. Figure 1 A schematic diagram of the structure of a multimodal embodied data acquisition and automatic annotation optimization system for industrial sites; Figure 2 A schematic diagram of a method for multimodal embodied data acquisition and automatic annotation in industrial settings; Figure 3 This is a schematic diagram of the multimodal annotation correction relationship.

Claims

1. A method for multimodal embodied data acquisition and automatic annotation optimization in industrial settings, characterized in that, include: Acquire embodied action information in the industrial field and match it with industrial operation semantics; Data acquisition is initiated when the triggering condition corresponding to the industrial operation semantics is met. Collect data from at least two types of sensors to form multimodal data; An initial annotation result is generated based on the first modality, and the initial annotation result is corrected based on the second modality; Consistency verification of annotation results is performed based on industrial physical constraint rules; Adjust subsequent data collection strategies through an edge-cloud collaboration mechanism.

2. The method according to claim 1, wherein the triggering condition includes a change in the action state or a change in the characteristics of the sensor signal.

3. The method according to claim 1, wherein the industrial physical constraint rules include process sequence constraints or force value threshold constraints.

4. An industrial field multimodal embodied data acquisition and automatic annotation optimization system, comprising: Motion sensing module; Data trigger control module; Multimodal data acquisition module; Automatic annotation processing module; End-to-end cloud collaborative processing module; Used to implement the method of claim 1.

5. A computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method of any one of claims 1 to 3.

Images