Intelligent visual-based building block construction interaction system and method

Abstract

The application belongs to the technical field of computer vision, and discloses a visual-based intelligent building block construction interaction system and method, which comprises the following steps: aligning and preprocessing a top image sequence and a side view image sequence to form a construction observation sequence; structurally analyzing a target drawing and a target component list to generate a target component set, a target connection set and a target stage set; obtaining candidate components, candidate connection relationships and component representation information corresponding to the candidate components according to the construction observation sequence, and performing cross-time confirmation to form a stage state sequence; and extracting construction behavior events according to target structure information, and generating a comprehensive evaluation result in combination with structure corresponding results. The application improves the detailed degree and determination reliability of the building block construction process evaluation.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a vision-based intelligent block construction interaction system and method. Background Technology

[0002] Block-building tasks, which can simultaneously reflect a subject's spatial perception, execution sequence, motor coordination, error correction, and continuous completion ability, have been widely used in autism rehabilitation training, cognitive ability assessment in the elderly, and related interactive teaching scenarios. Current technologies largely rely on manual observation and recording of the block-building process, or only statically compare the final assembly after completion. While these methods can, to some extent, determine whether the task is completed, the speed of completion, and whether the final structure is generally correct, their focus is primarily on subjective impressions or the final result itself. They lack a stable and unified basis for processing continuous state changes such as the gradual addition of components, continuous changes in connection relationships, and dynamic adjustments in hierarchical positions during construction. Existing technologies typically struggle to establish an accurate correspondence between the target structural information corresponding to the target drawings and target component list and the actual state at each moment during the construction process, and also find it difficult to uniformly identify and continuously compare construction behaviors at different stages.

[0003] Furthermore, during the construction process of building blocks, the placement of new components, the formation or disappearance of connections, the adjustment of component positions, and changes in the sequence of stages all directly affect the judgment of construction capability. Existing technologies, lacking a process-oriented continuous state confirmation and temporal differential analysis mechanism, often fail to finely distinguish key behavioral changes during construction. Consequently, it is difficult to further identify which changes constitute effective assembly, which are temporary erroneous trials, which represent dismantling and correction, and which states should be categorized as stagnation or sequence switching. As a result, a large amount of effective behavioral information during the construction process cannot be reliably extracted and transformed into quantifiable evaluation criteria. Ultimately, the results regarding structural restoration, construction efficiency, construction strategies, and error correction capabilities remain at a relatively rough level, failing to meet the technical requirements of objectivity, consistency, and traceability in rehabilitation training, cognitive assessment, and intelligent interaction scenarios.

[0004] Therefore, how to accurately identify, confirm the timing and perform differential analysis of the continuous states in the process of building blocks, and combine them with the target structural relationships to form an objective and unified process evaluation result, has become a technical problem that urgently needs to be solved in this field.

[0005] In view of this, this application proposes a vision-based intelligent block construction interaction system and method to solve the above problems. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application aims to provide a vision-based intelligent block construction interaction system and method. By acquiring top image sequences and side view image sequences to form a construction observation sequence, the system performs structured analysis on target drawings and target component lists. Combined with stage state recognition, cross-time confirmation, event extraction, and result alignment evaluation processing, the system achieves objective identification and quantitative analysis of continuous states and key behavioral events in the block construction process, thereby improving the objectivity, consistency, and quantifiability of block construction interaction evaluation.

[0007] To achieve the above objectives, this application provides the following technical solution:

[0008] Align and preprocess the pre-acquired top and side view image sequences to form a constructed observation sequence;

[0009] The pre-acquired target drawings and target component list are structured and parsed to generate a set of target components, a set of target connections, and a set of target stages.

[0010] The constructed observation sequence is input into the pre-constructed stage state identification model to obtain candidate components, candidate connection relationships, and component representation information corresponding to the candidate components;

[0011] Based on the component characterization information and candidate connection relationships, candidate components are confirmed across time periods to form a stage state sequence;

[0012] Based on the target component set, target matching is performed on stable components in the stage state sequence to obtain target matching components; based on the target connection set, the connection relationship between target matching components is matched and confirmed to obtain matching connection relationship; based on the target stage set, adjacent stage states are compared, and based on the comparison result and matching connection relationship, construction behavior events are extracted.

[0013] The stage states of the last M consecutive observation times in the stage state sequence are merged to obtain the final construction state; the final construction state, the target component set, and the target connection set are aligned to obtain the structure correspondence result;

[0014] Based on the structural correspondence results and construction behavior events, generate structural fidelity scores, construction efficiency scores, construction strategy scores, error correction scores, and comprehensive evaluation results.

[0015] Furthermore, the methods for obtaining the target stage set include:

[0016] The target components that are in direct contact with the workbench are classified into the first target stage, and the first target stage is defined as the target stage that has been classified into the current target stage.

[0017] Among the target components that have never been assigned to the target stage, select target components that have a support connection with at least one target component that has been assigned to the target stage, and whose suspended connection ends will not appear when only the target components that have been assigned to the target stage are retained.

[0018] The selected target components are assigned to the next target stage, and the currently assigned target components are updated to all target components that have been assigned to each target stage up to the present.

[0019] Repeat the above filtering and categorization process until all target components have been categorized, resulting in the target stage set.

[0020] Furthermore, methods for constructing observation sequences include:

[0021] The acquisition time of each frame in the top image sequence is recorded as the reference time;

[0022] Find the side view image with the smallest time difference from each reference time in the side view image sequence, and whose time difference does not exceed the preset alignment time difference, and use it as the side view image for the corresponding reference time.

[0023] According to the preset observation interval, extract the image frames corresponding to each observation time from the aligned top image and side view image as observation frames;

[0024] Each observation frame is processed by performing workbench area cropping, perspective correction, and brightness normalization to form a constructed observation sequence.

[0025] Furthermore, methods for forming a phase state sequence include:

[0026] In adjacent observation times, the center distance and bounding box overlap ratio are calculated for candidate components with the same component category. When the center distance is not greater than the cross-time matching threshold and the bounding box overlap ratio is not lower than the overlap matching threshold, they are determined to be the same candidate component.

[0027] The consistency of the component representation information and candidate connection relationship of the same candidate component in several consecutive preset stable frames is checked. When the check is passed, it is determined to be a stable component, and the candidate connection relationship that has passed the check is determined as the connection relationship at the corresponding observation time.

[0028] Based on the stable components at each observation time and the connection relationship of the corresponding observation time, the stage state at each observation time is formed. The stage states at each observation time are arranged in chronological order to form a stage state sequence.

[0029] Furthermore, the construction behavior events include new splicing events, error test play events, demolition and correction events, stagnation events, and sequence switching events;

[0030] Methods for extracting constructive behavioral events include:

[0031] Compare the states of adjacent stages to obtain newly added and disappeared stable components;

[0032] Based on the matching status of the newly added stable components, extract the new splicing event and the error test release event;

[0033] Extract the removal correction event based on the correction situation after the disappearance of the stabilizing component;

[0034] Extract stagnant events based on the changes in events across multiple consecutive stages of the state;

[0035] Extract the sequence switching event based on the target stage number of the corresponding target component in the newly added splicing event.

[0036] Furthermore, based on the matching status of the newly added stable components, the newly added splicing events and error test release events are extracted as follows:

[0037] When a new stable component corresponds to a target matching component and forms a corresponding target connection relationship in the target connection set, the stage state change caused by the appearance of the new stable component will be determined as a new splicing event.

[0038] When a newly added stable component does not correspond to a target matching component, or does not form a corresponding target connection relationship, and disappears within the error duration threshold, the stage state change caused by the appearance of the newly added stable component will be identified as an error test release event.

[0039] Furthermore, methods for obtaining matching connection relationships include:

[0040] Obtain the hierarchical position and component boundary of any two target matching components;

[0041] Calculate the contact overlap ratio of the component boundaries of two target matching components in the connection direction corresponding to the target connection relationship in the target connection set;

[0042] The hierarchical relationship between the two target matching components is determined based on their hierarchical positions. Then, the angle difference between the direction of the line connecting the centers of the two target matching components and the direction of the connection relationship of the corresponding target is calculated.

[0043] When the hierarchical relationship is consistent with the hierarchical relationship of the corresponding connection relationship in the target connection set, the contact overlap ratio is not lower than the contact overlap threshold, and the angle difference is not greater than the direction deviation threshold, the matching connection relationship of the two target matching components is determined.

[0044] Furthermore, methods for obtaining the corresponding structural results include:

[0045] The final constructed state is obtained by merging the stage states of the last M consecutive observation times in the stage state sequence.

[0046] The final constructed state is matched with the target component set and the target connection set to obtain the structural correspondence result. The number of correctly matched components, the number of correctly matched connections, and the number of correctly matched hierarchical positions are counted.

[0047] Furthermore, methods for generating construction strategy scores include:

[0048] Retrieve the target stage number corresponding to the newly added splicing event;

[0049] When the target stage number corresponding to the next newly added splicing event is less than the target stage number corresponding to the previous newly added splicing event, it is recorded as a cross-stage rollback.

[0050] When the target stage number corresponding to a newly added splicing event is greater than the minimum target stage number of the target component to which the unmatched component belongs, it is recorded as a sequence switching event.

[0051] The number of event pairs in adjacent newly added splicing event pairs where the number of the next target stage is greater than or equal to the number of the previous target stage is counted, and the continuity of stage advancement is obtained by dividing the number of event pairs by the number of adjacent newly added splicing event pairs.

[0052] The construction strategy score is generated based on the continuity of phase advancement, the number of sequential switching events, and the number of cross-phase rollbacks.

[0053] Furthermore, methods for generating structural fidelity scores include:

[0054] The component matching ratio is determined by the ratio of the number of correctly matched components to the total number of target components; the connection matching ratio is determined by the ratio of the number of correctly matched connections to the total number of target connections; and the hierarchical matching ratio is determined by the ratio of the number of correctly matched hierarchical positions to the total number of target components.

[0055] The structural fidelity score is generated based on the component matching ratio, connection matching ratio, and hierarchy matching ratio.

[0056] This application provides a vision-based intelligent block construction interaction system, and the implementation of the vision-based intelligent block construction interaction method includes:

[0057] The observation construction module is used to align and preprocess the pre-acquired top image sequence and side view image sequence to form a constructed observation sequence;

[0058] The target parsing module is used to perform structured parsing of the pre-acquired target drawings and target component list to generate a target component set, a target connection set, and a target stage set.

[0059] The state recognition module is used to input the constructed observation sequence into the pre-constructed stage state recognition model to obtain candidate components, candidate connection relationships, and component representation information corresponding to the candidate components;

[0060] The timing confirmation module is used to confirm candidate components across time based on component characterization information and candidate connection relationships, forming a stage state sequence;

[0061] The event extraction module is used to perform target matching on stable components in the stage state sequence based on the target component set to obtain target matched components; to match and confirm the connection relationship between target matched components based on the target connection set to obtain matching connection relationship; and to compare adjacent stage states based on the target stage set, and extract construction behavior events based on the comparison result and matching connection relationship.

[0062] The result alignment module is used to merge the stage states of the last M consecutive observation times in the stage state sequence to obtain the final construction state; and to align the final construction state, the target component set, and the target connection set to obtain the structure correspondence result.

[0063] The evaluation generation module is used to generate structural fidelity scores, construction efficiency scores, construction strategy scores, error correction scores, and comprehensive evaluation results based on the structural correspondence results and construction behavior events.

[0064] The technical effects and advantages of the vision-based intelligent block construction interaction system and method proposed in this application are as follows:

[0065] First, this application aligns and preprocesses the top and side image sequences, and combines them with structured analysis of the target drawings and target component lists to form construction observation sequences, target component sets, target connection sets, and target stage sets, respectively. This organizes the scattered observation information and target structure information in existing technologies into unified input results required for subsequent processing, reducing subsequent judgment biases caused by inconsistent acquisition times and unorganized target structure information, and improving the input consistency for subsequent state recognition, event extraction, and result matching.

[0066] Second, this application obtains candidate components, candidate connection relationships, and component representation information corresponding to the candidate components based on the construction observation sequence, and further performs cross-time confirmation to form a stage state sequence. This can transform the discrete identification results at each observation time during the block construction process into stage state results with temporal continuity. Compared with the method of judging only based on single-time images or final results, this application can more accurately reflect the continuous evolution process of component states and connection relationships, thus providing a reliable basis for subsequent extraction of construction behavior events.

[0067] Third, this application performs target matching on stable components in the stage state sequence based on the target component set, confirms the connection relationships between target-matched components based on the target connection set, and compares the states of adjacent stages based on the target stage set. Based on the comparison results and matching connection relationships, it extracts construction behavior events, including new splicing events, error trial placement events, dismantling and correction events, stagnation events, and sequence switching events, from the construction process. This not only identifies different types of process behaviors that are difficult to distinguish in existing technologies, but also avoids mixing up state changes with different evaluation significance, allowing for a more complete expression of effective behavioral information during the construction process, and improving the detail and reliability of process evaluation.

[0068] Fourth, this application merges the last consecutive stages in the stage state sequence to obtain the final construction state, and then performs structural correspondence matching between the final construction state and the target component set and target connection set to obtain the structural correspondence result. Based on the structural correspondence result and construction behavior events, it generates a structural restoration score, a construction efficiency score, a construction strategy score, an error correction score, and a comprehensive evaluation result. This allows both the final structural result and the process behavior result to be included in the evaluation criteria, enabling the output to reflect both the final construction completion status and the execution characteristics and error correction capabilities during the construction process. This overcomes the limitation of existing technologies that can only make rough judgments, and is more conducive to forming objective, unified, and repeatable evaluation results in rehabilitation training, cognitive assessment, and intelligent interaction scenarios. Attached Figure Description

[0069] Figure 1 This is a flowchart of a vision-based intelligent block construction and interaction method according to Embodiment 1 of this application;

[0070] Figure 2 This is a flowchart of the construction behavior event acquisition process in Embodiment 1 of this application;

[0071] Figure 3 This is a block diagram of a vision-based intelligent block building interaction system according to Embodiment 2 of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] Example 1:

[0074] See Figure 1As shown, this embodiment provides a vision-based intelligent block construction interaction method, including: first, performing structured analysis on the target block task; then, continuously acquiring images of the entire block construction process and recognizing stage states; subsequently, extracting construction behavior events based on changes in adjacent stage states; and combining the structural correspondence between the final construction result and the target block task to generate a structure restoration score, a construction efficiency score, a construction strategy score, and an error correction score; and finally, generating a comprehensive evaluation result.

[0075] In this embodiment, the target building block task includes at least target drawings and a target component list; the target drawings provide the target planar position, target hierarchical position, and target connection relationship of each target component; the target component list provides the component category corresponding to each target component; the construction process data includes at least a top image sequence and a side view image sequence; the stage status includes at least a set of stable components, the planar position, hierarchical position, and connection relationship of each stable component; the component representation information corresponding to the candidate component includes at least the component category, planar position, and hierarchical position; the construction behavior events include at least new splicing events, error trial placement events, dismantling and correction events, stagnation events, and sequence switching events. This embodiment also pre-establishes a standardized rule building block library, which records the component category identifier, standard planar outline dimensions, standard layer height dimensions, main direction identifier, and outer outline rectangular boundary of various building block components; the component category refers to the building block type identifier pre-registered in the standardized rule building block library.

[0076] The method for establishing a standardized rule block library includes: pre-registering all types of block components allowed for the same target block task; collecting the top view and side view height of each type of block component in a standard placement posture; recording the standard planar outline dimensions, standard layer height dimensions, and outer outline rectangular boundaries of the block component; determining the main direction identifier based on the direction of the longest side in the standard planar outline; determining the main direction identifier for block-shaped components according to the preset assembly orientation; and configuring a unique component category identifier for each type of block component. During the target drawing parsing, candidate component identification, hierarchical position identification, and connection relationship matching processes, the standardized rule block data corresponding to the same component category identifier is called.

[0077] Images of the construction process are continuously acquired by a top camera mounted above the workbench and a side camera mounted on the side of the workbench, forming a top image sequence and a side image sequence, respectively. The acquisition time of each frame in the top image sequence is recorded as the reference time. Then, the side image in the side image sequence with the smallest time difference from each reference time and not exceeding the preset alignment time difference is selected as the side image for the corresponding reference time, thus completing the unified sampling time alignment of the two images. The sampling period of the top image is determined by the acquisition frequency of the top camera, and the preset alignment time difference is half of the sampling period of the top image. Subsequently, according to the preset observation interval, the image frames corresponding to each observation time are extracted from the top image and side image after the unified sampling time alignment is completed, forming observation frames. Workbench area cropping, perspective correction, and brightness normalization are performed on each observation frame to form a construction observation sequence.

[0078] The method for setting the preset observation interval includes: statistically analyzing the time interval between two consecutive valid splicing actions in historical completed samples to form an action interval sample set; valid splicing actions refer to splicing actions in historical samples that have been manually marked as correctly placed components and form target connection relationships in subsequent stages; sorting the action interval sample set from smallest to largest and reading the corresponding value of the 10th percentile and the median value; taking the minimum value between the corresponding value of the 10th percentile and the median value as the preset observation interval to ensure that the observation frequency can cover most valid splicing actions without excessively increasing the number of redundant frames.

[0079] The method for implementing workbench area cropping, perspective correction, and brightness normalization includes: setting positioning markers at the four corners of the workbench and identifying the positions of the four positioning markers in the observation frame; performing perspective transformation based on the correspondence between the four positioning markers and a preset standard rectangular area to obtain an image consistent with the standard workbench coordinate system; then statistically analyzing the average brightness of the perspective-corrected image and adjusting this average brightness to the historical standard average brightness to complete brightness normalization. The historical standard average brightness is obtained by statistically analyzing the average brightness of workbench observation frames under historical standard acquisition conditions. The workbench coordinate system is a two-dimensional coordinate system established with the upper left corner of the workbench plane as the origin, extending horizontally along the width of the workbench and vertically along the length of the workbench. The preset standard rectangular area is generated from the standard boundary dimensions of the workbench and serves as the target mapping area after perspective correction.

[0080] After obtaining the target drawings and target component list, the target building block task is subjected to structured parsing to form a target structure description; the target structure description includes the target component set, the target connection set, and the target stage set.

[0081] Specifically, a unique target component number is assigned to each target component in the target drawing, and the component category, target planar position, target hierarchical position, and target connection relationship of the target component are recorded to form a target component set and a target connection set. The target planar position is represented by the center coordinates in the workbench coordinate system; the target hierarchical position is represented by a bottom-up hierarchical number; the target connection relationship includes at least the numbers of the two connected target components, the connection direction code, and the hierarchical relationship after the connection; the connection direction code is used to characterize the relative connection direction between the two target components, and the hierarchical relationship after the connection is used to characterize the upper and lower hierarchical correspondence of the two target components after the connection.

[0082] The target stage set is obtained based on the target component set and the target connection set. Specifically, target components that directly contact the workbench are assigned to the first target stage, and the first target stage is identified as the currently assigned target stage; the stage sequence of the first target stage is denoted as target stage number one. From the target components not yet assigned to a target stage, target components that have a supporting connection to at least one target component currently assigned to a target stage are selected, and those that do not have suspended connection ends when only the target components currently assigned to a target stage are retained; the selected target components are assigned to the next target stage, and assigned corresponding target stage numbers according to the assignment order; then the currently assigned target stages are updated to include all target components currently assigned to each target stage. Subsequently, based on the updated currently assigned target stages, the above selection and assignment process is repeated for the remaining target components not yet assigned to a target stage until all target components are assigned, resulting in the target stage set and the target stage number corresponding to each target component.

[0083] A support connection refers to a relationship of mutual support between two target components, or a lateral snap-fit ​​relationship recorded by the target connection set, and the lateral snap-fit ​​relationship has a corresponding connection direction code and hierarchical relationship; a suspended connection end refers to a connection end that should form a support connection with the preceding target component in the target drawing, but still exists when only the target components that have been included in the stage are retained.

[0084] After obtaining the constructed observation sequence and target structure description, stage state recognition is performed at each observation time. Specifically, the top image and side view image corresponding to each observation time are input into the stage state recognition model to obtain candidate components, candidate connection relationships, and component representation information corresponding to the candidate components; the component representation information includes component category, planar position, and hierarchical position.

[0085] Candidate components refer to the component objects to be confirmed identified by the stage state recognition model at a single observation time. The stage state recognition model adopts a multi-input multi-output neural network model, including a top image processing branch, a side image processing branch, and a relationship determination branch.

[0086] The top image processing branch is used to extract the contour and texture features of candidate parts from the top image, and output the part category, top bounding box and top center position of the candidate parts, where the top center position is used as the planar position of the candidate parts. The side image processing branch is used to extract the height direction features from the region in the side image corresponding to the candidate parts, and output the hierarchical position of the candidate parts.

[0087] The relationship determination branch is used to output candidate connection relationships based on the component category, planar position, hierarchical position and boundary features of the candidate component pair. The boundary features include at least the top bounding box size, the outer contour rectangle boundary size and the main direction identifier. The position features include at least the center coordinates of the candidate component and the relative position difference between the candidate component pairs.

[0088] The output format of candidate connection relationships includes the starting component number, the ending component number, the connection direction code, and the connection establishment mark. The starting component number and the ending component number are temporary identifiers of the corresponding candidate components in the candidate component set at the same observation time. The connection direction code adopts six-directional codes: up, down, left, right, front, and back. The connection establishment mark adopts a binary mark, with a value of 1 indicating that the connection is established and a value of 0 indicating that the connection is not established.

[0089] The method for cross-time confirmation of the same candidate component includes: for each candidate component in the previous observation time, selecting candidate components of the same component category in the subsequent observation time, and calculating the center distance and bounding box overlap ratio between the candidate component and each selected candidate component; when the center distance is not greater than the cross-time matching threshold and the bounding box overlap ratio is not lower than the overlap matching threshold, the corresponding candidate components are determined to be the same candidate component. The bounding box overlap ratio is the ratio of the area of ​​the overlapping region of the bounding boxes of the two candidate components to the area of ​​the smaller bounding box.

[0090] A consistency check is performed on the identification results of the same candidate component across several consecutive preset stable frames. The consistency check includes: determining whether the component category of the candidate component is consistent across these frames; determining whether the hierarchical position of the candidate component is consistent across these frames; calculating the Euclidean distance between the center coordinates of the candidate component's planar position at adjacent observation times; and determining that the planar position passes the consistency check when all Euclidean distances are less than or equal to the position stability threshold. Finally, the consistency of candidate connection relationships is determined. Consistent candidate connection relationships mean that the set of connection object numbers, connection direction codes, and connection establishment markers for the corresponding candidate components remain consistent across adjacent observation times.

[0091] When both the component characterization information and the candidate connection relationship pass the consistency check, the candidate component is determined as a stable component, and the candidate connection relationship that passes the consistency check is determined as the connection relationship at each corresponding observation time; when either fails the consistency check, the candidate component is not determined as a stable component.

[0092] After establishing the stage states at each observation time, target matching is performed on the stable components at the current observation time to obtain target-matched components and unmatched components. Specifically, for each stable component at the current observation time, candidate target components with the same component category and target level position are first searched in the target component set. Then, the Euclidean distance between the planar position of the stable component and the target planar positions of each candidate target component is calculated, and the candidate target component with the smallest distance is selected as the matching target component. When the minimum distance is not greater than the position matching threshold, the stable component is recorded as the target-matched component; otherwise, the stable component is recorded as the unmatched component. To avoid multiple stable components matching the same target component, all matching pairs are sorted in ascending order of minimum distance, and matching pairs that do not have duplicate target component occupation are retained in turn, while the remaining matching pairs that have duplicate target component occupation are deleted.

[0093] After obtaining the target matching components, the matching connection relationship at the current observation time is acquired. Specifically, for any two target matching components, the hierarchical position and component boundary of each component are first acquired; then, the contact overlap ratio of the component boundaries of the two target matching components in the connection direction of the corresponding target connection relationship in the target connection set is calculated; then, the hierarchical relationship between the two components is determined based on their hierarchical positions, and the angle difference between the direction of the center line connecting the two target matching components and the connection direction of the corresponding target connection relationship is calculated; when the hierarchical relationship is consistent with the hierarchical relationship of the corresponding connection relationship in the target connection set, the contact overlap ratio is not lower than the contact overlap threshold, and the angle difference is not greater than the direction deviation threshold, the matching connection relationship of the two target matching components is determined; otherwise, it is not determined to be a matching connection relationship.

[0094] See Figure 2 As shown, after forming the stage state sequence, constructive behavior events are extracted. Specifically, the stage states at two adjacent observation times are compared to obtain newly added stable components and disappeared stable components.

[0095] When a newly added stable component corresponds to a target matching component and forms a corresponding target connection relationship in the target connection set, the stage state change caused by the appearance of the newly added stable component will be determined as a new splicing event.

[0096] When a newly added stable component does not correspond to a target matching component, or does not form a corresponding target connection relationship, and the newly added stable component disappears when the error duration does not exceed the preset error duration threshold, the change in the stage state caused by the appearance of the newly added stable component will be determined as an error test release event; the error duration is the time interval between the moment when the newly added stable component is first determined to be a stable component and the moment when the newly added stable component disappears from the stage state.

[0097] When an existing stabilizing component disappears, and a new stabilizing component appears within a preset correction time, and this new stabilizing component corresponds to a target matching component and forms a corresponding target connection relationship, and the planar position of the new stabilizing component is located within the correction area corresponding to the disappeared stabilizing component, this combined change of the disappearance of the disappeared stabilizing component and the appearance of the new stabilizing component is determined as a removal correction event; the correction area corresponding to the disappeared stabilizing component is a circular area centered on the planar position of the disappeared stabilizing component before its disappearance and with a radius of twice the position matching threshold.

[0098] When no new splicing events or dismantling / correction events occur within multiple observation periods corresponding to the continuous stagnation duration threshold, and there are still target components in the target component set that are not matched with the target component, the time period is determined as a stagnation event.

[0099] When the target stage number of the target component to which a newly added splicing event belongs is greater than the minimum target stage number to which the target component to which the unmatched target component belongs, the newly added splicing event is determined as a sequence switching event.

[0100] The reasoning method of the stage state recognition model includes first inputting the top image into the top image processing branch to obtain the component category, top bounding box and top center position corresponding to the candidate component, then determining the corresponding region based on the projection position of the top bounding box in the side view image, and inputting the corresponding region and the side view image into the side view image processing branch to obtain the hierarchical position of the candidate component.

[0101] Subsequently, candidate component pairs are constructed based on candidate components at the same observation time. The component category, planar position, hierarchical position, and boundary features of the candidate component pairs are input into the relationship determination branch to obtain candidate connection relationships. Finally, the candidate components, candidate connection relationships, and the component category, planar position, and hierarchical position corresponding to the candidate components are merged to form the stage state recognition result at the current observation time.

[0102] The projection position of the top outer frame in the side view image is determined by the preset mapping relationship between the workbench coordinate system and the side view coordinate system. The preset mapping relationship is obtained by calibrating the corresponding positions of the four corner positioning marks of the workbench in the top image and the side view image. The side view coordinate system is a two-dimensional coordinate system established with the upper left corner of the side view image imaging plane as the origin, with the horizontal direction along the width of the workbench projection and the vertical direction along the height direction.

[0103] The training method for the stage state recognition model includes first establishing a training sample set, and then dividing the training sample set into a training subset, a validation subset, and a test subset in an 8:1:1 ratio. Each training sample includes a top image, a side view image, top bounding box annotations of each component in the top image, component category annotations of each component, top center position annotations of each component, hierarchical position annotations of each component, and connection relationship annotations between each component. The connection relationship annotations are recorded in units of candidate component pairs, and include at least the starting component number, the ending component number, the connection direction code, and the connection establishment mark.

[0104] The training sample set is then preprocessed, including workbench area cropping, perspective correction, brightness normalization, image size unification, and generation of corresponding regions in the side view image. Subsequently, the stage state recognition model is trained by simultaneously inputting the top image and the side view image into the stage state recognition model, and outputting the component category, top bounding box, top center position, hierarchical position, and candidate connection relationship corresponding to the candidate component.

[0105] The component category recognition error is calculated based on the component category labeling. The position regression error is calculated based on the top bounding box labeling and the top center position labeling. The hierarchical recognition error is calculated based on the hierarchical position labeling. The connection direction recognition error and connection establishment recognition error are calculated based on the connection relationship labeling. The model parameters are then updated based on the weighted results of the component category recognition error, position regression error, hierarchical recognition error, connection direction recognition error, and connection establishment recognition error. The position regression error corresponds to the prediction error of the top bounding box and the top center position, the hierarchical recognition error corresponds to the hierarchical position prediction error, the connection direction recognition error corresponds to the connection direction encoding prediction error, and the connection establishment recognition error corresponds to the connection establishment marker prediction error. The training weights corresponding to each error term are determined based on the numerical scale of each error term in the validation subset and the importance of the recognition task. First, each error term is normalized. Then, the initial training weights are allocated according to the order of influence of component category, position, hierarchical position, connection direction, and connection establishment marker on the stage state recognition result. The initial training weights are then normalized so that the sum of all training weights is one.

[0106] When the decrease in validation set loss is less than a preset convergence threshold for 10 consecutive rounds, the current model training is stopped. The preset convergence threshold is determined based on the historical fluctuation of validation set loss in consecutive training rounds, and is selected as the loss change boundary that can distinguish between normal decrease and stagnation. In some implementations, this loss change boundary can be set as the low percentile value of the historical fluctuation of validation set loss. The validation set loss is the error statistics of the stage state recognition model on the validation subset. After training is completed, a stage state recognition model for block construction stage state recognition is obtained.

[0107] The method for setting the position stability threshold, cross-time matching threshold, and overlap matching threshold includes reading the position fluctuation value of the same component under continuous observation time, the center distance sample across adjacent observation time, and the bounding box overlap ratio sample from the historical correctly stitched samples. The position fluctuation value is the Euclidean distance between the center coordinates of the same component in the plane at adjacent observation time. The position fluctuation value samples are sorted from smallest to largest and the corresponding value at the 95th percentile is taken as the position stability threshold. The center distance sample across adjacent observation time is sorted from smallest to largest and the corresponding value at the 95th percentile is taken as the cross-time matching threshold. The bounding box overlap ratio sample is sorted from smallest to largest and the corresponding value at the 10th percentile is taken as the overlap matching threshold. Then, the number of consecutive observation frames corresponding to the same component from its first appearance to its stable state in the historical correctly stitched samples is counted, and the number of consecutive observation frames with the highest frequency of occurrence is taken as the preset stable frame number.

[0108] The method for setting the position matching threshold includes: reading the planar distance between the correctly spliced ​​part and the corresponding target part in the historical correctly spliced ​​samples to form a correct matching distance sample set; sorting the correct matching distance sample set in ascending order, and taking the value corresponding to the 95th percentile as the position matching threshold.

[0109] The standard planar contour dimensions recorded in the standardized rule block library are used to determine the boundary range of the component in the top image; the standard layer height dimensions are used to determine the layer range of the component in the side view image; the main direction identifier is used to determine the connection direction encoding; the outer contour rectangle boundary is used to calculate the contact overlap ratio; the component boundary is represented by the outer contour rectangle boundary of each component in the standardized rule block library; when the component is a long strip, the long side direction is taken as the main direction boundary; when the component is a square component, the four sides of equal length rectangle boundary is taken; when the component is a corner component, the minimum circumscribed rectangle boundary is taken.

[0110] The methods for setting the contact overlap threshold and the direction deviation threshold include: reading the contact overlap ratio and the angle difference of the correctly connected component pairs in the historical correctly connected samples to form an overlap sample set and an angle sample set respectively; sorting the overlap sample set in ascending order and taking the value corresponding to the 10th percentile as the contact overlap threshold; sorting the angle sample set in ascending order and taking the value corresponding to the 90th percentile as the direction deviation threshold.

[0111] The method for setting the error duration threshold, preset correction duration, and stagnation duration threshold includes: reading error test-release events, dismantling correction events, and stagnation events from historical manually labeled samples, and respectively calculating the duration of error test-release events, the time interval from the disappearance of the faulty component to the appearance of the correct component in dismantling correction events, and the duration of stagnation events to form three types of duration sample sets; then sorting the error duration sample set in ascending order and taking the 90th percentile value as the error duration threshold; sorting the dismantling correction time interval sample set in ascending order and taking the 90th percentile value as the preset correction duration; and sorting the stagnation duration sample set in ascending order and taking the 80th percentile value as the stagnation duration threshold.

[0112] After obtaining all construction events, the stage states of the last M consecutive observation times are merged to obtain the final construction state; M is a positive integer; stable components that exist in the last M consecutive observation times are retained, and the median of the horizontal and vertical coordinates of their planar positions are taken, the mode of their hierarchical positions is taken, and the mode of their connection relationships is taken; when there are multiple modes of hierarchical positions, the hierarchical position that appears more frequently in the last two observation times is taken as the final hierarchical position; when there are multiple modes of connection relationships, the connection relationship that matches the target connection relationship more times is taken as the final connection relationship; then, the final construction state is matched with the target component set and the target connection set to obtain the structure correspondence result; the value of M is determined based on the number of consecutive observation frames in the final structure preservation stage in the historical completed samples, specifically, the number of consecutive observation frames corresponding to the final correct structure remaining unchanged in the historical completed samples is counted to form the final preservation frame sample set, and the number of frames that can cover the final stable preservation state is selected from the final preservation frame sample set as M.

[0113] The method for generating the structural fidelity score includes: first, dividing the number of correctly matched components by the total number of target components to obtain the component matching ratio; then, dividing the number of correctly matched connections by the total number of target connection relationships to obtain the connection matching ratio; subsequently, dividing the number of correctly matched hierarchical positions by the total number of target components to obtain the hierarchical matching ratio; determining the total number of structural fidelity evaluation items based on the total number of target components, the total number of target connection relationships, and the sum of the total number of target components, where the first total number of target components corresponds to a component matching evaluation item, the total number of target connection relationships corresponds to a connection matching evaluation item, and the second total number of target components corresponds to a hierarchical matching evaluation item; dividing the total number of target components by the total number of structural fidelity evaluation items to obtain the component matching weight, dividing the total number of target connection relationships by the total number of structural fidelity evaluation items to obtain the connection matching weight, and dividing the total number of target components by the total number of structural fidelity evaluation items to obtain the hierarchical matching weight; finally, multiplying the component matching ratio, connection matching ratio, and hierarchical matching ratio by their respective weights and the full percentage score to obtain the structural fidelity score.

[0114] The method for determining the number of correctly matched connections includes: pairing all correctly matched components together, reading their connection relationships in the final constructed state, and comparing them with the target connection relationships between the corresponding target components; when the connection directions and hierarchical relationships are consistent, the component pair is recorded as a correctly matched connection relationship. The method for determining the number of correctly matched hierarchical positions includes: comparing the final hierarchical position of all correctly matched components one by one with the target hierarchical position of the corresponding target component; when the two are consistent, the component is recorded in the number of correctly matched hierarchical positions.

[0115] The method for generating the construction efficiency score includes: when the total number of new splicing events is not zero, firstly, reading the occurrence time corresponding to each new splicing event in the new splicing event sequence, and determining the time interval between the earliest and latest occurrence times as the total completion time; then dividing the total completion time by the total number of new splicing events to obtain the average splicing time per unit component; subsequently, sorting the total completion times in the historical completion samples from smallest to largest, and using the corresponding values ​​of the 20th percentile, 40th percentile, 60th percentile, and 80th percentile as four dividing values, dividing the total completion time into the first, second, third, fourth, and fifth time intervals from shortest to longest, and assigning them values ​​of 50, 40, and 50 respectively. The time scores are 10, 30, 20, and 10 points respectively. Then, the average splicing time of each unit in the historical completed samples is sorted from smallest to largest, and the corresponding values ​​of the 20th percentile, 40th percentile, 60th percentile, and 80th percentile are used as four dividing values ​​to divide the average splicing time of each unit into the first efficiency interval, the second efficiency interval, the third efficiency interval, the fourth efficiency interval, and the fifth efficiency interval, respectively, and assigned efficiency scores of 50, 40, 30, 20, and 10 points respectively. Finally, the time score corresponding to the total completion time is directly added to the efficiency score corresponding to the average splicing time of each unit to obtain the construction efficiency score. When the total number of new splicing events is zero, the construction efficiency score is recorded as zero.

[0116] The method for generating the construction strategy score includes: first, obtaining the target stage number corresponding to each newly added splicing event; when the target stage number corresponding to the subsequent newly added splicing event is less than the target stage number corresponding to the previous newly added splicing event, the stage number rollback is recorded as a cross-stage rollback; when the target stage number corresponding to a certain newly added splicing event is greater than the minimum target stage number to which the target component to which the unmatched component belongs, the newly added splicing event is recorded as a sequence switching event; then, statistics are calculated for adjacent newly added splicing event pairs where the subsequent target stage number is greater than or equal to... The number of event pairs numbered in the previous target stage is used to calculate the stage progress continuity by dividing the number of event pairs by the number of adjacent newly added splicing event pairs. Then, the stage progress continuity is multiplied by 50 points, the result of subtracting the number of sequence switching events from 1 and dividing it by the total number of newly added splicing events is multiplied by 30 points, and the result of subtracting the number of cross-stage rollbacks from 1 and dividing it by the total number of newly added splicing events is multiplied by 20 points. These three results are then added together to obtain the construction strategy score. When the total number of newly added splicing events is zero or the number of adjacent newly added splicing event pairs is zero, the construction strategy score is recorded as zero.

[0117] The method for generating error correction scores includes: when the total number of new splicing events is not zero and the number of error test events is not zero, first count the total number of all error test events, then count the total number of all dismantling and correction events; then, for each error test event, search backward within a preset correction time period to see if there is a new splicing event that overlaps with the correction area corresponding to the error test event and has the same target component category; the correction area corresponding to the error test event is a circular area centered on the planar position when the new stable component corresponding to the error test event is first determined as a stable component, with a radius of twice the position matching threshold; then, for each error test event, search backward within a preset correction time period to see if there is a new splicing event whose planar position of the corresponding new stable component is located within the correction area corresponding to the error test event, and whose component category is the same as the component category of the new stable component corresponding to the error test event; When an error occurs, the test-run event is recorded as a successful correction event; when it does not occur, it is recorded as an uncorrected event. The successful correction ratio is obtained by dividing the number of successful correction events by the number of error test-run events. Then, the error control component is obtained by multiplying the result of subtracting the number of error test-run events from the total number of new splicing events by 1 and multiplying by 40. The positive correction component is obtained by multiplying the result of dividing the number of dismantling correction events by the number of error test-run events by 20. The successful correction ratio is multiplied by 40 to obtain the successful correction component. Finally, the three components are added together to obtain the error correction score. When the number of dismantling correction events is greater than the number of error test-run events, the number of error test-run events is used as the upper limit for calculating the number of dismantling correction events. When the number of error test-run events is zero, the error correction score is directly recorded as 100 points. When the total number of new splicing events is zero, the error correction score is recorded as zero points.

[0118] After obtaining the structural fidelity score, construction efficiency score, construction strategy score, and error correction score, a comprehensive evaluation result is generated according to the overall weight. Specifically, the structural fidelity score, construction efficiency score, construction strategy score, and error correction score are multiplied by their corresponding comprehensive weights, and the products are then summed to obtain the comprehensive evaluation result. The method for setting the comprehensive weights includes: first, reading the individual human scores for structural fidelity, construction efficiency, construction strategy, and error correction from historical human scoring samples, as well as the corresponding comprehensive human scores; then, calculating the correlation coefficient between each individual human score and the comprehensive human score, and determining the initial weights corresponding to the structural fidelity score, construction efficiency score, construction strategy score, and error correction score according to the size of the correlation coefficients; then, normalizing each initial weight to obtain the comprehensive weight; finally, an evaluation report containing the structural fidelity score, construction efficiency score, construction strategy score, error correction score, and comprehensive evaluation result is output.

[0119] In this embodiment, the historical completed samples, historical correctly assembled samples, historical correctly connected samples, and historical manually scored samples are all selected from historical sample data formed using the same standardized rule block library, the same data acquisition device deployment method, and the same type of target block task.

[0120] Example 2:

[0121] See Figure 3 As shown, this embodiment provides a vision-based intelligent block construction interaction system. Implementing the vision-based intelligent block construction interaction method includes:

[0122] The observation construction module is used to align and preprocess the pre-acquired top image sequence and side view image sequence to form a constructed observation sequence;

[0123] The target parsing module is used to perform structured parsing of the pre-acquired target drawings and target component list to generate a target component set, a target connection set, and a target stage set.

[0124] The state recognition module is used to input the constructed observation sequence into the pre-constructed stage state recognition model to obtain candidate components, candidate connection relationships, and component representation information corresponding to the candidate components;

[0125] The timing confirmation module is used to confirm candidate components across time based on component characterization information and candidate connection relationships, forming a stage state sequence;

[0126] The event extraction module is used to perform target matching on stable components in the stage state sequence based on the target component set to obtain target matched components; to match and confirm the connection relationship between target matched components based on the target connection set to obtain matching connection relationship; and to compare adjacent stage states based on the target stage set, and extract construction behavior events based on the comparison result and matching connection relationship.

[0127] The result alignment module is used to merge the stage states of the last M consecutive observation times in the stage state sequence to obtain the final construction state; and to align the final construction state, the target component set, and the target connection set to obtain the structure correspondence result.

[0128] The evaluation generation module is used to generate structural fidelity scores, construction efficiency scores, construction strategy scores, error correction scores, and comprehensive evaluation results based on the structural correspondence results and construction behavior events.

[0129] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0130] In conclusion, the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A vision-based intelligent block construction interaction method, characterized in that, include: Align and preprocess the pre-acquired top and side view image sequences to form a constructed observation sequence; The pre-acquired target drawings and target component list are structured and parsed to generate a set of target components, a set of target connections, and a set of target stages. The constructed observation sequence is input into the pre-constructed stage state identification model to obtain candidate components, candidate connection relationships, and component representation information corresponding to the candidate components; Based on the component characterization information and candidate connection relationships, candidate components are confirmed across time periods to form a stage state sequence; Based on the target component set, target matching is performed on stable components in the stage state sequence to obtain target matching components; based on the target connection set, the connection relationship between target matching components is matched and confirmed to obtain matching connection relationship; based on the target stage set, adjacent stage states are compared, and based on the comparison result and matching connection relationship, construction behavior events are extracted. The final constructed state is obtained by merging the stage states of the last M consecutive observation times in the stage state sequence. Align the final construction state, the target component set, and the target connection set to obtain the corresponding structural result; Based on the structural correspondence results and construction behavior events, generate structural fidelity scores, construction efficiency scores, construction strategy scores, error correction scores, and comprehensive evaluation results.

2. The vision-based intelligent block construction interaction method according to claim 1, characterized in that, The methods for obtaining the target stage set include: The target components that are in direct contact with the workbench are classified into the first target stage, and the first target stage is defined as the target stage that has been classified into the current target stage. Among the target components that have never been assigned to the target stage, select target components that have a support connection with at least one target component that has been assigned to the target stage, and whose suspended connection ends will not appear when only the target components that have been assigned to the target stage are retained. The selected target components are assigned to the next target stage, and the currently assigned target components are updated to all target components that have been assigned to each target stage up to the present. Repeat the above filtering and categorization process until all target components have been categorized, resulting in the target stage set.

3. The vision-based intelligent block construction interaction method according to claim 1, characterized in that, Methods for constructing observation sequences include: The acquisition time of each frame in the top image sequence is recorded as the reference time; Find the side view image with the smallest time difference from each reference time in the side view image sequence, and whose time difference does not exceed the preset alignment time difference, and use it as the side view image for the corresponding reference time. According to the preset observation interval, extract the image frames corresponding to each observation time from the aligned top image and side view image as observation frames; Each observation frame is processed by performing workbench area cropping, perspective correction, and brightness normalization to form a constructed observation sequence.

4. The vision-based intelligent block construction interaction method according to claim 1, characterized in that, Methods for forming a phase state sequence include: In adjacent observation times, the center distance and bounding box overlap ratio are calculated for candidate components with the same component category. When the center distance is not greater than the cross-time matching threshold and the bounding box overlap ratio is not lower than the overlap matching threshold, they are determined to be the same candidate component. The consistency of the component representation information and candidate connection relationship of the same candidate component in several consecutive preset stable frames is checked. When the check is passed, it is determined to be a stable component, and the candidate connection relationship that has passed the check is determined as the connection relationship at the corresponding observation time. Based on the stable components at each observation time and the connection relationship of the corresponding observation time, the stage state at each observation time is formed. The stage states at each observation time are arranged in chronological order to form a stage state sequence.

5. The vision-based intelligent block construction interaction method according to claim 1, characterized in that, Construction behavior events include new splicing events, error test play events, demolition and correction events, stagnation events, and sequence switching events; Methods for extracting constructive behavioral events include: Compare the states of adjacent stages to obtain newly added and disappeared stable components; Based on the matching status of the newly added stable components, extract the new splicing event and the error test release event; Extract the removal correction event based on the correction situation after the disappearance of the stabilizing component; Extract stagnant events based on the changes in events across multiple consecutive stages of the state; Extract the sequence switching event based on the target stage number of the corresponding target component in the newly added splicing event.

6. The vision-based intelligent block construction interaction method according to claim 5, characterized in that, The extraction of new splicing events and error test release events based on the matching status of the newly added stable components includes: When a new stable component corresponds to a target matching component and forms a corresponding target connection relationship in the target connection set, the stage state change caused by the appearance of the new stable component will be determined as a new splicing event. When a newly added stable component does not correspond to a target matching component, or does not form a corresponding target connection relationship, and disappears within the error duration threshold, the stage state change caused by the appearance of the newly added stable component will be identified as an error test release event.

7. The vision-based intelligent block construction interaction method according to claim 1, characterized in that, Methods for obtaining matching connection relationships include: Obtain the hierarchical position and component boundary of any two target matching components; Calculate the contact overlap ratio of the component boundaries of two target matching components in the connection direction corresponding to the target connection relationship in the target connection set; The hierarchical relationship between the two target matching components is determined based on their hierarchical positions. Then, the angle difference between the direction of the line connecting the centers of the two target matching components and the direction of the connection relationship of the corresponding target is calculated. When the hierarchical relationship is consistent with the hierarchical relationship of the corresponding connection relationship in the target connection set, the contact overlap ratio is not lower than the contact overlap threshold, and the angle difference is not greater than the direction deviation threshold, the matching connection relationship of the two target matching components is determined.

8. The vision-based intelligent block construction interaction method according to claim 1, characterized in that, Methods for obtaining the corresponding results of the structure include: The final constructed state is obtained by merging the stage states of the last M consecutive observation times in the stage state sequence. The final constructed state is matched with the target component set and the target connection set to obtain the structural correspondence result. The number of correctly matched components, the number of correctly matched connections, and the number of correctly matched hierarchical positions are counted.

9. The vision-based intelligent block construction interaction method according to claim 5, characterized in that, Methods for generating construction strategy scores include: Retrieve the target stage number corresponding to the newly added splicing event; When the target stage number corresponding to the next newly added splicing event is less than the target stage number corresponding to the previous newly added splicing event, it is recorded as a cross-stage rollback. When the target stage number corresponding to a newly added splicing event is greater than the minimum target stage number of the target component to which the unmatched component belongs, it is recorded as a sequence switching event. The number of event pairs in adjacent newly added splicing event pairs where the number of the next target stage is greater than or equal to the number of the previous target stage is counted, and the continuity of stage advancement is obtained by dividing the number of event pairs by the number of adjacent newly added splicing event pairs. The construction strategy score is generated based on the continuity of phase advancement, the number of sequential switching events, and the number of cross-phase rollbacks.

10. A vision-based intelligent block construction interaction method according to claim 8, characterized in that, Methods for generating structure fidelity scores include: The component matching ratio is determined by the ratio of the number of correctly matched components to the total number of target components; the connection matching ratio is determined by the ratio of the number of correctly matched connections to the total number of target connections; and the hierarchical matching ratio is determined by the ratio of the number of correctly matched hierarchical positions to the total number of target components. The structural fidelity score is generated based on the component matching ratio, connection matching ratio, and hierarchy matching ratio.

11. A vision-based intelligent block construction interaction system, implementing the vision-based intelligent block construction interaction method according to any one of claims 1-10, characterized in that, include: The observation construction module is used to align and preprocess the pre-acquired top image sequence and side view image sequence to form a constructed observation sequence; The target parsing module is used to perform structured parsing of the pre-acquired target drawings and target component list to generate a target component set, a target connection set, and a target stage set. The state recognition module is used to input the constructed observation sequence into the pre-constructed stage state recognition model to obtain candidate components, candidate connection relationships, and component representation information corresponding to the candidate components; The timing confirmation module is used to confirm candidate components across time based on component characterization information and candidate connection relationships, forming a stage state sequence; The event extraction module is used to perform target matching on stable components in the stage state sequence based on the target component set to obtain the target matching components; and to match and confirm the connection relationships between the target matching components based on the target connection set to obtain the matching connection relationships. Based on the target stage set, the states of adjacent stages are compared, and construction behavior events are extracted based on the comparison results and matching connection relationships. The result alignment module is used to merge the stage states of the last M consecutive observation times in the stage state sequence to obtain the final constructed state. Align the final construction state, the target component set, and the target connection set to obtain the corresponding structural result; The evaluation generation module is used to generate structural fidelity scores, construction efficiency scores, construction strategy scores, error correction scores, and comprehensive evaluation results based on the structural correspondence results and construction behavior events.

Images