Building outer wall defect identification method based on multi-modal data fusion

Abstract

The application discloses a building outer wall defect identification method based on multi-modal data fusion, and relates to the technical field of computer vision and artificial intelligence, and comprises the following steps: collecting image data of a building outer wall under a continuous time sequence, marking a brightness change area and a corresponding spatial position in each time frame, and constructing a continuous change sequence.The application identifies a periodic and consistent change process by segmenting and statistically tracking the brightness change in the continuous time sequence, distinguishes between false changes and real cracks by combining continuity analysis, improves identification accuracy, simultaneously reconstructs the brightness change amplitude in the false change interval, converts the continuous change into a segmented form, weakens the continuity, and thus reduces the interference on defect evolution judgment and improves the stability and consistency of the detection result.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a method for identifying defects in building exterior walls based on multimodal data fusion. Background Technology

[0002] Multimodal data fusion-based building facade defect identification refers to the process of inspecting building facades that moves beyond relying on information from a single source. Instead, it simultaneously collects and comprehensively utilizes multiple types of data (such as visible light images, infrared thermal imaging, depth information, or laser point clouds), and through cross-data type correlation analysis and collaborative processing, identifies and locates defects on the facade surface such as cracks, hollow areas, detachment, and water seepage. Its core lies in leveraging the complementary relationships in the physical properties of different data. For example, visible light information is used to judge the appearance, infrared information is used to reveal internal temperature abnormalities, and depth information is used to characterize structural deformation. This improves the accuracy and stability of defect identification under complex environmental conditions, reduces false positives and missed detections, and makes the inspection results more comprehensive and reliable. It is suitable for intelligent inspection and engineering safety assessment scenarios of high-rise building facades.

[0003] The existing technology has the following shortcomings: In existing technologies, the identification of exterior wall defects based on continuous image acquisition is easily affected by dynamic changes in external lighting. When the exterior wall surface has strong reflective properties, periodic reflective flickering occurs during continuous changes in the angle of sunlight, causing the brightness of local areas to fluctuate regularly over time, forming pseudo-variable trajectories resembling linear extensions in continuous images. These trajectories, in terms of spatial morphology and temporal evolution, are highly similar to the crack propagation process, easily leading to misinterpretation as a gradual crack expansion trend and thus misjudgment as continuous structural deterioration. These problems interfere with defect evolution assessment, reduce the accuracy of identification results, and consequently affect the reliability of subsequent detection decisions.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method for identifying defects in building exterior walls based on multimodal data fusion, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying defects in building exterior walls based on multimodal data fusion, comprising the following steps: Image data of building exterior walls are collected in a continuous time series. The brightness change areas and their corresponding spatial locations in each time frame are marked to construct a continuous change sequence containing temporal relationships. Based on the continuous change sequence, the recurrence of brightness changes in the time series is segmented and statistically analyzed. The change segments that show repetitive change characteristics are merged to form a set of periodic change segments. Based on the set of periodic change segments, the positional changes of each change segment in space are continuously tracked, the change process that moves smoothly along a fixed direction is selected, and the corresponding directional change features are extracted. By combining the characteristics of directional changes, the continuity of each change process in the time series is analyzed, the change process that maintains continuous movement and lacks sudden change characteristics is identified, and the pseudo-change interval is determined. Based on the pseudo-change interval, the brightness change amplitude within the corresponding time range is reconstructed. By adjusting the distribution relationship between the continuous enhancement segment and the interval segment, the continuous change is transformed into a segmented change, thereby reducing the interference of pseudo-change on defect identification.

[0007] Preferably, for the organization and processing of brightness variation regions and spatial locations in a continuous time series, a continuous variation sequence with a temporal order is constructed. The specific steps are as follows: Image data from a continuous time series is acquired and time-stamped. Brightness change areas in each time frame are extracted and their corresponding spatial locations are recorded to form an original set of change records. Spatial location matching of brightness change regions in adjacent time frames is carried out in the original change record set. Brightness change regions with overlapping or boundary connections are connected to form continuous change segments and record the spatial location change path. The sequence of time nodes within the continuous change segments is organized, and the brightness change information is combined with the spatial location information for recording. The independent recording state between different change segments is maintained to obtain a set of change segments. The temporal sequence of the change fragment set is summarized and a continuous index is established. The brightness change areas and spatial locations corresponding to each time node are uniformly arranged to form a continuous change sequence.

[0008] Preferably, the spatial positions corresponding to each time node in a continuously changing segment are recorded using a unified coordinate representation, and the order of time nodes and the correspondence between spatial positions are kept consistent. At the same time, the spatial position matching process is limited to include overlap determination and boundary connection determination to ensure the continuous expression of the continuously changing segment in both time and spatial dimensions.

[0009] Preferably, the recurring occurrences of brightness changes in a continuous sequence are organized to form a set of periodic change segments with temporal regularity. The specific steps are as follows: Extract the brightness change regions corresponding to each time node in the continuous change sequence, and record the occurrence of the brightness change regions at different time nodes in chronological order, forming a brightness change time record trajectory with spatial location as the index; The time distribution of the brightness change time record trajectory is divided into segments. Time nodes with continuous brightness change relationships are divided into brightness change segments, and intervals without brightness change are marked to obtain a segmented structure. Organize the time interval relationships of each brightness change segment in the segmented structure, group the brightness change segments whose time intervals show a repetitive distribution characteristic, form a set of change segments, and record the corresponding spatial location range; The set of change segments is compiled and arranged in chronological order. The set of change segments that show recurring change characteristics is then selected to form a set of periodic change segments.

[0010] Preferably, the brightness change segments in the set of change segments are arranged in order of time interval, and the brightness change segments with consistent time intervals are grouped together. Brightness change segments with repetitive time intervals are combined into the same set of change segments, while maintaining the consistency of the corresponding spatial location range, so as to enhance the unified expression of the set of periodic change segments in the time and space dimensions.

[0011] Preferably, the spatial position changes of the changing segments in the set of periodic changing segments are continuously tracked, and the changing processes that move smoothly along a fixed direction are selected and the directional change features are extracted. The specific steps are as follows: Expand the time nodes within the time range corresponding to each periodic change segment in the set of periodic change segments, extract the spatial coordinates of the brightness change area and arrange them in chronological order to form a spatial position change sequence; Construct spatial displacement segments between adjacent time nodes in a spatial position change sequence, record the position coordinates of the starting time node and the current time node and determine the spatial movement direction to form a set of continuous spatial displacement segments; Compare the directional changes of adjacent spatial displacement segments in a continuous set of spatial displacement segments, merge spatial displacement segments with the same direction into a movement interval and record the time node range, and separate spatial displacement segments with different directional changes. Organize the spatial location change paths within the movement interval and arrange them in chronological order, summarize the path direction expression and extract the direction change features to form a set of direction change features.

[0012] Preferably, the spatial position change path within the movement interval is organized to ensure directional consistency. Spatial displacement segments with consistent directions are continuously merged, and the merged spatial paths are given a unified directional expression. At the same time, the corresponding change process is recorded in conjunction with the time node range to form a set of directional change features with continuous directional characteristics.

[0013] Preferably, a time continuity analysis is performed on the change process corresponding to the directional change characteristics to identify change processes that maintain continuous movement and lack sudden change characteristics, and to determine pseudo-change intervals. The specific steps are as follows: The time range corresponding to each change process in the set of directional change features is expanded, the brightness change information and spatial location information of each time node are extracted and arranged in chronological order to form a continuous time series; Organize the brightness change relationship between adjacent time nodes in a continuous time series, divide the continuous change segment and the abrupt change segment and record the difference in the change amplitude to obtain the segmented time structure; Based on the corresponding segmented time structure and directional change characteristics, the spatial movement path in the directional change characteristics is matched with the continuous change segment, and stable movement segments that maintain consistent direction and continuous spatial position are selected. Summarize stable movement segments and arrange them in chronological order. Merge continuous stable movement segments that are not separated by abrupt change segments and determine the corresponding time range as pseudo-change intervals.

[0014] Preferably, the determination of continuous change segments includes continuous recording of the difference in brightness change amplitude between adjacent time nodes, the screening of stable movement segments includes change processes with consistent spatial movement path direction and continuous time sequence arrangement, and pseudo change intervals are formed by combining the time ranges of multiple stable movement segments.

[0015] Preferably, the brightness change amplitude within the pseudo-change interval is reconstructed, the distribution relationship between the continuous enhancement segment and the interval segment is adjusted, and the segmented change expression is achieved, including the following steps: Expand all time nodes corresponding to the pseudo-change interval and extract the brightness change amplitude. Form a brightness change amplitude sequence in chronological order and record the time node interval relationship. Analyze the amplitude change relationship between adjacent time nodes in the brightness change amplitude sequence, divide the continuously enhanced segment into interval segments, and record the corresponding time node range of each segment; Adjust the arrangement of continuous enhancement segments and interval segments, compress the time nodes in the continuous enhancement segments and allocate the removed time nodes to adjacent interval segments to form a segmented structure; The segments are reorganized and arranged in a new time sequence, with continuously enhanced segments and interval segments alternating to obtain a segmented change sequence.

[0016] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention segments and statistically analyzes the recurrence of brightness changes in a continuous time series, and continuously tracks changes in spatial location. This allows for the effective identification and differentiation of periodic changes that move along a fixed direction. Furthermore, it analyzes the continuity of these changes, constructing a unified discrimination criterion from both temporal and spatial dimensions. This distinguishes pseudo-changes caused by illumination variations from the actual crack propagation process, avoids misjudging reflective trajectories as structural damage trends, improves the accuracy of defect identification results, and makes the detection results more closely reflect the actual condition of the exterior wall.

[0017] This invention reconstructs the brightness change amplitude for the identified pseudo-change intervals. By adjusting the distribution relationship between continuous enhancement segments and interval segments in the time series, the originally continuous change process is transformed into a segmented presentation. This weakens the continuity of pseudo-changes in terms of time expression, making it difficult for them to form a continuously developing trend in subsequent identification processes. This reduces the interference of pseudo-changes on defect evolution judgment, improves the stability of the overall identification process, and enhances the adaptability under complex lighting conditions, ensuring that the detection results remain consistent under different time conditions. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a flowchart of the method for identifying building exterior wall defects based on multimodal data fusion according to the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0021] This invention provides, for example Figure 1 The building exterior wall defect identification method based on multimodal data fusion shown includes the following steps: Image data of building exterior walls are collected in a continuous time series. The brightness change areas and their corresponding spatial locations in each time frame are marked to construct a continuous change sequence containing temporal relationships. By meticulously organizing the image changes of the building's exterior walls over a continuous time process, the correspondence between brightness variation areas in the temporal and spatial dimensions is established layer by layer, creating a continuous connection between each time point and thus forming a continuous change sequence that can be used for subsequent analysis. The specific implementation steps are as follows: Within a pre-defined time acquisition range, continuous image acquisition is performed on the target area of ​​the building's exterior wall. Each frame of the image is assigned a unique time identifier, arranging all images into a time sequence. After image acquisition, the pixel brightness distribution in each frame is compared frame by frame. The brightness values ​​at corresponding positions in the current time frame are compared with those in the previous time frame. The locations where brightness values ​​change are extracted and combined to form brightness change regions. Simultaneously, the specific spatial location of each brightness change region in the image is recorded, expressed in image coordinates, establishing a clear correspondence between brightness change regions and spatial locations. During continuous time progression, the brightness change regions extracted from each time frame and their corresponding spatial locations are recorded frame by frame and arranged sequentially, thus forming an original set of change records containing time sequence information.

[0022] After the original set of change records is formed, the brightness change regions between adjacent time frames are matched one by one. The brightness change regions in the current time frame are compared with the brightness change regions in the previous time frame in terms of spatial position. When two brightness change regions have an overlapping relationship or a boundary connection relationship in spatial position, they are regarded as the continuation of the same change process at different time nodes and are connected. This allows the brightness change regions scattered in different time frames to be connected into continuous change segments in chronological order. During the connection process, the spatial position changes of each time node in each change segment are recorded so that the change segment can completely reflect the spatial movement path of the brightness change region in the process of time progression. At the same time, new change segments are created separately for brightness change regions that cannot form a connection relationship with the previous time frame, so that all brightness change regions can be included in the corresponding change segments.

[0023] After constructing the continuous change segments, the data within each segment is uniformly organized. Each segment is rearranged according to the order of its contained time nodes, so that each time node in the segment corresponds to a continuous time series. At each time node, the brightness change information is combined with the corresponding spatial location information and recorded, so that each time node has a complete description of brightness change and location change. In this process, the time distribution between different change segments is distinguished, so that different change segments within the same time range are kept in an independent recording state, avoiding data overlap between different change segments, thus forming a set of multiple change segments that exist in parallel in the time dimension and are independent of each other in the spatial dimension.

[0024] After the above-mentioned set of change segments is organized, all change segments are uniformly summarized and arranged in chronological order. A continuous index is created for each time node, so that any time position can correspond to a specific brightness change area and its spatial location. During the summarization process, the temporal connection relationship between each change segment is sequentially arranged so that different change segments form a continuous coverage relationship in the time dimension, while maintaining the independent expression of each change segment in the spatial path. Through the above summarization and arrangement process, a continuous change sequence containing temporal order relationship is formed. This continuous change sequence completely records the evolution trajectory of the brightness change area in the continuous time process and retains the spatial location change information corresponding to each time node, providing a continuous and structurally clear data foundation for subsequent further processing based on temporal change characteristics and spatial movement characteristics.

[0025] Based on the continuous change sequence, the recurrence of brightness changes in the time series is segmented and statistically analyzed. The change segments that show repetitive change characteristics are merged to form a set of periodic change segments. This study focuses on the recurring nature of brightness changes over time in a continuous sequence. By segmenting and structuring the change segments within the time series, segments exhibiting recurring characteristics can be uniformly grouped, forming a set of periodic change segments with temporal regularity. This provides a clear data organization foundation for subsequent change selection. The specific steps are as follows: Based on the established temporal sequence in the continuous change sequence, the brightness change region corresponding to each time node is expanded node by node, and the occurrence of the brightness change region at different time nodes is recorded in chronological order. During the recording process, the spatial position of each brightness change region is used as the correlation basis, and brightness change regions appearing in the same spatial location range at different time nodes are marked accordingly, thus forming a brightness change time recording trajectory indexed by spatial position. This ensures that each recording trajectory can reflect the occurrence of brightness changes in the same spatial region during the time progression. On this basis, the occurrence time points of brightness changes in each recording trajectory are arranged continuously, so that the distribution of brightness changes in the time dimension is fully expressed.

[0026] After constructing the brightness change time recording trajectory, the time distribution in each recording trajectory is divided into segments. The parts where there is a continuity of brightness change between adjacent time nodes are divided into the same time segment. At the same time, the intervals between time nodes where no brightness change occurs are marked. This decomposes each recording trajectory into a structure composed of multiple alternating brightness change segments and interval segments. During the division process, the start and end times of each brightness change segment are recorded, and the continuity of brightness change within the segment is organized. This ensures that each brightness change segment has a clear time range description, while retaining the time interval information between segments. This provides a segmented data foundation for subsequent statistical analysis of recurrence.

[0027] After obtaining segmented brightness change segments, the temporal distribution relationship between multiple brightness change segments in each recorded trajectory is organized. Brightness change segments with similar time interval characteristics are compared and analyzed. When the time interval between multiple brightness change segments repeats during continuous time progression, these brightness change segments are grouped into the same set of change segments. The time interval order of each brightness change segment in the set is uniformly arranged so that the set can reflect the recurring pattern of brightness changes in the time dimension. At the same time, during the merging process, the spatial location range contained in each set of change segments is uniformly recorded so that the set can reflect both the repetitive characteristics in time and maintain the consistency in spatial location.

[0028] After merging the various change segment sets, the temporal distribution structure within each set is comprehensively organized. The brightness change segments within each set are rearranged in chronological order, and the time intervals between segments are uniformly expressed, forming a regularly distributed temporal structure within the set. Based on this, all change segment sets are uniformly summarized, and sets exhibiting recurring change characteristics in the time dimension are selected. These sets are identified as periodic change segment sets. This ensures that the periodic change segment sets not only contain multiple brightness change segments that repeat in time but also reflect the cyclical relationship of these segments in the process of time progression. This provides temporally regular input data for subsequent further analysis based on directional and continuity features.

[0029] Based on the set of periodic change segments, the positional changes of each change segment in space are continuously tracked, the change process that moves smoothly along a fixed direction is selected, and the corresponding directional change features are extracted. By decomposing and organizing the spatial position changes of each segment in a set of periodic change segments during continuous time progression, the positional evolution of each segment in the time dimension can be continuously recorded. Based on this, change processes with consistent movement trends are extracted, thus forming a stable representation of directional change characteristics. The specific steps are as follows: For each time range of a periodic variation segment in the set of periodic variation segments, all corresponding time nodes within that time range are expanded one by one, and the spatial coordinates of the brightness variation region are extracted at each time node. The spatial position adopts a unified two-dimensional coordinate representation method, and the center position of each brightness variation region in the image is determined as the representative position of the region. At the same time, the horizontal and vertical position values ​​of this position in the image coordinates are recorded, so that each time node corresponds to a clear spatial position expression. After all time nodes are processed, the spatial positions of each time node within the same variation segment are arranged in chronological order to form a continuous spatial position change sequence. This sequence completely describes the spatial movement path of the brightness variation region within the time range and maintains the consistency of the coordinate representation method throughout the entire sequence, so that subsequent processing can be carried out under a unified spatial reference.

[0030] Based on the spatial position change sequence, the positional change relationship between adjacent time nodes is unfolded segment by segment. The spatial position of the current time node is connected one-to-one with the spatial position of the previous time node to form a clear spatial displacement record. In each spatial displacement record, the position coordinates of the starting time node and the position coordinates of the current time node are recorded respectively, and the spatial movement direction is determined according to the relative change relationship between the two positions. This direction is expressed in a unified directional description form. At the same time, the change trend of spatial displacement in the horizontal direction and the change trend in the vertical direction are recorded, so that each spatial displacement has a complete positional change description. After processing all time nodes, the entire spatial position change sequence is divided into multiple spatial displacement segments arranged continuously in chronological order. Each spatial displacement segment maintains the same expression structure, thus providing a unified data foundation for subsequent directional consistency analysis.

[0031] Based on the directional change relationship between each spatial displacement segment, all spatial displacement segments within the same change segment are continuously compared. The directional changes of adjacent spatial displacement segments are compared one by one. When multiple consecutive spatial displacement segments maintain consistency in directional expression and do not exhibit directional reversal or deviation during time progression, these spatial displacement segments are merged into the same movement interval, and the start and end time nodes of this movement interval are recorded. At the same time, the sequential relationship of all spatial displacement segments within this movement interval is preserved, so that the movement interval can fully reflect the spatial movement of the brightness change area within a continuous time range. For spatial displacement segments that differ in directional change, the continuation of the current movement interval is terminated, and a new movement interval is re-established, so that spatial displacement segments with different directional changes are separated into different movement intervals. Through the above processing, each movement interval corresponds to a continuous spatial movement process with consistent direction, thereby filtering out the change process that moves smoothly along a fixed direction.

[0032] For each selected movement interval, the spatial position change path within each interval is comprehensively organized. The spatial positions of all time nodes within the interval are arranged chronologically, and the directional expressions of each spatial displacement segment in the path are summarized, forming a continuous spatial path in a single direction for the entire movement interval. Simultaneously, the spatial position information corresponding to each time node and the spatial displacement relationship between adjacent nodes are retained in this path, ensuring that the path reflects both the overall movement direction and the continuous state of the movement process over time. After completing the path organization, the directional information of each movement interval is extracted as the directional change feature of the change process, and the directional change features are uniformly recorded, ensuring that each change process has a clear directional expression and time range description. This forms a set of directional change features that can be used for subsequent analysis, providing a stable spatial behavior basis for further identification of pseudo-changes.

[0033] By combining the characteristics of directional changes, the continuity of each change process in the time series is analyzed, the change process that maintains continuous movement and lacks sudden change characteristics is identified, and the pseudo-change interval is determined. By combining the directional change features extracted from the preceding sequence, the continuity of each change process in the continuous time series is analyzed layer by layer. Spatial movement behavior and temporal change behavior are uniformly correlated, thereby identifying change processes that maintain continuous movement and lack sudden change characteristics. Based on this, the corresponding pseudo-change intervals are determined, enabling subsequent processing to specifically suppress such changes. The specific steps are as follows: Around the set of directional change features, each change process is recorded, and the time range corresponding to each change process is expanded one by one. Within this time range, the brightness change information and spatial location information of all time nodes are extracted. The brightness change state of each time node is arranged in chronological order, while keeping the correspondence of spatial location in each time node unchanged. This makes each change process a continuous time series containing brightness change information and spatial location information. In this time series, the time interval between each time node is recorded consistently, so that the entire change process forms a continuous arrangement in the time dimension, thus providing a complete time expansion basis for subsequent continuous analysis.

[0034] For continuous time series, the brightness change relationship between each time node is organized segment by segment. The brightness change information of the current time node is compared with the brightness change information of the previous time node, and the difference in the change amplitude between the two is recorded. At the same time, the change amplitude is continuously recorded during the time process, so that the brightness change within each time interval has a clear trend description. In the process of organization, the part of brightness change amplitude that remains continuously changing between adjacent time nodes is divided into continuous change segment, and the part of brightness change amplitude that jumps between adjacent time nodes is marked separately as abrupt change segment. The entire time series is divided into a structure composed of alternating continuous change segment and abrupt change segment, thus providing a segmented time structure expression for subsequent change process screening.

[0035] By focusing on the correlation between continuous change segments and directional change characteristics, a joint analysis is performed on the time segments within each change process. The spatial movement paths recorded in the directional change characteristics are matched one-to-one with the continuous change segments in the time series. When the spatial position change path of a continuous change segment is consistent with the directional change characteristics within its corresponding time range, the continuous change segment is identified as a stable movement segment, and its start and end time nodes are recorded. At the same time, the continuity of spatial position change within this time segment is maintained, so that the segment can simultaneously reflect the continuous change state in the time dimension and the continuous movement state in the spatial dimension. If the spatial position change is interrupted or the directional change is changed within the continuous change segment, the continuation of the current segment is terminated, so that it does not participate in the merging of stable movement segments, thereby selecting change processes that simultaneously satisfy continuous temporal change and consistent spatial direction.

[0036] For the selected stable moving segments, their distribution in the overall time series is uniformly organized. The stable moving segments are arranged in chronological order, and the time intervals between adjacent segments are recorded. When multiple stable moving segments show a continuous connection during the time progression without being interrupted by abrupt segments, these segments are merged into the same change process. The start and end time nodes of this change process in the entire time series are uniformly marked to form a complete time range expression. Based on this, all change processes that meet the characteristics of continuous movement and lack abrupt change are summarized, and their corresponding time ranges are determined as pseudo-change intervals. The pseudo-change intervals can accurately reflect the change behavior of brightness changes that show continuous movement but lack abrupt change characteristics during the time progression, thus providing a clear time positioning basis for subsequent processing of pseudo-changes.

[0037] Based on the pseudo-change interval, the brightness change amplitude within the corresponding time range is reconstructed. By adjusting the distribution relationship between the continuous enhancement segment and the interval segment, the continuous change is transformed into a segmented change, thereby reducing the interference of pseudo-change on defect identification. By decomposing and reorganizing the distribution of brightness changes over time within the pseudo-change interval, the originally continuous change process is reorganized into an expression composed of multiple separated change segments. This weakens the continuous representation of brightness changes in the time series, thereby reducing the interference of pseudo-changes on defect identification results. The specific implementation steps are as follows: Around the established pseudo-change interval, all time nodes covered by the interval are expanded one by one, and each time node is numbered and recorded according to the original time sequence. The corresponding brightness change amplitude value is extracted at each time node, and the value is stored in a one-to-one correspondence with the time node number. This makes the entire pseudo-change interval a brightness change amplitude sequence arranged continuously in time sequence. In this sequence, each time node has a unique number, a corresponding brightness change amplitude, and a clear time position. At the same time, the fixed time interval between adjacent time nodes is recorded, so that the entire sequence maintains a continuous and uniform progression relationship in the time dimension. Through the above processing, the distribution of brightness change amplitude in the pseudo-change interval in the time dimension is fully expressed, and a clear data arrangement basis is provided for subsequent segmentation processing.

[0038] Based on the brightness variation amplitude sequence, the amplitude variation relationship between adjacent time nodes is analyzed node by node. The brightness variation amplitude of the current time node is compared with that of the previous time node, and the variation is classified and recorded according to the trend. When the brightness variation amplitude shows a continuous increase or decrease in multiple consecutive time nodes, these time nodes are classified into the same continuous enhancement segment, and the start and end time node numbers of the segment are recorded. At the same time, the order relationship of the brightness variation amplitude of each time node within the segment is preserved. When the brightness variation amplitude remains basically stable in consecutive time nodes or the variation amplitude shows an alternating state between adjacent nodes, these time nodes are divided into interval segments, and the corresponding time node range is also recorded. This results in the entire brightness variation amplitude sequence being divided into a structure of multiple continuous enhancement segments and multiple interval segments arranged alternately, ensuring that each time node is uniquely classified into a certain segment.

[0039] Based on the arrangement of continuous enhancement segments and interval segments in the time series, each segment is reorganized. While maintaining the order of time nodes within each segment, the time span of continuous enhancement segments is compressed. Specifically, this is achieved by reducing the number of time nodes involved in the expression within the continuous enhancement segment. Some time nodes that were originally arranged continuously are extracted at fixed intervals, and only a portion of these time nodes are retained to express the continuous enhancement process. At the same time, the time nodes that are not retained are removed from the continuous enhancement segment and redistributed to adjacent interval segments. This makes the distribution of interval segments in the time series more dispersed, thus forming multiple continuous enhancement segments separated by interval segments in the overall time series. This artificially splits the originally continuous brightness change process into multiple time-discontinuous change segments.

[0040] For all segments after reorganization, the brightness change amplitude sequence is reconstructed as a whole. All continuous enhancement segments and interval segments are arranged in a unified order according to the new time nodes. This makes the entire sequence a structure composed of multiple dispersed change segments while keeping the total number of time nodes unchanged. In this structure, each continuous enhancement segment is separated by interval segments, so that the brightness change appears in segments as time progresses. At the same time, the original change trend is still retained within each continuous enhancement segment. Through the above reconstruction process, the continuous changes within the pseudo-change interval are transformed into segmented changes, thereby weakening the overall coherence of continuous changes in the time series and reducing its interference with the judgment of change trends during defect identification.

[0041] This invention segmentally analyzes the recurrence of brightness changes in a continuous time series and continuously tracks spatial changes. This allows for the effective identification and differentiation of periodic changes that move along a fixed direction. Furthermore, it analyzes the continuity of these changes, constructing a unified discrimination criterion from both temporal and spatial dimensions. This distinguishes between pseudo-changes caused by illumination variations and genuine crack propagation processes, preventing the misjudgment of reflected light trajectories as structural damage trends. This improves the accuracy of defect identification results and makes the detection results more closely reflect the actual condition of the exterior wall. This invention reconstructs the brightness change amplitude for the identified pseudo-change intervals. By adjusting the distribution relationship between continuous enhancement segments and interval segments in the time series, the originally continuous change process is transformed into a segmented presentation. This weakens the continuity of pseudo-changes in terms of time expression, making it difficult for them to form a continuously developing trend in subsequent identification processes. This reduces the interference of pseudo-changes on defect evolution judgment, improves the stability of the overall identification process, and enhances the adaptability under complex lighting conditions, ensuring that the detection results remain consistent under different time conditions.

[0042] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for identifying building exterior wall defects based on multimodal data fusion, characterized in that, Includes the following steps: Image data of building exterior walls were collected in a continuous time series. The brightness change areas and their corresponding spatial locations in each time frame were marked to construct a continuous change sequence. Based on the continuous change sequence, the recurrence of brightness changes in the time series is segmented and statistically analyzed. The change segments that show repetitive change characteristics are merged to form a set of periodic change segments. Based on the set of periodic change segments, the positional changes of each change segment in space are continuously tracked, the change process that moves smoothly along a fixed direction is selected, and the corresponding directional change features are extracted. By combining the characteristics of directional changes, the continuity of each change process in the time series is analyzed, the change process that maintains continuous movement and lacks sudden change characteristics is identified, and the pseudo-change interval is determined. Based on the pseudo-change interval, the brightness change amplitude within the corresponding time range is reconstructed. By adjusting the distribution relationship between the continuous enhancement segment and the interval segment, the continuous change is transformed into a segmented change.

2. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 1, characterized in that, To organize and process the brightness variation regions and spatial locations in continuous time series, a continuous variation sequence is constructed. The specific steps are as follows: Image data from a continuous time series is acquired and time-stamped. Brightness change areas in each time frame are extracted and their corresponding spatial locations are recorded to form an original set of change records. Spatial location matching of brightness change regions in adjacent time frames is carried out in the original change record set. Brightness change regions with overlapping or boundary connections are connected to form continuous change segments and record the spatial location change path. The sequence of time nodes within the continuous change segments is organized, and the brightness change information is combined with the spatial location information for recording. The independent recording state between different change segments is maintained to obtain a set of change segments. The temporal sequence of the change fragment set is summarized and a continuous index is established. The brightness change areas and spatial locations corresponding to each time node are uniformly arranged to form a continuous change sequence.

3. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 2, characterized in that, The spatial positions corresponding to each time node in a continuously changing segment are recorded using a unified coordinate representation, and the order of time nodes and the correspondence between spatial positions are kept consistent. At the same time, the spatial position matching process is limited to include the determination of overlap and boundary connection.

4. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 2, characterized in that, The recurring patterns of brightness changes in a continuous sequence are organized to form a set of periodic change segments. The specific steps are as follows: Extract the brightness change regions corresponding to each time node in the continuous change sequence, and record the occurrence of the brightness change regions at different time nodes in chronological order, forming a brightness change time record trajectory with spatial location as the index; The time distribution of the brightness change time record trajectory is divided into segments. Time nodes with continuous brightness change relationships are divided into brightness change segments, and intervals without brightness change are marked to obtain a segmented structure. Organize the time interval relationships of each brightness change segment in the segmented structure, group the brightness change segments whose time intervals show a repetitive distribution characteristic, form a set of change segments, and record the corresponding spatial location range; The set of change segments is compiled and arranged in chronological order. The set of change segments that show recurring change characteristics is then selected to form a set of periodic change segments.

5. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 4, characterized in that, The brightness change segments in the set of change segments are arranged in order of time interval, and the brightness change segments with consistent time intervals are grouped together. Brightness change segments with repeated time intervals are grouped into the same set of change segments.

6. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 4, characterized in that, The spatial position changes of the changing segments in a set of periodic changing segments are continuously tracked. The changing processes that move smoothly along a fixed direction are selected, and the directional change features are extracted. The specific steps are as follows: Expand the time nodes within the time range corresponding to each periodic change segment in the set of periodic change segments, extract the spatial coordinates of the brightness change area and arrange them in chronological order to form a spatial position change sequence; Construct spatial displacement segments between adjacent time nodes in a spatial position change sequence, record the position coordinates of the starting time node and the current time node and determine the spatial movement direction to form a set of continuous spatial displacement segments; Compare the directional changes of adjacent spatial displacement segments in a continuous set of spatial displacement segments, merge spatial displacement segments with the same direction into a movement interval and record the time node range, and separate spatial displacement segments with different directional changes. Organize the spatial location change paths within the movement interval and arrange them in chronological order, summarize the path direction expression and extract the direction change features to form a set of direction change features.

7. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 6, characterized in that, The spatial position change paths within the movement interval are organized to ensure directional consistency. Spatial displacement segments with consistent directions are continuously merged, and the merged spatial paths are given a unified directional expression. At the same time, the corresponding change process is recorded in conjunction with the time node range to form a set of directional change features.

8. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 6, characterized in that, A time continuity analysis is performed on the change process corresponding to the directional change characteristics to identify change processes that maintain continuous movement and lack sudden change characteristics, and to determine pseudo-change intervals. The specific steps are as follows: The time range corresponding to each change process in the set of directional change features is expanded, the brightness change information and spatial location information of each time node are extracted and arranged in chronological order to form a continuous time series; Organize the brightness change relationship between adjacent time nodes in a continuous time series, divide the continuous change segment and the abrupt change segment and record the difference in the change amplitude to obtain the segmented time structure; Based on the corresponding segmented time structure and directional change characteristics, the spatial movement path in the directional change characteristics is matched with the continuous change segment, and stable movement segments that maintain consistent direction and continuous spatial position are selected. Summarize stable movement segments and arrange them in chronological order. Merge continuous stable movement segments that are not separated by abrupt change segments and determine the corresponding time range as pseudo-change intervals.

9. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 8, characterized in that, The determination of continuous change segments includes continuous recording of the difference in brightness change amplitude between adjacent time nodes. The selection of stable movement segments includes change processes with consistent spatial movement path direction and continuous time sequence arrangement. Pseudo-change intervals are formed by the combination of time ranges of multiple stable movement segments.

10. The method for identifying building exterior wall defects based on multimodal data fusion according to claim 8, characterized in that, The brightness variation amplitude within the pseudo-variation interval is reconstructed, and the distribution relationship between continuous enhancement segments and interval segments is adjusted to achieve segmented variation expression. This includes the following steps: Expand all time nodes corresponding to the pseudo-change interval and extract the brightness change amplitude. Form a brightness change amplitude sequence in chronological order and record the time node interval relationship. Analyze the amplitude change relationship between adjacent time nodes in the brightness change amplitude sequence, divide the continuously enhanced segment into interval segments, and record the corresponding time node range of each segment; Adjust the arrangement of continuous enhancement segments and interval segments, compress the time nodes in the continuous enhancement segments and allocate the removed time nodes to adjacent interval segments to form a segmented structure; The segments are reorganized and arranged in a new time sequence, with continuously enhanced segments and interval segments alternating to obtain a segmented change sequence.

Images