A method for monitoring hazard sources of semi-trailers based on image fusion

Abstract

The present invention relates to the field of intelligent detection technology, specifically a method for monitoring hazard sources of semi-trailers based on image fusion, comprising the following steps: obtaining acceleration and roughness level flow to filter disturbance image frames, aggregating to generate a spliced ​​image set, extracting boundaries to filter dense perspectives, fusing inter-frame features to generate image bands, analyzing boundary curvature and texture changes to locate unstable areas, and outputting hazard source monitoring results. In the present invention, disturbance period image frames are located and filtered by peak co-occurrence positioning, scene coverage is enhanced by combining angle grouping and lens aggregation, key perspective images are extracted based on boundary closure and regional density, inter-frame line segment matching and connectivity models are constructed to achieve high consistency fusion, unstable areas are identified by geometric changes in structural boundaries, and high-variation blocks are located by combining grayscale and texture co-variation trends, thereby achieving accurate positioning of abnormal boundaries and spatial expression of potential risk features, thereby improving the detection integrity and analysis accuracy of disturbance areas.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and in particular to a method for monitoring hazard sources of a semi-trailer based on image fusion. Background Art

[0002] The field of intelligent detection technology involves the use of modern perception technologies, image processing, artificial intelligence, and automated control to identify, locate, track, analyze, and judge target objects. Core aspects include target feature extraction, state recognition, environmental modeling, and intelligent decision-making. It is widely used in scenarios such as industrial production, transportation, and security monitoring. Within this field, multi-source image fusion technology based on machine vision has become a research hotspot, playing a particularly important role in achieving high-precision detection and analysis in dynamic and complex environments. Traditional methods for monitoring hazardous sources on semi-trailers rely on single-camera image recognition to detect potential hazards around the semi-trailer. These methods typically employ inter-frame change analysis to identify moving objects or target detection algorithms to identify specific categories of risky objects. These methods primarily rely on single-view image data acquisition, coupled with object classifiers for processing steps such as static image recognition, region boundary extraction, or feature point matching. They often identify potential hazardous sources through methods such as image differential analysis and edge detection algorithms. However, due to limitations in field of view, image clarity, and lighting variations, these methods have limited robustness and coverage in identifying hazardous sources in real-world road transportation scenarios, making them difficult to meet monitoring requirements in highly dynamic scenarios. Therefore, developing a monitoring method based on image fusion and integrating multiple image information to enhance the comprehensive perception of the environment around the semi-trailer has become an important technical path to improve the level of hazard source identification.

[0003] Using single-view images as input, scene coverage is significantly limited. The ability to judge the persistence of disturbances in inter-frame change analysis is weak. There is a lack of a dynamic correlation extraction mechanism between the event period and the image, which can easily lead to gaps in the identification of continuous risk areas. Edge extraction and target detection rely on the stability of image clarity, and the accuracy decreases under complex background interference. The expression of regional features is limited to the content of a single frame, and it is impossible to effectively construct the correspondence between regional structures in time-series images. When the environment changes drastically or the target state mutates, the result deviation increases, and it is easy to miss the recognition of key areas, disconnect between image frames, and interrupt the risk transmission path, affecting the dynamic understanding of the scene and the ability to model the complete space. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a method for monitoring hazard sources of semi-trailers based on image fusion.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: a method for monitoring hazard sources of semi-trailers based on image fusion, comprising the following steps:

[0006] S1: Obtain the vehicle acceleration sequence and wheel-end laser roughness level stream, extract the peak co-occurrence segment images, determine the continuous disturbance period through the time overlap interval, filter the image frame numbers within the period, aggregate them into image sets by angle and lens source, and generate a stitched image set;

[0007] S2: extracting image frame boundary segments from the stitched image set, determining the number of closed segments, calculating the closed boundary coverage ratio, comparing the ratio with the average ratio of the entire frame, screening boundary-intensive viewing angles, and generating a stitching source viewing angle screening group;

[0008] S3: Filtering a group of image frame sequences based on the stitching source perspective, identifying the start and end coordinates of adjacent frame line segments and matching image center points, calculating the inter-frame translation direction angle and edge connectivity, fusing and correcting misalignment indexes, and generating a continuous image strip;

[0009] S4: detecting the change in the turning angle of each closed edge segment of the vehicle bottom layer in the continuous image strip, screening the change in contour curvature, determining the spatial overlap between the boundary distortion segment and the symmetry axis offset area, and generating an unstable boundary set;

[0010] S5: Obtain the grayscale change blocks of the corresponding spliced ​​image from the unstable boundary set, analyze the texture direction consistency, extract the overlapping position of the edge line density change and the texture level transition area, and generate the hazard source monitoring result.

[0011] As a further solution of the present invention, the stitched image set includes peak co-occurrence area images, grouped image angle labels, and corresponding frame sets of source lenses; the stitched source perspective screening group includes high boundary density perspective numbers, closed boundary coverage ratio data, and image frame boundary closure statistics; the continuous image band includes inter-image frame displacement direction angles, edge connectivity indicators, and layer dislocation correction parameters; the unstable boundary set includes high curvature turning line segments, edge distortion areas, and symmetry axis offset blocks; the hazard source monitoring results include grayscale change blocks, edge line density distribution, and texture level mutation areas.

[0012] As a further solution of the present invention, the continuous disturbance period is defined as the time interval during which the vehicle acceleration sequence and the peak segments in the wheel-end laser roughening flow continuously overlap in time, reflecting that the vehicle is subjected to continuous excitation disturbance;

[0013] The definition of the boundary-dense perspective refers to the target camera perspective in which the coverage ratio of closed boundary segments in the stitched image set is higher than the average level of the whole frame, reflecting the dense distribution of object edge information;

[0014] The definition of the correction misalignment index refers to the process of adjusting and relocating the mismatched and misaligned frame numbers in the frame sequence by analyzing the line segment position relationship and translation direction consistency between image frames.

[0015] As a further solution of the present invention, the specific steps of S1 are:

[0016] S101: Obtain the vehicle acceleration sequence and wheel-end laser roughness level stream, extract the peak points, compare the timestamps of the two, select the peak pairs with high time overlap, record the corresponding start and end times, and generate a disturbance time interval sequence;

[0017] S102: Based on the disturbance time interval sequence, extract the image frame numbers within the interval, obtain corresponding images, group them according to the angle field in the image name, remove images without angle information, arrange them in order of numbers, and generate an angle-grouped image set;

[0018] S103: calling the angle-grouped image set, extracting the lens identifiers of the image frames, grouping the image frames with the same identifier into an image set, and stitching each group of images in numerical order to generate a stitched image set;

[0019] The peak pair with high temporal overlap is defined as a pair of peaks whose timestamp difference between the peak point of the vehicle acceleration sequence and the peak point of the wheel-end laser roughness level flow is within a set threshold and reflects the same disturbance event;

[0020] The definition of the image without angle information refers to an image frame whose image file name does not include angle field identification information for distinguishing the shooting angle of view.

[0021] As a further solution of the present invention, the specific steps of S2 are:

[0022] S201: Calling all image frames in the stitched image set, extracting boundary line segment information in the image, determining whether the start and end pixel positions of the line segment are closed, counting the number of closed line segments and pixel ranges in each frame of the image, calculating the coverage ratio of the closed boundary in the image, and generating a closed boundary coverage result;

[0023] S202: Based on the closed boundary coverage result, construct continuous frame regions according to the image number, summarize the average closed boundary coverage ratio in each group of regions, compare it with the average coverage ratio of the entire frame, filter the covered image number segments, and generate a preferred image segment number set;

[0024] S203: Call the preferred image segment number set, extract the view identification field of the corresponding image frame, count the number and distribution density of the image frames in the view group, determine whether the density is higher than the average density of the entire group, filter the view numbers with more concentrated image frame distribution, and generate a stitching source view filter group.

[0025] As a further solution of the present invention, the calculation formula for the coverage ratio value of the closed boundary in the image is specifically:

[0026] ;

[0027] in, represents the coverage ratio of the closed boundary in the i-th frame image, Pi represents the total number of pixels in the i-th frame image, Represents the number of closed boundary segments in the i-th frame image, Represents the number of pixels in the area enclosed by the jth closed boundary segment in the i-th frame image, and They represent the horizontal coordinate values ​​of the starting point and the ending point of the jth closed boundary segment in the i-th frame image, and Respectively represent the vertical coordinate values ​​of the starting point and ending point of the line segment.

[0028] As a further solution of the present invention, the specific steps of S3 are:

[0029] S301: Calling the image frame sequence in the stitching source view filter group, extracting the start and end coordinate data of the boundary line segments in each pair of adjacent frames, calculating the position offset of the corresponding points of the line segments based on the coordinates of the image center point, and constructing an inter-frame line segment pairing structure to generate an inter-frame line segment matching coordinate difference set;

[0030] S302: identifying the translation direction angles of adjacent image frames based on the inter-frame line segment matching coordinate difference set, extracting the edge coordinate segments between the paired line segments, comparing the grayscale continuity and edge strength of corresponding pixels, calculating the connectivity feature values ​​of the image frames in the edge region, and generating an edge connectivity parameter set;

[0031] S303: Calling the edge connectivity parameter group, sequentially fusing all image frames to construct a whole image layer, locating the layer misalignment area according to the directional angle offset and connectivity change, recording the corresponding position index and adjusting the pixel mapping position to obtain a continuous image band.

[0032] As a further solution of the present invention, the calculation formula of the connectivity feature value of the image frame in the edge area is specifically:

[0033] ;

[0034] in, represents the connectivity feature value of the k-th pair of image frames in the edge region, Represents the number of pixels involved in connectivity calculation in the k-th pair of frames, and Respectively represent the grayscale values ​​of the mth edge pixel in the previous frame and the next frame in the kth pair of image frames, and are the edge strength values ​​of the mth edge pixel in the previous frame and the next frame respectively, and It is the horizontal and vertical coordinate deviation of the m-th pixel at the junction of the previous and next frames.

[0035] As a further solution of the present invention, the specific steps of S4 are:

[0036] S401: calling the vehicle bottom layer image in the continuous image strip, extracting the pixel sequence of each closed boundary line segment in the image, sequentially calculating the angle change values ​​of the line segment turning points, and counting the interval range of the turning angle change to generate a turning angle change result;

[0037] S402: Extracting the contour segments of each boundary line segment within the turning angle change interval based on the turning angle change result, calling the image edge region position range, determining whether the start and end coordinates of the contour segment are within the region, filtering out matching segment numbers, and generating edge curvature distribution segments;

[0038] S403: Calling the edge curvature distribution segment, extracting pixel coverage positions of the coordinate area of ​​the boundary distortion segment and the symmetry axis offset area corresponding to the image, performing an overlapping range judgment operation on the two areas in the same image in spatial position, obtaining a set of overlapping area numbers, and generating an unstable boundary set;

[0039] The definition of the matching segment number refers to the sequence number or number corresponding to the contour segment whose starting and ending coordinates of the turning angle change section in the boundary line segment are within the image edge area and meet the screening conditions;

[0040] The definition of the pixel coverage position refers to a two-dimensional coordinate set defined by the distribution of pixel points within a specified area in the image, which represents the actual occupation range of the target area in the image space.

[0041] As a further solution of the present invention, the specific steps of S5 are:

[0042] S501: Calling each coordinate block in the unstable boundary concentration, extracting the grayscale value distribution of the block in the corresponding area in the stitched image, calculating the direction of the grayscale difference between adjacent pixels in the block, and counting the concentration degree of the key direction, obtaining the ratio of the number of pixels in the main direction in the block, and generating a texture direction concentration result;

[0043] S502: Extracting edge line pixel sets in the image block based on the texture direction concentration result, extracting and sorting the number of edge lines in the unit area by row and column indexes, recording the continuous rising and falling segments in the edge density change sequence curve, and locating the position of the texture direction mutation area to generate the edge density change segment;

[0044] S503: Call the edge density change segment, obtain the texture mutation position coordinates corresponding to each segment, calculate the coverage relationship with the unstable boundary area, determine whether there is continuous space overlap, and record the corresponding area number and position index to generate the hazard source monitoring result;

[0045] The definition of the texture direction mutation area position refers to the pixel area coordinate position where the main texture direction in the image block changes direction in spatial distribution and the grayscale gradient direction changes;

[0046] The definition of continuous spatial overlap refers to a two-dimensional region where the texture mutation position in the edge density variation segment and the unstable boundary region are spatially continuous and overlapped at multiple points.

[0047] Compared with the prior art, the advantages and positive effects of the present invention are:

[0048] In the present invention, image frames of disturbance periods are screened through peak co-occurrence positioning, scene coverage is enhanced by combining angle grouping and lens aggregation, key perspective images are extracted based on boundary closure and regional density, inter-frame line segment matching and connectivity models are constructed to achieve high consistency fusion, unstable areas are identified through geometric changes in structural boundaries, and high-variance blocks are located by combining grayscale and texture co-variation trends, so as to achieve accurate positioning of abnormal boundaries and spatial expression of potential risk characteristics, thereby improving the detection integrity and analysis accuracy of disturbance areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 It is a schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0052] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0053] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0054] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0055] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0056] See also Figure 1 A method for monitoring hazard sources of semi-trailers based on image fusion includes the following steps:

[0057] S1: Obtain the vehicle acceleration sequence and wheel-end laser roughness level stream, extract the co-occurrence segment images of the two peaks, determine the continuous disturbance period based on the time overlap interval of the peak points, filter all image frame numbers within the period and group the images by angle, aggregate the bound frames into image sets according to the source lens, and generate a stitched image set;

[0058] S2: Call all image frames in the stitched image set, extract boundary segments from the image and measure the number of closed segments, calculate the closed boundary coverage ratio in the continuous area and determine whether it is better than the average ratio of the entire frame, select the boundary density perspective group as the starting channel, and generate the stitching source perspective screening group;

[0059] S3: Call the image frame sequence in the stitching source perspective screening group, identify the start and end coordinates of the line segments in adjacent frames and perform corresponding matching based on the image center point, calculate the inter-frame image translation direction angle and edge boundary connectivity, perform full-frame fusion and correct the position index of the layer misalignment to generate a continuous image strip;

[0060] S4: Call the vehicle bottom layer in the continuous image strip to detect the turning angle variation range of each closed edge segment in the image, screen whether the contour curvature variation segment is distributed in the edge area, and perform spatial overlap judgment on the boundary distortion segment and the symmetry axis offset area in the same area to generate an unstable boundary set;

[0061] S5: Call the unstable boundary concentrated coordinate block, obtain the grayscale change block of the corresponding area from the stitched image and analyze the consistency of the texture direction, extract the edge line density change trend within the block and the texture block level transition area and perform overlap positioning judgment to generate the hazard source monitoring results.

[0062] The stitched image set includes peak co-occurrence area images, grouped image angle labels, and corresponding frame sets of source lenses. The stitched source perspective screening group includes high boundary density perspective numbers, closed boundary coverage ratio data, and image frame boundary closure statistics. The continuous image band includes the displacement direction angle between image frames, edge connectivity indicators, and layer dislocation correction parameters. The unstable boundary set includes high curvature turning line segments, edge distortion areas, and symmetry axis offset blocks. The hazard source monitoring results include grayscale change blocks, edge line density distribution, and texture level mutation areas.

[0063] The specific steps of S1 are:

[0064] S101: Obtain the vehicle acceleration sequence and wheel-end laser roughness level stream, extract the peak points, compare the timestamps of the two, select the peak pairs with high time overlap, record the corresponding start and end times, and generate a disturbance time interval sequence;

[0065] Before obtaining the on-board acceleration sequence and wheel-end laser roughness grade flow, the vehicle needs to be equipped with a high-frequency sampling acceleration sensor and a laser device with real-time road scanning capability. The acceleration sensor is installed in the middle of the vehicle body structure to reflect the overall vehicle body vibration as much as possible, and the laser device is set at the position of the wheel close to the contact surface to obtain the surface undulation of the road in real time. When the vehicle passes through the road section, the two devices collect data independently and then import them into the processing module for data sorting. The system first suppresses the noise of the original acceleration sequence, removes the jitter information caused by low-frequency fluctuations and equipment errors, and uses window sliding analysis to judge the change trend. When the acceleration change in the continuous sample value is greater than the set initial reference range, it is judged as an abnormal fluctuation, and the corresponding time point is recorded as the acceleration peak point. For the laser roughness flow, the continuous point difference value calculation is also performed. When the surface undulations of adjacent data points exceed the average state level, it is determined to be a roughness mutation point, and the time of the point is extracted. The system stores all peaks extracted from the two sequences in two lists according to the time dimension, and then matches them pair by pair. The points with the time difference between the two timestamps within a short time window are determined to be valid coincidence points. The time window needs to be set according to the vehicle running speed and sampling interval. For example, under medium-speed driving conditions, the time window can be set to a smaller second-level interval range. When valid coincidence points appear continuously and the interval time between them does not exceed the set threshold, the system classifies these consecutive points as the same disturbance event, and uses the earliest coincidence point time as the interval starting point and the latest coincidence point time as the end point. Finally, a set of disturbance time intervals indicating possible changes in vehicle road feel or abnormal structural response are obtained. These time intervals are recorded to guide image extraction.

[0066] S102: Based on the disturbance time interval sequence, extract the image frame numbers within the interval, obtain the corresponding images, group them according to the angle field in the image name, eliminate images without angle information, arrange them in order of numbers, and generate a set of angle-grouped images;

[0067] After acquiring the disturbance time interval sequence, the system enters the image frame extraction phase. It first calls the image acquisition records corresponding to the vehicle's operation process and associates each image frame with a specific time point using the established image timestamp index table. Then, based on the start and end times of the disturbance interval, it filters all image frame numbers within that time range and loads the corresponding image files into the processing list. Image file names typically contain multiple fields, such as frame number, camera number, and shooting angle. The shooting angle field is an important basis for grouping. The system performs string analysis on the image name to extract the angle value. If the angle field is not detected in the image name, the image is not included in the subsequent processing flow to avoid grouping errors. After extraction, all images containing valid angle fields are grouped according to their angle values. Each angle group contains multiple image files, which are snapshots taken at the same shooting angle during the disturbance period. To ensure consistency in subsequent processing, the system sorts the image files within each angle group by frame number from smallest to largest, forming an ordered image set. The order of images within each set is consistent with the actual time sequence of the vehicle passing through the road section, providing basic data for lens identification extraction and image stitching.

[0068] S103: Calling the angle-grouped image set, extracting the lens identifiers of the image frames, grouping the image frames with the same identifier into an image set, and stitching each group of images in numerical order to generate a stitched image set;

[0069] After completing the angle-grouped image collection, the system needs to further extract the lens identification information of each image to identify which specific camera the image comes from. The lens identification usually exists in the form of a specific string in the image file name, or is recorded in the metadata of the image file. The processing module identifies the grouped images one by one and divides the images with the same lens identification into the same group. Each group of images comes from the same camera device and has similar shooting angles and focal length parameters, which is conducive to the consistency processing of subsequent image stitching. After completing the lens identification grouping, the system then numbers and sorts the images in the group to ensure that the image order is consistent with the vehicle operation. The images are stitched together in a unified manner, for example, all images are arranged horizontally one by one, the image edges are aligned during stitching, and blurred transition processing is performed on the stitching edges to reduce seam marks. The system also detects and adjusts the image size to ensure that all images have consistent size parameters during the stitching process. If there are individual image sizes that are inconsistent, image scaling or cropping operations are performed for unified processing. After the images under all lens groups are stitched together, a group of continuous stitched images divided by angle and with consistent lenses are formed for further image feature extraction or anomaly recognition processes.

[0070] The specific steps of S2 are:

[0071] S201: Call all image frames in the stitched image set, extract boundary line segment information in the image, determine whether the start and end pixel positions of the line segment are closed, count the number of closed line segments and pixel range in each frame of the image, calculate the coverage ratio of the closed boundary in the image, and generate a closed boundary coverage result;

[0072] The calculation formula for the coverage ratio of the closed boundary in the image is:

[0073] ;

[0074] in, represents the coverage ratio of the closed boundary in the i-th frame image, Pi represents the total number of pixels in the i-th frame image, Represents the number of closed boundary segments in the i-th frame image, Represents the number of pixels in the area enclosed by the jth closed boundary segment in the i-th frame image, and They represent the horizontal coordinate values ​​of the starting point and the ending point of the jth closed boundary segment in the i-th frame image, and Respectively represent the vertical coordinate values ​​of the starting point and the ending point of the line segment;

[0075] It represents the coverage ratio of the closed boundary in the i-th frame image, which is an indicator for evaluating the relative proportion of the closed area of ​​the structural contour in the image. Pi represents the total number of pixels in the image. In a standard high-definition image with an image size of 1920×1080 pixels, this value is 2073600. is the number of closed boundary segments in the i-th frame image. After using the boundary detector to identify the image contour segments, the start and end points of all segments are closed. The judgment logic is that the Euclidean distance between the first and last pixels of the segment is less than the set closing tolerance. The tolerance setting value is 3 pixels. The number of segments that meet the closing condition is counted. Here, it is set =3 items;

[0076] It represents the pixel area of ​​the area enclosed by the jth closed boundary line segment in the i-th frame image. The area extraction method is achieved by area filling and pixel counting. The first closed boundary A i1 The enclosed area is 46,000 pixels. The second A i2 The area is 35500 pixels, the third A i3 The area is 25,000 pixels;

[0077] and are the horizontal coordinate values ​​of the starting point and the ending point of the jth line segment, respectively, which are obtained by extracting the first and last position coordinates from the contour coordinate sequence. If A i1 The horizontal coordinate of the starting point of the line segment is 612, and the horizontal coordinate of the ending point is 648, then =612, =648;

[0078] and are the ordinate values ​​of the starting point and the ending point of the jth line segment, respectively. The extraction method is the same as the horizontal coordinate. If the corresponding =300, =285, then the correction value of the first boundary is calculated as:

[0079] ;

[0080] A i1 =46000, the adjusted area is ;

[0081] Similarly, for A i2 calculate, =400, =438, =510, =498, the correction term is calculated as:

[0082] ;

[0083] To A i3 , the x difference is 32, the y difference is 10, and the correction term is:

[0084] ;

[0085] The sum of the areas after adjusting the three boundaries is:

[0086] 45988.38+35489.34+24991.06=106468.78;

[0087] Bring in Calculation formula:

[0088] ;

[0089] The results show that the proportion of closed boundaries in the i-th frame image to the total image pixels is 5.13%, indicating the relative distribution density of closed structures in the entire image. This value will be used in the subsequent steps to screen and cluster the structural density segments.

[0090] Definition of the coverage ratio of the closed boundary in the image:

[0091] The closed boundary coverage ratio in an image is the ratio of the pixel area enclosed by closed boundary segments to the total number of pixels in the entire image. This metric represents the spatial proportion of closed structures in an image and is a measure of the complexity of the image structure. This value is a dimensionless quantity ranging from 0 to 1.

[0092] Calculation formula:

[0093] ;

[0094] Formula operation principle (plain text description):

[0095] The formula is processed and normalized from three aspects:

[0096] Closed area pixel area processing:

[0097] Indicates the number of pixels enclosed by the jth closed boundary segment in the i-th frame image. This value is obtained by counting the pixels in the contour closed area. The unit is number of pixels and it is the basic measurement value of the effective closed area.

[0098] Geometric correction term processing:

[0099] The coordinate differences between the starting and ending points of each closed boundary segment are expressed as | - |with| - |, the two items are multiplied to get an approximate estimate of the pixel area, the square root of which is multiplied by 1 / 2 as the area correction term. Its function is to adjust the area misjudgment that may be caused by closure error. The unit is still pixel.

[0100] Total image pixel normalization processing:

[0101] Pi represents the total number of pixels in the image, and its unit is the number of pixels. When the image size is 1920×1080, its value is 2073600. This term is used as the denominator to normalize the sum of the entire area, so that the final result is is a dimensionless ratio value.

[0102] Parameter dimension normalization description:

[0103] Pi: The unit is the number of pixels, the normalized denominator, used to convert area differences into proportions; : The unit is the number of pixels, the original area area, used to count the pixels in the closed area; - : The unit is pixel, horizontal distance; - : The unit is pixel, vertical distance; approximate estimate of the correction area; : dimensionless, the final output is the normalized ratio value covered by the closed boundary in the image.

[0104] All intermediate calculation items are unified on the pixel scale and finally divided by the total number of pixels Pi in the image to achieve normalization and convert to a standard ratio The higher the ratio, the denser the closed boundaries in the image; if the value is close to 0, it means that the image structure contour is relatively sparse.

[0105] S202: Based on the closed boundary coverage results, construct continuous frame regions according to the image numbers, summarize the average closed boundary coverage ratio in each group of regions, compare it with the average coverage ratio of the entire frame, select the covered image number segments, and generate a preferred image segment number set;

[0106] According to the closed boundary coverage results, the system uses the image number as the index to arrange the coverage ratio values ​​of all image frames in order, and constructs a continuous area mapping curve in which the coverage ratio changes with the image frame. The continuous frame image is then divided into multiple frame segments, and each segment is composed of several image frames with consecutive numbers. For example, the frame numbers from 100 to 120 constitute a frame segment. The system sums the coverage ratio of each frame image in the segment and divides it by the number of frames to obtain the average coverage ratio of the segment. The total coverage ratio of all image frames in the entire group of images and the number of images are then counted to calculate the average coverage ratio of the full-frame image. The average coverage ratio of each frame segment is then compared with the average value of the full frame. If the average coverage ratio of the frame segment is For example, if the coverage ratio is greater than the average coverage ratio of the whole frame multiplied by the set weight threshold, the frame segment is determined to be a preferred image segment. The weight threshold is set to 1.2, which means that it needs to reach more than 120% of the full-frame average to be determined as a preferred segment. This value is set according to the importance of the closed area in the scene recognition task. In the experimental samples, the average coverage ratio is usually between 10% and 20%. In the actual comparison, for example, the frame segment average is 18% and the full-frame average is 14%. Since 18% is greater than 1.2 times of 14%, that is, 16.8%, it is determined that the segment meets the preferred requirements. All image number segments that meet this judgment condition are screened and recorded to form the final preferred image segment number set.

[0107] S203: Calling the preferred image segment number set, extracting the view identification field of the corresponding image frame, counting the number and distribution density of the image frames in the view group, determining whether the density is higher than the average density of the entire group, selecting the view numbers with more concentrated image frame distribution, and generating a stitching source view filter group;

[0108] After calling the preferred image segment number set, the system calls the image file for each image number and reads the view identification field in the image name or metadata. This field exists in the form of an angle value or a specific direction code, such as the view identification is "angle_60" or "front_left". The system groups the image frames with the identification field, counts the number of image frames in each view group, and calculates the difference between the maximum and minimum image numbers in the group as the image number span. The number of image frames is divided by the number span to obtain the distribution density. This value represents the degree of image concentration distribution at a certain view, and then the density of the view group is obtained. The average density is compared with the average density of all view groupings. The average density is obtained by dividing the sum of the number of all view images by the sum of all view number spans. If the distribution density of a certain view group is greater than the set multiple of the average density, for example, it is set to 1.3 times, then it is judged that the image frame distribution of this view group is more concentrated. The multiple threshold is set as the basic reference value for the stability and density requirements of the regional image number distribution. In actual tests, when more than 8 frames of image frames are concentrated in the same view within a span of 10 frame numbers, the view number meets the screening criteria. Finally, all view numbers that meet the above density requirements are recorded to generate a stitching source view screening group.

[0109] The specific steps of S3 are:

[0110] S301: Calling the image frame sequence in the stitching source view filter group, extracting the start and end coordinate data of the boundary line segments in each pair of adjacent frames, calculating the position offset of the corresponding points of the line segments based on the coordinates of the image center point, and constructing an inter-frame line segment pairing structure to generate an inter-frame line segment matching coordinate difference set;

[0111] The image frame sequence in the stitching source perspective screening group is called, and the system extracts the data content of any two adjacent image frames one by one. In each image frame, the boundary of the entire image area is first identified, and the boundary line segments are locked by detecting the grayscale gradient mutation area. Then, the pixel coordinate values ​​of the starting and ending positions of all line segments are extracted. These coordinate values ​​are expressed in the horizontal and vertical coordinates of the image pixel matrix. After the extraction is completed, the system uses the center point of the image as the reference point to calculate the horizontal and vertical distances from the two end points of each boundary line segment to the center point, thereby obtaining the relative position offset of the line segment in the image coordinate system. The offset is given by the absolute value of the difference between the two point coordinates. For example, the coordinates of the center point are (960, 540) and the starting point of the line segment is (1010, 550), the horizontal offset is +50 pixels and the vertical offset is +10 pixels. The system performs the same offset calculation process for all extracted boundary segments in each pair of image frames, and then determines whether there are corresponding segments in the previous and next frames based on the proximity of the starting coordinates. If the pixel distance between the starting and ending points of the two segments is within the set tolerance range, for example, set to within 12 pixels, the pairing is considered successful. The tolerance value is set based on the image resolution and camera angle of view, which can effectively eliminate errors caused by non-structural jitter. The coordinate differences corresponding to all successfully paired segments are recorded, and the pixel differences between the starting and ending points in the horizontal and vertical directions are recorded respectively. These differences constitute the inter-frame segment matching coordinate difference set, which is used to describe the position changes of the same structural boundary in different frames.

[0112] S302: Identifying the translation direction angles of adjacent image frames based on the inter-frame line segment matching coordinate difference set, extracting the edge coordinate segments between the paired line segments, comparing the grayscale continuity and edge strength of the corresponding pixels, calculating the connectivity feature values ​​of the image frames in the edge region, and generating an edge connectivity parameter set;

[0113] The calculation formula of the connectivity eigenvalue of the image frame in the edge area is as follows:

[0114] ;

[0115] in, represents the connectivity feature value of the k-th pair of image frames in the edge region, Represents the number of pixels involved in connectivity calculation in the k-th pair of frames, and Respectively represent the grayscale values ​​of the mth edge pixel in the previous frame and the next frame in the kth pair of image frames, and are the edge strength values ​​of the mth edge pixel in the previous frame and the next frame respectively, and is the horizontal and vertical coordinate deviation of the m-th pixel at the junction of the previous and next frames;

[0116] Parameter source and quantification process:

[0117] Pixel grayscale value and : In an 8-bit grayscale image acquired by the image acquisition system, the pixel grayscale value range is [0, 255]. Boundary pixels are measured in the previous and next frames and their values ​​are obtained.

[0118] Edge strength value 、 :Use Sobel operator to calculate the image gradient amplitude. After the Sobel operator performs convolution on the pixel area, the maximum gradient value can reach , where M=255, m=0, and the maximum value is about 360. Therefore, the gradient intensity value range is [0,360], which is the image edge intensity.

[0119] Coordinate deviation 、 : The coordinate difference after positioning by pixel matching algorithm (such as optical flow or feature matching), calculated in pixels, and the value range is usually [0,5] pixels.

[0120] Number of pixels It is the total number of valid pixels in the edge area, which is counted by the connected domain extraction algorithm. A typical area contains 50 to 200 pixels.

[0121] Parameter setting basis and fluctuation items:

[0122] The Sobel threshold does not have a weighting factor set; its intensity is measured by the actual gradient size. This can vary with image clarity and noise level.

[0123] Coordinate deviation does not introduce weight coefficients because deformation is difficult to preset.

[0124] Calculation process:

[0125] Assume that a pair of frame edge regions are identified The following data is obtained by monitoring pixels (taking the average value as an example):

[0126] Average grayscale: previous frame , after frame ⇒ ;

[0127] Average edge strength: previous frame , after frame ⇒ ;

[0128] Mean coordinate deviation: , .

[0129] Substitute the values ​​into the refinement:

[0130] Total difference sum and calculation:

[0131] ;

[0132] Sum of coordinate differences:

[0133] ;

[0134] Denominator calculation:

[0135] ;

[0136] Value calculation:

[0137] ;

[0138] [[ID=!29]]Explanation of the intermediate value: [[ID=3!1]]

[0139] The numerator is the grayscale of the adjacent edge area + the total change in edge strength, representing the change intensity of the image content.

[0140] The denominator is composed of the weighted sum of the number of pixels and the coordinate offset. The larger the offset, the weaker the connectivity determination. This item reflects the looseness of the regional connectivity.

[0141] 21.06 represents the overall connectivity characteristic value of the adjacent edge area of this frame.

[0142] Analysis of the meaning represented by the numerical result:

[0143] This numerical value indicates that in this pair of images, on average, each pixel contributes approximately 21.06 unit changes in (grayscale + edge), divided by the equivalent number of pixels after connectivity correction. A higher value means that the edge area changes more strongly between frames and the connectivity is lower; on the contrary, a lower value means weaker changes and better adjacent edge connectivity.

[0144] Application of connectivity evaluation: [[ID=5!0]]

[0145] This can be used to judge whether the adjacent edge is continuous, and the threshold can be set according to historical data. If <T_low≈10, it is determined that the connectivity is good. If >T_high≈30, it is determined that there is a missing edge or occlusion.

[0146] Association between the result and the steps:

[0147] The total coordinate deviation of 350 in step 2 is used for the denominator, further affecting the value; the total change of 2500 obtained in step 1 affects the numerator; step 3 combines the two to obtain , which is used to evaluate the adjacent edge connectivity characteristics.

[0148] Summary:

[0149] The formula quantifies inter-frame edge connectivity by combining grayscale difference, edge intensity difference, and coordinate offset. The selected parameter values ​​are derived from the acquisition and calculation process, without any assumptions. The deviation and grayscale gradient are based on actual image processing results; coordinate movement is monitored; and the offset range and gradient amplitude are derived from actual image processing devices and algorithm outputs.

[0150] Parameter dimension normalization method:

[0151] To ensure comparability between different physical quantity units and eliminate dimensionality effects, the following parameters need to be normalized in the connectivity eigenvalue formula:

[0152] Gray value difference ;

[0153] Dimension: pure number (unit range is 0 to 255);

[0154] Normalization method: Use 255 as the normalization factor and normalize it to the range [0,1];

[0155] Normalization formula:

[0156] ;

[0157] Edge strength difference term ;

[0158] Dimension: pure number (unit range 0 to 360, based on the upper limit of Sobel gradient theory);

[0159] Normalization method: Use 360 ​​as the normalization factor and normalize it to the interval [0,1];

[0160] Normalization formula:

[0161] ;

[0162] Coordinate deviation term ;

[0163] Dimension: Pixel;

[0164] Normalization method: Considering the maximum possible displacement value is set to 5 pixels (the maximum deviation is measured by optical flow tracking and other methods), 5 is used as the standard for normalization;

[0165] Normalization formula:

[0166] ;

[0167] Note: The denominator is 10 because the maximum sum of the two terms is 10 (5+5);

[0168] Number of participating pixels ;

[0169] Dimension: number of pixels (count);

[0170] Normalization processing method: No normalization, as an amplification factor of the overall distribution range, retaining its absolute value to express the actual meaning of the size of the connected area;

[0171] The connectivity eigenvalue of the image frame in the edge area is defined as:

[0172] The connectivity feature value of the image frame in the edge area (denoted as ) is a quantitative indicator to measure the structural consistency and matching accuracy of two consecutive frames in the seam or overlapping area in an image sequence.

[0173] This metric comprehensively considers grayscale consistency, edge gradient changes, and the spatial consistency of edge pixels, reflecting the continuity of image content between frames. Smaller values ​​indicate less change in the edge region between consecutive frames, indicating better connectivity. Larger values ​​indicate discontinuity or drastic changes in the region, indicating issues such as occlusion, displacement, or texture loss.

[0174] Operation principle:

[0175] Calculation of difference items;

[0176] In each pair of consecutive image frames, the pixels in the edge region are first extracted. For each edge pixel, the absolute difference between its grayscale value and the edge intensity between frames is calculated.

[0177] Position change assessment;

[0178] The spatial coordinate deviation of the corresponding pixel in the previous and next frames is further calculated, and the difference in the horizontal and vertical directions is taken respectively.

[0179] synthetic molecules;

[0180] The grayscale difference and edge intensity difference of all adjacent pixels are summed to form a cumulative change intensity value, which reflects the total amount of visual difference between frames.

[0181] composite denominator;

[0182] The total number of edge pixels and the coordinate deviation of all pixels are added together to form a structural adjustment term, which is used to represent the displacement or alignment degree between frames.

[0183] Eigenvalue generation;

[0184] The total amount of change intensity is used as the numerator, and the function of position adjustment and regional scale is used as the denominator. The two are divided to form the connectivity eigenvalue The overall expression is:

[0185] Total change degree ÷ area complexity and deformation degree adjustment items;

[0186] Structurally, this formula has the ability to discern the degree of change in image regions between frames, producing lower eigenvalues ​​in regions that maintain consistency and significantly higher results in regions with breaks, occlusions, or sudden changes in illumination.

[0187] S303: Calling the edge connectivity parameter group, sequentially fusing all image frames to construct a whole image layer, locating the layer misalignment area based on the angular offset and connectivity change, recording the corresponding position index and adjusting the pixel mapping position to obtain a continuous image band;

[0188] Call the edge connectivity parameter group, the system loads all image frames in sequence according to the frame number, and stitches them together in sequence in the unified layer framework to build the entire image layer. During the stitching process, each time a frame of image is loaded, it is necessary to refer to the translation direction angle and connectivity value of the previous frame. If the angle offset between the direction angle and the previous frame exceeds the set angle difference threshold, for example, it is set to 15 degrees, or the connectivity value decreases by more than the set ratio threshold compared to the previous frame, such as the decrease ratio is greater than 0.25, it is considered that the current image frame has a risk of dislocation during the stitching process, and the system records the position index of this image frame in the layer and Marked as a misaligned area, after discovering the misaligned area, the system traces back the boundary line segment position of the previous frame and the next frame image, and readjusts the splicing starting pixel position of the current frame to align it with the boundary line segment of the previous frame. The mapping position adjustment operation is performed based on the matching coordinate difference of the previous and next frames, and the overall image of the current frame is corrected for horizontal and vertical displacement. The displacement amount is equal to the average amount of the difference between the previous and next frame segments. After the correction, the connectivity value is judged again. If it is increased to more than 90% of the connectivity value of the previous frame, it is confirmed that the correction is effective. After completing the sequential splicing and position adjustment of all frames, the system obtains a complete and continuous image band.

[0189] The specific steps of S4 are:

[0190] S401: Calling the vehicle bottom layer image in the continuous image strip, extracting the pixel sequence of each closed boundary line segment in the image, sequentially calculating the angle change value of the line segment turning point, and counting the range of the turning angle change to generate the turning angle change result;

[0191] The vehicle bottom layer image in the continuous image band is called, and the system extracts closed boundary line segments one by one for each image. Each line segment obtains the set of continuous pixel points that it constitutes through edge detection, and arranges these points in the order of image pixel positions to form a pixel sequence. On this basis, the system performs angle analysis operations, and obtains the angle change values ​​of continuous turning points by calculating the angle between each pixel point and its two adjacent pixel points. The angle calculation is based on two vectors composed of three points, and the angle change value range is recorded as the actual angle value. The system traverses all turning points in the entire line segment, outputs all angle change values ​​in order to form an angle change list, and then lists them. The angle values ​​in the table are divided into intervals, and the angle change range is divided into four intervals: 0 to 30 degrees, 30 to 60 degrees, 60 to 90 degrees, and greater than 90 degrees. The division standard is set according to the actual image resolution and the possible bending degree of the structural contour. For example, if there are multiple turning points with an angle change greater than 60 degrees in a boundary segment, the segment can be considered to have obvious contour structural features. The system records the number of turning points corresponding to each angle interval and marks their position in the entire segment. At the same time, the angle change range of all closed boundary segments is summarized to form the turning angle change structural data of all closed boundaries in the image, and finally the turning angle change results are summarized.

[0192] S402: Based on the turning angle change result, the contour segments of each boundary line segment within the turning angle change range are extracted, the image edge region position range is called, and the start and end coordinates of the contour segments are determined to be within the region. The matching segment numbers are screened out to generate edge curvature distribution segments.

[0193] Based on the turning angle change results, the system extracts turning segments from each closed boundary line segment whose angle changes fall within a specified interval. The specific operation is to screen out contour segments composed of multiple consecutive angle change points. In this contour segment, the boundary continuity between the start and end points meets the length condition. For example, the system sets the minimum length of the contour segment to 15 pixels to avoid interference from short noise segments. The system then calls the image edge region position range. This region is predefined to form a rectangular area band near the image boundary. Its width is set according to the total image width. For example, if the image width is 1920 pixels, the edge band width is set to 60 pixels, that is, 60 pixels on each side of the image constitute the edge region. The system determines whether the start and end points of each contour segment are simultaneously located within the edge region. The judgment criterion is whether the horizontal or vertical coordinates of the two endpoints meet the pixel position condition within the boundary interval. If the condition is met, the contour segment is determined to be at the image edge. The contour segments that meet the condition are numbered and recorded. All numbers are summarized to form the edge curvature distribution segment. This distribution segment reflects the contour area with obvious curvature features in the vehicle bottom image and located at the image boundary, which can be used as an input for subsequent structural stability analysis.

[0194] S403: Calling the edge curvature distribution segment, extracting the pixel coverage position of the coordinate area of ​​the corresponding boundary distortion segment and the symmetry axis offset area in the image, performing an overlapping range judgment operation on the two areas in the same image in spatial position, obtaining a set of overlapping area numbers, and generating an unstable boundary set;

[0195] The edge curvature distribution segment is called, and the system locates the specific coordinate position of each contour segment in the image and represents it as a rectangular bounding box area. This area covers all the pixels of the contour segment and is recorded as the coordinate area of ​​the boundary distortion segment. At the same time, the system calls the pixel coverage position information of the symmetry axis offset area. The symmetry axis is predetermined on both sides of the central axis of the image by analyzing the pixel density change and the contour symmetry characteristics. The offset area is a rectangular area within a certain pixel range on both sides of the symmetry axis. The pixel range is proportionally set according to the image size. For example, if the image width is 1920 pixels, the offset area width is set to 200 pixels, that is, on the central axis. An evaluation zone is formed within 100 pixels on both sides of the line. The system will make an overlap judgment on the rectangular coordinate frame corresponding to the boundary distortion segment and the symmetry axis offset area in the image space. The judgment process is carried out by comparing the horizontal and vertical coordinates of the two rectangular areas. If there is an overlapping area between the two rectangular areas in the horizontal or vertical direction, it is determined that there is a spatial overlap relationship. The system further determines whether the overlapping area is greater than the set area threshold, which is set to 20% of the contour segment area. If the overlapping area exceeds this ratio, the boundary segment number is marked as an unstable area number. All numbers that meet the overlapping area conditions are extracted and sorted one by one to finally generate an unstable boundary set.

[0196] The specific steps of S5 are:

[0197] S501: Call each coordinate block in the unstable boundary concentration, extract the grayscale value distribution of the block in the corresponding area in the stitched image, calculate the direction of the grayscale difference between adjacent pixels in the block, and count the concentration degree of the key direction, obtain the ratio of the number of pixels in the main direction in the block, and generate the texture direction concentration result;

[0198] Call each coordinate block in the unstable boundary set, the system locates the block area corresponding to the stitched image one by one, extracts the grayscale value data of all pixels in the area, and the grayscale value extraction range covers all pixels in the block and constructs a two-dimensional grayscale matrix in row and column order. Then the system calculates the grayscale difference of each pair of adjacent pixels in the block, takes the absolute value of the difference, and determines the direction of grayscale change. It counts the horizontal, vertical, diagonal and other directions separately, and counts the number of pixel pairs with grayscale changes in each direction. Then, it calculates the ratio of pixel pairs in each direction to the total change pairs. The ratio reflects the direction of grayscale difference change. The system uses the number of pixels corresponding to the direction with the highest proportion in all directions as the number of pixels in the main direction, and records the total number of pixels in the block at the same time. The system divides the number of pixels in the main direction by the total number of all grayscale change pixel pairs in the block to obtain the ratio of the number of pixels in the main direction. The ratio ranges from 0 to 1, reflecting the concentration trend of the grayscale texture of the block in a certain direction. For example, in a certain block, the total number of grayscale change pixel pairs is 2500 pairs, of which 1700 pairs are in the horizontal change direction, and the ratio is 0.68. The system summarizes the ratios of all blocks to obtain the texture direction concentration result.

[0199] S502: Based on the texture direction concentration result, edge line pixel sets in the image block are extracted, the number of edge lines in the unit area is extracted and sorted by row and column indexes, the continuous rising and falling segments in the edge density change sequence curve are recorded, and the position of the texture direction mutation area is located to generate the edge density change segment;

[0200] According to the texture direction concentration result, the system locates the area block with the most concentrated grayscale texture direction in each block area, and further extracts the edge line pixel set of the corresponding image. The edge line pixels are composed of image grayscale mutation points. The system scans the two-dimensional image matrix according to row and column indexes, and forms unit area blocks every certain number of pixels. For example, every 10×10 pixels is defined as a unit area block, and the number of edge line pixels in each area block is counted, and the number is used as the edge density value of the area. Then, the edge density values ​​of all unit areas are sorted in row and column order to construct a one-dimensional edge density change sequence. Each item in the sequence corresponds to the number of edge lines in a unit area. The system performs increasing and decreasing trend analysis on the sequence, marks the starting and ending positions of all continuous rising segments and continuous falling segments, and the judgment standard is adjacent areas. Whether the direction of change of the edge density value of the domain is consistent and continues to reach more than three areas. For example, in the unit areas numbered A1 to A5, if the edge density values ​​are 12, 15, 18, 22, and 20, then A1 to A4 is a continuous rising segment, and A4 to A5 is a falling segment. The system records the numbers and start and end area indexes of these segments respectively, and extracts the position of the block where the main direction suddenly changes from the texture direction concentration result, that is, the area where the difference in the main direction ratio between adjacent blocks is greater than the set threshold. The threshold can be set to 0.25, which means that the main direction change degree reaches more than 25% before it is judged as a texture mutation. The system locates the number of such mutation blocks, combines the numbers of the continuously changing edge density segments, and cross-compares them to screen out the segments where the block numbers and the unit area density change trends appear synchronously, and finally generates the edge density change segments.

[0201] S503: Call the edge density change segment, obtain the coordinates of the texture mutation position corresponding to each interval, calculate the coverage relationship with the unstable boundary area, determine whether there is continuous space overlap, and record the corresponding area number and position index to generate the hazard source monitoring result;

[0202] The system calls the edge density change segment and obtains the coordinate information of the texture mutation position in each edge density change segment segment by segment. The image coordinate area corresponding to each segment is represented by a rectangle, and its coverage is calculated from the pixel range of the unit area. For example, a unit area numbered B7 has a pixel range from 700 to 800 on the horizontal coordinate and 400 to 500 on the vertical coordinate. The system uses this as a basis to generate the spatial position coordinates of the corresponding texture mutation. The system then overlaps the coordinates of this area with each coordinate area in the unstable boundary area. By comparing the start and end pixel values ​​of the two areas in the horizontal and vertical directions, it determines whether there is a pixel coverage relationship that crosses between the areas. When the two areas cross in both the horizontal and vertical directions, it is determined that there is spatial overlap. The system further determines whether this spatial overlap occurs continuously between multiple adjacent areas. For example, if three consecutive numbered edge density change segments overlap with the unstable boundary segment and the overlapping areas are non-zero, it is determined to be a continuous spatial overlap relationship. The system records the area numbers that meet the conditions and marks the position index on the original image. Finally, the block numbers corresponding to these continuous spatial overlaps and their actual position indexes in the image are compiled into a table to generate the hazard source monitoring results.

[0203] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for monitoring hazard sources of semi-trailers based on image fusion, characterized in that: The following steps are involved: S1: Obtain the vehicle acceleration sequence and wheel-end laser roughness level stream, extract the peak co-occurrence segment images, determine the continuous disturbance period through the time overlap interval, filter the image frame numbers within the period, aggregate them into image sets by angle and lens source, and generate a stitched image set; S2: extracting image frame boundary segments from the stitched image set, determining the number of closed segments, calculating the closed boundary coverage ratio, comparing the ratio with the average ratio of the entire frame, screening boundary-intensive viewing angles, and generating a stitching source viewing angle screening group; S3: Filtering a sequence of image frames based on the stitching source perspective, identifying the start and end coordinates of line segments in adjacent frames and matching the image center points, calculating the inter-frame translation direction angle and edge connectivity, fusing and correcting the misalignment indexes, and generating a continuous image strip. The edge connectivity represents the inter-frame variability of the edge region. Correcting the misalignment indexes refers to the process of adjusting and relocating the mismatched and misaligned frame numbers in the frame sequence by analyzing the inter-frame line segment position relationship and translation direction consistency. S4: detecting the change in the turning angle of each closed edge segment of the vehicle bottom layer in the continuous image strip, screening the change in contour curvature, determining the spatial overlap between the boundary distortion segment and the symmetry axis offset area, and generating an unstable boundary set, including: S402: Based on the change in turning angle, extract the contour segments of each boundary line segment within the turning angle change interval, call the image edge region position range, determine whether the start and end coordinates of the contour segments are within the region, filter out the numbers corresponding to the contour segments whose start and end coordinates of the turning angle change segments are within the image edge region and meet the screening conditions, and generate edge curvature distribution segments; S403: Calling the edge curvature distribution segment, extracting pixel coverage positions of the coordinate area of ​​the boundary distortion segment and the symmetry axis offset area corresponding to the image, performing an overlapping range judgment operation on the two areas in the same image in spatial position, obtaining a set of overlapping area numbers, and generating an unstable boundary set; S5: Obtain the grayscale change blocks of the corresponding spliced ​​image from the unstable boundary set, analyze the texture direction consistency, extract the overlap position of the edge line density change and the texture level mutation area, and generate the hazard source monitoring results, including: S502: Extract edge line pixel sets in the image block based on texture direction consistency, extract and sort the number of edge lines in the unit area by row and column index, record the continuous rising and falling segments in the edge density change sequence curve, and locate the position of the texture direction mutation area to generate the edge density change segment; S503: Call the edge density change segment, obtain the texture mutation position coordinates corresponding to each segment, calculate the coverage relationship with the unstable boundary area, determine whether there is continuous space overlap, and record the corresponding area number and position index to generate the hazard source monitoring result; The continuous spatial overlap refers to a two-dimensional region in which the texture mutation position in the edge density variation section and the unstable boundary region are continuously and spatially overlapped at multiple points.

2. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1 is characterized in that: The stitched image set includes peak co-occurrence area images, grouped image angle labels, and corresponding frame sets of source lenses; the stitched source perspective screening group includes high boundary density perspective numbers, closed boundary coverage ratio data, and image frame boundary closure statistics; the continuous image band includes the displacement direction angle between image frames, edge connectivity indicators, and layer dislocation correction parameters; the unstable boundary set includes high curvature turning line segments, edge distortion areas, and symmetry axis offset blocks; the hazard source monitoring results include grayscale change blocks, edge line density distribution, and texture level mutation areas.

3. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1 is characterized in that: The continuous disturbance period refers to the time interval in which the vehicle acceleration sequence and the peak segments in the wheel-end laser roughness flow continuously overlap in time, reflecting that the vehicle is subjected to continuous excitation disturbance; The boundary-dense viewing angle refers to a target camera viewing angle in which the coverage ratio of closed boundary line segments in the stitched image set is higher than the average level of the entire frame, reflecting the dense distribution of object edge information.

4. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1 is characterized in that: The specific steps of S1 are: S101: Obtain the vehicle acceleration sequence and wheel-end laser roughness level stream, extract the peak points, compare the timestamps of the two, select the peak pairs with high time overlap, record the corresponding start and end times, and generate a disturbance time interval sequence; S102: Based on the disturbance time interval sequence, extract the image frame numbers within the interval, obtain corresponding images, group them according to the angle field in the image name, remove images without angle information, arrange them in order of numbers, and generate an angle-grouped image set; S103: calling the angle-grouped image set, extracting the lens identifiers of the image frames, grouping the image frames with the same identifier into an image set, and stitching each group of images in numerical order to generate a stitched image set; The peak pairs with high temporal overlap refer to the peak pairs whose timestamp differences between the peak points of the vehicle acceleration sequence and the peak points of the wheel-end laser roughness level flow are within a set threshold and reflect the same disturbance event; The image without angle information refers to an image frame whose image file name does not include angle field identification information for distinguishing shooting angles.

5. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1 is characterized in that: The specific steps of S2 are: S201: Calling all image frames in the stitched image set, extracting boundary line segment information in the image, determining whether the start and end pixel positions of the line segment are closed, counting the number of closed line segments and pixel ranges in each frame of the image, calculating the coverage ratio of the closed boundary in the image, and generating a closed boundary coverage result; S202: Based on the closed boundary coverage result, construct continuous frame regions according to the image number, summarize the average closed boundary coverage ratio in each group of regions, compare it with the average coverage ratio of the entire frame, filter the covered image number segments, and generate a preferred image segment number set; S203: Call the preferred image segment number set, extract the view identification field of the corresponding image frame, count the number and distribution density of the image frames in the view group, determine whether the density is higher than the average density of the entire group, filter the view numbers with more concentrated image frame distribution, and generate a stitching source view filter group.

6. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 5 is characterized in that: The calculation formula for the coverage ratio of the closed boundary in the image is specifically: ； in, represents the coverage ratio of the closed boundary in the i-th frame image, Pi represents the total number of pixels in the i-th frame image, Represents the number of closed boundary segments in the i-th frame image, Represents the number of pixels in the area enclosed by the jth closed boundary segment in the i-th frame image, and They represent the horizontal coordinate values ​​of the starting point and the ending point of the jth closed boundary segment in the i-th frame image, and Respectively represent the vertical coordinate values ​​of the starting point and ending point of the line segment.

7. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1 is characterized in that: The specific steps of S3 are: S301: Calling the image frame sequence in the stitching source view filter group, extracting the start and end coordinate data of the boundary line segments in each pair of adjacent frames, calculating the position offset of the corresponding points of the line segments based on the coordinates of the image center point, and constructing an inter-frame line segment pairing structure to generate an inter-frame line segment matching coordinate difference set; S302: identifying the translation direction angles of adjacent image frames based on the inter-frame line segment matching coordinate difference set, extracting the edge coordinate segments between the paired line segments, comparing the grayscale continuity and edge strength of corresponding pixels, calculating the connectivity feature values ​​of the image frames in the edge region, and generating an edge connectivity parameter set; S303: Calling the edge connectivity parameter group, sequentially fusing all image frames to construct a whole image layer, locating the layer misalignment area according to the directional angle offset and connectivity change, recording the corresponding position index and adjusting the pixel mapping position to obtain a continuous image band.

8. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 7 is characterized in that: The calculation formula of the connectivity feature value of the image frame in the edge area is specifically: ； in, represents the connectivity feature value of the k-th pair of image frames in the edge region, Represents the number of pixels involved in connectivity calculation in the k-th pair of frames, and Respectively represent the grayscale values ​​of the mth edge pixel in the previous frame and the next frame in the kth pair of image frames, and are the edge strength values ​​of the mth edge pixel in the previous frame and the next frame respectively, and It is the horizontal and vertical coordinate deviation of the m-th pixel at the junction of the previous and next frames.

9. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1, characterized in that: The S4 also includes: S401: calling the vehicle bottom layer image in the continuous image strip, extracting the pixel sequence of each closed boundary line segment in the image, sequentially calculating the angle change values ​​of the line segment turning points, and counting the interval range of the turning angle change to generate a turning angle change result; The pixel coverage position refers to a two-dimensional coordinate set defined by the distribution of pixel points in a specified area in the image, which represents the actual occupation range of the target area in the image space.

10. The method for monitoring hazard sources of semi-trailers based on image fusion according to claim 1, characterized in that: The S5 also includes: S501: Calling each coordinate block in the unstable boundary concentration, extracting the grayscale value distribution of the block in the corresponding area in the stitched image, calculating the direction of the grayscale difference between adjacent pixels in the block, and counting the concentration degree of the key direction, obtaining the ratio of the number of pixels in the main direction in the block, and generating a texture direction concentration result; The position of the texture direction mutation area refers to the coordinate position of the pixel area where the main direction of the texture in the block undergoes a direction jump in spatial distribution and the grayscale gradient direction undergoes a mutation.

Images