Camera offset detection method and system based on image segmentation

Abstract

The invention provides a camera offset detection method and system based on image segmentation, and belongs to the technical field of image processing, and the method comprises the steps: inputting a to-be-detected railway image into an image segmentation model, and obtaining a plurality of rail region contours; a target contour is determined from all the rail area contours, and the target contour is divided into a plurality of convex areas in the preset direction when it is judged that concave points exist in the target contour; determining a target area from all the convex areas, and performing straight line fitting on contour points in the target area to obtain a straight line to be detected; and when it is determined that the camera deviates based on a comparison result of the to-be-detected straight line and the reference straight line, alarm information of camera deviation is generated. According to the invention, through image segmentation and contour analysis, the rail boundary is accurately extracted and the camera offset is detected, so that the accuracy of camera offset detection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a camera offset detection method and system based on image segmentation. Background Art

[0002] In the current railway monitoring and image acquisition system, real-time monitoring of rails is particularly important for ensuring railway traffic safety. Cameras are installed at fixed positions along the rails to capture images of the tracks and analyze these images to determine the rail status. However, in actual environments, cameras may deviate due to external forces, long-term use, or vibration, causing the shooting angle to change, thus affecting the accuracy of the image.

[0003] In the prior art, methods based on image feature point matching or template matching are usually used to detect the offset state of the camera. These methods rely on the stability of image features and the consistency of the background. Due to the complex environment along the railway, changes in lighting, weather, and rail edges will cause instability in image feature points. Existing methods are not effective in detecting rail areas and offset conditions in different environments, and are easily affected by noise, making it impossible to effectively identify camera offsets or falsely report offsets.

[0004] Therefore, how to improve the accuracy of camera offset detection has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a camera offset detection method, system, electronic device and storage medium based on image segmentation, which are used to solve the defects in the prior art and improve the accuracy of camera offset detection.

[0006] The present invention provides a camera offset detection method based on image segmentation, comprising the following steps: inputting a railway image to be detected into an image segmentation model to obtain multiple rail area contours; Determining a target contour from all the rail area contours, and when it is determined that there is a concave point in the target contour, dividing the target contour into a plurality of convex areas along a preset direction; Determine a target area from all the convex areas, and perform straight line fitting on contour points in the target area to obtain a straight line to be detected; When it is determined that the camera is offset based on the comparison result between the straight line to be detected and the reference straight line, alarm information of camera offset is generated.

[0007] According to a camera offset detection method based on image segmentation provided by the present invention, the target area is determined from all the convex areas, and contour points in the target area are fitted with straight lines to obtain a straight line to be detected, which specifically includes: Determine the convex region with the largest area from all the convex regions, and use the convex region with the largest area as the target region; Determining the long axis direction of the target area; Performing initial straight line fitting on all contour points of the target area along the long axis direction to obtain an intermediate straight line; According to the middle straight line, all contour points of the target area are divided into two parts, and all contour points in either part are screened to obtain screened contour points; Linear fitting is performed on all the screened contour points to obtain the straight line to be detected.

[0008] According to a camera offset detection method based on image segmentation provided by the present invention, screening all contour points in any part to obtain the screened contour points specifically includes: Determine the coordinates of all contour points in any of the parts; Determine the minimum and maximum ordinates from all ordinates; Determining a range of the ordinate based on the minimum ordinate and the maximum ordinate; Contour points whose ordinates belong to the ordinate range are determined from all the contour points in any part to obtain screened contour points.

[0009] According to a camera offset detection method based on image segmentation provided by the present invention, the camera offset is determined based on the comparison result between the straight line to be detected and the reference straight line, specifically comprising: Calculate the angle between the straight line to be detected and the preset direction to obtain the angle to be detected; Calculate the angle between the reference straight line and the preset direction to obtain a reference angle; Compare the straight line to be detected with the reference straight line to obtain a distance error, and compare the angle to be detected with the reference angle to obtain an angle error; When the distance error is greater than a preset first distance threshold, and / or when the angle error is greater than a preset angle threshold, it is determined that the camera is offset.

[0010] According to a camera offset detection method based on image segmentation provided by the present invention, the determining whether there is a concave point in the target contour specifically includes: Determine the rail area contour with the largest area from all the rail area contours, and use the rail area contour with the largest area as the target contour; Determine a circumscribed convex polygon corresponding to the target contour; Calculate the distance between each point on the target contour and the circumscribed convex polygon; When there are points on the target contour whose spacing exceeds a preset second distance threshold, it is determined that there are concave points in the target contour.

[0011] According to a camera offset detection method based on image segmentation provided by the present invention, the image segmentation model is trained by the following method: Collecting railway image samples under different weather conditions, and annotating the rail area in each railway image sample to construct an annotated data set; Dividing the labeled data set into a training set and a test set; The U-Net model is trained based on the training set to obtain a preliminary segmentation model, and the preliminary segmentation model is tested based on the test set to obtain the pre-trained segmentation model.

[0012] The present invention also provides a camera offset detection system based on image segmentation, comprising the following modules: The first processing module is used to input the railway image to be detected into the image segmentation model to obtain multiple rail area contours; A region division module, used for determining a target contour from all the rail region contours, and when it is determined that there is a concave point in the target contour, dividing the target contour into a plurality of convex regions along a preset direction; A fitting module, used for determining a target area from all the convex areas, and performing straight line fitting on the contour points in the target area to obtain a straight line to be detected; The second processing module is used to generate camera deviation warning information when it is determined that the camera is deviated based on the comparison result between the straight line to be detected and the reference straight line.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a camera offset detection method based on image segmentation as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the camera offset detection method based on image segmentation as described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned camera offset detection methods based on image segmentation.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: By inputting the railway image to be detected into the pre-trained segmentation model, multiple rail area contours are obtained, thereby realizing the automatic segmentation of the rail area in the image, improving the detection efficiency and providing a clear contour data basis. Then, by determining the target contour from all rail area contours and judging whether there are concave points, the analysis is focused on the main rail area with the largest area, and the concave areas that may affect the detection accuracy are identified. Subsequently, when there are concave points in the target contour, the target contour is divided into multiple convex areas along the preset direction, thereby simplifying the complex edge morphology and better reflecting the main structure of the target area, making the subsequent straight line fitting more accurate. By determining the target area with the largest area from all convex areas and performing straight line fitting on the contour points in the area, the straight line to be detected is obtained, thereby extracting the geometric features representing the main boundary of the rail, making the detection benchmark more stable and reliable. Next, based on the reference straight line corresponding to the straight line to be detected and the reference straight line corresponding to the reference railway image, the angle and distance errors are calculated to determine whether the camera is offset, thereby realizing accurate detection of camera position and angle changes, ensuring that the monitoring angle is consistent with the preset rail position. Finally, when the camera deviates, an alarm message of the camera deviation is generated, thereby effectively improving the accuracy of camera deviation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the camera offset detection method based on image segmentation provided by the present invention.

[0019] Figure 2 This is the second flow chart of the camera offset detection method based on image segmentation provided by the present invention.

[0020] Figure 3 This is the third flow chart of the camera offset detection method based on image segmentation provided by the present invention.

[0021] Figure 4 This is the fourth flow chart of the camera offset detection method based on image segmentation provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of a camera offset detection system based on image segmentation provided by the present invention.

[0023] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be noted that, in the description of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0026] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" means at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] Combine the following Figure 1-Figure 6 The present invention describes a camera offset detection method, system, electronic device and storage medium based on image segmentation.

[0028] Figure 1 is one of the flow charts of the camera offset detection method based on image segmentation provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps: Step 1: Input the railway image to be detected into the image segmentation model to obtain multiple rail area contours.

[0029] Specifically, in the embodiment of step 1, the railway image to be detected is first input into the trained segmentation model to obtain the possible rail area contour information in the image. Through this segmentation model, the target area in the railway image can be effectively extracted, making the subsequent area screening and straight line fitting more accurate, thereby improving the accuracy and reliability of the entire detection process. In order to ensure that the segmentation model has good generalization ability and cope with railway images under different weather conditions, lighting conditions and shooting angles, the training data set of the model needs to contain as many samples and scene changes as possible. Based on this, the pre-trained segmentation model is trained by the following method: Collect railway image samples under different weather conditions, annotate the railroad area in each railway image sample, and construct an annotated dataset; Divide the labeled data set into training set and test set; The U-Net model is trained based on the training set to obtain a preliminary segmentation model, and the preliminary segmentation model is tested based on the test set to obtain a pre-trained segmentation model.

[0030] Specifically, first of all, the purpose of collecting image samples is to adapt the segmentation model to a variety of environmental conditions and increase the robustness and generalization ability of the model. By collecting railway image samples under different weather conditions (such as sunny, cloudy, rainy, and foggy) and at different times (such as early morning, noon, evening, and night), the segmentation model can obtain image data in different environments during training, ensuring that it has a high adaptability to environmental changes. After obtaining the sample images, the railroad areas are manually labeled to construct an accurate labeled data set so that the model can identify the edge and shape features of the railroad in subsequent training, thereby improving the segmentation accuracy.

[0031] Next, the labeled dataset is divided into a training set and a test set to ensure that the segmentation model can not only learn the main features of the rail area during the training process, but also be verified through the test set after training to evaluate the generalization ability and accuracy of the model. The specific division ratio is usually 80% training set and 20% test set. This divided dataset structure can provide sufficient training data for the model, and the test set can also be used to evaluate the training effect to ensure that the model has a high segmentation ability on unseen data.

[0032] Then, the U-Net model is trained based on the training set, and finally a pre-trained segmentation model that can accurately segment the rail area is obtained. To adapt to the model input requirements, the original image and the annotated mask are scaled to a specific size of 224×224 and input into the U-Net model for training. During the training process, BCELoss is used as the loss function to measure the accuracy of the model in segmenting the rail area in each iteration. In order to further optimize the training process, the Adam optimization algorithm is used to dynamically adjust the learning rate of the model to accelerate convergence and improve model performance. The training parameters are set to iterate 100 epochs and batch_size is 8. After multiple rounds of iterative training, the model gradually learns the edge and shape features of the rail area, and finally obtains a preliminary segmentation model. After the training is completed, the model is verified by the test set to measure the segmentation effect and stability of the model, and the structure and parameters of the model are appropriately optimized according to the test results.

[0033] Step 2: Determine the target contour from the contours of all rail areas, and when it is determined that there are concave points in the target contour, divide the target contour into a plurality of convex areas along a preset direction.

[0034] In the embodiment of step 2, the focus is on determining the target contour by using multiple rail area contours generated by the segmentation model, and further determining whether there are concave points in the contour. The purpose of this step is to ensure that the subsequent camera offset detection focuses on the rail area with significant boundary features, thereby improving the accuracy and robustness of the detection.

[0035] When there are sunken points, it indicates that the contour contains irregular shapes and large edge fluctuations. In order to further process this shape, the target contour is divided along a preset direction (preferably in the direction parallel to the X-axis in this solution). The division method is based on segmenting the Y-coordinate range of the contour so that each segmented area can form a convex area. The result of this division is to generate multiple convex areas with relatively stable shapes along the Y-axis direction, which no longer contain significant sunken points or complex fluctuations, to ensure that even when the contour structure is complex, it can still focus on areas with relatively simple shapes and suitable for straight line fitting, thereby improving the accuracy and reliability of camera offset detection.

[0036] In one possible implementation, refer to Figure 2 , Figure 2 FIG. 2 is a flow chart of the camera offset detection method based on image segmentation provided by the present invention. Figure 2 As shown, step 2 specifically includes steps 21-24: Step 21: Determine the rail area contour with the largest area from all rail area contours, and use the rail area contour with the largest area as the target contour.

[0037] Specifically, in the process of determining the target contour, the area of ​​all contours generated by the segmentation model is first calculated, and then the contour of the rail area with the largest area is determined. The reason for choosing the rail area with the largest area as the target contour is that, in general, the main area of ​​the rail has a relatively stable boundary and a large coverage area in the camera's perspective, and is not easily affected by environmental factors or changes in shooting angles. Therefore, the contour with the largest area can not only stably present the main features of the rail area, but also can largely avoid misjudgment or noise interference caused by small contours or debris.

[0038] In this process, the effect of determining the contour with the largest area is significant. By focusing on the contour with the largest area, the recognition accuracy of the target contour can be improved in the subsequent detection process, ensuring that operations such as straight line fitting are based on a stable and accurate boundary foundation. In addition, this step also improves the robustness of the detection algorithm, and the detection results will not be easily disturbed even in the presence of partial occlusion or blurred edges.

[0039] Step 22: Determine the circumscribed convex polygon corresponding to the target contour.

[0040] In the embodiment of step 22, by determining the circumscribed convex polygon corresponding to the target contour, the concave-convex features of the target contour can be effectively determined, thereby identifying whether there is a sunken point. The purpose of this step is to provide a standard shape for the subsequent edge analysis, ensuring that the circumscribed convex polygon of the target contour can be used as a reference when determining the sunken point, thereby excluding smaller fluctuations in the shape and focusing on the main morphological features.

[0041] Specifically, the circumscribed convex polygon of the target contour with the largest area is first calculated. The circumscribed convex polygon is the minimum convex hull structure of the target contour, which can contain all contour points and eliminate the concave parts in the contour. In this way, a smooth and standard polygon envelope can be obtained for subsequent distance calculation. Calculating the circumscribed convex polygon of the target contour can not only simplify the analysis of complex edges, but also provide a stable reference shape for subsequent steps, so that even if there are slight bumps in the target contour, it will not interfere with the overall analysis.

[0042] Step 23: Calculate the distance between each point on the target contour and the circumscribed convex polygon.

[0043] In the embodiment of step 23, by calculating the distance between each point on the target contour and the circumscribed convex polygon, the significant concave points that may exist in the target contour can be identified, providing more accurate boundary information for camera offset detection. The core of this step is to use the circumscribed convex polygon as a reference shape and determine whether the target contour has significant concave points that deviate from the convex hull boundary in the local area through distance calculation, so that important morphological features in complex edges can be extracted.

[0044] Specifically, the perpendicular distance from each point on the target contour to the boundary of the circumscribed convex polygon is first calculated, and the distance between these points and the circumscribed convex polygon is recorded. To achieve this, the system constructs a distance map between all points on the target contour and the boundary of the convex polygon to obtain a complete distance distribution.

[0045] Step 24: When there are points on the target contour whose spacing exceeds a preset second distance threshold, it is determined that there are concave points in the target contour.

[0046] In the embodiment of step 24, by judging whether there are points on the target contour that exceed the preset distance threshold, it can be determined whether there are significant concave points in the contour, providing conditional support for subsequent convex area division and straight line fitting operations. The purpose of this step is to identify the part of the target contour that obviously deviates from the convex hull shape, and further refine the contour shape through this judgment to facilitate the subsequent geometric feature extraction and fitting processing.

[0047] Specifically, in the previous step, the distance distribution from each point on the target contour to the circumscribed convex polygon has been calculated. Based on this distance information, the distance value of each point is compared with the preset second distance threshold. When the distance value of some contour points exceeds the threshold, these points are identified as the location of the sunken points. The setting of this threshold is based on the statistical analysis of the rail contour features, ensuring that the contour will only be judged as a sunken point when it deviates significantly from the convex hull shape, and will not be misjudged due to slight boundary fluctuations.

[0048] Step 3: Determine the target area from all convex areas, and perform straight line fitting on the contour points in the target area to obtain the straight line to be detected.

[0049] In the embodiment of step 3, by determining the convex area with the largest area from multiple convex areas and performing straight line fitting on its contour points, a stable and representative straight line to be detected can be obtained, providing an accurate geometric basis for camera offset detection. The purpose of this step is to select the most representative area in the complex rail contour for straight line fitting to ensure that the fitting result can reliably reflect the actual direction of the rail area, thereby improving the accuracy of offset detection.

[0050] In one possible implementation, refer to Figure 3 , Figure 3 FIG. 3 is a flow chart of the camera offset detection method based on image segmentation provided by the present invention. Figure 3 As shown, step 3 specifically includes steps 31-35: Step 31: Determine the convex region with the largest area from all convex regions, and use the convex region with the largest area as the target region.

[0051] Specifically, among the multiple convex areas divided in step 3, the convex area with the largest area usually matches the main structure of the rail most closely. Selecting this area as the target area can better extract the main boundary features of the rail and reduce the offset error caused by the secondary area. By selecting the area size, other small areas such as debris or irregular edges can be filtered out. Although these small areas appear in the rail segmentation results, they are not representative and may cause noise interference.

[0052] In the implementation of step 31, the system calculates the area of ​​all convex regions, compares the area sizes, and finally selects the region with the largest area as the target region. This process ensures that the target region has a strong representativeness and provides a clearer boundary shape for subsequent operations. By focusing on the convex region with the largest area, subsequent straight line fitting can be performed on a stable boundary, reducing the interference of complex edges or messy small areas.

[0053] Step 32: Determine the long axis direction of the target area.

[0054] Specifically, after selecting the convex area with the largest area as the target area in step 31, the major axis direction of the area is calculated in step 32 to determine its main morphological direction. The major axis direction is determined by calculating the distribution range of the contour points in the target area, finding the maximum difference between the X and Y axes, and thus determining the direction of maximum extension. This direction usually corresponds to the actual direction of the rails, so it is highly accurate and representative as a fitting direction. In the process of determining the major axis direction, algorithms such as principal component analysis (PCA) can be used to analyze the distribution characteristics of the contour points to ensure that the calculation results of the direction accurately reflect the geometric morphology of the area. Through this operation, the major axis direction is used as a geometric reference, so that the subsequent straight line fitting can focus on the main extension direction of the target area.

[0055] Step 33: Perform initial straight line fitting on all contour points of the target area along the long axis direction to obtain an intermediate straight line.

[0056] Specifically, after the long axis direction of the target area has been determined in step 32, the system performs a preliminary fitting on the contour points in the target area to generate a middle straight line. by is the independent variable, As the dependent variable, we can construct a straight line equation , where the slope and the intercept The least square method is used to determine the minimum sum of the distances from all points to the straight line. Through this fitting method, the system can obtain an intermediate straight line that fits the overall contour of the area as closely as possible, ensuring that subsequent operations are carried out based on the actual direction of the target area.

[0057] Step 34: Divide all contour points of the target area into two parts according to the middle straight line, and filter all contour points in either part to obtain filtered contour points.

[0058] In the embodiment of step 34, by reasonably dividing all contour points of the target area based on the middle straight line and further screening the core contour points, a representative point set can be effectively extracted to ensure that the subsequent straight line fitting process is more stable and accurate. The purpose of this step is to group and screen the contour points using the middle straight line, focus on the core structure part of the target area, and thus reduce the influence of peripheral interference points on the fitting results.

[0059] In a possible implementation, step 34 specifically includes the following steps: Determine the coordinates of all contour points in any of the parts; Determine the minimum and maximum ordinates from all ordinates; Based on the minimum ordinate and the maximum ordinate, determine the ordinate range; Contour points whose ordinates belong to the range of the ordinate are determined from all the contour points in any part, and the filtered contour points are obtained.

[0060] Specifically, first extract the straight line in the middle All contour points of any part of the two sides (such as the left part or the right part, the left part is preferred in this solution) , these points form the point set to be processed for further screening operations. Next, calculate the distribution range of the ordinates of these point sets, that is, find the maximum ordinate among them and the minimum ordinate , and define the filter range based on this.

[0061] Furthermore, based on the calculated maximum and minimum ordinates, the ordinate range for screening is determined to exclude possible noise points at both ends. Specifically, the screening condition is set to retain points that meet the following ordinate range: ; and remove points outside the range of the ordinate, that is, remove the edge points at both ends. The screening conditions set in this way ensure that only the core area points whose ordinates are outside the upper and lower 10% range are retained, thereby avoiding the extreme values ​​of the endpoints from interfering with the fitting process.

[0062] Finally, based on the above screening conditions, a core contour point set is obtained, and the ordinates of all points are within the set threshold range. These screened points constitute the main boundary features of the target area, eliminating the noise points and deviation points at both ends, and providing accurate data input for subsequent straight line fitting. Through this screening process, the system can focus on the main contour morphological features during fitting, avoiding interference from irregular edges or small undulations.

[0063] Step 35: Perform straight line fitting on all the filtered contour points to obtain the straight line to be detected.

[0064] Specifically, step 34 has already included the core contour points that meet the conditions in the target area. After the screening is completed, the interference points at both ends are eliminated, leaving a set of points with strong representativeness. Next, use these filtered core point sets to perform straight line fitting. is the dependent variable, As the independent variable, the least square method is used to fit a straight line equation , where the slope and the intercept The calculation of is to minimize the sum of the distances from all the screened points to the fitting line. This fitting method can ensure that the final straight line fits the main contour points after screening closely, eliminates the interference of edge or noise points, and makes the fitting result more stable and accurate.

[0065] Through this straight line fitting, the final straight line to be detected is It becomes the geometric reference in the subsequent camera offset detection. This straight line accurately reflects the core direction of the target area and can provide reliable data support for detecting whether the camera is offset. After eliminating the interference points in the fitting process, the straight line fitting is based only on the main boundary features of the target area, which greatly reduces the error and improves the accuracy and robustness of the fitting.

[0066] Step 4: When it is determined that the camera is offset based on the comparison result between the straight line to be detected and the reference straight line, an alarm message of camera offset is generated.

[0067] In the embodiment of step 4, by making an offset judgment based on the straight line to be detected and the reference straight line in the reference railway image, the actual offset of the camera can be accurately evaluated, thereby ensuring the effective monitoring of the rail area by the monitoring system. The purpose of this step is to compare the distance and angle of the straight line to be detected with the reference straight line to identify whether the camera has deviated from the preset position and to judge the degree of offset, thereby ensuring the accuracy of the image captured in the rail area.

[0068] It should be noted that the specific method of obtaining the reference straight line corresponding to the reference railway image is consistent with the specific method of obtaining the straight line to be detected corresponding to the railway image to be detected, that is, it is obtained through the method of steps 1 to 3, so no further details are given here.

[0069] In one possible implementation, refer to Figure 4 , Figure 4 FIG. 4 is a flow chart of the camera offset detection method based on image segmentation provided by the present invention. Figure 4 As shown, step 4 specifically includes steps 41-44: Step 41: Calculate the angle between the straight line to be detected and the preset direction to obtain the angle to be detected.

[0070] Step 42: Calculate the angle between the reference straight line and the preset direction to obtain a reference angle.

[0071] Step 43: Compare the straight line to be detected with the reference straight line to obtain a distance error, and compare the angle to be detected with the reference angle to obtain an angle error.

[0072] Step 44: When the distance error is greater than a preset first distance threshold, and / or when the angle error is greater than a preset angle threshold, it is determined that the camera is offset.

[0073] In the embodiment of steps 41 to 44, by respectively calculating the angle, distance deviation and comparison threshold of the straight line to be detected and the reference straight line, it is possible to accurately determine whether the camera has shifted. The purpose of this series of steps is to use the angle and distance difference between the straight line to be detected and the reference straight line to determine whether the actual shooting angle and position of the camera after installation have shifted, thereby ensuring that the camera's monitoring image of the rail area meets the preset requirements.

[0074] In step 41, firstly based on the equation of the straight line to be detected , calculate the angle between the straight line to be detected and the preset direction (preferably the positive half axis of the X axis in this solution) to obtain the angle to be detected. Specifically, this angle can be obtained by the slope of the straight line to be detected The inverse tangent value of is determined, that is The angle to be detected reflects the actual direction of the rail area in the current image, and is the angle of the rail captured by the camera in the current shooting state.

[0075] Next, in step 42, the angle of the reference straight line, i.e., the reference angle, is calculated, and the preset direction of the rail in the reference image is obtained in the same way. This step calculates the angle by the slope of the reference straight line, i.e., to ensure that the system can use the base angle as a reference to determine the deviation of the current shooting angle.

[0076] In step 43, by comparing the angle and distance between the straight line to be detected and the reference straight line, the angle error between the angle to be detected and the reference angle is calculated, and the distance error between the two straight lines in space is calculated. It reflects the deviation of the camera shooting angle from the reference angle. The distance error is obtained by calculating the vertical distance between two straight lines, which indicates the degree of deviation of the shooting position.

[0077] Finally, in step 44, the calculated angle error is compared with the set angle threshold to determine whether the angle deviation exceeds the set standard. At the same time, the distance error is compared with the first distance threshold to determine the degree of position offset. If the distance error is greater than the first distance threshold, or the angle error is greater than the angle threshold, the system determines that the camera has offset. This judgment result triggers the alarm mechanism and generates an alarm message of camera offset, so as to promptly remind the system to adjust and calibrate.

[0078] Furthermore, in step 44, the system has determined whether the camera is offset by comparing the angle error and distance error between the straight line to be detected and the reference straight line. When the detection result shows that the angle error exceeds the set angle threshold, or the distance error exceeds the first distance threshold, the system determines that the camera has deviated from the preset position or angle. In this case, the system generates an alarm message of camera offset, marking and prompting this offset state. The alarm message usually includes the camera offset type (position offset or angle offset), the specific value of the offset (such as angle deviation and distance deviation), and timestamp information, so that the administrator can accurately understand the camera offset and take corresponding measures.

[0079] The effect of generating alarm information is that it provides the system with an active monitoring and feedback mechanism, ensuring that relevant personnel can be notified immediately when the camera deviates. Through this alarm, maintenance personnel can respond quickly to recalibrate and adjust the camera to restore the accuracy of monitoring. At the same time, the record of alarm information can also be used for future maintenance to help analyze the frequency and possible causes of camera deviation and provide data support for system optimization.

[0080] Reference Figure 5 , Figure 5 : is a structural schematic diagram of a camera offset detection system based on image segmentation provided by the present invention, the system comprises: The first processing module is used to input the railway image to be detected into the image segmentation model to obtain multiple rail area contours; A region division module is used to determine the target contour from all rail region contours, and when it is determined that there are concave points in the target contour, divide the target contour into a plurality of convex regions along a preset direction; A fitting module is used to determine the target area from all convex areas and perform straight line fitting on the contour points in the target area to obtain a straight line to be detected; The second processing module is used to generate camera deviation warning information when it is determined that the camera is deviated based on the comparison result between the straight line to be detected and the reference straight line.

[0081] In a possible implementation, the fitting module is further used for: Determine the convex region with the largest area from all convex regions, and use the convex region with the largest area as the target region; Determine the long axis direction of the target area; Perform initial straight line fitting on all contour points of the target area along the long axis direction to obtain an intermediate straight line; According to the middle straight line, all contour points of the target area are divided into two parts, and all contour points in any one of the parts are screened to obtain screened contour points; Perform straight line fitting on all the filtered contour points to obtain the straight line to be detected.

[0082] In a possible implementation, the fitting module is further used for: Determine the coordinates of all contour points in any of the parts; Determine the minimum and maximum ordinates from all ordinates; Based on the minimum ordinate and the maximum ordinate, determine the ordinate range; Contour points whose ordinates belong to the range of the ordinate are determined from all the contour points in any part, and the filtered contour points are obtained.

[0083] In a possible implementation manner, the second processing module is further configured to: Calculate the angle between the straight line to be detected and the preset direction to obtain the angle to be detected; Calculate the angle between the reference straight line and the preset direction to obtain the reference angle; The straight line to be detected is compared with the reference straight line to obtain the distance error, and the angle to be detected is compared with the reference angle to obtain the angle error; When the distance error is greater than a preset first distance threshold, and / or when the angle error is greater than a preset angle threshold, it is determined that the camera is offset.

[0084] In a possible implementation manner, the region division module is further used to: Determine the rail area contour with the largest area from all rail area contours, and use the rail area contour with the largest area as the target contour; Determine the circumscribed convex polygon corresponding to the target contour; Calculate the distance between each point on the target contour and the circumscribed convex polygon; When there are points on the target contour whose spacing exceeds a preset second distance threshold, it is determined that there are concave points in the target contour.

[0085] In a possible implementation, the system further includes a model training module, which is used to: Collect railway image samples under different weather conditions, annotate the railroad area in each railway image sample, and construct an annotated dataset; Divide the labeled data set into training set and test set; The U-Net model is trained based on the training set to obtain a preliminary segmentation model, and the preliminary segmentation model is tested based on the test set to obtain a pre-trained segmentation model.

[0086] It should be noted that the camera offset detection system based on image segmentation provided by the present invention can execute the camera offset detection method based on image segmentation in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0087] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610 (processor), a communication interface 620 (CommunicationsInterface), a memory 630 (memory) and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the camera offset detection method based on image segmentation, the method comprising: inputting the railway image to be detected into the image segmentation model to obtain multiple rail area contours; determining the target contour from all rail area contours, and when it is determined that there are concave points in the target contour, dividing the target contour into multiple convex areas along a preset direction; determining the target area from all convex areas, and performing straight line fitting on the contour points in the target area to obtain the straight line to be detected; when it is determined that the camera is offset based on the comparison result between the straight line to be detected and the reference straight line, generating a warning message of camera offset.

[0088] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0089] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer can execute the camera offset detection method based on image segmentation provided in the above-mentioned embodiments, the method including: inputting a railway image to be detected into an image segmentation model to obtain multiple rail area contours; determining a target contour from all rail area contours, and when it is determined that there are concave points in the target contour, dividing the target contour into multiple convex areas along a preset direction; determining a target area from all convex areas, and performing straight line fitting on the contour points in the target area to obtain a straight line to be detected; when it is determined that the camera is offset based on the comparison result of the straight line to be detected and the reference straight line, generating an alarm message of camera offset.

[0090] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by the processor 610, is implemented to execute the camera offset detection method based on image segmentation provided in the above-mentioned embodiments, the method comprising: inputting a railway image to be detected into an image segmentation model to obtain a plurality of rail area contours; determining a target contour from all rail area contours, and when it is determined that there are sunken points in the target contour, dividing the target contour into a plurality of convex areas along a preset direction; determining a target area from all convex areas, and performing straight line fitting on the contour points in the target area to obtain a straight line to be detected; and generating a camera offset alarm message when it is determined that the camera is offset based on the comparison result between the straight line to be detected and the reference straight line.

[0091] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0092] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A camera offset detection method based on image segmentation, characterized in that: include: Inputting the railway image to be detected into the image segmentation model to obtain multiple rail area contours; Determining a target contour from all the rail area contours, and when it is determined that there is a concave point in the target contour, dividing the target contour into a plurality of convex areas along a preset direction; Determine a target area from all the convex areas, and perform straight line fitting on contour points in the target area to obtain a straight line to be detected; When it is determined that the camera is offset based on the comparison result between the straight line to be detected and the reference straight line, alarm information of camera offset is generated.

2. The camera offset detection method based on image segmentation according to claim 1, characterized in that: The step of determining a target region from all the convex regions and performing straight line fitting on contour points in the target region to obtain a straight line to be detected specifically includes: Determine the convex region with the largest area from all the convex regions, and use the convex region with the largest area as the target region; Determining the long axis direction of the target area; Performing initial straight line fitting on all contour points of the target area along the long axis direction to obtain an intermediate straight line; According to the middle straight line, all contour points of the target area are divided into two parts, and all contour points in either part are screened to obtain screened contour points; Linear fitting is performed on all the screened contour points to obtain the straight line to be detected.

3. The camera offset detection method based on image segmentation according to claim 2, characterized in that: The screening of all contour points in any part to obtain the screened contour points specifically includes: Determine the coordinates of all contour points in any of the parts; Determine the minimum and maximum ordinates from all ordinates; Determining a range of the ordinate based on the minimum ordinate and the maximum ordinate; Contour points whose ordinates belong to the ordinate range are determined from all the contour points in any part to obtain screened contour points.

4. The camera offset detection method based on image segmentation according to claim 1, characterized in that: The determining that the camera is offset based on the comparison result between the straight line to be detected and the reference straight line specifically includes: Calculate the angle between the straight line to be detected and the preset direction to obtain the angle to be detected; Calculate the angle between the reference straight line and the preset direction to obtain a reference angle; Compare the straight line to be detected with the reference straight line to obtain a distance error, and compare the angle to be detected with the reference angle to obtain an angle error; When the distance error is greater than a preset first distance threshold, and / or when the angle error is greater than a preset angle threshold, it is determined that the camera is offset.

5. The camera offset detection method based on image segmentation according to claim 1, characterized in that: The determining whether there is a concave point in the target contour specifically includes: Determine the rail area contour with the largest area from all the rail area contours, and use the rail area contour with the largest area as the target contour; Determine a circumscribed convex polygon corresponding to the target contour; Calculate the distance between each point on the target contour and the circumscribed convex polygon; When there are points on the target contour whose spacing exceeds a preset second distance threshold, it is determined that there are concave points in the target contour.

6. The camera offset detection method based on image segmentation according to claim 1, characterized in that: The image segmentation model is trained by the following method: Collecting railway image samples under different weather conditions, and annotating the rail area in each railway image sample to construct an annotated data set; Dividing the labeled data set into a training set and a test set; The U-Net model is trained based on the training set to obtain a preliminary segmentation model, and the preliminary segmentation model is tested based on the test set to obtain the pre-trained segmentation model.

7. A camera offset detection system based on image segmentation, characterized in that: include: The first processing module is used to input the railway image to be detected into the image segmentation model to obtain multiple rail area contours; A region division module, used for determining a target contour from all the rail region contours, and when it is determined that there is a concave point in the target contour, dividing the target contour into a plurality of convex regions along a preset direction; A fitting module, used for determining a target area from all the convex areas, and performing straight line fitting on the contour points in the target area to obtain a straight line to be detected; The second processing module is used to generate camera deviation warning information when it is determined that the camera is deviated based on the comparison result between the straight line to be detected and the reference straight line.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the camera offset detection method based on image segmentation is implemented as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the camera offset detection method based on image segmentation as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the camera offset detection method based on image segmentation as claimed in any one of claims 1 to 6 is implemented.

Images