A semantic segmentation-based wide-angle image intelligent segmentation and correction method and system

Abstract

The application discloses a wide-angle image intelligent segmentation and correction method and system based on semantic segmentation, and relates to the technical field of image processing. The method comprises the following steps: acquiring a wide-angle image to be processed; performing semantic segmentation on the wide-angle image to be processed to obtain an image segmentation result of a target region and a background region, performing morphological shrinkage processing on the target region, and obtaining a path search segmentation result; searching for a target avoidance segmentation path based on the distribution relationship between the target region and the background region; dividing the regions according to the target avoidance segmentation path, performing fisheye correction on a distortion correction region, keeping the original image state or performing light correction on a low-distortion region, and obtaining a regionally processed image; and respectively identifying and counting each region to generate a target statistical result. Thus, the influence of a segmentation boundary passing through a target and edge distortion on counting can be reduced, and the accuracy and scene adaptability of wide-angle image target statistics are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for intelligent segmentation and correction of wide-angle images based on semantic segmentation. Background Technology

[0002] In agricultural livestock management, the number of livestock is a crucial foundation for daily production management, inventory verification, feeding planning, and monitoring of abnormal conditions. Taking pigsty management as an example, traditional manual inventory methods typically rely on personnel entering the farming area for observation and counting. This is not only inefficient but also susceptible to errors due to factors such as pig obstruction, aggregation, movement, and ambient lighting conditions. With the development of image processing and computer vision technologies, using cameras to capture images of the farming area and automatically performing target detection, recognition, and counting based on image recognition methods has gradually become an important technological means to improve the automation level of livestock management. Wide-angle or fisheye cameras, in particular, have a large field of view, allowing them to cover a large farming area with a smaller number of cameras. Therefore, they have high application value in scenarios with limited space and densely distributed targets, such as pigsties and poultry houses.

[0003] However, while wide-angle or fisheye cameras expand the field of view, they typically introduce significant image distortion, especially at the image edges. The contours, scale, and shape of targets are easily stretched or compressed, affecting the accuracy of subsequent target recognition and counting. Existing techniques employ two main approaches: one is to perform global fisheye correction on the entire wide-angle image. However, this requires uniform mapping of the entire image, resulting in significant computational overhead. Furthermore, the distortion in the central image region is relatively small, making overall correction prone to unnecessary processing costs. Another approach divides the image into different regions using fixed-region segmentation or simple geometric rules before processing. However, this method often fails to adequately consider the actual distribution of targets in the image. Segmentation boundaries may cross over pigs waiting to be identified, causing the same target to be segmented into different regions, leading to duplicate counting, missed detections, or decreased recognition accuracy. Simultaneously, the number, location, and density of targets vary significantly across different farming scenarios, making fixed segmentation methods unsuitable for the image processing needs of different scenarios.

[0004] Therefore, in wide-angle image target recognition and counting, obvious distortion in image edge regions, easy passage of targets through fixed segmentation boundaries, and poor adaptability of segmentation correction methods to different scenes have become urgent problems to be solved. Summary of the Invention

[0005] This application provides a semantic segmentation-based intelligent segmentation and correction method, system, computer device, and storage medium for wide-angle images, aiming to solve the problems of obvious distortion in image edge regions, easy passage of targets through fixed segmentation boundaries, and poor adaptability of segmentation and correction methods to different scenes in existing wide-angle image target recognition and counting technologies.

[0006] Firstly, a method for intelligent segmentation and correction of wide-angle images based on semantic segmentation, the method comprising: Acquire the wide-angle image to be processed; Semantic segmentation is performed on the wide-angle image to be processed to obtain the image segmentation results of the target region and the background region; Morphological shrinkage processing is performed on the target region in the image segmentation result to obtain the path search segmentation result; Based on the distribution relationship between the target region and the background region in the path search segmentation results, at least one target avoidance segmentation path is obtained. The target avoidance segmentation path is located within the background region, or the overlap between the target avoidance segmentation path and the target region is less than a preset overlap threshold or is the smallest among the candidate paths. The wide-angle image to be processed is divided into regions based on the at least one target avoidance segmentation path to obtain region division results, which include distortion correction regions and low distortion regions. Based on the region division results, fisheye correction is performed on the distortion correction region, and the low distortion region is kept in its original image state or subjected to light correction, to obtain a region-processed image. Target identification and counting are performed on each region of the segmented image, and target statistics are generated based on the counting results of each region.

[0007] Optionally, in the above scheme, performing semantic segmentation on the wide-angle image to be processed to obtain image segmentation results of the target region and the background region includes: The wide-angle image to be processed is input into a preset semantic segmentation model to obtain the semantic category information corresponding to each pixel in the wide-angle image to be processed. Based on the semantic category information, determine the set of target pixels and the set of non-target pixels corresponding to the target to be counted; A binary segmentation image is generated based on the target pixel set and the non-target pixel set, and the binary segmentation image is used as the image segmentation result. The binary segmentation image includes the target region and the background region.

[0008] Optionally, in the above scheme, performing morphological shrinkage processing on the target region in the image segmentation result to obtain the path search segmentation result includes: The morphological shrinkage kernel parameters are determined based on at least one of the image size, target region area, and target region distribution density of the wide-angle image to be processed. Based on the morphological shrinkage kernel parameters, the target region in the image segmentation result is subjected to morphological erosion processing to obtain the intermediate segmentation result after shrinkage of the target region. Based on the intermediate segmentation results after the target region is shrunk, the path search segmentation results used to search for target avoidance segmentation paths are determined.

[0009] Optionally, in the above scheme, the step of searching for at least one target avoidance segmentation path based on the distribution relationship between the target region and the background region in the path search segmentation results includes: Based on the path search segmentation results, determine the set of background candidate points on the preset starting boundary; The starting path point of the target avoidance segmentation path is determined based on the background candidate point set; Using the starting path point as the initial state, a multi-row path state set is constructed according to the preset path extension direction. The path state set includes multiple candidate path points corresponding to each row. Based on the reachability relationship between adjacent candidate path points, calculate the cumulative path cost corresponding to each candidate path point to obtain the path cost calculation result. Based on the path cost calculation results, path backtracking is performed to obtain the target avoidance segmentation path.

[0010] In the above scheme, optionally, the step of calculating the cumulative path cost corresponding to each candidate path point based on the reachability relationship between adjacent candidate path points to obtain the path cost calculation result includes: Based on whether each candidate path point belongs to the background region in the path search segmentation result, the region cost corresponding to each candidate path point is determined. The movement cost corresponding to each candidate path point is determined based on the horizontal offset of the current row candidate path point relative to the previous row candidate path point. The cumulative path cost corresponding to the candidate path point in the current row is determined based on the area cost, the movement cost, and the cumulative path cost corresponding to the reachable candidate path point in the previous row. The path cost calculation result is obtained based on the cumulative path cost corresponding to each candidate path point, so that the target avoidance segmentation path tends to the background area and the lateral offset is limited.

[0011] Optionally, in the above scheme, the target avoidance segmentation path includes a first target avoidance segmentation path and a second target avoidance segmentation path; The step of dividing the wide-angle image to be processed into regions based on the at least one target avoidance segmentation path to obtain region division results includes: The first segmentation boundary is determined based on the first target avoidance segmentation path, and the second segmentation boundary is determined based on the second target avoidance segmentation path; Based on the first segmentation boundary and the second segmentation boundary, the wide-angle image to be processed is divided into a first edge region, a middle region, and a second edge region; The distortion correction region is determined based on the first edge region and the second edge region, and the low distortion region is determined based on the middle region, thus obtaining the region division result.

[0012] Optionally, in the above scheme, the method further includes: If a target avoidance segmentation path that is completely located within the background area cannot be obtained based on the cumulative path cost, the target overlap degree corresponding to each candidate path is determined based on the number of overlapping pixels between each candidate path and the target area. Based on the degree of target overlap, determine the candidate path with the minimum degree of target overlap from all candidate paths; The candidate path with the lowest degree of target overlap is determined as the target avoidance and segmentation path.

[0013] Optionally, in the above scheme, the step of performing fisheye correction processing on the distortion correction region based on the region division result, and maintaining the original image state or performing light correction processing on the low distortion region to obtain a region-processed image includes: The location of each region in the wide-angle image to be processed is determined based on the region division results. The distortion level information corresponding to each region is determined based on the region's location. Based on the distortion degree information, a region processing strategy is determined for each region. The region processing strategy includes fisheye correction, light correction, or maintaining the original image state. Each region is processed according to the region processing strategy to obtain the region-processed image, wherein the correction intensity corresponding to the distortion correction region is greater than the correction intensity corresponding to the low distortion region.

[0014] Optionally, in the above scheme, the step of performing target recognition and counting on each region of the segmented image, and generating target statistical results based on the counting results of each region, includes: Target recognition is performed on the processed images corresponding to the distortion correction region and the processed images corresponding to the low distortion region, respectively, to obtain the target recognition results for each region; Based on the target recognition results for each region, determine the region counting results for each region. Based on the region division results and the target avoidance segmentation path, the target recognition results at the boundary of adjacent regions are verified to obtain the verified region count results. The verified region count results are summarized to generate the target statistical results.

[0015] Secondly, a wide-angle image intelligent segmentation and correction system based on semantic segmentation, the system comprising: The image acquisition module is used to acquire the wide-angle image to be processed; The semantic segmentation module is used to perform semantic segmentation on the wide-angle image to be processed, and obtain the image segmentation results of the target region and the background region. The morphological processing module is used to perform morphological shrinkage processing on the target region in the image segmentation result to obtain the path search segmentation result; The path search module is used to search for at least one target avoidance segmentation path based on the distribution relationship between the target area and the background area in the path search segmentation result. The target avoidance segmentation path is located in the background area, or the overlap between the target avoidance segmentation path and the target area is less than a preset overlap threshold or is the smallest among the candidate paths. The region segmentation module is used to segment the wide-angle image to be processed into regions based on the at least one target avoidance segmentation path, and obtain the region segmentation result, which includes a distortion correction region and a low distortion region. The region-based correction module is used to perform fisheye correction processing on the distortion correction region according to the region division result, and to maintain the original image state or perform light correction processing on the low distortion region to obtain a region-based processed image. The target statistics module is used to identify and count targets in each region of the segmented image, and generate target statistics results based on the counting results of each region.

[0016] Compared with the prior art, this application has at least the following beneficial effects: Based on further analysis and research of existing technical problems, this application recognizes that existing technologies in wide-angle image target recognition and counting suffer from significant distortion in image edge regions, easy penetration of targets by fixed segmentation boundaries, and poor adaptability of segmentation correction methods to different scenes. By acquiring the wide-angle image to be processed and performing semantic segmentation, the target region corresponding to the target to be counted and the background region where the non-target is located can be determined first, so that subsequent image segmentation no longer depends on fixed positions or simple geometric rules. On this basis, by performing morphological shrinkage processing on the target region in the image segmentation result, a path search segmentation result more suitable for path search can be obtained, reducing the impact of target boundary noise on the segmentation path determination process. Furthermore, based on the distribution relationship between the target region and the background region in the path search segmentation result, a target avoidance segmentation path is searched, so that the target avoidance segmentation path is preferentially located in the background region, or, when it cannot completely avoid the target region, its degree of overlap with the target region is adjusted. By satisfying the avoidance requirements, the probability of the same target being segmented into different regions due to the segmentation boundary crossing the target is reduced. Subsequently, the region is divided according to the target avoidance segmentation path, and fisheye correction is performed on the distortion correction region, while the original image state is maintained or a light correction is performed on the low distortion region. This can differentiate the processing for the characteristics of wide-angle image edge region with obvious distortion and center region with low distortion, avoiding unnecessary computational overhead caused by global correction of the entire image. Finally, target recognition and counting are performed separately in each region of the segmented image, and target statistical results are generated. This can improve the accuracy of wide-angle image target recognition and counting and the adaptability to different target distribution scenes, while reducing target segmentation, improving target deformation in edge region and reducing correction computation. This solves the problems of obvious distortion in image edge region, easy crossing of target by fixed segmentation boundary and poor adaptability of segmentation correction method to different scenes in wide-angle image target recognition and counting. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a semantic segmentation-based intelligent segmentation and correction method for wide-angle images, provided as an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In one embodiment, such as Figure 1As shown, a semantic segmentation-based intelligent segmentation and correction method for wide-angle images is provided. In one implementation, this method can be applied to target recognition and counting processes after images are acquired by a wide-angle camera or fisheye camera. The wide-angle image to be processed can be an image of a breeding area, a storage area, a public area, or other scene images that require target counting. Taking a breeding area as an example, the wide-angle image to be processed can be an image containing pigs captured by a wide-angle camera inside a pigsty. Because wide-angle cameras or fisheye cameras have a large field of view, the edge areas of the wide-angle image to be processed usually have obvious geometric distortion, while the distortion degree near the center of the image is relatively low.

[0020] After acquiring the wide-angle image to be processed, semantic segmentation can be performed to obtain image segmentation results for the target region and the background region. The target region is used to represent the image position of the target to be counted in the wide-angle image, and the background region is used to represent the region other than the target to be counted. The image segmentation result can be a binary segmentation image or a pixel-level segmentation result including target region identifiers and background region identifiers. For example, the pixels corresponding to the target to be counted can be marked as the target region, and the remaining pixels can be marked as the background region, thus providing a basis for target distribution for subsequent target avoidance segmentation path search.

[0021] After obtaining the image segmentation result, morphological shrinkage processing can be performed on the target region in the segmentation result to obtain the path search segmentation result. Morphological shrinkage processing can be morphological erosion or other morphological processing methods that shrink the target region boundary inward. By shrinking the target region, the influence of target edge noise on path search can be reduced, and a processing margin that facilitates path search can be formed between the target region and the background region, enabling the subsequent path search process to determine a relatively stable segmentation path under the constraint of target distribution.

[0022] After obtaining the path search segmentation results, at least one target avoidance segmentation path can be searched based on the distribution relationship between the target region and the background region in the path search segmentation results. The target avoidance segmentation path can extend along the height direction of the wide-angle image to be processed, or it can extend along other preset path extension directions. The target avoidance segmentation path is preferentially located within the background region; when the target is dense and there is no feasible path that is completely located within the background region, a path with an overlap degree of less than a preset overlap threshold with the target region can be selected, or the path with the smallest overlap degree with the target region among the candidate paths can be selected. Thus, the target avoidance segmentation path can be adaptively determined according to the actual distribution of the target in the current image, rather than determined according to a fixed position or simple geometric rules.

[0023] After obtaining at least one target avoidance segmentation path, the wide-angle image to be processed can be divided into regions based on this path, resulting in region segmentation results. These results include distortion-corrected regions and low-distortion regions. Distortion-corrected regions can be areas near the image edges with high distortion levels, while low-distortion regions can be areas near the image center with low distortion levels. During region segmentation, the target avoidance segmentation path serves as the segmentation boundary or boundary determination basis between different regions, ensuring that the segmentation boundary avoids the target as much as possible, reducing the probability of the same target being segmented into different regions.

[0024] After obtaining the region segmentation results, fisheye correction can be performed on the distortion correction areas based on the segmentation results, while low-distortion areas can maintain their original image state or undergo light correction processing to obtain a region-processed image. Fisheye correction processing determines the correction mapping relationship based on camera calibration parameters, distortion model parameters, region location, and region distortion degree, and then uses this correction mapping relationship to perform pixel remapping on the distortion correction areas. For low-distortion areas, due to their lower distortion degree, they can either maintain their original image state or undergo light correction processing with a lower correction intensity than the distortion correction areas.

[0025] After obtaining the segmented image, target recognition and counting can be performed on each region of the segmented image, and target statistics can be generated based on the counting results of each region. Target recognition can be implemented using target detection models, semantic segmentation models, instance segmentation models, or other visual recognition algorithms. After each region is recognized separately, the counting results of each region can be summarized and verified by combining them with the target recognition results at the boundaries of adjacent regions to reduce duplicate counting or missed detections.

[0026] The above implementation method first uses semantic segmentation to obtain the target region and background region, and then searches for target avoidance segmentation paths based on the target distribution, so that the region division process can avoid the target and reduce duplicate counting and missed detection caused by fixed segmentation boundaries crossing the target. At the same time, different processing strategies are adopted for the distortion correction region and the low distortion region according to the region division results, which can improve the target deformation in the edge region while reducing the computational overhead of global correction, thereby improving the accuracy and scene adaptability of target recognition and counting in wide-angle images.

[0027] In one implementation, when performing semantic segmentation on a wide-angle image, the image can be input into a preset semantic segmentation model to obtain semantic category information corresponding to each pixel in the image. The preset semantic segmentation model can be a convolutional neural network model, an encoder-decoder structure model, a lightweight semantic segmentation model, or other models capable of outputting pixel-level category results. The semantic category information can include the target category and background category to be counted, or it can include the target category, ground category, railing category, wall category, and other non-target categories.

[0028] After obtaining the semantic category information corresponding to each pixel, the target pixel set and non-target pixel set corresponding to the target to be counted can be determined based on the semantic category information. The target pixel set can be composed of pixels whose semantic category belongs to the target to be counted, and the non-target pixel set can be composed of pixels whose semantic category does not belong to the target to be counted. Taking a pigsty inventory scenario as an example, pixels whose semantic category belongs to pigs can be used as the target pixel set, and pixels corresponding to fences, ground, walls, feeding facilities, and other non-pig areas can be used as the non-target pixel set.

[0029] After obtaining the target pixel set and the non-target pixel set, a binary segmentation image can be generated based on these sets, and this binary segmentation image is used as the image segmentation result. In the binary segmentation image, pixels corresponding to the target pixel set can be assigned a first pixel value, and pixels corresponding to the non-target pixel set can be assigned a second pixel value. For example, the target region can be set to white, and the background region to black; alternatively, the opposite pixel assignment method can be used depending on the specific processing requirements. Binary segmentation images can represent the positional relationship between the target and the background with low data complexity, facilitating subsequent morphological shrinkage processing and path searching.

[0030] Through the above implementation, the semantic segmentation step can separate the target to be counted in the wide-angle image to be processed from the background, forming an image segmentation result in which the target region and the background region clearly correspond. This allows the generation of subsequent target avoidance segmentation paths to have a pixel-level target distribution basis, avoiding the problem of relying solely on fixed geometric positions for region division while ignoring the actual distribution of the target.

[0031] In one implementation, when performing morphological shrinkage processing on the target region in the image segmentation result, the morphological shrinkage kernel parameters can be determined based on at least one of the image size of the wide-angle image to be processed, the area of ​​the target region, and the distribution density of the target region. The morphological shrinkage kernel parameters may include at least one of the erosion kernel size, the erosion kernel shape, and the number of erosion iterations. The erosion kernel size can be set according to the image resolution, the average scale of the targets, the spacing between targets, and the density of the targets. For example, when the target size is large or the image resolution is high, a larger erosion kernel size can be used; when the targets are dense or the spacing between targets is small, a smaller erosion kernel size can be used to avoid excessive shrinkage of the target region.

[0032] After determining the morphological shrinkage kernel parameters, morphological erosion can be performed on the target region in the image segmentation result based on these parameters to obtain an intermediate segmentation result after target region shrinkage. Morphological erosion shrinks the target region edges inward, converting some target edge pixels into pixels corresponding to the background region. The erosion kernel size is preferably three to ten pixels, but can also be adaptively adjusted according to the application scenario. Erosion reduces the interference of semantic segmentation edge spikes, adhered boundaries, and local noise on path search.

[0033] After obtaining the intermediate segmentation results after shrinking the target region, the path search segmentation results used to search for target avoidance segmentation paths can be determined based on these intermediate segmentation results. The path search segmentation results can be obtained directly from the intermediate segmentation results after target region shrinkage, or they can be obtained by performing connected component filtering, hole filling, edge smoothing, or noise removal on the intermediate segmentation results. The target region and background region in the path search segmentation results are used to calculate whether the candidate path is passable and the path cost of the candidate path.

[0034] Through the above implementation, morphological shrinkage processing can adaptively adjust the processing scale according to image size and target distribution characteristics, making the path search segmentation results more suitable for target avoidance segmentation path search. On the one hand, it can reduce the impact of semantic segmentation edge noise on path search; on the other hand, it can provide background passage space for candidate paths, thereby improving the stability and adaptability of target avoidance segmentation path search.

[0035] In one implementation, when searching for a target avoidance segmentation path, a set of background candidate points on a preset starting boundary can be determined based on the path search segmentation results. The preset starting boundary can be the top boundary of the wide-angle image to be processed, or it can be the left boundary, right boundary, or other boundaries that match the path extension direction. When the target avoidance segmentation path extends along the image height direction, multiple background candidate points can be determined on the top boundary of the image. Background candidate points can be pixels belonging to the background region in the path search segmentation results, or pixels whose target region proportion within a preset neighborhood is below a threshold.

[0036] After obtaining the set of background candidate points, the starting path point for the target avoidance segmentation path can be determined based on this set. The starting path point can preferentially select background candidate points close to a preset reference position. For example, when a vertical segmentation path needs to be formed, the search can begin from the top center of the image and proceed to the left and right sides, selecting the pixel that first satisfies the background conditions as the starting path point. This method ensures that the starting path point both meets the target avoidance requirements and is as close as possible to the preset region division position.

[0037] After determining the starting path point, a multi-row path state set can be constructed using the starting path point as the initial state, following a preset path extension direction. The path state set includes multiple candidate path points corresponding to each row. Taking a search from the top to the bottom of the image as an example, multiple candidate path points can be set for each row. These candidate path points can be obtained by extending from the previous row's path points in three directions: lower left, lower right, and lower left, or they can be determined within a preset horizontal search window. The reachability relationships between candidate path points are used to limit lateral jumps in the path, ensuring the target avoidance segmentation path remains continuous and smooth.

[0038] After constructing a multi-row path state set, the cumulative path cost for each candidate path point can be calculated based on the reachability relationships between adjacent candidate path points, yielding the path cost calculation result. The cumulative path cost can include the region cost of the candidate path point itself, the movement cost between adjacent rows, and the cumulative path cost of the candidate path points reachable from the previous row. The region cost indicates whether the candidate path point is in the background region, while the movement cost indicates the degree of lateral offset of the path. Through the cumulative path cost, it is possible to evaluate whether the path tends towards the background region and whether it has a small lateral sway at the entire path level.

[0039] After obtaining the path cost calculation results, path backtracking can be performed based on these results to obtain the target avoidance segmentation path. For example, the candidate path point with the minimum cumulative path cost at the termination boundary can be selected as the termination path point, and backtracking can be performed row by row along the optimal reachable path point recorded in the previous row to obtain the complete path from the starting boundary to the termination boundary. The target avoidance segmentation path obtained in this way can be used as the boundary basis for region division.

[0040] Through the above implementation, the target avoidance segmentation path search process is based on the path search segmentation results and the cumulative path cost. It is not a simple line-by-line greedy selection of path points. It can take into account both target avoidance and path smoothness at the global path level, so that the target avoidance segmentation path obtained by the search is located in the background area as much as possible, and reduces path abrupt changes caused by complex target distribution or local occlusion, thereby improving the stability of region division.

[0041] In one implementation, when calculating the cumulative path cost for each candidate path point, the region cost for each candidate path point can be determined based on whether it belongs to the background region in the path search segmentation result. A lower region cost can be assigned when a candidate path point belongs to the background region; a higher region cost can be assigned when a candidate path point belongs to the target region; and when a candidate path point is located in the neighborhood of the target region boundary, the region cost can be determined based on its distance from the target region or the proportion of target pixels in the neighborhood. A lower region cost indicates that the candidate path point is more suitable as a path point on the target avoidance segmentation path.

[0042] After determining the regional cost, the movement cost for each candidate path point can be determined based on the lateral offset of the current row candidate path point relative to the previous row candidate path point. The lateral offset can be used to characterize the degree of horizontal movement of the path as it extends from the previous row to the current row. To encourage the path to extend vertically, a slight penalty can be applied to lateral movement, and a lower penalty can be applied to downward movement, thereby encouraging the target avoidance segmentation path to remain as smooth and vertical as possible while satisfying the avoidance target.

[0043] After determining the region cost and movement cost, the cumulative path cost of the current candidate path point can be determined based on the region cost, movement cost, and the cumulative path cost of the previous row's reachable candidate path points. For each candidate path point in the current row, a path state with a lower cumulative cost can be selected from the previous row's reachable candidate path points. The cumulative path cost of the previous row, the region cost, and the movement cost of the current candidate path point are then merged to obtain the cumulative path cost of the current candidate path point.

[0044] After calculating the cumulative path cost for each row of candidate path points, the path cost calculation result can be obtained based on the cumulative path cost corresponding to each candidate path point. The path cost calculation result can include the cumulative path cost of each candidate path point and the corresponding optimal reachable path point in the previous row. This path cost calculation result can be used for subsequent path backtracking to obtain target avoidance segmentation paths that tend towards the background region and have limited lateral offset.

[0045] Through the above implementation, the regional cost makes the target avoidance segmentation path tend towards the background area, the movement cost limits the lateral offset of the target avoidance segmentation path, and the cumulative path cost of the candidate path points reachable in the previous row gives the path selection a global cumulative evaluation capability. Thus, a balance can be achieved between avoiding target crossing and maintaining path smoothness, improving the reliability of the target avoidance segmentation path.

[0046] In one implementation, the target avoidance segmentation path may include a first target avoidance segmentation path and a second target avoidance segmentation path. The first and second target avoidance segmentation paths may be located on either side of the central region of the wide-angle image to be processed, or they may be determined based on the distortion distribution and target distribution of the wide-angle image. Both the first and second target avoidance segmentation paths can be obtained through path search segmentation results and cumulative path cost search.

[0047] After obtaining the first target avoidance segmentation path and the second target avoidance segmentation path, the first segmentation boundary can be determined based on the first target avoidance segmentation path, and the second segmentation boundary can be determined based on the second target avoidance segmentation path. The first and second segmentation boundaries can be obtained directly using the corresponding target avoidance segmentation paths, or they can be obtained after smoothing, interpolation, or boundary expansion processing based on the target avoidance segmentation paths. Boundary expansion processing can be used to form more stable region division boundaries, avoiding the influence of discontinuity in individual pixel paths or local jitter on the region division results.

[0048] After determining the first and second segmentation boundaries, the wide-angle image to be processed can be divided into a first edge region, a middle region, and a second edge region based on these boundaries. The first and second edge regions are typically located on either side of the wide-angle image, corresponding to edge regions with higher distortion; the middle region is typically located between the two segmentation boundaries, corresponding to the central image region with lower distortion. Since the first and second segmentation boundaries are adaptively determined based on the target distribution, the probability of the boundaries crossing the target being analyzed can be reduced during region division.

[0049] After image segmentation, distortion correction regions can be determined based on the first and second edge regions, and low-distortion regions can be determined based on the intermediate region, resulting in the region segmentation results. Distortion correction regions can be used for subsequent focused fisheye correction processing, while low-distortion regions can be used for subsequent processing that preserves the original image state or performs minor correction. If only one edge region in the application scenario exhibits significant distortion, that edge region can be designated as the distortion correction region, and the remaining regions can be designated as low-distortion regions.

[0050] By using the above implementation method, two target avoidance segmentation paths can be used to form a first edge region, a middle region, and a second edge region. This not only adapts to the characteristics of wide-angle images where the left and right edges have heavier distortion and the middle has lighter distortion, but also reduces the probability of the segmentation boundary crossing the target by using the target avoidance segmentation path, thus providing a clear and stable regional basis for subsequent regional correction and partition counting.

[0051] In one implementation, when a target avoidance segmentation path entirely within the background region cannot be obtained based on the cumulative path cost, the target overlap degree corresponding to each candidate path can be determined based on the number of overlapping pixels between each candidate path and the target region. The number of overlapping pixels between the candidate path and the target region can be obtained by matching the pixel positions covered by the candidate path with the target region in the image segmentation result or path search segmentation result. The target overlap degree can be either the number of overlapping pixels or the ratio between the number of overlapping pixels and the total number of pixels in the candidate path.

[0052] After determining the degree of target overlap for each candidate path, the candidate path with the lowest degree of target overlap can be selected from among all candidate paths. If there are multiple candidate paths with the same degree of target overlap, a selection can be made based on at least one of the following: cumulative path cost, path smoothness, offset from the preset region boundary, or path length, to obtain a candidate path that is more suitable for region division.

[0053] After identifying the candidate path with the lowest target overlap, this path can be designated as the target avoidance segmentation path. This approach is suitable for image scenarios where targets are densely packed, there are few background gaps between targets, or targets are severely occluded. In such scenarios, continuous paths entirely within the background region may not exist; minimizing target overlap can reduce the target crossing range when complete target avoidance is not possible.

[0054] By implementing the above method, even when there is no path that is completely located within the background area, it is still possible to determine the available target avoidance segmentation path by minimizing the target overlap, thus avoiding the problem of being unable to perform region division due to path search failure, and minimizing the degree of segmentation boundary cutting of the target, thereby improving the robustness of image segmentation and statistical processes in dense target scenes.

[0055] In one implementation, when performing region segmentation processing based on the region segmentation results, the location of each region within the wide-angle image to be processed can be determined first based on the region segmentation results. The region location can include the region's distance relative to the image center, the region's boundary coordinates, the region's width, the region's height, and the region's distribution range in the image coordinate system. Since the degree of distortion in wide-angle or fisheye images is usually related to pixel location, the region location can serve as an important basis for determining the degree of region distortion.

[0056] After determining the location of each region, the degree of distortion for each region can be determined based on that location. This distortion degree can be determined using camera calibration parameters, distortion models, the distance from the region to the image center, the field of view of the region, or a preset distortion distribution rule. For example, the first and second edge regions closer to the image edge may have higher distortion levels, while the middle region closer to the image center may have lower distortion levels.

[0057] After determining the degree of distortion, a corresponding region processing strategy can be determined for each region. Region processing strategies include fisheye correction, lightweight correction, or maintaining the original image state. For distortion-prone regions with high distortion levels, fisheye correction can be chosen as the region processing strategy; for low-distortion regions with low distortion levels, lightweight correction or maintaining the original image state can be chosen as the region processing strategy. Lightweight correction can employ a lower correction intensity, a simplified correction model, or a correction method with less remapping computation.

[0058] After determining the region processing strategy, each region can be processed according to the strategy to obtain a region-processed image. The correction intensity for distortion correction regions is greater than that for low-distortion regions. For distortion correction regions, remapping correction based on a fisheye distortion model can be used; for low-distortion regions, remapping may not be performed, or only local scale adjustment, slight perspective correction, or low-intensity distortion correction may be applied. The processed regions can be used independently for target recognition, or they can be stitched together to form a region-processed image.

[0059] Through the above implementation, regional correction processing can employ differentiated processing strategies based on the degree of distortion in different regions, effectively improving the target deformation in edge regions while avoiding unnecessary high-intensity correction in low-distortion regions. This reduces the computational overhead of global fisheye correction and improves the accuracy of target recognition and counting in edge-distorted regions.

[0060] In one implementation, after obtaining the region-processed image, target recognition can be performed on the processed image corresponding to the distortion-corrected region and the processed image corresponding to the low-distortion region, respectively, to obtain the target recognition result for each region. The target recognition result may include at least one of the following: target detection box, target contour, target center point, target category, target confidence score, and target instance identifier. Target recognition can be implemented using a detection model, an instance segmentation model, or a counting model.

[0061] After obtaining the target recognition results for each region, the region count results for each region can be determined based on these results. For example, the number of target detection boxes, target instances, or target center points in each region can be counted to obtain the region count results. For target recognition results with low confidence, a pre-set confidence threshold can be used for filtering to reduce the impact of false detections on the region count results.

[0062] After obtaining the region counting results, boundary target verification can be performed on the target recognition results at the boundaries of adjacent regions based on the region division results and the target avoidance segmentation path, resulting in verified region counting results. Boundary target verification can include determining whether there are target recognition results near the boundaries of adjacent regions that are spatially close, have continuous target outlines, overlapping detection boxes, or whose target center point distance is less than a preset distance threshold. If there are recognition results at the boundaries of adjacent regions that are suspected to be the same target, one of the recognition results can be retained based on confidence level, target area, target center position, or region affiliation rules, or adjacent recognition results can be merged.

[0063] After boundary target verification is completed, the verified region counting results can be summarized to generate target statistics. Target statistics can include the total number of targets in the wide-angle image to be processed, or the number of targets in each region, the number of targets in each category, the number of abnormal targets, or target distribution information. Taking a livestock farming area as an example, target statistics can be used to represent the number of pigs in a pigsty and the regional distribution of pigs in the image.

[0064] By implementing the above method, target identification and counting are performed separately in each region, and then the targets at the boundary are verified by combining the region division results and target avoidance segmentation paths. This can further reduce the risk of duplicate counting or missed detection at the boundary of adjacent regions, making the final target statistics results more accurate and reliable.

[0065] In one implementation, the wide-angle image intelligent segmentation and correction system based on semantic segmentation includes an image acquisition module, a semantic segmentation module, a morphological processing module, a path search module, a region division module, a region-specific correction module, and a target statistics module. These modules can be deployed in the same image processing device or distributed across image acquisition terminals, edge computing devices, servers, or cloud processing platforms. Intermediate processing results can be transmitted between modules via data bus, communication interface, software call interface, or inter-process communication.

[0066] The image acquisition module is used to acquire the wide-angle image to be processed. This wide-angle image can originate from a wide-angle camera, fisheye camera, image frames from a video stream, an image database, or a remote monitoring device. The image acquisition module can also perform resizing, format conversion, frame extraction, or quality filtering on the acquired image to obtain a wide-angle image suitable for subsequent processing.

[0067] The semantic segmentation module performs semantic segmentation on the wide-angle image to be processed, obtaining image segmentation results for the target region and background region. The semantic segmentation module can call a preset semantic segmentation model to output pixel-level semantic category information and generate a binary segmented image based on this information. The target region and background region in the binary segmented image can be used to represent the target and non-target regions to be statistically analyzed, respectively.

[0068] The morphological processing module performs morphological shrinkage on the target region in the image segmentation result to obtain the path search segmentation result. The morphological processing module can determine the morphological shrinkage kernel parameters according to the image size, target region area, or target region distribution density, and perform erosion processing on the target region based on the morphological shrinkage kernel parameters to shrink the target region boundary inward, thereby generating a path search segmentation result suitable for path search.

[0069] The path search module is used to search for at least one target avoidance segmentation path based on the distribution relationship between the target region and the background region in the path search segmentation results. The path search module can determine a set of background candidate points on a preset starting boundary, determine the starting path point based on the background candidate point set, construct a multi-row path state set, calculate the cumulative path cost corresponding to each candidate path point, and backtrack to obtain the target avoidance segmentation path based on the path cost calculation results. The target avoidance segmentation path is preferentially located within the background region; if no path exists that is completely located within the background region, the path with a degree of overlap with the target region less than a preset overlap threshold or the smallest among the candidate paths can be selected.

[0070] The region segmentation module is used to segment the wide-angle image to be processed into regions based on at least one target avoidance segmentation path, obtaining the region segmentation result. The region segmentation result includes a distortion correction region and a low distortion region. The region segmentation module can form a first edge region, a middle region, and a second edge region based on a first target avoidance segmentation path and a second target avoidance segmentation path, and determine the first edge region and the second edge region as the distortion correction region, and the middle region as the low distortion region.

[0071] The region-based correction module performs fisheye correction on the distortion correction regions based on the region division results, and maintains the original image state or performs light correction on the low-distortion regions, resulting in a region-processed image. The module can determine the degree of distortion based on the location of each region in the wide-angle image to be processed, and determine the region processing strategy based on this information, ensuring that the correction intensity for the distortion correction regions is greater than that for the low-distortion regions.

[0072] The target statistics module is used to perform target recognition and counting in each region of the segmented image, and generate target statistics results based on the counting results of each region. The module can perform target recognition in distortion-corrected regions and low-distortion regions separately, obtaining the target recognition results for each region; determine the region counting results based on the target recognition results; perform boundary target verification on the target recognition results at the boundaries of adjacent regions; and summarize the verified region counting results to generate the target statistics results.

[0073] Through the aforementioned system architecture, the image acquisition module, semantic segmentation module, morphological processing module, path search module, region division module, sub-region correction module, and target statistics module work together sequentially to adaptively generate target avoidance segmentation paths based on the distribution of the target and background regions. Sub-region correction and target statistics are then performed based on these target avoidance segmentation paths. This reduces the risk of duplicate counting or missed detections caused by segmentation boundaries crossing targets, while also reducing the computational overhead of global fisheye correction, thus improving the accuracy and processing efficiency of target statistics results for wide-angle images.

[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for intelligent segmentation and correction of wide-angle images based on semantic segmentation, characterized in that, The method includes: Acquire the wide-angle image to be processed; Semantic segmentation is performed on the wide-angle image to be processed to obtain the image segmentation results of the target region and the background region; Morphological shrinkage processing is performed on the target region in the image segmentation result to obtain the path search segmentation result; Based on the distribution relationship between the target region and the background region in the path search segmentation results, at least one target avoidance segmentation path is obtained. The target avoidance segmentation path is located within the background region, or the overlap between the target avoidance segmentation path and the target region is less than a preset overlap threshold or is the smallest among the candidate paths. The wide-angle image to be processed is divided into regions based on the at least one target avoidance segmentation path to obtain region division results, which include distortion correction regions and low distortion regions. Based on the region division results, fisheye correction is performed on the distortion correction region, and the low distortion region is kept in its original image state or subjected to light correction, to obtain a region-processed image. Target identification and counting are performed on each region of the segmented image, and target statistics are generated based on the counting results of each region.

2. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 1, characterized in that, The semantic segmentation of the wide-angle image to be processed, to obtain the image segmentation results of the target region and the background region, includes: The wide-angle image to be processed is input into a preset semantic segmentation model to obtain the semantic category information corresponding to each pixel in the wide-angle image to be processed. Based on the semantic category information, determine the set of target pixels and the set of non-target pixels corresponding to the target to be counted; A binary segmentation image is generated based on the target pixel set and the non-target pixel set, and the binary segmentation image is used as the image segmentation result. The binary segmentation image includes the target region and the background region.

3. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 1, characterized in that, The step of performing morphological shrinkage processing on the target region in the image segmentation result to obtain the path search segmentation result includes: The morphological shrinkage kernel parameters are determined based on at least one of the image size, target region area, and target region distribution density of the wide-angle image to be processed; Based on the morphological shrinkage kernel parameters, the target region in the image segmentation result is subjected to morphological erosion processing to obtain the intermediate segmentation result after shrinkage of the target region. Based on the intermediate segmentation results after the target region is shrunk, the path search segmentation results used to search for target avoidance segmentation paths are determined.

4. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 1, characterized in that, The step of searching for at least one target avoidance segmentation path based on the distribution relationship between the target region and the background region in the path search segmentation results includes: Based on the path search segmentation results, determine the set of background candidate points on the preset starting boundary; The starting path point of the target avoidance segmentation path is determined based on the background candidate point set; Using the starting path point as the initial state, a multi-row path state set is constructed according to the preset path extension direction. The path state set includes multiple candidate path points corresponding to each row. Based on the reachability relationship between adjacent candidate path points, calculate the cumulative path cost corresponding to each candidate path point to obtain the path cost calculation result. Based on the path cost calculation results, path backtracking is performed to obtain the target avoidance segmentation path.

5. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 4, characterized in that, The step of calculating the cumulative path cost corresponding to each candidate path point based on the reachability relationship between adjacent candidate path points, and obtaining the path cost calculation result, includes: Based on whether each candidate path point belongs to the background region in the path search segmentation result, the region cost corresponding to each candidate path point is determined. The movement cost corresponding to each candidate path point is determined based on the horizontal offset of the current row candidate path point relative to the previous row candidate path point. The cumulative path cost corresponding to the candidate path point in the current row is determined based on the area cost, the movement cost, and the cumulative path cost corresponding to the reachable candidate path point in the previous row. The path cost calculation result is obtained based on the cumulative path cost corresponding to each candidate path point, so that the target avoidance segmentation path tends to the background area and the lateral offset is limited.

6. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 1, characterized in that, The target avoidance segmentation path includes a first target avoidance segmentation path and a second target avoidance segmentation path; The step of dividing the wide-angle image to be processed into regions based on the at least one target avoidance segmentation path to obtain region division results includes: The first segmentation boundary is determined based on the first target avoidance segmentation path, and the second segmentation boundary is determined based on the second target avoidance segmentation path; Based on the first segmentation boundary and the second segmentation boundary, the wide-angle image to be processed is divided into a first edge region, a middle region, and a second edge region; The distortion correction region is determined based on the first edge region and the second edge region, and the low distortion region is determined based on the middle region, thus obtaining the region division result.

7. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 5, characterized in that, The method further includes: If a target avoidance segmentation path that is completely located within the background area cannot be obtained based on the cumulative path cost, the target overlap degree corresponding to each candidate path is determined based on the number of overlapping pixels between each candidate path and the target area. Based on the degree of target overlap, determine the candidate path with the minimum degree of target overlap from all candidate paths; The candidate path with the lowest degree of target overlap is determined as the target avoidance and segmentation path.

8. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 1, characterized in that, The step of performing fisheye correction on the distortion correction region based on the region division result, and maintaining the original image state or performing light correction on the low distortion region to obtain a region-processed image includes: The location of each region in the wide-angle image to be processed is determined based on the region division results. The distortion level information corresponding to each region is determined based on the region's location. Based on the distortion degree information, a region processing strategy is determined for each region. The region processing strategy includes fisheye correction, light correction, or maintaining the original image state. Each region is processed according to the region processing strategy to obtain the region-processed image, wherein the correction intensity corresponding to the distortion correction region is greater than the correction intensity corresponding to the low distortion region.

9. The wide-angle image intelligent segmentation and correction method based on semantic segmentation according to claim 8, characterized in that, The step of performing target recognition and counting on each region of the segmented image, and generating target statistical results based on the counting results of each region, includes: Target recognition is performed on the processed images corresponding to the distortion correction region and the processed images corresponding to the low distortion region, respectively, to obtain the target recognition results for each region; Based on the target recognition results for each region, determine the region counting results for each region. Based on the region division results and the target avoidance segmentation path, the target recognition results at the boundary of adjacent regions are verified to obtain the verified region count results. The verified region count results are summarized to generate the target statistical results.

10. A wide-angle image intelligent segmentation and correction system based on semantic segmentation, characterized in that, The system includes: The image acquisition module is used to acquire the wide-angle image to be processed; The semantic segmentation module is used to perform semantic segmentation on the wide-angle image to be processed, and obtain the image segmentation results of the target region and the background region. The morphological processing module is used to perform morphological shrinkage processing on the target region in the image segmentation result to obtain the path search segmentation result; The path search module is used to search for at least one target avoidance segmentation path based on the distribution relationship between the target area and the background area in the path search segmentation result. The target avoidance segmentation path is located in the background area, or the overlap between the target avoidance segmentation path and the target area is less than a preset overlap threshold or is the smallest among the candidate paths. The region segmentation module is used to segment the wide-angle image to be processed into regions based on the at least one target avoidance segmentation path, and obtain the region segmentation result, which includes a distortion correction region and a low distortion region. The region-based correction module is used to perform fisheye correction processing on the distortion correction region according to the region division result, and to maintain the original image state or perform light correction processing on the low distortion region to obtain a region-based processed image. The target statistics module is used to identify and count targets in each region of the segmented image, and generate target statistics results based on the counting results of each region.

Images