Automatic partition detection method and system based on editable biological behavior observation area

Abstract

The application discloses an automatic partition detection method and system based on an editable biological behavior observation area, obtains a representative frame of a monocular overhead experiment video, outputs a candidate observation area frame through an observation area target detection model, removes and filters the candidate observation area frame to obtain an effective observation area frame, performs row and column sorting and size normalization on the obtained effective observation area frame to obtain an initial region of interest, and can obtain a confirmed region of interest based on an operation instruction, and then obtains an overall cropping area, and determines relative region of interest coordinates of the confirmed region of interest relative to the overall cropping area, so that automatic detection, removal and filtering, row and column stable numbering, editable confirmation and relative coordinate output in the overall cropping area of the region of interest are realized, thereby providing a region benchmark for subsequent various recognition algorithms.

Description

Technical Field

[0001] This invention belongs to the field of aquaculture image region configuration technology. Specifically, it relates to an automatic partitioning detection method and system for editable biological behavior observation regions in experimental scenarios with multiple tanks, multiple compartments, or multiple observation grids. Background Technology

[0002] In experiments on the behavior of aquatic organisms such as fish, shrimp, and crabs, researchers often arrange multiple tanks, compartments, or observation frames simultaneously within a single overhead view, and record the activity trajectory, aggregation status, feeding response, or dwell time in each area.

[0003] To separate and locate regions within the images of multiple tanks, compartments, or observation frames, current technologies typically rely on manual selection of ROIs (Regions of Interest) one by one, which is cumbersome. Another approach is to pre-define coordinates under fixed camera and tank placement conditions. This method requires repeated configuration when the experimental tank moves slightly, the camera angle changes, the resolution changes, or batch videos are replaced, and it is prone to region numbering confusion, coordinate offsets, and subsequent statistical assignment errors. Yet another region localization method uses traditional threshold segmentation, which sets global or adaptive thresholds for image grayscale values, color components, or brightness differences. After obtaining a binary image, morphological processing and connected component filtering are used to determine candidate regions. This method is prone to region boundary breakage, adhesion, or false detection when there is reflection from transparent containers, uneven local lighting, shadows, or insufficient contrast at region edges.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background technology of this application, and therefore may include prior art that is not known to those skilled in the art. Summary of the Invention

[0005] This invention proposes an automatic partitioning detection method and system based on editable biological behavior observation regions to solve the technical problems of existing technologies, such as cumbersome operation, chaotic region numbering, coordinate offset, broken region boundaries, adhesion, or false detection.

[0006] To achieve the above-mentioned invention / design objectives, the present invention adopts the following technical solution:

[0007] An automatic partitioning detection method based on editable biological behavior observation regions, the method comprising:

[0008] Obtain representative frames from the monocular top-down experimental video;

[0009] Obtain the number of preset observation areas;

[0010] The representative frame is detected by the observation area target detection model to obtain a set of candidate observation area boxes. The candidate observation area boxes in the set of candidate observation area boxes are deduplicated and geometrically filtered to obtain a number of effective candidate observation area boxes that are no less than the preset number of observation areas.

[0011] The effective candidate observation region boxes are sorted by row and column and their sizes are normalized to obtain the initial region of interest;

[0012] The initial region of interest is displayed as an editable graphic object, and the region of interest is confirmed in response to an operation command;

[0013] Generate an overall clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and determine the relative region of interest coordinates of the confirmed regions of interest relative to the overall clipping region.

[0014] As described above, the automatic partitioning detection method based on editable biological behavior observation regions involves reducing the detection threshold of the target detection model in the observation region when the number of valid candidate observation region boxes obtained by deduplication and geometric filtering of the candidate observation region box set is lower than the preset number of observation regions. This is done to obtain a supplementary candidate observation region box set. The candidate observation region box set and the supplementary candidate observation region box set are then merged and deduplicated and geometrically filtered to obtain valid candidate observation region boxes. This process continues until the number of valid candidate observation region boxes is not less than the preset number of observation regions.

[0015] The automatic partitioning detection method based on editable biological behavior observation regions, as described above, obtains the coordinates of the upper left corner, width, and height of the candidate observation region box, and calculates the coordinates of the center point, width, height, and area of ​​the candidate observation region box for set filtering and row / column sorting.

[0016] The automatic partitioning detection method based on editable biological behavior observation regions as described above includes the following deduplication method: obtaining the confidence level of the candidate observation region boxes, processing the candidate observation region boxes in descending order of confidence level, calculating the intersection-union ratio (IUU) between the candidate observation region boxes, and retaining the candidate observation region boxes with high confidence level and removing the candidate observation region boxes with low confidence level when the IUU of two candidate observation region boxes is higher than a preset deduplication threshold.

[0017] The automatic partitioning detection method based on editable biological behavior observation regions, as described above, includes the following row and column sorting method:

[0018] The valid candidate observation region boxes are processed in ascending order of their center point ordinates. The median of the center point ordinates of the valid candidate observation region boxes already included in the current row is used as the reference ordinate for that row. When the difference between the center point ordinate of the valid candidate observation region box to be processed and the reference ordinate is lower than the preset row grouping tolerance, the valid candidate observation region box is included in the current row; otherwise, a new row is created. The rows are sorted in ascending order of their reference ordinates to determine the row number, and within each row, the column number is determined in ascending order of the center point abscissa of the valid candidate observation region box. A continuous region of interest number is generated based on the row number and column number.

[0019] The automatic partitioning detection method based on editable biological behavior observation regions, as described above, includes the following size normalization method:

[0020] The width and height of the unified region of interest are determined based on the mean, median, or truncated mean of the set of widths and heights of the effective candidate observation region boxes.

[0021] As described above, the automatic partitioning detection method based on editable biological behavior observation regions includes one or more of the following operation instructions: dragging, corner scaling, boundary constraint, deletion, regeneration, size adjustment, or position adjustment. After executing the operation instructions, the confirmed regions of interest are reordered according to row and column sorting and consecutive numbers are generated.

[0022] The automatic partitioning detection method based on editable biological behavior observation areas, as described above, generates relative region of interest coordinates based on the confirmed regions of interest as follows: calculate the minimum bounding rectangle of all confirmed regions of interest, determine the upper left corner, lower right corner, width, and height of the minimum bounding rectangle, and convert the original coordinates of each confirmed region of interest into relative coordinates relative to the upper left corner of the minimum bounding rectangle.

[0023] An automated zoning detection system based on editable biological behavior observation regions, the system comprising:

[0024] The frame acquisition unit is used to acquire representative frames of the monocular top-down experimental video.

[0025] The candidate detection unit is used to detect the representative frame using the observation area target detection model to obtain a set of candidate observation area boxes;

[0026] The effective candidate observation region box determination unit is used to perform deduplication and geometric filtering on the candidate observation region boxes of the candidate observation region box set to obtain an effective candidate observation region box with a number not less than the preset number of observation regions.

[0027] The row and column sorting and size normalization unit is used to sort the effective candidate observation region box by rows and columns and normalize its size to obtain the initial region of interest.

[0028] An editable unit is used to display the initial region of interest as an editable graphic object, and to confirm the region of interest in response to an operation command;

[0029] The coordinate transformation output unit is used to generate an overall clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and to determine the relative region of interest coordinates of the confirmed regions of interest relative to the overall clipping region.

[0030] In the system described above, the effective candidate observation region box determination unit is used to reduce the detection threshold of the target detection model in the observation region and perform supplementary detection to obtain a supplementary candidate observation region box set when the number of effective candidate observation region boxes obtained by deduplication and geometric filtering of the candidate observation region boxes in the candidate observation region box set is lower than the preset number of observation regions. The unit then merges the candidate observation region box set and the supplementary candidate observation region box set and performs deduplication and geometric filtering to obtain effective candidate observation region boxes until the number of effective candidate observation region boxes is not less than the preset number of observation regions.

[0031] Compared with existing technologies, the advantages and positive effects of this invention are as follows: This invention uses an automatic partitioning detection method for editable biological behavior observation regions to obtain representative frames of monocular top-down experimental videos; obtains a preset number of observation regions; uses an observation region target detection model to detect representative frames to obtain a set of candidate observation region boxes; performs deduplication and geometric filtering on the candidate observation region boxes in the candidate observation region box set to obtain a number of effective candidate observation region boxes that are no less than the preset number of observation regions; sorts the effective candidate observation region boxes by rows and columns and normalizes their sizes to obtain an initial region of interest; displays the initial region of interest as an editable graphic object, and obtains a confirmed region of interest in response to operation commands; generates an overall cropping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and determines the relative region of interest coordinates of the confirmed regions of interest relative to the overall cropping region. This invention reduces repetitive bounding box selection in multi-observation area experiments; it can automatically adapt to changes in camera position and observation container placement; it does not rely on lighting or edge contrast; this invention sorts the obtained effective observation area bounding boxes by rows and columns and normalizes their size to obtain an initial region of interest, and can obtain a confirmed region of interest based on operation commands, thereby obtaining the overall cropped region, and determining the relative coordinates of the confirmed region of interest relative to the overall cropped region. It realizes automatic detection, deduplication filtering, stable row and column numbering, editable confirmation, and output of relative coordinates within the overall cropped region of interest, thus providing a regional benchmark for subsequent recognition algorithms.

[0032] Other features and advantages of the present invention will become clearer after reading the detailed embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of an automatic partitioning detection method based on an editable biological behavior observation region, which is a specific embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the system according to a specific embodiment of the present invention.

[0036] Figure 3a This is a schematic diagram of the initial ROI in a specific embodiment of the present invention.

[0037] Figure 3b This is a schematic diagram confirming the ROI for a specific embodiment of the present invention.

[0038] Figure 3c This is a schematic diagram of the relative ROI of a specific embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of the row and column sorting and uniform initial ROI of the multi-row, multi-column observation area in a specific embodiment of the present invention.

[0040] Figure 5 This is a schematic diagram illustrating how the detection results of fish or feed are assigned to a specific region based on relative ROI coordinates and how regional statistical data is output, according to a specific embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] In the description of this invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0043] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. In the description of embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0044] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0045] This embodiment focuses on the automatic generation, manual correction, and relative coordinate output of multiple observation regions. After reading the representative frame, candidate observation region boxes are obtained through the observation region target detection model. When the number of candidates is insufficient, the detection threshold is automatically lowered for supplementary detection. Subsequently, area filtering, width and height filtering, deduplication and merging, row and column sorting, and quantity filtering are performed on the candidate observation region boxes. Then, an initial ROI (Region of Interest) of uniform size is generated based on the statistical values ​​of the width and height of the candidate observation region boxes, and it is displayed to the user as an editable graphic object. After user confirmation, the system reads the coordinates of the confirmed ROI (Region of Interest), calculates the overall cropped area, and outputs a set of relative ROI (Region of Interest) coordinates, forming a coordinate reference that can be reused for subsequent fish detection, feed detection, feeding recognition, or behavior statistics.

[0046] An automatic partitioning detection method based on editable biological behavior observation regions:

[0047] Obtain representative frames from the monocular top-down experimental video;

[0048] Obtain the number of preset observation areas;

[0049] The target detection model of the observation area detects the representative frame to obtain a set of candidate observation area boxes. The candidate observation area boxes in the set of candidate observation area boxes are deduplicated and geometrically filtered to obtain a number of effective candidate observation area boxes that are no less than the preset number of observation areas.

[0050] The initial region of interest is obtained by sorting the effective candidate observation regions by rows and columns and normalizing their sizes.

[0051] The initial region of interest is displayed as an editable graphic object, and the region of interest is confirmed in response to operation commands;

[0052] Generate a global clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and determine the relative region of interest coordinates of the confirmed regions of interest relative to the global clipping region.

[0053] The coordinates of the relative region of interest are used as the basis for region attribution in subsequent target detection results or experimental data.

[0054] Representative frames were selected from the experimental video sequence.

[0055] The representative frame can be the first frame.

[0056] If the first frame contains reflections, occlusions, or blurriness, a frame with higher sharpness can be selected from several candidate frames. Sharpness can be evaluated by Laplacian variance, edge intensity, or luminance stability.

[0057] If the first frame meets the requirements for clarity and integrity, then the first frame is directly used as the representative frame.

[0058] The number of preset observation areas is consistent with the number of experimental water tanks or observation grids.

[0059] The representative frame is input into the observation region detection model, which outputs several candidate observation region boxes. The observation region detection model can employ existing general object detection networks and be trained or fine-tuned using samples from the aquatic experimental observation region. The observation region detection model is used to output candidate observation region boxes, and the network structure itself is not a necessary constraint.

[0060] Each candidate observation region box is represented only by the coordinates of the top left corner, width, height, and confidence level, avoiding the use of complex multi-point contour descriptions.

[0061] ;

[0062] in, This represents the i-th candidate observation region bounding box output by the model. and The x and y coordinates of the top left corner of the candidate observation region in the frame coordinate system; and These represent the width and height of the candidate observation region bounding box, respectively. To assess confidence, this quintuple, derived from the output of the observation region detection model, describes the location, scale, and reliability of candidate observation region boxes. It serves as the foundational data for subsequent quantity assessment, supplementary detection, geometric filtering, and initial ROI (Region of Interest) localization.

[0063] In some embodiments, when the number of valid candidate observation region boxes obtained by deduplication and geometric filtering of the candidate observation region boxes in the candidate observation region box set is lower than the preset number of observation regions, the detection threshold of the target detection model in the observation region is lowered to perform supplementary detection to obtain a supplementary candidate observation region box set. The candidate observation region box set and the supplementary candidate observation region box set are merged and then deduplicated and geometrically filtered to obtain valid candidate observation region boxes, until the number of valid candidate observation region boxes is not less than the preset number of observation regions.

[0064] In some embodiments, the deduplication method includes: obtaining the confidence level of candidate observation region boxes; processing the candidate observation region boxes in descending order of confidence level; calculating the intersection-union ratio (IUU) between candidate observation region boxes; and retaining the candidate observation region box with high confidence level and removing the candidate observation region box with low confidence level when the IUU of two candidate observation region boxes is higher than a preset deduplication threshold.

[0065] The target detection model for the observation region first performs observation region detection on representative frames using a first detection threshold, obtaining an initial set of candidate observation region boxes. Instead of directly using the number of original output boxes from the model as the final criterion, the model uses valid candidate observation region boxes, after initial detection, necessary supplementary detection, deduplication, and geometric filtering, which are no less than the preset number of observation regions, as the basis for subsequent judgments.

[0066] The candidate observation region set B1 obtained from the first detection is deduplicated; if a supplementary detection is triggered, the same deduplication rule is applied to the merged set of the first detection result and the supplementary detection result.

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] ;

[0072] .

[0073] in, This is the set of candidate observation region boxes obtained from the first detection. This represents the number of candidate observation regions in the initial observation. Let D be the set of candidate observation region boxes obtained under the second detection threshold; D represents the observation region detection model. and are respectively the initial detection threshold and the supplementary detection threshold, and k is the threshold reduction coefficient. By defining 0<k<1, it can be clarified that the second detection threshold is lower than the first detection threshold. B is the set of candidate observation area frames after merging and deduplication, and NMS is non-maximum suppression or an equivalent deduplication operator, is the deduplication threshold for the overlap degree of candidate observation area frames.

[0074] In this embodiment, the deduplicated candidate observation area frames are sent to the subsequent geometric filtering to avoid repeated numbering of the same observation area. NMS processes the candidate observation area frames in descending order of the confidence of the candidate observation area frames. When the Intersection over Union of two candidate observation area frames is not less than the deduplication threshold θ d , the candidate observation area frame with higher confidence is retained to avoid repeated numbering of the same observation area.

[0075] In some embodiments, the top-left corner coordinates, width and height of the candidate observation area frame are acquired, and the center point coordinates, width, height and area of the candidate observation area frame are calculated for set filtering and row-column sorting.

[0076] The deduplicated candidate observation area frames may still contain background false detections or abnormal shape frames. First, the center point coordinates, area, and aspect ratio of each candidate observation area frame are calculated, and then an effective candidate observation area frame index set is formed according to the area, aspect ratio, and confidence. Whether supplementary detection continues to be triggered and whether manual supplementation is required are both judged based on the filtered effective quantity.

[0077] .

[0078] wherein, is the center point of the i-th candidate observation area frame, and the center point is calculated from the top-left corner coordinates, width and height. This center point is only used for row-column grouping, sorting and initial ROI alignment of the observation area frames, and does not represent the position of fish bodies, feed or other subsequent detection targets, so it will not be confused with the target representative points in subsequent behavior recognition or feeding recognition.

[0079] as Figure 4 shown, the center point of the candidate observation area frame can take the geometric center of the candidate observation area frame, which is indicated by a red dot in the figure.

[0080] .

[0081] wherein, is the pixel area of the i-th candidate observation area frame, which is obtained by multiplying the width and height of the candidate observation area frame. This area value is sent to the subsequent minimum area condition, which is used to exclude false detection frames that are significantly smaller than the real observation area, and provides a scale basis for the candidate observation area frame for unified ROI size statistics.

[0082] The aspect ratio of candidate observation region boxes and the set of indices for effective candidate observation region boxes are defined as follows:

[0083] ;

[0084] ;

[0085] ;

[0086] in, Let be the aspect ratio of the i-th candidate observation region bounding box. This is the set of candidate observation region bounding box indices retained after filtering by area, aspect ratio, and confidence level. This represents the number of valid candidate observation region boxes after filtering. This is the minimum area threshold used to exclude false positives due to small areas. and These are the lower and upper limits of the aspect ratio, used to exclude candidate observation regions with abnormal shapes. This is the minimum confidence threshold for the geometric filtering stage after deduplication, used to exclude candidate observation regions with insufficient confidence.

[0087] Among them, the area condition, aspect ratio condition, and confidence level condition are combined using a logical AND operation; only when all three conditions are met can the condition be satisfied. , and Only candidate observation region boxes with index i are included in the set of valid candidate observation region box indices. If any condition is not met, the candidate observation region box will not be retained. If the number of observation areas is still less than the preset number N, the system can continue to perform supplementary checks or prompt for manual supplementation; if If the value is greater than N, priority will be given to retaining candidate observation regions with higher confidence, areas and aspect ratios that are closer to the overall statistical characteristics, and more consistent row and column layouts.

[0088] In some embodiments, the row and column sorting method includes:

[0089] The effective candidate observation area boxes are processed in ascending order of their center point ordinates. The median of the center point ordinates of the effective candidate observation area boxes already included in the current row is used as the reference ordinate for that row. When the difference between the center point ordinate of the effective candidate observation area box to be processed and the reference ordinate is lower than the preset row grouping tolerance, the effective candidate observation area box is included in the current row; otherwise, a new row is created. The rows are sorted in ascending order of their reference ordinates to determine the row number, and within each row, the column number is determined in ascending order of the center point abscissa of the effective candidate observation area box. A continuous region of interest number is generated based on the row number and column number.

[0090] After filtering, the candidate observation region boxes are processed in ascending order of their center point ordinates. The first candidate observation region box is used to create the first row, and its center point ordinate is used as the initial reference ordinate for the first row. For subsequent candidate observation region boxes i, their center point ordinates are calculated. Reference y-axis of the current row The difference; when When, the candidate observation region box i is added to the current row; when When a new row is created, the reference ordinate of the current row is updated with the median of the ordinates of the center points of all candidate observation region boxes within that row. After row grouping is completed, the system sorts the rows in ascending order of their reference ordinates. Within each row, candidate observation region boxes are sorted in ascending order of their center point abscissas, with the smallest abscissa determining the first column, and the remaining candidate observation region boxes determining the subsequent column numbers accordingly. To avoid splitting the same row due to slight tilt or perspective errors, the row grouping tolerance is adjusted. The size of the observation area or a preset tolerance should be determined. To avoid splitting the same row due to slight tilt or perspective errors, a row grouping tolerance can be set; within the same row, the column number should be determined according to the horizontal coordinate from smallest to largest.

[0091] ;

[0092] ;

[0093] Where 'a' is the row number of the candidate observation region bounding box, and 'b' is the column number within that row. The number of rows and columns, Let be the ordinate of the center point of the i-th candidate observation region. Let be the reference ordinate for row a. This rule allows for row grouping tolerance. It enables interface labels, parameter table records, exported data, and subsequent region assignment results to use the same continuous numbering system.

[0094] like Figure 4 As shown, taking a three-row, four-column observation area as an example, the four candidate observation area boxes in the first row all satisfy the condition that the difference between them and the reference vertical coordinate of the first row is no greater than the row grouping tolerance, and are numbered ROI1 to ROI4 in ascending order of the horizontal coordinate of the center point; the second and third rows adopt the same rule and are numbered ROI5 to ROI8 and ROI9 to ROI12 in sequence. Figure 4 This is only used to illustrate the sorting and numbering mechanism in multi-row and multi-column scenarios. The actual number of rows, columns, and quantities in the observation area can be determined according to the configuration of the experimental setup.

[0095] In some embodiments, the size normalization method includes:

[0096] The width and height of the unified region of interest are determined based on the mean, median, or truncated mean of the set of widths and heights of the valid candidate observation regions.

[0097] Since the detection bounding box may be slightly larger or smaller, a uniform ROI size is generated based on the width and height of all qualified candidate observation region boxes. Preferably, the median is used to reduce the influence of outlier candidate observation region boxes; alternatively, the mean, truncated mean, or a manually set value can be used.

[0098] ;

[0099] ;

[0100] in, and These represent the uniform ROI width and uniform ROI height, respectively; med represents the median operator, which is preferentially used to reduce the impact of outlier candidate observation bounding boxes on the uniform size. The mean, truncated mean, or manually set values ​​can also be used depending on the consistency of the experimental containers.

[0101] like Figure 4 As shown, and These represent the width and height of the initial ROI with uniform dimensions, used to ensure that the initial ROIs after row and column sorting have consistent dimensions.

[0102] Using the sorted center point as a reference, an initial ROI of uniform size is generated. This initial ROI is only an automatically suggested result; users can still drag or scale it to make corrections later.

[0103] ;

[0104] in, For the j-th initial ROI, and These are the x and y coordinates of the center point of the j-th candidate observation region after sorting. A suggested region with uniform width and height is generated using this center point as the anchor point.

[0105] When the initial ROI crosses the image boundary, the system maintains a uniform width for the initial ROI. and uniform height The coordinates of the top left corner remain unchanged, and are translated to ensure that the ROI is fully within the range of the representative frame image. When the ROI of uniform size cannot be fully placed within the range of the representative frame image, the initial ROI is marked as an object to be manually confirmed and automatic cropping is not performed.

[0106] ;

[0107] .

[0108] in, and Let x and y be the x and y coordinates of the top-left corner of the j-th initial ROI in the frame coordinate system, respectively. and These represent the width and height of the frame image, respectively. and These are uniform ROI width and uniform ROI height, respectively. (Constraints) This indicates that the left and right boundaries of the initial ROI do not exceed the horizontal boundary representing the frame; constraints This means that the upper and lower boundaries of the initial ROI do not exceed the vertical boundary of the representative frame. The two constraints together limit the coordinates of the upper left corner of the initial ROI, ensuring that the initial ROI remains completely within the image range of the representative frame while maintaining a uniform size.

[0109] In some embodiments, the operation instructions include one or more of drag, corner scaling, boundary constraint, deletion, regeneration, size adjustment, or position adjustment. After executing the operation instructions, the confirmed regions of interest are reordered according to row and column sorting and consecutive numbers are generated.

[0110] The initial ROI is displayed as an editable graphical object, and its number is shown in the interface. Figure 3a The automatically generated initial ROI (green box) is shown. Figure 3b The text shows the user-confirmed ROI (green box) and the overall cropping area (red box) after editing. Figure 3c The relative ROI is shown with the top left corner of the overall cropped area (red box) as the origin. Figure 5 This displays the region assignment results of fish or feed targets in a relative coordinate system, along with corresponding region statistics. Users can execute commands such as moving, scaling, deleting, or regenerating ROIs. After user confirmation, the number of confirmed ROIs is counted, and the center point is calculated based on the coordinates of the top-left corner, width, and height of the confirmed ROIs. The rows and columns are then re-sorted according to the rules for sorting candidate observation region boxes, generating consecutive numbers.

[0111] ;

[0112] in, For the j-th confirmed ROI, and The coordinates of the top left corner of the area after user confirmation. and This refers to the confirmed width and height. If a user deletes or adds an ROI, it will be reordered and renumbered after confirmation to ensure consistency between the exported coordinates and the interface numbering.

[0113] .

[0114] in, This indicates the total number of confirmed ROIs. Indicates the number of elements in the set. This represents the set consisting of all confirmed ROIs. This reflects the final number of valid observation areas after the user completes editing and confirmation; when the user adds, deletes, or regenerates ROIs, the system recalculates using the updated confirmed ROI set. Based on this, subsequent overall cropping area calculations, relative ROI coordinate outputs, and numbering consistency checks are performed.

[0115] In some embodiments, the method for generating relative region of interest coordinates based on the confirmed regions of interest is as follows: calculate the minimum bounding rectangle of all confirmed regions of interest, determine the upper left corner, lower right corner, width, and height of the minimum bounding rectangle, and convert the original coordinates of each confirmed region of interest into relative coordinates relative to the upper left corner of the minimum bounding rectangle.

[0116] To reduce the background area for subsequent detection, the system treats the outer perimeter of all confirmed ROIs as the overall cropping region G. Compared to detecting directly on the entire image, this approach reduces the influence of pool walls, supports, blank areas, and irrelevant reflections.

[0117] The overall clipping region G is represented by its top-left corner coordinates, bottom-right corner coordinates, width, and height, and is determined by the bounding boundary of all confirmed ROIs. The calculation relationship is as follows:

[0118] ;

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] ;

[0124] .

[0125] in, and The coordinates of the top left corner of the overall cropping area. and The coordinates of the bottom right boundary. and This defines the width and height of the overall cropping area. The minimum and maximum values ​​mentioned above are calculated across all confirmed ROIs, ensuring that the cropping area encompasses all confirmed observation areas.

[0126] like Figure 3b As shown, the overall clipping region G is the smallest bounding rectangle enclosing all confirmed ROIs, and its top-left corner coordinates, bottom-right corner coordinates, width, and height are marked as follows: , , and .like Figure 3c As shown, the upper left corner of the overall cropping region G is taken as the relative coordinate origin. This will confirm that the ROI has been transformed to a relative coordinate system within the overall clipping region G.

[0127] Using the top-left corner coordinates of the overall cropping area as the relative origin, each confirmed ROI is converted into a relative ROI. A relative ROI only changes its top-left corner coordinates; its width and height remain unchanged.

[0128]

[0129] in, Let j be the j-th relative ROI. During the transformation, only the coordinates of the top-left corner of the confirmed ROI are subtracted from the coordinates of the top-left corner of the overall cropped region, while the width and height remain unchanged. Therefore, the target coordinates obtained by subsequent algorithms in the overall cropped image can be directly compared with the relative ROI. The original coordinates of the confirmed ROI are used to preserve its absolute position in the representative frame; the relative ROI coordinates are used for region attribution calculation in the subsequent overall cropped image. Since subsequent target detection is performed within the overall cropped region G, the target representative point and the relative ROI are located in the same coordinate system, allowing for direct region attribution determination without repeatedly performing the transformation from full-image coordinates to cropped coordinates for each target.

[0130] When a target representative point p is subsequently detected, the system determines its region affiliation based on the left-closed, right-open boundary of the relative ROI. If the representative point does not belong to any relative ROI, the system returns 0, indicating no valid affiliation.

[0131] ;

[0132] ;

[0133] ;

[0134] .

[0135] Where p is a representative point of the target to be detected later. and These represent the x and y coordinates of the point in the overall clipping coordinate system; and Let be the coordinates of the top-left corner of the j-th relative ROI. The left-closed, right-open boundary rule can prevent the same point from being assigned repeatedly when adjacent ROIs share a boundary; when ROIs overlap, the assignment can be determined first according to the principle of being closest to the center of the ROI.

[0136] like Figure 4 As shown, a representative point p can be selected within the detection frame of the fish or feed target, for example, the center point of the detection frame. When the representative point p is located within the j-th relative ROI, the target is assigned to the j-th region and denoted as [region name missing]. When the target point p is not located within any relative ROI, it is denoted as... .

[0137] like Figure 1 As shown, the automatic partitioning detection method based on editable biological behavior observation regions in this embodiment includes the following steps:

[0138] S1. Obtain representative frames from the monocular top-down experimental video.

[0139] S2. Obtain the number of preset observation areas.

[0140] S3. The target detection model of the observation area is used to detect the representative frame with a detection threshold to obtain a set of candidate observation area boxes. The candidate observation area boxes in the set of candidate observation area boxes are deduplicated and geometrically filtered to obtain the effective candidate observation area boxes.

[0141] S4. If the number of valid candidate observation area boxes is greater than or equal to the preset number of observation areas, proceed to step S6; otherwise, proceed to step S5.

[0142] S5. Using the target detection model in the observation area, a reduced detection threshold is used to detect the representative frame to obtain a set of supplementary observation region boxes. After merging the set of candidate observation region boxes and the set of supplementary observation region boxes, deduplication and geometric filtering are performed to obtain the effective candidate observation region boxes, and then proceed to step S4.

[0143] S6. Sort the effective candidate observation region boxes by rows and columns and normalize their sizes to obtain the initial region of interest.

[0144] S7. Display the initial region of interest as an editable graphic object, and confirm the region of interest in response to operation commands.

[0145] S8. Generate a global clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and determine the relative region of interest coordinates of the confirmed regions of interest relative to the global clipping region.

[0146] An automated zoning detection system based on editable biological behavior observation regions includes:

[0147] The frame acquisition unit is used to acquire representative frames from the monocular top-down experimental video.

[0148] Candidate detection units are used to detect representative frames using the observation area target detection model to obtain a set of candidate observation area boxes.

[0149] The effective candidate observation region box determination unit is used to perform deduplication and geometric filtering on the candidate observation region boxes in the candidate observation region box set to obtain an effective candidate observation region box with a number not less than the preset number of observation regions.

[0150] The row and column sorting and size normalization unit is used to sort the effective candidate observation area boxes by row and column and normalize their size to obtain the initial region of interest.

[0151] An editable unit is used to display the initial region of interest as an editable graphic object, and to confirm the region of interest in response to an operation command.

[0152] The coordinate transformation output unit is used to generate a global clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and to determine the relative region of interest coordinates of the confirmed regions of interest relative to the global clipping region.

[0153] In some embodiments, the effective candidate observation region box determination unit is used to reduce the detection threshold of the target detection model in the observation region and perform supplementary detection to obtain a supplementary candidate observation region box set when the number of effective candidate observation region boxes obtained by deduplication and geometric filtering of the candidate observation region boxes in the candidate observation region box set is lower than the preset number of observation regions. The unit then merges the candidate observation region box set and the supplementary candidate observation region box set and performs deduplication and geometric filtering to obtain effective candidate observation region boxes until the number of effective candidate observation region boxes is not less than the preset number of observation regions.

[0154] Obtain the coordinates of the top left corner, width, and height of the candidate observation region box, and calculate the coordinates of the center point, width, height, and area of ​​the candidate observation region box for set filtering and row / column sorting.

[0155] In some embodiments, the deduplication method of the effective candidate observation region box determination unit includes: obtaining the confidence level of the candidate observation region boxes, processing the candidate observation region boxes in descending order of confidence level, calculating the cross-union ratio between the candidate observation region boxes, and retaining the candidate observation region boxes with high confidence level and removing the candidate observation region boxes with low confidence level when the cross-union ratio of two candidate observation region boxes is higher than a preset deduplication threshold.

[0156] In some embodiments, the row and column sorting method for size-normalized cells includes:

[0157] The effective candidate observation area boxes are processed in ascending order of their center point ordinates. The median of the center point ordinates of the effective candidate observation area boxes already included in the current row is used as the reference ordinate for that row. When the difference between the center point ordinate of the effective candidate observation area box to be processed and the reference ordinate is lower than the preset row grouping tolerance, the effective candidate observation area box is included in the current row; otherwise, a new row is created. The rows are sorted in ascending order of their reference ordinates to determine the row number, and within each row, the column number is determined in ascending order of the center point abscissa of the effective candidate observation area box. A continuous region of interest number is generated based on the row number and column number.

[0158] In some embodiments, the method for normalizing the size of row and column sorting and size normalization units includes:

[0159] The width and height of the unified region of interest are determined based on the mean, median, or truncated mean of the set of widths and heights of the valid candidate observation regions.

[0160] In some embodiments, the operation instructions for the editable unit include one or more of drag, corner scaling, boundary constraint, deletion, regeneration, size adjustment, or position adjustment. After executing the operation instructions, the confirmed regions of interest are reordered according to row and column sorting and consecutive numbers are generated.

[0161] In some embodiments, the coordinate transformation output unit generates relative region of interest coordinates based on the confirmed region of interest by: calculating the minimum bounding rectangle of all confirmed regions of interest, determining the upper left corner, lower right corner, width, and height of the minimum bounding rectangle, and converting the original coordinates of each confirmed region of interest into relative coordinates relative to the upper left corner of the minimum bounding rectangle.

[0162] This embodiment can reduce the repetitive selection of bounding boxes in multi-observation area experiments; it is more adaptable to changes in camera position and observation container placement compared to fixed ROI; it is less dependent on lighting and edge contrast compared to traditional threshold segmentation; this embodiment transforms the detection results into editable, verifiable, and reusable coordinate references, which are convenient for subsequent fish detection, feed detection, behavior statistics, and experimental verification.

[0163] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions claimed by the present invention.

Claims

1. An automatic partitioning detection method based on editable biological behavior observation regions, characterized in that, The method includes: Obtain representative frames from the monocular top-down experimental video; Obtain the number of preset observation areas; The representative frame is detected by the observation area target detection model to obtain a set of candidate observation area boxes. The candidate observation area boxes in the set of candidate observation area boxes are deduplicated and geometrically filtered to obtain a number of effective candidate observation area boxes that are no less than the preset number of observation areas. The effective candidate observation region boxes are sorted by row and column and their sizes are normalized to obtain the initial region of interest; The initial region of interest is displayed as an editable graphic object, and the region of interest is confirmed in response to an operation command; Generate an overall clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and determine the relative region of interest coordinates of the confirmed regions of interest relative to the overall clipping region.

2. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, When the number of valid candidate observation region boxes obtained by deduplication and geometric filtering of the candidate observation region box set is lower than the preset number of observation regions, the detection threshold of the target detection model in the observation region is reduced to obtain a supplementary candidate observation region box set. The candidate observation region box set and the supplementary candidate observation region box set are merged and then deduplicated and geometrically filtered to obtain valid candidate observation region boxes, until the number of valid candidate observation region boxes is not less than the preset number of observation regions.

3. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, Obtain the coordinates of the top left corner, width, and height of the candidate observation region box, and calculate the coordinates of the center point, width, height, and area of ​​the candidate observation region box for set filtering and row / column sorting.

4. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, The deduplication method includes: obtaining the confidence level of the candidate observation region boxes; processing the candidate observation region boxes in descending order of their confidence levels; calculating the intersection-union ratio (IUU) between the candidate observation region boxes; and retaining the candidate observation region boxes with high confidence levels and removing the candidate observation region boxes with low confidence levels when the IUU of two candidate observation region boxes is higher than a preset deduplication threshold.

5. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, The row and column sorting method includes: The valid candidate observation region boxes are processed in ascending order of their center point ordinates. The median of the center point ordinates of the valid candidate observation region boxes already included in the current row is used as the reference ordinate for that row. When the difference between the center point ordinate of the valid candidate observation region box to be processed and the reference ordinate is lower than the preset row grouping tolerance, the valid candidate observation region box is included in the current row; otherwise, a new row is created. The rows are sorted in ascending order of their reference ordinates to determine the row number, and within each row, the column number is determined in ascending order of the center point abscissa of the valid candidate observation region box. A continuous region of interest number is generated based on the row number and column number.

6. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, The size normalization method includes: The width and height of the unified region of interest are determined based on the mean, median, or truncated mean of the set of widths and heights of the effective candidate observation region boxes.

7. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, The operation instructions include one or more of drag, corner scaling, boundary constraint, deletion, regeneration, size adjustment, or position adjustment. After the operation instructions are executed, the confirmed regions of interest are reordered according to row and column sorting and consecutive numbers are generated.

8. The automatic partitioning detection method based on editable biological behavior observation regions according to claim 1, characterized in that, The method for generating relative region of interest coordinates based on the confirmed regions of interest is as follows: calculate the minimum bounding rectangle of all confirmed regions of interest, determine the upper left corner, lower right corner, width, and height of the minimum bounding rectangle, and convert the original coordinates of each confirmed region of interest into relative coordinates relative to the upper left corner of the minimum bounding rectangle.

9. An automatic zoning detection system based on editable biological behavior observation regions, characterized in that, The system includes: The frame acquisition unit is used to acquire representative frames of the monocular top-down experimental video. The candidate detection unit is used to detect the representative frame using the observation area target detection model to obtain a set of candidate observation area boxes; The effective candidate observation region box determination unit is used to perform deduplication and geometric filtering on the candidate observation region boxes of the candidate observation region box set to obtain an effective candidate observation region box with a number not less than the preset number of observation regions. The row and column sorting and size normalization unit is used to sort the effective candidate observation region box by rows and columns and normalize its size to obtain the initial region of interest. An editable unit is used to display the initial region of interest as an editable graphic object, and to confirm the region of interest in response to an operation command; The coordinate transformation output unit is used to generate an overall clipping region that surrounds all confirmed regions of interest based on the confirmed regions of interest, and to determine the relative region of interest coordinates of the confirmed regions of interest relative to the overall clipping region.

10. The system according to claim 9, characterized in that, The effective candidate observation region box determination unit is used to reduce the detection threshold of the target detection model in the observation region and perform supplementary detection to obtain a supplementary candidate observation region box set when the number of effective candidate observation region boxes obtained by deduplication and geometric filtering of the candidate observation region boxes in the candidate observation region box set is lower than the preset number of observation regions. The unit then merges the candidate observation region box set and the supplementary candidate observation region box set and performs deduplication and geometric filtering to obtain effective candidate observation region boxes until the number of effective candidate observation region boxes is not less than the preset number of observation regions.

Images