A monitoring method based on cascade target detection

Abstract

The present application relates to the field of computer vision, and particularly relates to a monitoring method based on cascade target detection, comprising: acquiring a monitoring picture; identifying the monitoring picture through a trained first YOLO model, and cutting out target region images; identifying each target region image through a trained second YOLO model, and acquiring target information therein; collecting the target information to an upper computer; the target information comprises a region number, and the step of acquiring the region number comprises: locating target character information in each target region image through the trained second YOLO model; identifying the located target character information through OCR technology, and confirming the region number based on the identified information; the step of collecting the target information to the upper computer comprises: when any target region fails to be identified within a preset time length, collecting and transmitting target information corresponding to the target region to the upper computer, and after successful transmission, limiting other target information carrying a region number of the target region from being uploaded again.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically to a monitoring method based on cascaded target detection. Background Technology

[0002] Modernized three-dimensional cage farming has become the mainstream model for large-scale poultry farming. Compared with traditional free-range and single-layer cage farming, it has higher space utilization, higher stocking density, and higher standardization, which can effectively improve the production efficiency of poultry farming. In the process of three-dimensional poultry farming, the number of live chickens in a single cage and real-time egg production data are the core basic data for the farm's refined feeding and production capacity accounting. The accuracy of data statistics directly determines the farm's large-scale production efficiency.

[0003] Currently, most small, medium, and large-scale integrated poultry farms still rely on manual inspection and counting to calculate chicken inventory and egg production. However, this manual operation mode has inherent drawbacks: firstly, the numerous layers and dense number of cages in integrated farming make manual cage-by-cage checks extremely labor-intensive and inefficient, making it difficult to adapt to the pace of large-scale, high-density farming; secondly, manual counting and visual inspection are easily affected by factors such as staff condition, fatigue, and subjective judgment, leading to omissions, miscounts, and data discrepancies. Furthermore, manual data recording is lagging, failing to provide real-time feedback on the actual farming status of each cage, and thus failing to meet the refined, real-time, and precise management needs of modern farming.

[0004] To overcome the numerous drawbacks of manual operations, the industry is gradually adopting automated visual inspection methods to replace manual data collection in aquaculture. Existing visual inspection methods mostly employ a unified recognition and detection logic across the entire field, performing overall identification and statistics on targets such as chickens and eggs within the aquaculture scene. However, three-dimensional aquaculture scenarios involve multi-layered, multi-column modular cage structures with closely connected cage areas and highly homogeneous scenes. Existing full-field visual inspection methods can only complete the overall target quantity count, making it difficult to segment the identified target data into regions and accurately map the detection data to specific cage areas. This results in technical shortcomings such as inconsistent data attribution and inaccurate matching of data from individual cages.

[0005] Therefore, how to overcome the shortcomings of the existing technology is the subject of this invention. Summary of the Invention

[0006] The purpose of this invention is to provide a monitoring method based on cascaded target detection.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A monitoring method based on cascaded target detection includes:

[0009] Obtain surveillance footage;

[0010] The first YOLO model, after training, identifies the surveillance footage and crops out images of each target area.

[0011] The trained second YOLO model is used to identify images of each target region and obtain target information in each target region image.

[0012] The target information is aggregated and transmitted to the host computer.

[0013] The target information includes a region code, and the steps to obtain the region code include:

[0014] The target character information in each target region image is located using the trained second YOLO model;

[0015] The target character information is identified using OCR technology, and the region number of the corresponding target area image is confirmed based on the identified information.

[0016] The steps for summarizing target information and transmitting it to the host computer include:

[0017] If any target area fails to be identified within a preset time period (which can be adjusted as needed), the target information corresponding to that target area will be summarized and transmitted to the host computer. After successful transmission, other target information carrying the area number of that target area will be restricted from being uploaded to the host computer again, thus preventing duplicate counting and interference from abnormal data.

[0018] This application describes an indoor three-dimensional poultry farming scenario, but is not limited to being applicable only to this scenario.

[0019] For example, by training YOLO models cage.pt and items.pt, the cage.pt model (i.e., the first YOLO model) first identifies and crops images of each chicken cage area from the complete video footage (image frames) obtained by the inspection robot. Then, within the chicken cage area images, the items.pt model (i.e., the second YOLO model) identifies the chickens, eggs, and chicken cage numbers (e.g., number 1). The chicken cage area identified by the cage.pt model is defined as chicken cage number 1, and the corresponding number of chickens and eggs is counted, assigned to chicken cage number 1, and sent to the host computer. The following explanation can be based on an example from this application.

[0020] Understandably, the YOLO model can be trained on a dataset to accurately identify a specific target in an image, a process known as object detection. Model training itself is standard practice; a simple example is provided to aid understanding: Consider the entire chicken coop area (including the coop itself, its identification number, the chickens inside, and the conveyor belt used to carry and transport eggs) as the target to be detected. Data containing the entire chicken coop area is collected from the chicken farm. After data cleaning and filtering, the most frequently occurring chicken coop areas are manually labeled to ensure the accuracy of the trained cage.pt model in identifying the entire chicken coop area.

[0021] Understandably, the second YOLO model can combine with other technologies (such as OCR technology) to obtain target information in the images of each target region.

[0022] In indoor multi-level poultry farming scenarios, the cages are densely packed together. Due to factors such as cage mesh obstruction, target overlap, and uneven lighting, it is difficult to accurately assign the identified targets to the corresponding cage numbers when using a single visual YOLO model to detect chickens and eggs. This can also lead to false detections across cages and missed detections of small targets. Therefore, this application adopts a cascaded target detection scheme using a first YOLO model and a second YOLO model. Specifically, the first YOLO model first acquires images of each target region, such as anchoring individual cage region images within the image frame. Then, the second YOLO model identifies chickens, eggs, and numbered targets in each cage region image. This ensures that the collected data can be accurately assigned to the corresponding cage number (i.e., accurately assigned to the corresponding cage region), while reducing the probability of false detections across cages and missed detections of small targets.

[0023] In summary, the monitoring method in this application can divide the detected target data into regions and accurately map the target data to specific target regions, thus solving the problems of data confusion and inaccurate matching.

[0024] The target information includes the area code, and the steps to obtain the area code include:

[0025] The target character information in each target region image is located using the trained second YOLO model;

[0026] The target character information is identified using OCR technology, and the region number of the corresponding target area image is confirmed based on the identified information.

[0027] The target character information is not limited to numbers; it may also include letters, etc., all of which can be recognized using OCR technology. This application mainly focuses on the case where the target character information consists of numbers.

[0028] It should be noted that, taking the area code as the chicken coop location number (or chicken coop number) as an example (this application can be used as an illustration), the chicken coop location number consists of scattered independent numbers. When using global large-image OCR recognition, there are drawbacks such as low recognition efficiency and susceptibility to interference from complex backgrounds. If individual numbers are segmented and independently OCRed, it is impossible to restore the order of the numbers or match the corresponding chicken coop.

[0029] Regarding global large-image OCR recognition, the following explanation is provided: When using global large-image OCR, if the numbers in the chicken coop numbers are closely arranged from a direct viewpoint, the OCR can recognize and extract the number "168" (for example). However, when the robot's camera viewpoint is slightly to the left of the chicken coop numbers, the "1" and "6" are closely arranged due to the perspective, while the "6" and "8" are spaced apart. In this case, two sets of numbers, "16" and "8," may be recognized. This problem occurs more frequently in large-scale chicken coop scenarios with tens of thousands of chicken coop numbers and is a common issue in practical applications. To solve this problem, a QR code can be affixed to each chicken coop. The robot can then point its camera at the QR code to obtain information about each coop (including the coop number, the expected number of chickens, etc.). However, this method suffers from low recognition efficiency in large-scale chicken coop scenarios.

[0030] To address the aforementioned issues, this application uses a trained first YOLO model to identify surveillance footage and crop out images of each target region, thereby identifying key areas. Next, a trained second YOLO model is used to locate the specific region containing the target character information within the key area. Then, OCR technology is used to identify the target character information within that region, thus confirming the region number of the corresponding target area image. This approach balances the requirements of high recognition efficiency (compared to using QR code recognition) and high recognition accuracy (compared to using global large-image OCR recognition).

[0031] It's important to note that the second YOLO model can generally only locate the specific area of ​​the target character information, but cannot recognize it. Taking chicken cage location number recognition as an example, if the second YOLO model were used directly, all the numbers would need to be collected as images, manually labeled, and then used to train the model. However, chicken farms have tens of thousands of chicken cages, each with tens of thousands of numbers. This training method not only results in long training times and high labor costs, but also lacks universality. This is because although most standardized chicken farms in China have similar internal structures, the numbering is mostly set by the manufacturers themselves, leading to variations in style. Therefore, OCR technology is used for chicken cage location number recognition.

[0032] To clarify, besides the target character information—the chicken coop location number—the target area image may also contain other non-target character information. For example, the chicken coop may contain information such as equipment manufacturer names or trademarks. Based on this, the second YOLO model can be trained specifically for recognizing chicken coop location numbers to avoid misidentifying non-target character information. This targeted training is a standard training method for YOLO models and will not be elaborated upon here.

[0033] The steps for summarizing target information and transmitting it to the host computer include:

[0034] If any target area fails to be identified within a preset time period (which can be adjusted as needed), the target information corresponding to that target area will be summarized and transmitted to the host computer. After successful transmission, other target information carrying the area number of that target area will be restricted from being uploaded to the host computer again, thus preventing duplicate counting and interference from abnormal data.

[0035] For example, chicken coop area 1 has a continuously updated chicken coop registry 1. During one inspection, chicken coop area 1 generates several images. After these images are identified, chicken coop area 1 can no longer be identified until the next inspection. At this time, the updated chicken coop registry 1 is transmitted to the host computer. After successful upload, if a subsequent chicken coop registry containing area number 1 is encountered, its upload is rejected (e.g., through thread locks, the existing method), to avoid interfering with the determination of the number of eggs in chicken coop area 1, ensuring data consistency, and preventing duplicate uploads of chicken coop registry 1 after successful transmission.

[0036] Understandably, if the upload of registry 1 fails, the transmission will be automatically retried to ensure data transmission consistency.

[0037] A further technical solution involves locating target character information in each target region image using a trained second YOLO model, including the following steps:

[0038] Obtain the target bounding box of each target text in the image of each target region;

[0039] The target bounding boxes are sorted horizontally based on their x-coordinates, and their dimensions are simultaneously detected. Fixed fill margins are reserved around each target bounding box.

[0040] The sorted target bounding boxes are merged and stitched together according to their relative positions to form a continuous digital image without gaps.

[0041] For example, the target bounding box is a rectangular frame defined by an algorithm to enclose a single chicken coop number. A fixed width of padding margin is reserved outward in the four directions of the top, bottom, left, and right of each target bounding box to avoid the image cropping boundary being too close to the edge of the number. That is, a fixed padding margin is reserved around each target bounding box.

[0042] For example, the targets are horizontally sorted from left to right based on the x-coordinate of each target bounding box.

[0043] For example, after forming a continuous digital image without gaps, it is sent as the input image to the OCR model. The continuous digital image is recognized by OCR technology, and the unique region number of the corresponding target region image is confirmed based on the recognition information.

[0044] It should be noted that the spacing between the numbers in the area code may be large (depending on the settings of the farm owner or manufacturer). For example, the chicken cage position number "118" could easily be misinterpreted as "11" and "8", resulting in one chicken cage having two position numbers, making it impossible to monitor that chicken cage properly. The settings in this section ensure that one chicken cage corresponds to only one position number, further improving the accuracy of target character information recognition and eliminating background interference, making it suitable for efficient operation of edge devices.

[0045] Regarding the issue of identifying two cage positions for a single chicken cage, here's an example: If two numbers, such as "11" and "8", are identified in a chicken cage area, the system will first upload the count of chickens and eggs to the database for chicken cage number 11 (or register it in the chicken cage registry 11). Then, because the number "8" is also detected, the same data will be uploaded again to the database for chicken cage number 8, resulting in an error in the final aggregated data.

[0046] A further technical solution involves the following steps to aggregate the target information to the host computer:

[0047] Based on the corresponding region number, an independent local cache is established for each target region, and the target information in each target region is tracked across frames to continuously iterate and update the target information during the monitoring period of the target region.

[0048] Upload the target information for the terminated update to the host computer.

[0049] For example, based on the cage position numbers corresponding to each chicken cage area, a separate local cache is established for each chicken cage area, such as a separate chicken cage registry. This registry includes target information such as the corresponding cage position number, the number of chickens, and the number of eggs. Each chicken cage registry is configured with a unique cage position number, and the identified target information such as the number of chickens is encoded into the corresponding registry based on this cage position number. The chicken cage registry may also include corresponding image frames, such as one or two image frames corresponding to the maximum number of eggs and the maximum number of chickens detected.

[0050] Local caches are temporary data storage units specifically allocated for individual target regions. Each target region's local cache is isolated from the others (data is not shared) and is used only to store target information for that specific region. Relying on these dedicated local caches for each target region, continuous cross-frame correlation tracking of target information within the region is possible. Thus, within the monitoring period of the target region, the target information stored in the local cache is continuously iterated and updated based on the real-time detection results of each image frame, ensuring that the target information remains up-to-date or in its most accurate state throughout the monitoring period. For example, continuously tracking the number of chickens in each image frame of chicken coop location number 001, when the identified number of chickens exceeds the number of chickens currently in the chicken coop registry, the new identification result is added to the chicken coop registry to replace the previous identification result, thus updating the maximum number of chickens and maintaining the most accurate state.

[0051] For example, if no frame contains area number one for a period of time, it indicates that the monitoring cycle for the chicken coop area corresponding to area number one has ended, the relevant information is stopped being updated, and the target information for which updates have been terminated is uploaded to the host computer.

[0052] A further technical solution is that the target area is the chicken coop area, the target area image is the chicken coop area image, the area number is the chicken coop number, and the target information also includes the number of eggs and the number of chickens.

[0053] The steps for summarizing target information and transmitting it to the host computer include:

[0054] Based on the chicken cage number, an independent local cache is established for each chicken cage area, and the target information in each chicken cage area is tracked across frames.

[0055] During the monitoring period in the chicken coop area, the maximum number of eggs and the maximum number of chickens identified are continuously updated iteratively until the target information is no longer updated.

[0056] Upload the target information for the terminated update to the host computer.

[0057] It should be noted that, taking the chicken coop area as an example, the chickens in the coop move around, causing them to easily obscure each other, ultimately making it difficult for a single frame to reflect the true number of chickens in the coop. Therefore, this section implements cross-frame tracking, continuously iterating and updating the maximum number of chickens that can be identified within the chicken coop area's monitoring cycle until the target information is no longer updated. This ensures that the information uploaded to the host computer is accurate, further improving the accuracy of target information recognition.

[0058] It should also be noted that, compared to identifying chickens in different image frames and then combining the chickens in each image frame to obtain the final number of chickens identified, the statistical method in this application is more convenient and requires less computation. Here is a brief example of the former method: if there are four chickens in the first image frame, five chickens in the second image frame, and one of the four chickens in the first image frame is not identified in the second image frame, then it is determined that there are six chickens.

[0059] For example, in a series of image frames of a video, the chicken coop area No. 001 appears repeatedly in multiple frames. Affected by factors such as changes in ambient lighting and partial occlusion of objects, the number of chickens detected in the coop varies in different image frames. Therefore, this solution selects the maximum number of chickens detected in all image frames corresponding to the coop (determined by the coop number) as the final statistical data of the number of chickens in the coop, and uploads this data to the host computer.

[0060] In a further technical solution, the area code consists of numbers;

[0061] For the area numbers uploaded to the host computer, a number queue is established from front to back based on the obtained time. The area numbers of each adjacent area in the number queue are compared. If the absolute value of the difference between the two is greater than the first threshold, it is determined that the area number in the latter area is incorrect, and an alarm message can be issued.

[0062] Here's an example: During an inspection, there will be several image frames for a chicken coop. The number of eggs in the image frames numbered 1 is compared, and the image frame with the most eggs is selected and uploaded to the host computer (uploading is completed once the judgment is complete). This is the target area image to be uploaded to the host computer (hereinafter referred to as chicken coop area image 1, or chicken coop registry 1). The other image frames are discarded. When a second image frame needs to be uploaded to the host computer, this is the adjacent target area image of chicken coop area image 1. If it is chicken coop area image 2, there is no problem. If it is chicken coop area image 10, it is a skipped frame, and the corresponding area number 10 is incorrect.

[0063] When an area number is determined to be incorrect, this information will be recorded and aggregated to the host computer. Monitoring personnel can check these error messages through the host computer, thereby prompting manual on-site inspection. In other words, alarm information can refer to the reminder information aggregated on the host computer, and the specific alarm form is not limited.

[0064] It should be noted that visual inspection and recognition technologies (such as OCR technology) are limited by environmental and network factors, and may sometimes experience false detections or missed detections. For example, poor lighting, heavy dust, or chicken feathers obscuring the area can lead to errors in numbering the identification areas, thus affecting recognition accuracy. For instance, the chicken cage numbers identified by a chicken farm inspection robot should be sequential, such as 006, 007, 008, ..., but sometimes environmental factors can cause errors in optical character extraction, resulting in skipped numbering, such as identifying 911 after 115.

[0065] Based on the aforementioned issue of skipped numbering, this section sets up a queue of region numbers uploaded to the host computer, arranged from front to back according to the time of acquisition. It then compares adjacent region numbers in the queue. If the absolute value of the difference between the two numbers exceeds a first threshold, it is determined that the subsequent region number is incorrect, and an alarm can be issued (this data can be automatically recorded for manual review). For example, with the first threshold set to 3, if the previous identified number is 017 and the next identified number is 019, the difference is 2. This is determined to be due to data loss during network or data transmission, thus eliminating the need for an alarm.

[0066] To further clarify, data classified as "loss of network data transmission" will not proceed to the manual review process. However, data classified as "skipped sequence number frame" will trigger the manual review process for that frame and its corresponding image. The reasons are as follows:

[0067] When the difference in the number between adjacent frames is small (e.g., less than or equal to 2), it is usually caused by intermittent network failure. Since the robot has to inspect the chicken cages multiple times a day and update the data in real time, the impact of such problems is small. If such problems require a manual review process, it will lead to a significant increase in labor costs in large-scale chicken coop scenarios.

[0068] When the difference between adjacent frames is large, the factors causing this problem are generally that the numbering area is obscured or damaged, the network is disconnected for a long time, or the lighting facilities are malfunctioning. In this case, a manual review process needs to be triggered to avoid causing continuous and large-scale identification errors.

[0069] A further technical solution involves using OCR technology to identify the located target character information, obtaining an initial region number, and then updating the initial region number using a preset OCR character error correction mapping table to obtain the region number of the corresponding target region image.

[0070] The area code consists of numbers. For example, the chicken coop numbers are all composed of numbers. During recognition, due to factors such as dim lighting, thick dust, chicken feathers obscuring the area, or wear and tear on the chicken coop numbers over long-term use, some numbers may be recognized as letters (e.g., 005 may be recognized as 00S). In this case, these letters can be updated to numbers based on a preset OCR character error correction mapping table (e.g., updating S to 5), which further improves the accuracy of target information recognition.

[0071] The OCR character correction mapping table can be preset and updated as needed. For example, the OCR character correction mapping table includes: O→0, I→1, S→5, B→8.

[0072] A further technical solution is to remove the permission of erroneous area numbers that are judged to have deviations from the number comparison (without removing them from the number queue).

[0073] For two region numbers adjacent to the erroneous region number, the two are compared and the second threshold is used as the comparison judgment basis. The second threshold = the first threshold × (A+1), or the second threshold = the first threshold + A, where A is the number of erroneous region numbers between the two region numbers.

[0074] This section essentially establishes a valid numbering history queue mechanism. Each comparison is based on the most recent valid number in the queue. For example: if the region number of image 1 (chicken coop area) is 1, and the region number of image 2 (chicken coop area) is 4, the difference between the two is 3, which is greater than the first threshold of 2. Therefore, the region number of image 2 will not be included in subsequent numbering comparisons. Image 3 (chicken coop area) has a region number of 3. It is compared with 1, not with 4. The difference is 2, which is less than the second threshold of 4 (the first threshold multiplied by 2). This avoids the problem of the region number of image 2 affecting the determination of the region number of image 3.

[0075] Generally, if the first threshold is greater than 1, and the second threshold is equal to the first threshold × (A + 1), the number of error area numbers is less (compared to when the second threshold equals the first threshold + A), which can reduce the frequency of manual on-site verification. Conversely, if the second threshold equals the first threshold + A, the number of error area numbers is more, making it more suitable for scenarios requiring high reliability risk assessment.

[0076] For the first region number in the numbering queue, it can be compared with a preset value (such as 1). The comparison is based on the first threshold. If the region number is an incorrect region number, the next region number is taken as the first region number and compared with the preset value + B. B is the number of incorrect region numbers before the first region number. The comparison is based on the first threshold.

[0077] A further technical solution separates the optical character recognition task and the network upload task for aggregating target information to the host computer from the main thread of video stream processing. Independent OCR asynchronous queues and network upload asynchronous queues are configured respectively. The target area image data to be optically recognized and the target information to be aggregated and uploaded to the host computer are respectively sent to their respective matched asynchronous queues for background scheduling and processing.

[0078] It's important to note that in real-time video streaming scenarios, the video stream processing rate can easily become mismatched with the Optical Character Recognition (OCR) processing rate and the network upload processing rate, potentially causing stuttering and frame delays. Therefore, this section separates the time-consuming OCR processing and network upload operations from the main video stream thread, constructing independent asynchronous processing queues: an OCR asynchronous queue and a network upload asynchronous queue. Data to be recognized and uploaded are placed into their respective asynchronous queues for background asynchronous scheduling, preventing time-consuming operations from blocking the real-time video stream processing flow and ensuring the real-time performance and smoothness of the video stream. This asynchronous queue setup is standard practice; the specific method for constructing asynchronous queues can be found in existing documentation.

[0079] The terms "first," "second," etc., used in this article do not specifically refer to order or sequence, nor are they intended to limit this case; they are merely used to distinguish components or operations described using the same technical terms.

[0080] The terms "connection" or "positioning" as used in this article can refer to two or more components or devices making direct physical contact with each other, or making indirect physical contact with each other, or to two or more components or devices operating or moving with each other.

[0081] The terms “include,” “including,” and “have” used in this article are all open-ended, meaning they include but are not limited to.

[0082] Unless otherwise specified, the terms used herein generally have their ordinary meaning in the context of the art, the subject matter, and the specific context. Certain terms used to describe this case will be discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art in describing this case.

[0083] The terms “front,” “back,” “up,” “down,” “left,” and “right” used in this article are directional terms. In this case, they are only used to describe the positional relationship between the structures and are not intended to limit the specific direction of the protection scheme or its actual implementation.

[0084] The working principle and advantages of this invention are as follows:

[0085] In indoor multi-level poultry farming scenarios, the cages are densely packed together. Due to factors such as cage mesh obstruction, target overlap, and uneven lighting, it is difficult to accurately assign the identified targets to their corresponding cage numbers when using a single visual YOLO model to detect chickens and eggs. This can also lead to false detections across cages and missed detections of small targets. Therefore, this application adopts a cascaded target detection scheme using a first YOLO model and a second YOLO model. Specifically, the first YOLO model first acquires images of each target region, such as anchoring individual cage region images within the image frame. Then, the second YOLO model identifies chickens, eggs, and numbered targets within each cage region image. This ensures that the collected data can be accurately assigned to the corresponding cage number, while reducing the probability of false detections across cages and missed detections of small targets.

[0086] In summary, the monitoring method in this application can divide the detected target data into regions and accurately map the target data to specific target regions, thus solving the problems of data confusion and inaccurate matching.

[0087] Furthermore, the monitoring method in this application is configured such that when any target area fails to be identified within a preset time period (which can be adjusted as needed), the target information corresponding to that target area is aggregated and transmitted to the host computer. After successful transmission, other target information carrying the area number of that target area is restricted from being uploaded to the host computer again, thus preventing duplicate counting and interference from abnormal data. Attached Figure Description

[0088] Figure 1 This is a flowchart of a monitoring method based on cascaded target detection according to an embodiment of the present invention. Detailed Implementation

[0089] The present invention will be clearly described below with illustrations and detailed description. Any person skilled in the art who understands the embodiments of the present invention can make changes and modifications based on the technology taught in the present invention without departing from the spirit and scope of the present invention.

[0090] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the scope of this work. Singular forms such as “a,” “this,” “this,” “the,” and “the” as used herein also include plural forms.

[0091] See Figure 1 A monitoring method based on cascaded target detection, comprising:

[0092] Obtain surveillance footage;

[0093] The first YOLO model, after training, identifies the surveillance footage and crops out images of each target area.

[0094] The trained second YOLO model is used to identify images of each target region and obtain target information in each target region image.

[0095] The target information is aggregated and sent to the host computer.

[0096] This embodiment uses an indoor three-dimensional poultry farming scenario for illustration, but it is not limited to being applicable only to this scenario.

[0097] For example, by training YOLO models cage.pt and items.pt, the cage.pt model (i.e., the first YOLO model) first identifies and crops images of each chicken cage area from the complete video footage (image frames) obtained by the inspection robot. Then, within the chicken cage area images, the items.pt model (i.e., the second YOLO model) identifies the chickens, eggs, and chicken cage numbers (e.g., number 1). The chicken cage area identified by the cage.pt model is defined as chicken cage number 1. The number of chickens and eggs identified are counted, assigned to chicken cage number 1, and sent to the host computer. The following description uses this example from this embodiment.

[0098] Understandably, the YOLO model can be trained on a dataset to accurately identify a specific target in an image, a process known as object detection. Model training itself is standard practice; a simple example is provided to aid understanding: Consider the entire chicken coop area (including the coop itself, its identification number, the chickens inside, and the conveyor belt used to carry and transport eggs) as the target to be detected. Data containing the entire chicken coop area is collected from the chicken farm. After data cleaning and filtering, the most frequently occurring chicken coop areas are manually labeled to ensure the accuracy of the trained cage.pt model in identifying the entire chicken coop area.

[0099] Understandably, the second YOLO model can combine with other technologies (such as OCR technology) to obtain target information in the images of each target region.

[0100] In indoor multi-level poultry farming scenarios, the cages are densely packed together. Due to factors such as cage mesh obstruction, target overlap, and uneven lighting, it is difficult to accurately assign the identified targets to the corresponding cage numbers when using a single visual YOLO model to detect chickens and eggs. This can also lead to false detections across cages and missed detections of small targets. Therefore, this embodiment employs a cascaded target detection scheme using a first YOLO model and a second YOLO model. Specifically, the first YOLO model first acquires images of each target region, such as anchoring individual cage region images within the image frame. Then, the second YOLO model identifies chickens, eggs, and numbered targets within each cage region image. This ensures that the collected data can be accurately assigned to the corresponding cage number (i.e., accurately assigned to the corresponding cage region), while reducing the probability of false detections across cages and missed detections of small targets.

[0101] In summary, the monitoring method in this embodiment can divide the detected target data into regions and accurately map the target data to specific target regions, thus solving the problems of data confusion and inaccurate matching.

[0102] In one embodiment of this application, the target information includes a region number, and the step of obtaining the region number includes:

[0103] The target character information in each target region image is located using the trained second YOLO model;

[0104] The target character information is identified using OCR technology, and the region number of the corresponding target area image is confirmed based on the identified information.

[0105] The target character information is not limited to numbers; it can also include letters, etc., all of which can be recognized using OCR technology. This embodiment mainly illustrates the case where the target character information consists of numbers.

[0106] It should be noted that, using the area code as the chicken coop location number (or chicken coop number) as an example (this embodiment can be used for illustration), the chicken coop location number consists of scattered independent numbers. When using global large-image OCR recognition, there are drawbacks such as low recognition efficiency and susceptibility to interference from complex backgrounds. If individual numbers are segmented and independently OCRed, it is impossible to restore the order of the numbers or match the corresponding chicken coop.

[0107] Regarding global large-image OCR recognition, the following explanation is provided: When using global large-image OCR, if the numbers in the chicken coop numbers are closely arranged from a direct viewpoint, the OCR can recognize and extract the number "168" (for example). However, when the robot's camera viewpoint is slightly to the left of the chicken coop numbers, the "1" and "6" are closely arranged due to the perspective, while the "6" and "8" are spaced apart. In this case, two sets of numbers, "16" and "8," may be recognized. This problem occurs more frequently in large-scale chicken coop scenarios with tens of thousands of chicken coop numbers and is a common issue in practical applications. To solve this problem, a QR code can be affixed to each chicken coop. The robot can then point its camera at the QR code to obtain information about each coop (including the coop number, the expected number of chickens, etc.). However, this method suffers from low recognition efficiency in large-scale chicken coop scenarios.

[0108] To address the aforementioned issues, this embodiment uses a trained first YOLO model to identify the monitoring screen and crop out images of each target region, thereby identifying key areas. Next, a trained second YOLO model is used to locate the specific region containing the target character information within the key area. Then, OCR technology is used to identify the target character information in that region, thus confirming the region number of the corresponding target area image. This approach balances the requirements of high recognition efficiency (compared to using QR code recognition) and high recognition accuracy (compared to using global large-image OCR recognition).

[0109] It's important to note that the second YOLO model can generally only locate the specific area of ​​the target character information, but cannot recognize it. Taking chicken cage location number recognition as an example, if the second YOLO model were used directly, all the numbers would need to be collected as images, manually labeled, and then used to train the model. However, chicken farms have tens of thousands of chicken cages, each with tens of thousands of numbers. This training method not only results in long training times and high labor costs, but also lacks universality. This is because although most standardized chicken farms in China have similar internal structures, the numbering is mostly set by the manufacturers themselves, leading to variations in style. Therefore, OCR technology is used for chicken cage location number recognition.

[0110] To clarify, besides the target character information—the chicken coop location number—the target area image may also contain other non-target character information. For example, the chicken coop may contain information such as equipment manufacturer names or trademarks. Based on this, the second YOLO model can be trained specifically for recognizing chicken coop location numbers to avoid misidentifying non-target character information. This targeted training is a standard training method for YOLO models and will not be elaborated upon here.

[0111] In one embodiment of this application, the step of locating target character information in each target region image using a trained second YOLO model includes:

[0112] Obtain the target bounding box of each target text in the image of each target region;

[0113] The target bounding boxes are sorted horizontally based on their x-coordinates, and their dimensions are simultaneously detected. Fixed fill margins are reserved around each target bounding box.

[0114] The sorted target bounding boxes are merged and stitched together according to their relative positions to form a continuous digital image without gaps.

[0115] For example, the target bounding box is a rectangular frame defined by an algorithm to enclose a single chicken coop number. A fixed width of padding margin is reserved outward in the four directions of the top, bottom, left, and right of each target bounding box to avoid the image cropping boundary being too close to the edge of the number. That is, a fixed padding margin is reserved around each target bounding box.

[0116] For example, the targets are horizontally sorted from left to right based on the x-coordinate of each target bounding box.

[0117] For example, after forming a continuous digital image without gaps, it is sent as the input image to the OCR model. The continuous digital image is recognized by OCR technology, and the unique region number of the corresponding target region image is confirmed based on the recognition information.

[0118] It should be noted that the spacing between the numbers in the area code may be large (depending on the settings of the farm owner or manufacturer). For example, the chicken cage position number "118" could easily be misinterpreted as "11" and "8", resulting in one chicken cage having two position numbers, making it impossible to monitor that chicken cage properly. The settings in this section ensure that one chicken cage corresponds to only one position number, further improving the accuracy of target character information recognition and eliminating background interference, making it suitable for efficient operation of edge devices.

[0119] Regarding the issue of identifying two cage positions for a single chicken cage, here's an example: If two numbers, such as "11" and "8", are identified in a chicken cage area, the system will first upload the count of chickens and eggs to the database for chicken cage number 11 (or register it in the chicken cage registry 11). Then, because the number "8" is also detected, the same data will be uploaded again to the database for chicken cage number 8, resulting in an error in the final aggregated data.

[0120] In one embodiment of this application, the step of summarizing the target information to the host computer includes:

[0121] Based on the corresponding region number, an independent local cache is established for each target region, and the target information in each target region is tracked across frames to continuously iterate and update the target information during the monitoring period of the target region.

[0122] Upload the target information for the terminated update to the host computer.

[0123] For example, based on the cage position numbers corresponding to each chicken cage area, a separate local cache is established for each chicken cage area, such as a separate chicken cage registry. This registry includes target information such as the corresponding cage position number, the number of chickens, and the number of eggs. Each chicken cage registry is configured with a unique cage position number, and the identified target information such as the number of chickens is encoded into the corresponding registry based on this cage position number. The chicken cage registry may also include corresponding image frames, such as one or two image frames corresponding to the maximum number of eggs and the maximum number of chickens detected.

[0124] Local caches are temporary data storage units specifically allocated for individual target regions. Each target region's local cache is isolated from the others (data is not shared) and is used only to store target information for that specific region. Relying on these dedicated local caches for each target region, continuous cross-frame correlation tracking of target information within the region is possible. Thus, within the monitoring period of the target region, the target information stored in the local cache is continuously iterated and updated based on the real-time detection results of each image frame, ensuring that the target information remains up-to-date or in its most accurate state throughout the monitoring period. For example, continuously tracking the number of chickens in each image frame of chicken coop location number 001, when the identified number of chickens exceeds the number of chickens currently in the chicken coop registry, the new identification result is added to the chicken coop registry to replace the previous identification result, thus updating the maximum number of chickens and maintaining the most accurate state.

[0125] For example, if no frame contains area number one for a period of time, it indicates that the monitoring cycle for the chicken coop area corresponding to area number one has ended, the relevant information is stopped being updated, and the target information for which updates have been terminated is uploaded to the host computer.

[0126] In one embodiment of this application, the target area is a chicken coop area, the target area image is a chicken coop area image, the area number is a chicken coop number, and the target information also includes the number of eggs and the number of chickens.

[0127] The steps for summarizing target information and transmitting it to the host computer include:

[0128] Based on the chicken cage number, an independent local cache is established for each chicken cage area, and the target information in each chicken cage area is tracked across frames.

[0129] During the monitoring period in the chicken coop area, the maximum number of eggs and the maximum number of chickens identified are continuously updated iteratively until the target information is no longer updated.

[0130] Upload the target information for the terminated update to the host computer.

[0131] It should be noted that, taking the chicken coop area as an example, the chickens in the coop move around, causing them to easily obscure each other, ultimately making it difficult for a single frame to reflect the true number of chickens in the coop. Therefore, this section implements cross-frame tracking, continuously iterating and updating the maximum number of chickens that can be identified within the chicken coop area's monitoring cycle until the target information is no longer updated. This ensures that the information uploaded to the host computer is accurate, further improving the accuracy of target information recognition.

[0132] It should also be noted that compared to identifying chickens in different image frames and then combining the chickens in each image frame to obtain the final number of chickens identified, the statistical method in this embodiment is more convenient and requires less computation. Here is a brief example of the former method: if there are four chickens in the first image frame, five chickens in the second image frame, and one of the four chickens in the first image frame is not identified in the second image frame, then it is determined that there are six chickens.

[0133] For example, in a series of image frames of a video, the chicken coop area No. 001 appears repeatedly in multiple frames. Affected by factors such as changes in ambient lighting and partial occlusion of objects, the number of chickens detected in the coop varies in different image frames. Therefore, this solution selects the maximum number of chickens detected in all image frames corresponding to the coop (determined by the coop number) as the final statistical data of the number of chickens in the coop, and uploads this data to the host computer.

[0134] In one embodiment of this application, the area number consists of numbers;

[0135] For the area numbers uploaded to the host computer, a number queue is established from front to back based on the obtained time. The area numbers of each adjacent area in the number queue are compared. If the absolute value of the difference between the two is greater than the first threshold, it is determined that the area number in the latter area is incorrect, and an alarm message can be issued.

[0136] Here's an example: During an inspection, there will be several image frames for a chicken coop. The number of eggs in the image frames numbered 1 is compared, and the image frame with the most eggs is selected and uploaded to the host computer (uploading is completed once the judgment is complete). This is the target area image to be uploaded to the host computer (hereinafter referred to as chicken coop area image 1, or chicken coop registry 1). The other image frames are discarded. When a second image frame needs to be uploaded to the host computer, this is the adjacent target area image of chicken coop area image 1. If it is chicken coop area image 2, there is no problem. If it is chicken coop area image 10, it is a skipped frame, and the corresponding area number 10 is incorrect.

[0137] When an area number is determined to be incorrect, this information will be recorded and aggregated to the host computer. Monitoring personnel can check these error messages through the host computer, thereby prompting manual on-site inspection. In other words, alarm information can refer to the reminder information aggregated on the host computer, and the specific alarm form is not limited.

[0138] It should be noted that visual inspection and recognition technologies (such as OCR technology) are limited by environmental and network factors, and may sometimes experience false detections or missed detections. For example, poor lighting, heavy dust, or chicken feathers obscuring the area can lead to errors in numbering the identification areas, thus affecting recognition accuracy. For instance, the chicken cage numbers identified by a chicken farm inspection robot should be sequential, such as 006, 007, 008, ..., but sometimes environmental factors can cause errors in optical character extraction, resulting in skipped numbering, such as identifying 911 after 115.

[0139] Based on the aforementioned issue of skipped numbering, this section sets up a queue of region numbers uploaded to the host computer, arranged from front to back according to the time of acquisition. It then compares adjacent region numbers in the queue. If the absolute value of the difference between the two numbers exceeds a first threshold, it is determined that the subsequent region number is incorrect, and an alarm can be issued (this data can be automatically recorded for manual review). For example, with the first threshold set to 3, if the previous identified number is 017 and the next identified number is 019, the difference is 2. This is determined to be due to data loss during network or data transmission, thus eliminating the need for an alarm.

[0140] To further clarify, data classified as "loss of network data transmission" will not proceed to the manual review process. However, data classified as "skipped sequence number frame" will trigger the manual review process for that frame and its corresponding image. The reasons are as follows:

[0141] When the difference in the number between adjacent frames is small (e.g., less than or equal to 2), it is usually caused by intermittent network failure. Since the robot has to inspect the chicken cages multiple times a day and update the data in real time, the impact of such problems is small. If such problems require a manual review process, it will lead to a significant increase in labor costs in large-scale chicken coop scenarios.

[0142] When the difference between adjacent frames is large, the factors causing this problem are generally that the numbering area is obscured or damaged, the network is disconnected for a long time, or the lighting facilities are malfunctioning. In this case, a manual review process needs to be triggered to avoid causing continuous and large-scale identification errors.

[0143] In one embodiment of this application, the target character information is identified by OCR technology to obtain an initial region number, and then the initial region number is updated by a preset OCR character error correction mapping table to obtain the region number of the corresponding target region image.

[0144] The area code consists of numbers. For example, the chicken coop numbers are all composed of numbers. During recognition, due to factors such as dim lighting, thick dust, chicken feathers obscuring the area, or wear and tear on the chicken coop numbers over long-term use, some numbers may be recognized as letters (e.g., 005 may be recognized as 00S). In this case, these letters can be updated to numbers based on a preset OCR character error correction mapping table (e.g., updating S to 5), which further improves the accuracy of target information recognition.

[0145] The OCR character correction mapping table can be preset and updated as needed. For example, the OCR character correction mapping table includes: O→0, I→1, S→5, B→8.

[0146] In one embodiment of this application, for the erroneous area number that is determined to have a deviation, its permission to participate in the number comparison is cancelled (it is not necessary to remove it from the number queue).

[0147] For two region numbers adjacent to the erroneous region number, the two are compared and the second threshold is used as the comparison judgment basis. The second threshold = the first threshold × (A+1), or the second threshold = the first threshold + A, where A is the number of erroneous region numbers between the two region numbers.

[0148] This section essentially establishes a valid numbering history queue mechanism. Each comparison is based on the most recent valid number in the queue. For example: if the region number of image 1 (chicken coop area) is 1, and the region number of image 2 (chicken coop area) is 4, the difference between the two is 3, which is greater than the first threshold of 2. Therefore, the region number of image 2 will not be included in subsequent numbering comparisons. Image 3 (chicken coop area) has a region number of 3. It is compared with 1, not with 4. The difference is 2, which is less than the second threshold of 4 (the first threshold multiplied by 2). This avoids the problem of the region number of image 2 affecting the determination of the region number of image 3.

[0149] Generally, if the first threshold is greater than 1, and the second threshold is equal to the first threshold × (A + 1), the number of error area numbers is less (compared to when the second threshold equals the first threshold + A), which can reduce the frequency of manual on-site verification. Conversely, if the second threshold equals the first threshold + A, the number of error area numbers is more, making it more suitable for scenarios requiring high reliability risk assessment.

[0150] For the first region number in the numbering queue, it can be compared with a preset value (such as 1). The comparison is based on the first threshold. If the region number is an incorrect region number, the next region number is taken as the first region number and compared with the preset value + B. B is the number of incorrect region numbers before the first region number. The comparison is based on the first threshold.

[0151] In one embodiment of this application, the optical character recognition task and the network upload task for summarizing target information to the host computer are separated from the main thread of video stream processing. Independent OCR asynchronous queues and network upload asynchronous queues are configured respectively. The target area image data to be optically recognized and the target information to be summarized and uploaded to the host computer are respectively sent to their respective matched asynchronous queues for background scheduling processing.

[0152] It's important to note that in real-time video streaming scenarios, the video stream processing rate can easily become mismatched with the Optical Character Recognition (OCR) processing rate and the network upload processing rate, potentially causing stuttering and frame delays. Therefore, this section separates the time-consuming OCR processing and network upload operations from the main video stream thread, constructing independent asynchronous processing queues: an OCR asynchronous queue and a network upload asynchronous queue. Data to be recognized and uploaded are placed into their respective asynchronous queues for background asynchronous scheduling, preventing time-consuming operations from blocking the real-time video stream processing flow and ensuring the real-time performance and smoothness of the video stream. This asynchronous queue setup is standard practice; the specific method for constructing asynchronous queues can be found in existing documentation.

[0153] In one embodiment of this application, the step of summarizing the target information to the host computer includes:

[0154] If any target area fails to be identified within a preset time period (which can be adjusted as needed), the target information corresponding to that target area will be summarized and transmitted to the host computer. After successful transmission, other target information carrying the area number of that target area will be restricted from being uploaded to the host computer again, thus preventing duplicate counting and interference from abnormal data.

[0155] For example, chicken coop area 1 has a continuously updated chicken coop registry 1. During one inspection, chicken coop area 1 generates several images. After these images are identified, chicken coop area 1 can no longer be identified until the next inspection. At this time, the updated chicken coop registry 1 is transmitted to the host computer. After successful upload, if a subsequent chicken coop registry containing area number 1 is encountered, its upload is rejected (e.g., through thread locks, the existing method), to avoid interfering with the determination of the number of eggs in chicken coop area 1, ensuring data consistency, and preventing duplicate uploads of chicken coop registry 1 after successful transmission.

[0156] Understandably, if the upload of registry 1 fails, the transmission will be automatically retried to ensure data transmission consistency.

[0157] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A monitoring method based on cascaded target detection, characterized in that: include: Obtain surveillance footage; The first YOLO model, after training, identifies the surveillance footage and crops out images of each target area. The trained second YOLO model is used to identify images of each target region and obtain target information in each target region image. The target information is aggregated and transmitted to the host computer. The target information includes a region code, and the steps to obtain the region code include: The target character information in each target region image is located using the trained second YOLO model; The target character information is identified using OCR technology, and the region number of the corresponding target area image is confirmed based on the identified information. The steps for summarizing target information and transmitting it to the host computer include: If any target area fails to be identified within a preset time period, the target information corresponding to that target area will be summarized and transmitted to the host computer. After successful transmission, other target information carrying the area number of that target area will be restricted from being uploaded to the host computer again. The area number consists of numbers. For the area number uploaded to the host computer, a number queue is established from front to back according to the time of acquisition. The area numbers of each adjacent area in the number queue are compared. If the absolute value of the difference between the two is greater than the first threshold, it is determined that the area number in the latter area is incorrect. For erroneous area numbers where the determination is biased, their permission to participate in number comparison will be revoked; For two region numbers adjacent to the erroneous region number, the two are compared and the second threshold is used as the comparison judgment basis. The second threshold = the first threshold × (A+1), or the second threshold = the first threshold + A, where A is the number of erroneous region numbers between the two region numbers. The steps for locating target character information in each target region image using the trained second YOLO model include: Obtain the target bounding box of each target text in the image of each target region; The target bounding boxes are sorted horizontally based on their x-coordinates, and their dimensions are simultaneously detected. Fixed fill margins are reserved around each target bounding box. The sorted target bounding boxes are merged and stitched together according to their relative positions to form a continuous digital image without gaps. The steps for summarizing target information and transmitting it to the host computer include: Based on the corresponding region number, an independent local cache is established for each target region, and the target information in each target region is tracked across frames to continuously iterate and update the target information during the monitoring period of the target region. Upload the target information for the terminated update to the host computer; The target area is the chicken coop area, the target area image is the chicken coop area image, the area number is the chicken coop number, and the target information also includes the number of eggs and the number of chickens. The steps for summarizing target information and transmitting it to the host computer include: Based on the chicken cage number, an independent local cache is established for each chicken cage area, and the target information in each chicken cage area is tracked across frames. During the monitoring period in the chicken coop area, the maximum number of eggs and the maximum number of chickens identified are continuously updated iteratively until the target information is no longer updated. Upload the target information for the terminated update to the host computer; The target character information is identified by OCR technology to obtain the initial region number. Then, the initial region number is updated by a preset OCR character error correction mapping table to obtain the region number of the corresponding target region image. The optical character recognition task and the network upload task for summarizing target information to the host computer are separated from the main thread of video stream processing. Independent OCR asynchronous queues and network upload asynchronous queues are configured respectively. The target area image data to be recognized by optical characters and the target information to be summarized and uploaded to the host computer are respectively sent to their respective matched asynchronous queues for background scheduling and processing.

Images