Target object detection and prompt method and system and storage medium
By using deep learning in video streams to detect the position and label status of battery modules, the problems of module placement and missing labels were solved, achieving efficient and low-cost automated detection and improving the accuracy and efficiency of the battery manufacturing process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2026-03-13
AI Technical Summary
Traditional target detection methods in video streams face challenges in terms of processing speed, accuracy, and real-time performance. In particular, during the manual packaging process in the battery manufacturing industry, modules are easily placed backwards or labels are missed, leading to product defects and high costs.
A deep learning-based object detection method is adopted to acquire images from video streams, identify target objects and determine that they are located in regions of interest, detect label status, and output status prompts, including visual or auditory feedback.
It improves the accuracy of testing during the battery module packaging process, reduces human error, lowers hardware costs, is suitable for large-scale deployment, and improves production efficiency and product quality.
Smart Images
Figure CN121661314A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to computer vision and image processing technology, and in particular to a method, system, and storage medium for detecting and prompting target objects in video streams. Background Technology
[0002] In fields such as video surveillance, autonomous driving, and industrial automation, the detection and recognition of specific targets in videos is of great significance. Traditional target detection methods often rely on static image processing, while target detection and prompting in real-time video streams face greater challenges, including processing speed, accuracy, and real-time performance.
[0003] In particular, in the battery manufacturing industry, ensuring the correct orientation of the positive and negative terminals and the proper affixing of labels during module packaging is crucial. Traditional manual packaging is prone to errors due to human error, such as modules being placed upside down or labels being missing, leading to product defects and customer complaints. Furthermore, manual inspection carries the risk of missed inspections and is costly. While introducing automated testing equipment can improve accuracy, its high cost makes it unsuitable for large-scale deployment. Summary of the Invention
[0004] To address the aforementioned problems, this invention provides a method, system, and storage medium for target object detection and alerting. This method can effectively detect target objects in a video stream and provide status alerts after the target object is detected.
[0005] In a first aspect, this application provides a target object detection and prompting method, comprising: acquiring a video stream; acquiring one or more images to be tested based on the video stream; detecting whether the images to be tested include a target object; in response to detecting that the images to be tested include the target object, determining whether the target object is located in a region of interest (ROI); in response to determining that the target object is located in the ROI, detecting the label of the target object; and outputting a status prompt of the target object based on the detection result.
[0006] The beneficial effects of the method in the first aspect of this application are as follows: acquiring one or more images to be tested based on a video stream; detecting whether the images to be tested include a target object (module); by determining whether the target object is located in a region of interest (ROI) and detecting the label of the target object, it is possible to determine whether the module is within the ROI, whether it is unlabeled, and whether it is placed upside down, thereby solving the problem that in the traditional manual packaging process of battery modules, the module is easily placed upside down or the label is missing due to human factors, thus improving work efficiency.
[0007] Secondly, this application provides a target object detection and alerting system, comprising:
[0008] An image acquisition device is configured to acquire video streams from one or more pre-set locations;
[0009] A frame extraction device is configured to perform frame extraction processing on the video stream to obtain one or more images to be tested;
[0010] A first detection device is configured to detect whether the image to be tested includes a target object;
[0011] The second detection device is configured to determine whether the target object is located in a pre-defined region of interest (ROI) in response to detecting that the image to be tested includes the target object.
[0012] A third detection device is configured to detect the label of the target object in response to determining that the target object is located in the ROI; and
[0013] An output device is configured to output a status indication of the target object, based at least on the detection of the tag of the target object.
[0014] The beneficial effects of the system of the second aspect of this application are as follows: the image acquisition device of the system acquires video streams at one or more pre-set locations; the frame extraction device performs frame extraction processing on the video streams to obtain one or more images to be tested; the detection device detects whether the images to be tested include a target object (module). By determining whether the target object is located in the region of interest (ROI) and detecting the label of the target object, the system can determine whether the module is within the ROI, whether it is unlabeled, and whether it is placed upside down, etc., thereby solving the problem that in the traditional manual packaging process of battery modules, the module is easily placed upside down or the label is missing due to human factors. Furthermore, the system of the second aspect of this application is suitable for large-scale deployment and saves hardware costs. Attached Figure Description
[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0016] Figure 1 A flowchart illustrating a target object detection and prompting method according to an embodiment of the present disclosure is shown schematically;
[0017] Figure 2 A block diagram of a target object detection and prompting system according to an embodiment of the present disclosure is shown schematically. Detailed Implementation
[0018] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.
[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.
[0020] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.
[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0022] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.
[0023] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).
[0024] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.
[0025] Figure 1 A flowchart illustrating a target object detection and prompting method according to an embodiment of the present disclosure is shown schematically. Figure 1 The process includes at least some of the following steps.
[0026] Step S1: Acquire the video stream: First, acquire a real-time video stream using a camera or other video capture device. The video stream can be a continuous sequence of frames, each containing image data.
[0027] Step S2 involves acquiring one or more images to be tested based on the video stream: extracting the images to be tested from the video stream. This can be achieved by extracting frames at fixed time intervals or by triggering events such as motion detection.
[0028] Step S3 detects whether the image to be tested includes the target object: Each image to be tested is analyzed using a pre-trained deep learning model (such as a convolutional neural network CNN) to identify and locate the target object in the image. A large amount of labeled data is used during model training to ensure high accuracy.
[0029] Step S4, in response to detecting that the image to be tested includes the target object, determines whether the target object is located within a region of interest (ROI): Once the target object is detected, the system calculates the position of the target object and determines whether it is located within a predefined ROI. The ROI can be static or dynamically adjusted based on the scene.
[0030] In response to determining that the target object is located within the ROI, step S5 detects the label of the target object: if the target object is located within the ROI, the system will further analyze the characteristics of the target object to identify its label (e.g., vehicle, pedestrian, specific object, etc.).
[0031] Step S6, based on the detection results, outputs a status prompt for the target object: Finally, according to the detection results and tags of the target object, the system will output a corresponding status prompt. The prompt can be visual (e.g., highlighting the target object on the screen), auditory (e.g., emitting an alarm sound), or through other means (e.g., sending a notification to a mobile device).
[0032] Specifically, according to embodiments of this disclosure, a use case is provided for using the method to detect the positive and negative polarity orientation of modules and whether labels are correctly attached during the packaging process. The steps in the method according to embodiments of this disclosure can be implemented as follows: Video stream acquisition: Acquire real-time video stream from a surveillance camera using the RTSP protocol. Preprocess the acquired video stream images, including image cropping, scaling, and rotation, to adapt to the input requirements of the detection model. Object detection: Perform object detection on the preprocessed image using a trained YOLO-v5 model to identify modules and labels. Matching and judgment: Match modules and labels based on the coordinates of the detection results. Determine whether the module is within the ROI, whether it is unlabeled, and whether it is placed backwards. Result output: Write the detection results to a database and push them to an early warning platform to achieve a closed-loop business process.
[0033] In object detection, a suitable YOLO-v5 model version (YOLO-v5s, YOLO-v5m, YOLO-v5l, YOLO-v5x) can be selected, and the model size can be chosen according to requirements and resources. Furthermore, the YOLO-v5 model is an exemplary and not limiting algorithm. Those skilled in the art will be able to conceive of other possible object detection algorithms that existed before or were developed after the filing date of this disclosure and apply them to the methods disclosed in the embodiments of this disclosure.
[0034] In embodiments according to this disclosure, if the aspect ratio of the label is detected to be greater than or equal to a predetermined ratio, the system will output a status prompt indicating that the target object's position has been reversed. This means that if the label's orientation is not as expected, the system can identify this and prompt for adjustment.
[0035] In addition, if a label is missing or does not match a pre-stored label template, the system will output a status message indicating that the target object is not labeled. This helps to promptly detect missing or incorrect labels, ensuring that each battery module is correctly labeled.
[0036] Before detecting the target object, the system rotates the image under test based on the Euclidean distance between the center coordinates of the target object and the center coordinates of the label, and a predetermined Euclidean distance. This step ensures image consistency, making subsequent detection more accurate.
[0037] In embodiments according to this disclosure, the target object is one or more battery modules. Automated monitoring of the battery module status can significantly improve production efficiency and product quality.
[0038] Furthermore, if one or more battery modules are determined to be located within the ROI, the system will also detect whether these battery modules include a first predetermined number of first components. If a battery module does not include the first number of first components, the system will output a defective status message. This helps to promptly detect and address defects in the production process.
[0039] In embodiments according to this disclosure, the first component may be a protective cover or a bolt. These components are crucial to the integrity and safety of the battery module. Automated detection of the presence of these components ensures the quality and performance of the battery module.
[0040] In embodiments according to this disclosure, the first quantity is set based on the number of battery cells contained in the battery module. For example, if a battery module contains 10 battery cells, then a corresponding number of protective covers or bolts may be required. The system will detect the component integrity of each module based on this set quantity.
[0041] In embodiments according to this disclosure, when more than a predetermined threshold number of battery modules are located within an ROI, the system automatically detects the labels on these modules. This step helps to promptly identify missing or incorrect labels, ensuring that each battery module is correctly affixed.
[0042] In embodiments according to this disclosure, if a label is detected to be absent or does not match a pre-stored label template in a second consecutive number of test images, the system will output a status message indicating that the target object is not labeled. This continuous detection mechanism can improve the accuracy and reliability of detection.
[0043] In embodiments according to this disclosure, the video stream is a real-time video stream, meaning the system can monitor the production and packaging process of battery modules in real time. Real-time monitoring allows for timely detection of problems and the implementation of appropriate measures, thereby improving production efficiency and product quality.
[0044] In embodiments according to this disclosure, the system uses target detection technology to determine whether the detected result is within the ROI. The criteria include, but are not limited to: a. Determining whether the label is within the ROI by detecting the intersection area between the label position and the ROI. b. Saving modules and labels within the same ROI separately. c. Checking if the number of protective covers is 6, the number of wire harness connection ports is 13, and the number of bolts is 12; discrepancies are marked as appearance defects. d. Determining the number of modules and labels; if the number is too small or the number of labels is less than half the number of modules, no logical judgment is performed. e. Ensuring one label per module based on the Euclidean distance between the relative positions of modules and labels, and determining if any modules are unlabeled. f. Determining the overall orientation of the module using the aspect ratio of the module label, calculating the average offset angle, and fine-tuning the module angle. g. Layering modules according to their center coordinates and determining if any modules are placed backwards based on the orientation set for each layer. h. Outputting a single-image judgment result, marking NG results and triggering an alarm.
[0045] According to embodiments of this disclosure, the status prompts include, but are not limited to: when it is detected that a module is not labeled or is placed upside down, the system pushes a message to the abnormal result feedback platform and the group where the relevant person in charge is located, and saves the entire violation process to a designated path for archiving, so as to realize real-time alarm for the violation event.
[0046] Figure 2A block diagram of a target object detection and alerting system 200 according to an embodiment of the present disclosure is schematically shown. The target object detection and alerting system 200 exemplarily includes the following devices: Image acquisition device 201: configured to acquire a video stream from one or more pre-defined locations. This may include cameras installed on a production line, in a warehouse, or in a monitored area. Frame extraction device 202: configured to perform frame extraction processing on the video stream to acquire one or more images to be tested. Frame extraction may be timed or event-triggered. First detection device 203: configured to detect whether the image to be tested includes a target object. This may involve using an image recognition algorithm to identify the target object in the image. Second detection device 204: configured to determine whether the target object is located in a pre-defined region of interest (ROI) in response to detecting that the image to be tested includes a target object. The ROI may be static or dynamically adjusted based on the scene. Third detection device 205: configured to detect a label on the target object in response to determining that the target object is located in the ROI. This may include recognizing text, symbols, or QR codes on the label. Output device 206: configured to output a status prompt for the target object based at least in part on the detection of the tag on the target object. The prompt may be visual, auditory, or otherwise, such as sending a notification to a control center or mobile device. Devices in the system according to this disclosure may be electrically linked to each other or otherwise communicatively linked. In one embodiment, these devices constitute independent hardware; in another embodiment, these devices are locationally distributed functional modules.
[0047] This disclosure also provides a non-transitory computer-readable storage medium storing a specific instruction set that enables a processor to implement methods for a target object detection and alerting system when executing these instructions. These methods cover a range of automated monitoring functions, from image acquisition and target detection to status alerting.
[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A method for detecting and indicating target objects, characterized in that, The method includes: Acquire video stream; One or more images to be tested are acquired based on the video stream; Detect whether the image to be tested includes a target object; In response to detecting that the image to be tested includes the target object, determine whether the target object is located in the region of interest (ROI); In response to determining that the target object is located in the ROI, the label of the target object is detected; Based on the detection results, a status prompt for the target object is output.
2. The method according to claim 1, characterized in that, The output of the target object's state includes: based on detecting that the aspect ratio of the label is greater than or equal to a predetermined ratio, outputting a status prompt indicating that the target object's position is reversed.
3. The method according to claim 1, characterized in that, The output of the target object's status includes: outputting a status prompt indicating that the target object is not labeled based on whether the label does not exist or the label does not match a pre-stored label template.
4. The method according to claim 1, characterized in that, Before detecting whether the image to be tested includes a target object, the method further includes: rotating the image to be tested based on the Euclidean distance between the center coordinates of the target object and the center coordinates of the label and a predetermined Euclidean distance.
5. The method according to any one of claims 1-4, characterized in that, The target object is one or more battery modules.
6. The method according to claim 5, characterized in that, Also includes: In response to determining that the one or more battery modules are located in the ROI, it is detected whether the one or more battery modules include a first predetermined number of first components; If the one or more battery modules do not include a first number of first components, output a status message indicating that the one or more battery modules are defective.
7. The method according to claim 6, characterized in that, The first component is a protective cover or a bolt.
8. The method according to claim 6, characterized in that, The first quantity is set according to the number of battery cells contained in the battery module.
9. The method according to claim 5, characterized in that, Also includes: In response to determining that more than a predetermined threshold number of the battery modules are located in the ROI, the tags of the battery modules are detected.
10. The method according to claim 1, characterized in that, The output of the target object's status also includes: based on the detection that the label is absent or the label does not match the pre-stored label template in a second consecutive number of test images, outputting a status prompt that the target object is not labeled.
11. The method according to any one of claims 1-10, wherein the video stream is a real-time video stream.
12. A target object detection and alerting system, characterized in that, include: An image acquisition device is configured to acquire video streams from one or more pre-set locations; A frame extraction device is configured to perform frame extraction processing on the video stream to obtain one or more images to be tested; A first detection device is configured to detect whether the image to be tested includes a target object; The second detection device is configured to determine whether the target object is located in a pre-defined region of interest (ROI) in response to detecting that the image to be tested includes the target object. A third detection device is configured to detect the label of the target object in response to determining that the target object is located in the ROI; as well as An output device is configured to output a status indication of the target object, based at least on the detection of the tag of the target object.
13. A non-transitory computer-readable storage medium storing instructions, characterized in that, When the instruction is executed by the processor, the processor performs the method according to any one of claims 1-11.