Rescue vehicle stacked article counting method based on visual identification

Abstract

The invention discloses a rescue carriage stacked article counting method based on visual identification, a camera is installed above a drawer of a rescue carriage, the camera is used for acquiring a picture in the pulled drawer, and the method comprises the steps of image acquisition; preprocessing the image; detecting a medicine box target; calculating the stacking thickness of the medicines; carrying out lamination separation and counting: carrying out pixel-level segmentation on each layer of medicine boxes by adopting Mask R-CNN (Region Convolutional Neural Network) instance segmentation; a layer gap is optimized through corrosion expansion operation by means of morphological operation, and adjacent medicine boxes are separated; and carrying out weighted fusion on the detection frames for target detection, depth estimation and instance segmentation to obtain the number of fused drugs. According to the invention, the camera is used for collecting the images of the stacked medicines, the real-time dynamic counting of the stacked medicines is realized through the target detection algorithm and depth estimation, and the reliability of medicine management in an emergency scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for counting stacked items in a rescue vehicle based on visual recognition. Background Art

[0002] In medical emergency scenarios, emergency vehicles are crucial for storing emergency medicines and equipment. Rapid access and precise management of medicines within them are directly related to the patient's life safety. Traditional emergency vehicle medicine management relies heavily on manual counting or simple visual inspections, especially when medicines are stacked in drawers. This presents the following technical pain points: 1. Difficulty in counting stacked medicines: When similar medicines are stacked in multiple layers, manual counting is prone to omissions, double counting or estimation errors, which may delay rescue efforts, especially in emergency situations. 2. Lack of real-time inventory management: Existing methods cannot dynamically monitor drug consumption and require frequent manual inventory, which is inefficient and prone to drug shortages or expiration due to negligence. 3. Insufficient intelligence: Some solutions use RFID or barcode technology, but each drug needs to be individually labeled, which is costly and not suitable for unpackaged drugs (such as ampoules, tablets, etc.). Summary of the Invention

[0003] The technical problem to be solved by the present invention is that traditional emergency vehicle medicine management mostly relies on manual counting or simple visual inspection. In order to solve the above problem, a method for counting stacked items in emergency vehicles based on visual recognition is provided.

[0004] The object of the present invention is achieved in the following manner: A method for counting stacked items in a rescue vehicle based on visual recognition, wherein a camera is installed above a drawer of the rescue vehicle, and the camera is used to capture an image inside an opened drawer. The method comprises the following steps: S1: Image acquisition: Use a camera to capture the original image and RGB-D data of the stacked medicines in the drawer of the emergency vehicle; S2: Image preprocessing: Adaptive histogram equalization is performed on the collected images using the CLAHE algorithm to enhance the contrast between the edges of the medicine box and the text; and Gaussian filtering is used to eliminate image noise; S3: Medicine box target detection: Use the Yolov8 model to locate the medicine box bounding box in the preprocessed image, identify the medicine category and output the height reference value of a single medicine box; S4: Calculation of drug stacking thickness: Generate a depth map using monocular or RGB-D data based on the MiDaS depth estimation algorithm to quantify the vertical height of the stack. Use Canny edge detection to extract the contour differences between layers, and combine the depth map to calculate the number of stacked layers, where the number of layers = total height / height of a single drug box. S5: Layer separation and counting: Mask R-CNN instance segmentation is used to perform pixel-level segmentation on each layer of drug boxes. Morphological operations such as corrosion and dilation are then used to optimize the layer gaps and separate adjacent drug boxes. The counting includes: if the layers of medicine boxes are independent: total number = sum of the number of medicine boxes in each layer; If the same medicine is stacked regularly: total quantity = number of layers * base quantity of each layer; S6: Result output: Perform weighted fusion on the detection frames of target detection, depth estimation, and instance segmentation to obtain the number of fused drugs.

[0005] The framework of the Yolov8 model includes a Backbone feature extraction network, a Neck feature fusion network, and a Decoupled Head decoupling head connected in sequence; the Backbone feature extraction network is a CSPDarknet structure, and the Neck feature fusion network is a PANet+BiFPN.

[0006] The training configuration of the Yolov8 model includes an adaptive anchor box and an AdamW optimizer. The adaptive anchor box automatically calculates the anchor box size of the dataset through K-means clustering, reducing manual parameter adjustment. The AdamW optimizer combines dynamic learning rate adjustment with weight decay to avoid gradient oscillation.

[0007] The beneficial effects of the present invention are as follows: the present invention uses a camera to capture images of stacked medicines, and realizes real-time dynamic counting of stacked medicines through target detection algorithms and depth estimation, thereby improving the reliability of medicine management in emergency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0009] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0010] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same technical meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0011] like Figure 1 As shown, a method for counting stacked items in a rescue vehicle based on visual recognition is provided. A camera is installed above a drawer of the rescue vehicle, and the camera is used to capture images inside the opened drawer. The method includes the following steps: S1: Image acquisition: Use a camera to capture the original image and RGB-D data of the stacked medicines in the drawer of the emergency vehicle; S2: Image preprocessing: Adaptive histogram equalization is performed on the collected images using the CLAHE algorithm to enhance the contrast between the edges of the medicine box and the text; and Gaussian filtering is used to eliminate image noise; S3: Medicine box target detection: Use the Yolov8 model to locate the medicine box bounding box in the preprocessed image, identify the medicine category and output the height reference value of a single medicine box; S4: Calculation of drug stacking thickness: Generate a depth map using monocular or RGB-D data based on the MiDaS depth estimation algorithm to quantify the vertical height of the stack. Use Canny edge detection to extract the contour differences between layers, and combine the depth map to calculate the number of stacked layers, where the number of layers = total height / height of a single drug box. S5: Layer separation and counting: Mask R-CNN instance segmentation is used to perform pixel-level segmentation on each layer of drug boxes. Morphological operations such as corrosion and dilation are then used to optimize the layer gaps and separate adjacent drug boxes. The counting includes: if the layers of medicine boxes are independent: total number = sum of the number of medicine boxes in each layer; If the same type of medicine is stacked regularly: total quantity = number of layers * base quantity of each layer.

[0012] S6: Perform weighted fusion on the detection frames of target detection, depth estimation, and instance segmentation to obtain the number of fused drugs.

[0013] The Yolov8 model framework consists of a sequentially connected backbone feature extraction network, a neck feature fusion network, and a decoupled head. The backbone feature extraction network uses a CSPDarknet structure, based on the Darknet architecture of the Cross-Stage Local Network (CSP), to efficiently extract multi-scale features. The neck feature fusion network is a PANet+BiFPN bidirectional feature pyramid network that fuses deep and shallow features to enhance semantic and positional information. The decoupled head separates the classification and regression branches to avoid task interference and improve detection head accuracy.

[0014] Model export of the Yolov8 model: convert the PyTorch model into a standardized format suitable for edge device deployment.

[0015] The training configuration of the Yolov8 model includes an adaptive anchor box and an AdamW optimizer. The adaptive anchor box automatically calculates the anchor box size of the dataset through K-means clustering, reducing manual parameter adjustment. The AdamW optimizer combines dynamic learning rate adjustment with weight decay to avoid gradient oscillation.

[0016] The present invention uses a camera to capture images of stacked medicines, and implements real-time dynamic counting of stacked medicines through target detection algorithms and depth estimation, thereby improving the reliability of medicine management in emergency scenarios.

[0017] The above is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several changes and improvements can be made without departing from the overall concept of the present invention, and these should also be regarded as the scope of protection of the present invention.

Claims

1. A method for counting stacked items in a rescue vehicle based on visual recognition, wherein a camera is installed above a drawer of the rescue vehicle, and the camera is used to capture the image inside the open drawer, characterized in that: The method comprises the following steps: S1: Image acquisition: Use a camera to capture the original image and RGB-D data of the stacked medicines in the drawer of the emergency vehicle; S2: Image preprocessing: Adaptive histogram equalization is performed on the collected images using the CLAHE algorithm to enhance the contrast between the edges of the medicine box and the text; and Gaussian filtering is used to eliminate image noise; S3: Medicine box target detection: Use the Yolov8 model to locate the medicine box bounding box in the preprocessed image, identify the medicine category and output the height reference value of a single medicine box; S4: Calculation of drug stacking thickness: Generate a depth map using monocular or RGB-D data based on the MiDaS depth estimation algorithm to quantify the vertical height of the stack. Use Canny edge detection to extract the contour differences between layers, and combine the depth map to calculate the number of stacked layers, where the number of layers = total height / height of a single drug box. S5: Layer separation and counting: Mask R-CNN instance segmentation is used to perform pixel-level segmentation on each layer of drug boxes. Morphological operations such as corrosion and dilation are then used to optimize the layer gaps and separate adjacent drug boxes. The counting includes: if the layers of medicine boxes are independent: total number = sum of the number of medicine boxes in each layer; If the same medicine is stacked regularly: total quantity = number of layers * base quantity of each layer; S6: Result output: Perform weighted fusion on the detection frames of target detection, depth estimation, and instance segmentation to obtain the number of fused drugs.

2. The method for counting stacked items in a rescue vehicle based on visual recognition according to claim 1, characterized in that: The framework of the Yolov8 model includes a Backbone feature extraction network, a Neck feature fusion network, and a Decoupled Head decoupling head connected in sequence; the Backbone feature extraction network is a CSPDarknet structure, and the Neck feature fusion network is a PANet+BiFPN.

3. The method for counting stacked items in a rescue vehicle based on visual recognition according to claim 3, characterized in that: The training configuration of the Yolov8 model includes an adaptive anchor box and an AdamW optimizer. The adaptive anchor box automatically calculates the anchor box size of the dataset through K-means clustering, reducing manual parameter adjustment. The AdamW optimizer combines dynamic learning rate adjustment with weight decay to avoid gradient oscillation.

Images