Intelligent drawer article guiding method based on improved YOLOv5s model

Abstract

The invention belongs to the technical field of intelligent household equipment, and discloses an intelligent drawer article guiding method based on an improved YOLOv5s model, and the method comprises the steps: 1, carrying out the image collection and preprocessing; the method comprises the following steps of: 1, acquiring an article image, 2, performing instant identification and classification on the article image by using an improved YOLOv5s model, 3, storing information through a database management system, 4, acquiring a voice instruction of a user by a voice input module of an intelligent drawer system, and 5, performing keyword extraction and semantic analysis, and identifying an article searched by the user. Step 6, the user interacts with the intelligent drawer system through a voice command, and enters a guiding mode; and 7, recording and updating the use state and the position change of the article by the system. According to the invention, articles can be monitored and identified in the intelligent drawer in real time, rapid and accurate response is provided, the intelligent level of the intelligent drawer is greatly improved, and the user experience is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart home devices, and more specifically relates to a smart drawer item guidance method based on an improved YOLOv5s model. Background Art

[0002] With the rapid development of computer vision and deep learning, object detection and image recognition technologies have significantly improved in accuracy and speed. Mainstream deep learning frameworks such as TensorFlow, Adam optimizer, and PyTorch provide strong support, making image recognition applications possible on embedded systems. Deep learning breakthroughs in image classification, target detection, and image segmentation enable smart drawers to accurately identify and locate items, meeting users' daily needs for finding items.

[0003] The development of the Internet of Things has made smart home devices more and more popular. Devices can share information through the network to achieve remote control and data management. As part of the smart home, the smart finder can connect to user devices through the Internet of Things to improve user experience. The rise of edge computing provides a new way for embedded devices to process image data. By performing image processing and object recognition on local devices, it can reduce dependence on network bandwidth and improve real-time performance.

[0004] Traditional drawers only provide physical storage functions. Users need to remember and find the corresponding drawers to retrieve items. They lack intelligent management and cannot find the required items efficiently and accurately. Although existing image recognition models can be used to identify items and provide guidance for finding items, the accuracy of identifying items may still be affected in diverse and complex environments. In particular, the system may misidentify or miss items with similar shapes and colors. Some drawers are small and have uneven lighting, which poses a challenge to the quality of images captured by the camera and may lead to misjudgment. Summary of the invention

[0005] Based on the above technical problems, the present invention provides an intelligent drawer item guidance method based on an improved YOLOv5s model. The method adopts a deep learning model to perform item identification, improves detection accuracy, and realizes rapid guidance.

[0006] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0007] The present invention is an intelligent drawer item guidance method based on an improved YOLOv5s model, which specifically includes the following steps:

[0008] Step 1: When an item is placed in a drawer, the main control unit in the smart drawer system starts the image acquisition mode, and the camera built into the drawer takes images of the items in real time. The captured images are transmitted to the main control unit of the smart drawer system through the USB interface for image acquisition and preprocessing.

[0009] Step 2: Use the improved YOLOv5s model to instantly identify and classify the object images collected in step 1. The improved YOLOv5s model can efficiently detect and mark objects, ensuring rapid identification of different types of objects.

[0010] Step 3: After identifying different types of items in step 2, the smart drawer system inserts the relevant information of the items into the item information table and stores it through the database management system. Each item will be associated with a unique ID and the location data stored in the database management system for subsequent query and management;

[0011] Step 4: When the user is ready to find an item, the user issues a voice command, and the voice input module of the smart drawer system collects the user's voice command, detects the user's voice, and the smart drawer system converts the command into text;

[0012] Step 5: In the text instruction converted in step 4, the smart drawer system extracts keywords and performs semantic analysis to identify the items that the user is looking for;

[0013] Step 6: The user interacts with the smart drawer system through voice commands. The smart drawer system retrieves the item from the database management system according to the voice command, and prompts the user of the location of the item through the indicator light of the drawer or voice, and enters the guidance mode;

[0014] Step 7: The storage information of items is continuously updated and maintained. Any changes in the usage status and location of items will be recorded and updated by the system.

[0015] A further improvement of the present invention is that step 1 specifically comprises the following steps:

[0016] Step 1.1, the intelligent drawer system automatically recognizes whether the drawer is opened by triggering the switch. When the drawer is opened, the trigger switch senses the physical displacement, and the image sensor detects the state change. The control mainboard in the intelligent drawer system starts the image acquisition mode and wakes up the camera acquisition module to acquire images;

[0017] Step 1.2: The camera acquisition module quickly captures multiple images within a few seconds and selects the one with the highest definition for subsequent processing;

[0018] Step 1.3, transmitting the image with the highest definition selected in step 1.2 to the control main board via the USB interface;

[0019] Step 1.4: Preprocess the image, including removing random noise, cropping and scaling, and removing multiple backgrounds.

[0020] The further improvement of the present invention is that: in step 2, the improved YOLOv5s model includes an input, a backbone network (Backbone), a neck network (Neck) and a decoupled head (Decoupled head), the backbone network includes a Conv module, a C3 module and an SPPF module, the neck network includes a Conv module, a C3 module, a feature pyramid network (FPN) and a path aggregation network (PANet) are used, the decoupled head reduces the number of channels to 256 through a 1×1 convolution, two convolution layers with a convolution kernel size of 3×3, one for classification, and the other for positioning and confidence detection tasks, and the two convolution layers are parallel, two parallel 1×1 convolutions are used for separate positioning and confidence detection tasks, classification, positioning and confidence detection use different detection layers, the classification output dimension of the decoupled head is H×W×C, and the decoupled head is designed with a classification output dimension H×W×anchor×C, where H and W represent the height and width of the feature map, anchor represents the number of anchor boxes, and C represents the number of channels.

[0021] A further improvement of the present invention is that in the object recognition process of step 2, the improved YOLOv5s model instantly recognizes and classifies the collected object images, specifically including the following steps:

[0022] Step 2.1, establish a daily object detection dataset;

[0023] Step 2.2: Build a YOLOv5s model based on the Advanced-Adam optimization algorithm.

[0024] Step 2.3: Use the daily object detection dataset established in step 2.1 to train the YOLOv5s model constructed in step 2.2.

[0025] Step 2.4: Deploy the trained improved YOLOv5s model to the intelligent drawer system to achieve target detection.

[0026] A further improvement of the present invention is that the process of training the YOLOv5s model in step 2.3 is specifically as follows:

[0027] Step 2.3.1, initialize parameters and related variables;

[0028] Step 2.3.2: In each iteration, perform a quantitative update based on the current gradient calculation and correct the residual: k 、vk ), combined with quantized updates to reduce communication costs while maintaining the dynamics of momentum updates, where g k To reflect the gradient of the current gradient value, v k =g k -g k-1 In order to reflect the gradient difference of the gradient change trend, a quantization function Q is defined to compress the gradient update amount. The formula is as follows: Quantization function Q(x) = x+∈, where ||∈||≤δ||x||. Quantization reduces the communication cost of gradient update between distributed nodes.

[0029] Momentum Update:

[0030] m k =(1-β1)m k-1 +β1g k

[0031] v k =(1-β2)v k-1 +β2(g k -g k-1 )

[0032] Gradient update quantization:

[0033] Q(g k ) = g k +ε k ,||ε k ||≤δ||g k ||

[0034] Final update parameters:

[0035] θ k+1 =θ k -ηQ(m k );

[0036] Among them, β1 and β2 represent quantization functions, δ represents gradient, and η represents learning rate;

[0037] Step 2.3.3. Update momentum and model parameters while considering the influence of regularization terms: Dynamically adjust the learning rate:

[0038]

[0039] Among them, n k It is obtained through the second-order momentum calculation. This method can automatically adjust the step size at different stages of training, improve the convergence speed of the optimization process, and add an error feedback mechanism to correct the error introduced by quantization: t+1 =δ t -Q(δ t +et ), where e t It is the residual of each quantization. Combined with dynamic learning rate and error feedback, residual compensation is used to ensure that the dynamically adjusted learning rate will not affect the model performance due to quantization error. The update formula is integrated:

[0040]

[0041] Among them, θ t For the latest parameters, represents gradient quantization;

[0042] Introducing quantization updates and combining them with error compensation mechanisms can reduce the interference of communication and regularization on parameter updates:

[0043] θ k+1 =θ k -ηQ(m k )-λ||Q(θ k )||;

[0044] Step 2.3.4: Continue iterating until the YOLOv5s model converges.

[0045] A further improvement of the present invention is that the establishment of the data set in step 2.1 is specifically as follows: a large number of images of various objects are collected to establish the data set, and these images cover different lighting conditions, angles and background environments. The image annotation tool LabelImg is used to accurately annotate the target object in each image with a bounding box to ensure that the annotation of each category and position is accurate, and the data set is divided into a training set, a validation set and a test set to ensure that the performance and stability of the model can be comprehensively evaluated during the training process.

[0046] A further improvement of the present invention is that step 4 of searching for an item specifically includes the following steps:

[0047] Step 4.1: The user issues a voice command, and the microphone module of the smart drawer system collects the user's voice command signal;

[0048] Step 4.2: The intelligent drawer system pre-processes the voice command signal collected in step 4.1 to remove environmental noise and background sound and enhance voice clarity;

[0049] Step 4.3, convert the voice command signal from the time domain to the frequency domain through fast Fourier transform (FFT), and extract features for subsequent recognition;

[0050] Step 4.4: The smart drawer system calls the speech recognition API or the local speech recognition model to convert the collected voice command signal into a text command.

[0051] A further improvement of the present invention is that step 5 specifically includes the following steps:

[0052] Step 5.1, the smart drawer system extracts keywords through natural language processing (NLP) technology;

[0053] Step 5.2: Through the instruction classifier, the smart drawer system classifies the instructions into different categories: "search", "add", and "remove". The smart drawer system searches for items matching the instructions in the item information table and passes through the database management system.

[0054] The system compares and returns the location information of the item if there is a match. The search command and its result are stored in the user command table for subsequent query analysis and optimization of user experience. The smart drawer system feedbacks the found location. If the item is not in the drawer, it feedbacks "not found";

[0055] Step 5.3: The smart drawer system feeds back the query results to the user through voice to help the user find the location of the item.

[0056] A further improvement of the present invention is that in step 6, the smart drawer system guides the user's items to include the following steps:

[0057] Step 6.1, the main control unit sends a signal to the motor drive or LED light module to prepare to start position guidance; the position information queried from the database is usually a text description, such as "right side of the drawer".

[0058] Step 6.2, the smart drawer system converts text information into coordinates (for example, the left, middle, and right areas) in order to control the physical guidance device. According to the storage location of the item, the LED light in the corresponding area is controlled to prompt the user; for example, if the item is on the right side of the drawer, the LED light on the right side flashes to help the user identify it intuitively. In order to attract the user's attention, the LED light flashes at a certain frequency. The control motherboard controls the frequency of the light through the PWM signal. It is usually recommended to have a flashing frequency of 1-2 times / second to make it eye-catching and not dazzling.

[0059] Step 6.3: When the user successfully finds the item, he / she confirms that he / she has found it by voice or by pressing a button;

[0060] Step 6.4: After receiving the confirmation signal, the intelligent drawer system turns off the guidance mode and returns to the standby state;

[0061] Step 6.5: The smart drawer system updates the item status as "taken out" in the database management system and records the time of taking out to ensure that the database management system information is updated in real time.

[0062] A further improvement of the present invention is that the intelligent drawer system includes a control main board and several guide LED lights installed inside the drawer body. The control main board is the core control unit, responsible for coordination and management work, and is provided with stable power by the power supply line. The camera acquisition module is installed inside the drawer, and the image data in the drawer is collected and transmitted in real time through the control main board for object identification. The microphone module is connected to the control main board, responsible for receiving the user's voice command and transmitting the voice signal to the control main board for processing. After the user issues an instruction, the microphone module receives the user's voice command and transmits the voice signal to the control main board. The control main board analyzes the image in the drawer through the camera acquisition module and identifies the location of the target object. Once the location of the object is determined, the control main board commands the corresponding positions of the guide LED lights distributed inside the drawer to light up, guiding the user to quickly find the required items.

[0063] The beneficial effect of the present invention is that the present invention deploys the smart drawer with the optimization algorithm. After the deployment is completed, the system can monitor and identify items in the smart drawer in real time, provide fast and accurate response, greatly improve the intelligence level of the smart drawer, and greatly improve the user experience.

[0064] The present invention proposes the Advanced-Adam algorithm to modify the Nesterov momentum and adaptive optimization algorithm, and introduces decoupled weight decay, thereby achieving reduced calculation, faster convergence, and improved accuracy, making the outline of the object clearer, and ensuring that the smart drawer can efficiently complete tasks such as object identification, storage management, and voice interaction.

[0065] The quantization introduced by Advanced-Adam in the present invention is bidirectional, the staff sends updates to the parameter server, and the parameter server sends average updates to the staff. Then, the error feedback technology is used to reduce the impact of the quantization error, so that the location information of the object can be found more accurately.

[0066] The present invention can monitor items in real time, update storage status, and provide intelligent feedback, thereby improving user experience. Under the requirements of edge devices and low power consumption, the optimization of hardware architecture enables the system to run efficiently under limited computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flow chart of the present invention.

[0068] Figure 2 It is a schematic diagram of the interaction of the intelligent drawer system modules of the present invention.

[0069] Figure 3 It is a structural diagram of the intelligent drawer system of the present invention.

[0070] Figure 4Schematic diagram of the improved YOLOv5s model of the present invention.

[0071] Figure 5 Schematic diagram of the decoupling head of the present invention. DETAILED DESCRIPTION

[0072] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0073] like Figure 2-3As shown, the present invention provides an intelligent drawer system, which includes a control mainboard 5 and several guide LED lights 3 installed inside the drawer body 2. The control mainboard 5 is a core control unit responsible for coordination and management work, and is provided with stable power by the power supply line 6. The camera acquisition module 4 is installed inside the drawer, and the image data in the drawer is collected and transmitted in real time through the control mainboard 5 for object recognition. The microphone module 1 is connected to the control mainboard 5, responsible for receiving the user's voice command, and transmitting the voice signal to the control mainboard 5 for processing. After the user issues an instruction, the microphone module 1 receives the user's voice command and transmits the voice signal to the control mainboard 5. The control mainboard 5 analyzes the image in the drawer through the camera acquisition module 4 to identify the location of the target object. Once the location of the object is determined, the control mainboard 5 commands the corresponding position of the guide LED lights 3 distributed inside the drawer to light up, guiding the user to quickly find the required items. All these modules, including the camera, microphone module and LED light, are connected to the power supply line through the control mainboard to ensure that all parts of the system can operate normally and work efficiently together. The drawer body is an integral structure, and the positions of each module are reasonably arranged inside to ensure that the connection is stable and the wiring path is neat and orderly to avoid interference and damage. Through this close connection and coordinated workflow, the smart drawer can use image recognition technology to efficiently record and manage internal items, and help users quickly find the items they need through an intuitive guidance system. The system can monitor items in real time, update storage status, and provide intelligent feedback to enhance the user experience. Under the requirements of edge devices and low power consumption, the optimization of the hardware architecture enables the system to run efficiently with limited computing resources. The main control unit is a unit machine or a single-board computer, such as Raspberry Pi 4 or NVIDIA Jetson Nano, to ensure that the YOLO image recognition model can run smoothly and receive and process voice commands from the voice input module; the voice input module is a microphone array module; the processing unit includes an image sensor, a storage module, and an output module. The storage module uses an SD card and the image sensor is a camera; the power management module is a rechargeable lithium battery; and MySQL is used as the database management system.

[0074] like Figure 1 As shown, the present invention provides an intelligent drawer item guidance method based on an improved YOLOv5s model, and the intelligent drawer item guidance method specifically includes the following steps:

[0075] Step 1. When an item is placed in a drawer, the main control unit in the smart drawer system starts the image acquisition mode. The built-in camera in the drawer takes the image of the item in real time. The captured image is transmitted to the main control unit of the smart drawer system through the USB interface for image acquisition and preprocessing.

[0076] The specific steps include:

[0077] Step 1.1, the intelligent drawer system automatically recognizes whether the drawer is opened by triggering the switch. When the drawer is opened, the trigger switch senses the physical displacement, and the image sensor detects the state change. The control mainboard in the intelligent drawer system starts the image acquisition mode and wakes up the camera acquisition module to acquire images;

[0078] Step 1.2: The camera acquisition module quickly captures multiple images within a few seconds and selects the one with the highest definition for subsequent processing;

[0079] Step 1.3, transmitting the image with the highest definition selected in step 1.2 to the control main board via the USB interface; the USB interface has a high transmission speed and can meet the needs of real-time image processing.

[0080] Step 1.4: Preprocess the image, including removing random noise, cropping and scaling, and removing multiple backgrounds.

[0081] Step 2: Use the improved YOLOv5s model to instantly identify and classify the object images collected in step 1. The improved YOLOv5s model can efficiently detect and mark objects, ensuring rapid identification of different types of objects.

[0082] Step 3: After identifying different types of items in step 2, the smart drawer system inserts the relevant information of the items, such as name, type, location, confidence, etc., into the item information table and stores it through the database management system. Each item will be associated with a unique ID and the location data stored in the database management system for subsequent query and management;

[0083] Step 4: When the user is ready to find an item, the user issues a voice command. The voice input module of the smart drawer system, i.e., the microphone module, collects the user's voice command. After detecting the user's voice, the smart drawer system converts the command into text. Specifically, the steps include:

[0084] Step 4.1: The user issues a voice command, and the microphone module of the smart drawer system collects the user's voice command signal;

[0085] Step 4.2: The intelligent drawer system pre-processes the voice command signal collected in step 4.1 to remove environmental noise and background sound and enhance voice clarity;

[0086] Step 4.3, convert the voice command signal from the time domain to the frequency domain through fast Fourier transform (FFT), and extract features for subsequent recognition;

[0087] Step 4.4: The smart drawer system calls a speech recognition API, such as the Google Speech Recognition API or a local speech recognition model, to convert the collected voice command signal into a text command. For example, when the user says "Looking for a mobile phone charger", the system converts this sentence into the corresponding text format. To ensure the accuracy of recognition, the system can repeat the recognized text content to the user for confirmation through voice feedback, such as "You are looking for a mobile phone charger, right?" The user confirms with a simple "yes" or "no", and if there is an error, it can be re-recognized.

[0088] Step 5: In the text instructions converted in step 4, the smart drawer system extracts keywords and performs semantic analysis to identify the items the user is looking for. The specific steps include:

[0089] Step 5.1: The smart drawer system extracts keywords through natural language processing (NLP) technology, such as action words such as "find" and "search" and item names such as "mobile phone charger". These keywords help the system identify the user's intention and match it with the item information in the database.

[0090] Step 5.2: Through the instruction classifier, the smart drawer system classifies the instructions into different categories: "search", "add", and "remove". The smart drawer system searches for items matching the instructions in the item information table and passes through the database management system.

[0091] The system compares and returns the location information of the item if there is a match. The search command and its results are stored in the user command table for subsequent query analysis and optimization of user experience. The smart drawer system feedbacks the found location. If the item is not in the drawer, it feedbacks "not found"; in this scenario, the "search" type of command will trigger the database query process. The system searches for records matching the keywords in the item information table. The system retrieves items that match the name of "mobile phone charger" and are in the "in the drawer" state, and obtains their location information. If there are multiple matches in the database, the system will filter out the items that best meet the user's needs by time or location. For example, if multiple "chargers" are detected, the system may select the most recently placed record and return it to the user.

[0092] Step 5.3: The smart drawer system will give the user feedback on the query results through voice to help the user find the location of the item. For example, "The mobile phone charger is on the right side of the drawer." If no matching item is found, the system will give feedback "No relevant items found, do you need to continue searching?" to help the user understand the current status of the item. The system will enter the smart guidance process and further help the user find the location of the item through physical guidance.

[0093] Step 6: The user interacts with the smart drawer system through voice commands. The smart drawer system retrieves items from the database management system according to the voice commands, and prompts the user of the location of the items through the indicator light of the drawer or voice, and enters the guidance mode. The smart drawer system guides the user to the items in the following specific steps:

[0094] Step 6.1, the main control unit sends a signal to the motor drive or LED light module to prepare to start position guidance; the position information queried from the database is usually a text description, such as "right side of the drawer".

[0095] Step 6.2, the smart drawer system converts text information into coordinates such as left, middle, and right areas to control the physical guidance device. According to the storage location of the item, the LED light in the corresponding area is controlled to prompt the user; for example, if the item is on the right side of the drawer, the LED light on the right side flashes to help the user identify it intuitively. In order to attract the user's attention, the LED light flashes at a certain frequency. The control motherboard controls the frequency of the light through the PWM signal. It is usually recommended to have a flashing frequency of 1-2 times / second to make it eye-catching and not dazzling.

[0096] Step 6.3: When the user successfully finds the item, he / she confirms that he / she has found it by voice or by pressing a button;

[0097] Step 6.4: After receiving the confirmation signal, the intelligent drawer system turns off the guidance mode and returns to the standby state;

[0098] Step 6.5: The smart drawer system updates the item status as "taken out" in the database management system and records the time of taking out to ensure that the database management system information is updated in real time.

[0099] Step 7: As the items are used, the storage information is constantly updated and maintained. The usage status and location changes of the items will be recorded and updated by the system. Through intelligent management, the system can optimize the management of stored items and improve space utilization.

[0100] like Figure 4-5As shown, the improved YOLOv5s model of the present invention includes an input, a backbone network (Backbone), a neck network (Neck) and a decoupled head (Decoupled head), the backbone network includes a Conv module, a C3 module and an SPPF module, the neck network includes a Conv module, a C3 module, and a feature pyramid network (FPN) and a path aggregation network (PANet) are used. The decoupled head reduces the number of channels to 256 through a 1×1 convolution, and two convolution layers with a convolution kernel size of 3×3 are used for classification, and the other is used for positioning and confidence detection tasks, and the two convolution layers are parallel, and two parallel 1×1 convolutions are used for separate positioning and confidence detection tasks. Different detection layers are used for classification, positioning and confidence detection. The classification output dimension of the decoupled head is H×W×C, and the decoupled head is designed with a classification output dimension H×W×anchor×C, where H and W represent the height and width of the feature map, anchor represents the number of anchor boxes, and C represents the number of channels.

[0101] In the object recognition process of step 2, the improved YOLOv5s model instantly recognizes and classifies the collected object images, specifically including the following steps:

[0102] Step 2.1. Create a dataset for daily object detection.

[0103] The establishment of the dataset is as follows: a large number of images of various objects are collected to establish the dataset. These images cover different lighting conditions, angles and background environments. The image annotation tool LabelImg is used to accurately annotate the bounding box of the target object in each image to ensure that the annotation of each category and position is accurate. In order to enhance the diversity of the dataset and the generalization ability of the model, we also apply a variety of data enhancement techniques, including random cropping, rotation, scaling, color jitter and mirror flipping. These processing not only enriches the content of the dataset, but also effectively prevents the overfitting of the model. Finally, we divide the dataset into training set, validation set and test set to ensure that the performance and stability of the model can be fully evaluated during the training process.

[0104] Step 2.2: Build a YOLOv5s model based on the Advanced-Adam optimization algorithm.

[0105] Step 2.3: Use the daily object detection dataset established in step 2.1 to train the YOLOv5s model constructed in step 2.2. The specific process of training the YOLOv5s model is as follows:

[0106] Step 2.3.1, initialize parameters and related variables;

[0107] Step 2.3.2: In each iteration, perform a quantitative update based on the current gradient calculation and correct the residual: k 、v k ), combined with quantized updates to reduce communication costs while maintaining the dynamics of momentum updates, where g k To reflect the gradient of the current gradient value, v k =g k -g k-1 In order to reflect the gradient difference of the gradient change trend, a quantization function Q is defined to compress the gradient update amount. The formula is as follows: Quantization function Q(x) = x+∈, where ||∈||≤δ||x||. Quantization reduces the communication cost of gradient update between distributed nodes.

[0108] Momentum Update:

[0109] m k =(1-β1)m k-1 +β1g k

[0110] v k =(1-β2)v k-1 +β2(g k -g k-1 )

[0111] Gradient update quantization:

[0112] Q(g k ) = g k +ε k ,||ε k ||≤δ||g k ||

[0113] Final update parameters:

[0114] θ k+1 =θ k -ηQ(m k );

[0115] Among them, β1 and β2 represent quantization functions, δ represents gradient, and η represents learning rate;

[0116] Step 2.3.3. Update momentum and model parameters while considering the influence of regularization terms: Dynamically adjust the learning rate:

[0117]

[0118] Among them, n kIt is obtained through the second-order momentum calculation. This method can automatically adjust the step size at different stages of training, improve the convergence speed of the optimization process, and add an error feedback mechanism to correct the error introduced by quantization: t+1 =δ t -Q(δ t +e t ), where e t It is the residual of each quantization. Combined with dynamic learning rate and error feedback, residual compensation is used to ensure that the dynamically adjusted learning rate will not affect the model performance due to quantization error. The update formula is integrated:

[0119]

[0120] Among them, θ t For the latest parameters, represents gradient quantization;

[0121] Introducing quantization updates and combining them with error compensation mechanisms can reduce the interference of communication and regularization on parameter updates:

[0122] θ k+1 =θ k -ηQ(m k )-λ||Q(θ k )||;

[0123] Step 2.3.4: Continue iterating until the YOLOv5s model converges.

[0124] Step 2.4: Deploy the trained improved YOLOv5s model into the smart drawer system to achieve target detection.

[0125] Smart finder drawers can help users locate and extract items more quickly through IoT technology, intelligent recognition systems, and automated equipment, which is especially helpful for people with poor memory. These functions are not available in traditional smart drawers. They break through the limitations of traditional barcode recognition and use deep learning models and computer vision algorithms to identify items. Unlike traditional rule-based recognition methods, deep learning can automatically learn features from a large number of item images through training, and can quickly and accurately detect the type, size, and location of items, adapt to item recognition in various complex scenarios, and reduce the possibility of misidentification and missed recognition. As the use process progresses, the system can gradually adapt to the user's behavioral habits and item access patterns, and optimize recognition accuracy and classification efficiency. For example, the system can dynamically adjust the storage strategy based on factors such as the frequency of use and size of the item, thereby improving the overall level of intelligence in item management.

[0126] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An intelligent drawer item guidance method based on an improved YOLOv5s model, characterized in that: The smart drawer item guidance method specifically comprises the following steps: Step 1: When an item is placed in a drawer, the main control unit in the smart drawer system starts the image acquisition mode, and the built-in camera in the drawer takes images of the items in real time. The captured images are transmitted to the main control unit of the smart drawer system through the USB interface for image acquisition and preprocessing. Step 2: Use the improved YOLOv5s model to instantly identify and classify the object images collected in step 1. The improved YOLOv5s model detects and marks objects and identifies different types of objects. Step 3: After identifying different types of items in step 2, the smart drawer system inserts the relevant information of the items into the item information table of the database management system and stores it through the database management system. Each item will be associated with a unique ID and the location data stored in the database management system for subsequent query and management; Step 4: When the user is ready to find an item, the user issues a voice command, and the voice input module of the smart drawer system collects the user's voice command, detects the user's voice, and the smart drawer system converts the command into text; Step 5: In the text instruction converted in step 4, the smart drawer system extracts keywords and performs semantic analysis to identify the items that the user is looking for; Step 6: The user interacts with the smart drawer system through voice commands. The smart drawer system retrieves the item from the database management system according to the voice command, and prompts the user of the location of the item through the indicator light of the drawer or voice, and enters the guidance mode; Step 7: The storage information of items is continuously updated and maintained. The usage status and location changes of items will be recorded and updated by the system.

2. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 1, characterized in that: Step 1 specifically includes the following steps: Step 1.1, the intelligent drawer system automatically recognizes whether the drawer is opened by triggering the switch. When the drawer is opened, the trigger switch senses the physical displacement, and the image sensor detects the state change. The control mainboard in the intelligent drawer system starts the image acquisition mode and wakes up the camera acquisition module to acquire images; Step 1.2: The camera acquisition module quickly captures multiple images within a few seconds and selects the one with the highest definition for subsequent processing; Step 1.3, transmitting the image with the highest definition selected in step 1.2 to the control main board via the USB interface; Step 1.4: Preprocess the image, including removing random noise, cropping and scaling, and removing multiple backgrounds.

3. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 1, characterized in that: In step 2, the improved YOLOv5s model includes input, backbone network (Backbone), neck network (Neck) and decoupled head (Decoupled head), the backbone network includes Conv module, C3 module and SPPF module, the neck network includes Conv module, C3 module, feature pyramid network (FPN) and path aggregation network (PANet) are adopted, the decoupled head reduces the number of channels to 256 through 1×1 convolution, two convolution layers with a convolution kernel size of 3×3, one for classification, and the other for positioning and confidence detection tasks, and the two convolution layers are parallel, and two parallel 1×1 convolutions are used for separate positioning and confidence detection tasks. Different detection layers are used for classification, positioning and confidence detection. The classification output dimension of the decoupled head is H×W×C, and the decoupled head is designed with a classification output dimension H×W×anchor×C, where H and W represent the height and width of the feature map, anchor represents the number of anchor boxes, and C represents the number of channels.

4. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 2 is characterized in that: In the object recognition process of step 2, the improved YOLOv5s model instantly recognizes and classifies the collected object images, specifically including the following steps: Step 2.1, establish a daily object detection dataset; Step 2.2: Build a YOLOv5s model based on the Advanced-Adam optimization algorithm. Step 2.3: Use the daily object detection dataset established in step 2.1 to train the YOLOv5s model constructed in step 2.

2. Step 2.4: Deploy the trained improved YOLOv5s model to the smart drawer system to achieve target detection.

5. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 4 is characterized in that: Step 2.3 The process of training the YOLOv5s model is as follows: Step 2.3.1, initialize parameters and related variables; Step 2.3.2: In each iteration, perform a quantitative update based on the current gradient calculation and correct the residual: k 、v k ), combined with quantized updates to reduce communication costs while maintaining the dynamics of momentum updates, where g k To reflect the gradient of the current gradient value, v k =g k -g k-1 In order to reflect the gradient difference of the gradient change trend, a quantization function Q is defined to compress the gradient update amount. The formula is as follows: Quantization function in Momentum Update: m k =(1-β1)m k-1 +β1g k v k =(1-β2)v k-1 +β2(g k -g k-1 ) Gradient update quantization: Q(g k )=g k +e k ,||e k ||≤δ||g k || Final update parameters: i k+1 =θ k -ηQ(m k ); Among them, β1 and β2 represent quantization functions, δ represents gradient, and η represents learning rate; Step 2.3.

3. Update momentum and model parameters while considering the influence of regularization terms: Dynamically adjust the learning rate: Among them, n k It is obtained by calculating the second-order momentum, and an error feedback mechanism is added to correct the error introduced by quantization: t+1 =δ t -Q(δ t +e t ), where e t It is the residual of each quantization. Combined with dynamic learning rate and error feedback, residual compensation is used to ensure that the dynamically adjusted learning rate will not affect the model performance due to quantization error. The update formula is integrated: Among them, θ t For the latest parameters, represents gradient quantization; Introducing quantization updates and combining them with error compensation mechanisms can reduce the interference of communication and regularization on parameter updates: i k+1 =θ k -ηQ(m k )-λ||Q(θ k )||; Step 2.3.4: Continue iterating until the YOLOv5s model converges.

6. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 4 is characterized in that: The establishment of the data set in step 2.1 is specifically as follows: images of various objects are collected to establish a data set. These images cover different lighting conditions, angles and background environments. The image annotation tool LabelImg is used to annotate the bounding boxes of the target objects in each image to ensure that the annotations of each category and position are accurate, and the data set is divided into a training set, a validation set and a test set.

7. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 1, characterized in that: Step 4: Finding items includes the following steps: Step 4.1: The user issues a voice command, and the microphone module of the smart drawer system collects the user's voice command signal; Step 4.2: The intelligent drawer system pre-processes the voice command signal collected in step 4.1 to remove environmental noise and background sound and enhance voice clarity; Step 4.3, convert the voice command signal from the time domain to the frequency domain through fast Fourier transform (FFT), and extract features for subsequent recognition; Step 4.4: The smart drawer system calls the speech recognition API or the local speech recognition model to convert the collected voice command signal into a text command.

8. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 1, characterized in that: Step 5 specifically includes the following steps: Step 5.1, the smart drawer system extracts keywords through natural language processing (NLP) technology; Step 5.2: Through the command classifier, the smart drawer system classifies the commands into different categories of "search", "add", and "remove". The smart drawer system searches for items matching the commands in the item information table. After comparison with the database management system, if there is a match, the location information of the item is returned, and the search command and its result are stored in the user command table of the database management system for subsequent query analysis and optimization of user experience. The smart drawer system feedbacks the found location. If the item is not in the drawer, it feedbacks "not found". Step 5.3: The smart drawer system feeds back the query results to the user through voice to help the user find the location of the item.

9. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 1, characterized in that: In step 6, the smart drawer system guides the user to select items, which specifically includes the following steps: Step 6.1, the main control unit sends a signal to the motor drive or LED light module to prepare to start position guidance; Step 6.2, the intelligent drawer system converts the text information into coordinates according to the storage location of the items, and controls the LED lights in the corresponding area to prompt the user; Step 6.3: When the user successfully finds the item, he / she confirms that he / she has found it by voice or by pressing a button; Step 6.4: After receiving the confirmation signal, the intelligent drawer system turns off the guidance mode and returns to the standby state; Step 6.5: The smart drawer system updates the item status to "taken out" in the database management system and records the time of taking out to ensure that the database management system information is updated in real time.

10. The intelligent drawer item guidance method based on the improved YOLOv5s model according to claim 1, characterized in that: The intelligent drawer system comprises a control mainboard (5) and a plurality of guide LED lights (3) installed inside the drawer body (2). The control mainboard (5) is a core control unit responsible for coordination and management work, and is provided with stable power by a power supply line (6). The camera acquisition module (4) is installed inside the drawer, and the image data in the drawer is collected and transmitted in real time through the control mainboard (5) so as to identify the objects. The microphone module (1) is connected to the control mainboard (5), and is responsible for receiving the user's voice command and transmitting the voice signal to the control mainboard (5) for processing. After the user issues the command, the microphone module (1) receives the user's voice command and transmits the voice signal to the control mainboard (5). The control mainboard (5) analyzes the image in the drawer through the camera acquisition module (4) and identifies the location of the target object. Once the location of the object is determined, the control mainboard (5) commands the guide LED lights (3) distributed inside the drawer to light up at the corresponding positions, so as to guide the user to quickly find the desired object.

Images