Intelligent warehousing system with voice interaction function

Abstract

The invention discloses an intelligent warehousing system with a voice interaction function, which mainly comprises a raw material digital warehousing management system, a handheld terminal, a vehicle-mounted terminal and an intelligent maintenance system, and is characterized in that the raw material digital warehousing management system comprises a visual identification module, a Mecanum wheel trolley driving module and a man-machine interaction module; the visual identification module is used for calibrating cargo coordinates based on image processing, and the Mecanum wheel trolley driving module is used for conveying cargoes by adopting a mode of fusing motion control theory and a sensor algorithm; the man-machine interaction module is used for completing the functions of storage environment data broadcasting, automatic goods grabbing, voice intelligent interaction, mechanical arm remote control and the like through voice instructions. By means of the advantages of multi-task parallel processing and precise control, all the modules work cooperatively, so that the excellent performance is shown in warehouse management, and the warehouse management efficiency and reliability of the smoke enterprise are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of warehouse automation technology, and in particular to an intelligent warehouse system with a voice interaction function. Background Art

[0002] Industrial intelligence has gradually become popular in the warehousing scenarios of tobacco companies, mainly focusing on information construction pilots and experience exploration in several aspects such as operation automation, intelligent warehousing, and management visualization.

[0003] The intelligent warehousing technology currently used in tobacco companies, although it has introduced highly automated picking equipment and intelligent inventory databases, is still imperfect and incomplete, and there are many aspects that need to be improved. Summary of the Invention

[0004] In view of the above, the present invention aims to provide an intelligent warehousing system with voice interaction function to solve the technical problems mentioned above.

[0005] The technical solution adopted in the present invention is as follows:

[0006] The present invention provides an intelligent warehousing system with voice interaction function, which includes: a digital warehousing management system for raw materials, a handheld terminal, a vehicle-mounted terminal, and an intelligent maintenance system. The intelligent warehousing system generates warehousing instructions, in-warehouse instructions, and out-warehouse instructions, and combines preset strategies to guide operations to complete the following tasks: intelligent stack allocation, in-warehouse navigation, out-warehouse navigation, planned inventory, and statistical analysis. The digital warehousing management system for raw materials includes: a visual recognition module, a Mecanum wheel trolley drive module, and a human-computer interaction module.

[0007] The visual recognition module is used to perform image processing and calibrate cargo coordinates. In image processing, after the camera collects image data, it performs the following processing: RGB to YCbCr format conversion, binarization, edge detection, and skin color recognition;

[0008] The Mecanum wheel car drive module adopts chassis vector inverse kinematics algorithm and IMU sensor-based fusion processing algorithm;

[0009] The human-computer interaction module is used for users to interact through voice commands, including: when sending a command to start temperature and humidity detection, obtaining storage environment data and broadcasting it in the form of sound through the voice synthesis module; and controlling the robotic arm to switch the working mode through voice.

[0010] In at least one possible implementation, during the binarization process, the threshold is adjusted to make the image display part related to human skin color, wherein the yellow skin color recognition threshold is set to 77≤Cb≤127, 133≤Cr≤173.

[0011] In at least one possible implementation, the intelligent maintenance system includes the following four functional modules: environmental perception, early warning control, maintenance operations, and data analysis.

[0012] In at least one possible implementation, calibrating cargo coordinates includes:

[0013] Calculate the position of the pixel on the screen to obtain the coordinates of the pixel, and use the camera input enable as the counter synchronization switch to synchronize the counting with the frame counter. When the entire frame of pixel data is transmitted, the counter is cleared and the next frame counting begins. The x and y coordinate ranges are set to 1 < x < 480 and 1 < y < 272 based on the screen resolution. Convert the pixel coordinates to coordinates with the Mecanum wheel car drive module as the origin:

[0014] ;

[0015] in, and Indicates the offset of the Mecanum wheel car drive module relative to the pixel coordinate system;

[0016] Through the above conversion, the position of the cargo in the Mecanum wheel trolley drive module is obtained.

[0017] In at least one possible implementation, the Mecanum wheel trolley drive module establishes a velocity coordinate system with the chassis as the center, and derives the speed of each motor corresponding to the four wheels based on the relative relationship, as shown in the following formula:

[0018] ;

[0019] Among them, ω is the rotation speed of the car around the vertical ground direction, r x With r y are the horizontal distances from the wheel to the x and y directions respectively; V wi is the speed of each motor; V tx With V ty is the horizontal speed of the car in the x and y directions, and the speed of the four motors is obtained by solving.

[0020] In at least one possible implementation, the specific processing process of the voice module includes:

[0021] Perform A / D conversion and pre-processing on the collected voice signals, including: frequency band enhancement, spectrum flattening, digitization and endpoint detection;

[0022] Perform time-frequency, cepstral, and wavelet feature analysis on the processed speech signal, and extract features corresponding to timbre, language, and speech content;

[0023] The speech recognition model is used to match the keywords in the keyword list, and the words with the highest matching degree are obtained as the recognition result output; synchronously, the keyword list is dynamically edited.

[0024] In at least one possible implementation, the matching includes comparing the feature parameters of the extracted speech information with parameter models in a preset model library according to a set evaluation criterion to obtain the best matching result.

[0025] In at least one possible implementation, the mechanical structure of the intelligent warehousing system includes a bottom-up Mecanum wheel trolley chassis, a power supply and drive, a temperature and humidity sensor module, a human-computer interaction module, a lifting platform equipped with a camera, and a six-degree-of-freedom robotic arm.

[0026] Compared with existing technologies, the main design concept of the present invention is that it consists of a digital warehouse management system for raw materials, a handheld terminal, a vehicle-mounted terminal, and an intelligent maintenance system. By generating warehouse entry, storage, and outbound instructions, combined with preset strategies to guide operations, it can complete tasks such as intelligent stack allocation, entry navigation, outbound navigation, planned inventory, and statistical analysis. Among them, the digital warehouse management system for raw materials includes: a visual recognition module, a Mecanum wheel trolley drive module, and a human-computer interaction module; the visual recognition module calibrates cargo coordinates based on image processing, and the Mecanum wheel trolley drive module uses a fusion of motion control and sensor algorithms to achieve cargo transportation; the human-computer interaction module is used to complete functions such as warehouse environment data broadcasting, automatic cargo grabbing, voice intelligent interaction, and remote control of robotic arms through voice commands. The present invention leverages the advantages of multi-tasking parallel processing and precision control to enable various modules to work together, thereby demonstrating excellent performance in warehouse management and significantly improving the efficiency and reliability of tobacco companies' warehouse management. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below with reference to the accompanying drawings, in which:

[0028] Figure 1 A schematic diagram of an intelligent warehousing system with voice interaction function provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the processing flow of the intelligent warehousing system provided by an embodiment of the present invention;

[0030] Figure 3 This is a diagram of the speech recognition principle used in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0032] The present invention proposes an embodiment of an intelligent warehousing system with voice interaction function, specifically, Figures 1 to 3 As shown, it includes: a digital warehouse management system for raw materials, handheld terminals, vehicle-mounted terminals, and an intelligent maintenance system. These systems are mainly used to generate warehouse entry, in-warehouse, and outbound instructions. Through strategic guidance of operations, they implement intelligent stack allocation, entry navigation, outbound navigation, planned inventory, statistical analysis, etc. They support obtaining orders and unloading information from upstream systems to generate entry plans, implement intelligent stack allocation, and enable inbound navigation for transport vehicles to achieve operational support. They also support obtaining production tasks from upstream systems to generate outbound plans, and combine stack information to produce outbound navigation to achieve operational planning. Inventory: Based on the inventory tasks generated by the system, warehouse workers use vehicle-mounted terminals and handheld terminals to perform inventory operations. The system displays inventory information of the inventory materials and compares it with the physical inventory. Statistical analysis: The system has functions such as statistics and data query, analysis, and summary, and the query conditions can be combined flexibly and conveniently. Focusing on the nodes of warehousing, outbound, transfer, quality inspection, inventory and inventory status, the tobacco leaves are analyzed in terms of name, warehousing time, year, origin, type, form, structure, quantity, monthly mold content, quarterly standard cargo space utilization rate, corresponding warehouse / cargo space information and quality inspection information after warehousing. Secondly, the work order-driven clamping vehicle operation. The vehicle-mounted subsystem is installed on the onboard equipment of the clamping vehicle to guide the warehousing operation of the clamping vehicle and cooperate with the code scanning equipment on the clamping vehicle to collect the data of material in and out of the warehouse and transmit it back to the system and handheld mobile devices after the task is completed. Thirdly, the instruction-driven scanning operation. The system is installed on the mobile device side (supporting one-dimensional code, two-dimensional code scanning function and RFID recognition). It is mainly used for scanning codes such as material outbound, inbound, displacement, inventory, etc., collecting operation data, and supporting the generation of temporary operation instructions; and pushing these instructions to the forklift terminal or handheld terminal, and receiving warehouse operation warning information through the handheld terminal. By achieving wireless network coverage in the warehouse area, the mobile terminal can perform mobile operations and mobile maintenance work in the warehouse;

[0033] The raw material digital warehouse management system includes a visual recognition module, a Mecanum wheel trolley drive module, and a human-computer interaction module. The visual recognition module is used to perform image processing and calibrate cargo coordinates. In the image processing, after the camera collects image data, it performs processing including RGB to YCbCr format conversion, binarization, edge detection, skin color recognition, etc.; the Mecanum wheel trolley drive module adopts a chassis vector inverse kinematics algorithm and fusion processing based on an IMU sensor; the human-computer interaction module is used for users to interact with the system through voice commands. When a command is sent to start temperature and humidity detection, the system can obtain storage environment data and broadcast it in the form of sound through the voice synthesis module; the voice module can be used to control the robotic arm to switch working modes. In addition, through the Bluetooth connection between the mobile phone and the system, the user can intuitively and conveniently control the movement of the trolley to achieve rapid positioning and transportation of items in the warehouse.

[0034] In some embodiments, during the binarization process, the threshold is adjusted to display only the part of the image related to the human skin color. The threshold for yellow skin color recognition is 77≤Cb≤127, 133≤Cr≤173. The calibrated cargo coordinates use a mathematical formula to calculate the position of the pixel point on the screen, thereby obtaining the coordinates of the pixel point. The camera input is enabled as the counter synchronization switch, and the frame counter is used for synchronous counting. The original screen resolution is 480×272, and each frame contains 130560 pixels. After the entire frame of pixel data is transmitted, the counter is cleared and the next frame count begins. The x and y coordinate ranges are 1<x<480 and 1<y<272 (in pixels). The pixel coordinates are converted to coordinates with the Mecanum wheel trolley drive module as the origin:

[0035] ;

[0036] Where, and Represents the offset of the Mecanum wheel trolley drive module relative to the pixel coordinate system. Through the above conversion, the position of the cargo in the Mecanum wheel trolley drive module can be obtained. The Mecanum wheel trolley drive module establishes a velocity coordinate system with the chassis as the center. The speed of each motor is obtained based on the relative relationship. The specific formula is as follows:

[0037] ;

[0038] Where ω is the rotation speed of the car around the vertical ground direction, r x With r y are the horizontal distances from the wheel to the x and y directions respectively; V wi is the speed of each motor; V tx With V tyThe x and y horizontal speeds of the vehicle are obtained by solving the equations to obtain the rotational speeds of the four motors. The wheel speeds need to be adjusted based on the actual physical quantities and the PI algorithm. After calculating the motion posture required to reach the target location, the IMU provides the current position in real time during the actual motion process. The existing coordinates are refreshed every once in a while, and the inverse kinematics solution is re-performed to ensure that the vehicle can reach the target location with the correct posture.

[0039] The intelligent maintenance system comprises four functional modules: environmental perception, early warning and control, maintenance operations, and data analysis. Environmental perception utilizes various IoT sensor devices to comprehensively establish an environmental perception center, providing support for monitoring, early warning, maintenance management, and data analysis. Leveraging the early warning and control system, all maintenance operations are based on early warning and scheduling information, significantly improving operational accuracy and efficiency. Furthermore, using environmental temperature and humidity early warning information, the system can be integrated with environmental control equipment such as dehumidifiers and ventilators to achieve precise start and stop times, achieving the goal of automated inter-warehouse environmental control. Maintenance operations utilize a scheduling logic system centered on the maintenance operations center and supplemented by the early warning and control center. This includes proactive maintenance operations, survey-related operations, and reactive operations related to early warning processing, ensuring that all maintenance operations are initiated, supported, and evaluated by the system. The maintenance operations center serves as the initiator of all proactive maintenance operations. Data analysis utilizes big data tools to integrate and analyze equipment monitoring data, operational data, and tobacco leaf maintenance inspection data. The analysis results are presented to users in the form of charts and reports, helping them better understand the results. Provide support for business decisions based on the results of data analysis.

[0040] The data analysis function can realize the functions of "monthly log report", "data query" and "formula use warning", and at the same time complete the basic framework of data archive and data analysis model. By sorting and integrating various types of data and using a variety of information technology methods, a database including warehouse environment, stacking environment, maintenance mode, stacking method, tobacco raw material information, logistics operation information, etc. is constructed. It provides more complete, systematic and comprehensive data support for tobacco storage quality evaluation, tobacco storage quality warning and prediction, and personalized tobacco storage maintenance strategy. Two core temperature and humidity sensors are set at each stack:

[0041] Each warehouse is equipped with 5 wireless temperature and humidity sensors:

[0042] One insect monitoring point is set up in each warehouse. The intelligent insect monitoring device uses 4G / 5G wireless transmission, and the detection frequency can be adjusted according to demand.

[0043] One phosphine concentration sensor is installed in each compartment.

[0044] One LORA wireless gateway is installed on each floor of the alcoholization warehouse.

[0045] The specific processing process of the voice module is divided into five parts: voice preprocessing, feature extraction, model training, model matching and post-speech processing. The voice preprocessing is used to perform A / D conversion and other processing tasks on the collected voice signal, including increasing the frequency band, smoothing the spectrum, digitizing and endpoint detection processing. The voice chip first obtains the sound signal through the microphone (MIC: Microphone), and then performs spectrum analysis on the obtained signal, extracts features, and uses the voice recognition model to match the keywords in the keyword list (model training library) to find the words with the highest matching degree as the recognition result output. In this process, the keyword list can also be dynamically edited by the micro control unit (MCU: Micro Control Unit). The voice module uses the LD3320 chip, which has a speech recognition algorithm and voice data processing circuit (AD, DA) integrated inside. Module), the feature extraction is used for the speech signal after preprocessing, and it needs to perform feature analysis such as time-frequency domain, cepstrum domain and wavelet to obtain corresponding features such as timbre, language and speech content, the model training is used to continuously optimize the feature parameters extracted in the previous link, and establish a speech training model based on this, so that the model can more comprehensively reflect the speech features, the model matching is used to compare the feature parameters of the extracted speech information with the trained parameter model in the model library according to the evaluation criteria set in the template to obtain the best matching result, the post-speech processing is used to combine the relevant theoretical knowledge of machine learning to perform word meaning analysis, grammatical analysis and semantic understanding on the speech results output by pattern matching to improve the overall recognition performance of the system, the overall mechanical structure of the intelligent warehousing system includes a Mecanum wheel trolley chassis, a power supply and driver, a temperature and humidity sensor module, a human-computer interaction module, a lifting platform equipped with a camera, a six-degree-of-freedom robotic arm, etc. from bottom to top; the voice chip has two ways to recognize voice signals and output recognition results:

[0046] (1) The chip collects voice signal data for a predetermined time (e.g., 3 seconds). After the predetermined time is up, the chip stops collecting voice signal data. Then, only the extracted voice signal data is analyzed, feature extracted, and model library matched, and finally the recognition result is given;

[0047] (2) The chip uses endpoint detection technology (VAD: Voice Activity Detection) to collect voice signal data from the time the user starts to speak to the time they stop speaking, and then analyzes the collected voice signal data to extract features, match the model, and provide recognition results;

[0048] VAD technology determines the start and end times of human speech from a speech data stream, based on background sound. The speech recognition chip then processes, matches, and calculates the speech regions identified by VAD, and then outputs the recognition results. It's important to note that the LD3320 cannot actively output recognition results during the speech recognition process. The speech chip continuously recognizes and calculates the matching degree of the incoming speech signal. It only outputs a result when the incoming speech signal ends; otherwise, the chip remains in the speech recognition process and cannot output a result. For example, the speech chip's recognition list contains two keywords: "201" and "2017." A data sample for "2017" is collected. When the user says the syllable "1," the speech recognition chip's highest match is for the keyword "201." However, the chip cannot determine whether the user's speech signal continues. If it stops, it outputs "201." However, if the speech signal continues, as when the user says the syllable "7," the keyword "2017" has the highest match. At this point, the voice signal stops inputting and the chip gives the recognition result "2017".

[0049] The following human-machine interaction test uses the intelligent warehousing system of this invention. Using a Bluetooth connection, a Mecanum-wheeled cart can be remotely controlled, precisely guiding it to every corner of the warehouse. By integrating IMU (Inertial Measurement Unit) technology, the cart's real-time position and posture information is acquired. Intelligent voice recognition technology allows users to obtain information about the current temperature and humidity within the warehouse. If a stored item accidentally falls into a confined space, the user can use voice to switch the robotic arm to remote control mode, guiding it to accurately grasp and reposition the lost item. This solves the common problem of goods falling into confined spaces and preventing human intervention.

[0050] Automatic crawling test:

[0051] This test involves the collaboration of vision, robotic arms, and a Mecanum wheeled vehicle to automatically grasp, carry, and unload cargo:

[0052] The system implements functions such as automatic cargo handling and intelligent voice interaction. Leveraging the parallel processing and precision control advantages of FPGAs, the system enables various modules to work collaboratively, demonstrating exceptional performance in warehouse management. The system demonstrates high comprehensiveness and efficiency, providing innovative solutions to practical management issues in small and medium-sized warehouses, significantly improving the efficiency and accuracy of warehouse management and injecting new impetus into the development of the logistics industry.

[0053] In summary, the main design concept of the present invention is that it consists of a digital warehouse management system for raw materials, a handheld terminal, a vehicle-mounted terminal, and an intelligent maintenance system. By generating warehouse entry instructions, in-warehouse instructions, and outbound instructions, combined with preset strategies to guide operations, it can complete the following tasks: intelligent stack allocation, entry navigation, outbound navigation, planned inventory, and statistical analysis. The digital warehouse management system for raw materials includes: a visual recognition module, a Mecanum wheel trolley drive module, and a human-computer interaction module; the visual recognition module is used for image processing and calibration of cargo coordinates, and the Mecanum wheel trolley drive module uses a chassis vector inverse kinematics algorithm and a fusion processing algorithm based on an IMU sensor; the human-computer interaction module is used for user interaction through voice commands, including functions such as warehouse environment data broadcasting, automatic cargo grabbing, voice intelligent interaction, and remote control robotic arms. The present invention leverages the advantages of multi-task parallel processing and precision control to enable each module to work together, thereby demonstrating excellent performance in warehouse management and significantly improving the efficiency and reliability of tobacco companies' warehouse management.

[0054] If the expressions expressing directions are mentioned in the embodiments of the present invention, they are relative concepts based on the embodiments. In addition, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a, b and c, where a, b, c can be single or multiple.

[0055] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred modes can be reasonably combined and matched into a variety of equivalent schemes by those skilled in the art without departing from or changing the design ideas and technical effects of the present invention; therefore, the scope of implementation of the present invention is not limited to what is shown in the drawings. Any changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the description and drawings, should be within the scope of protection of the present invention.

Claims

1. An intelligent warehousing system with voice interaction function, characterized in that: include: A digital warehouse management system for raw materials, a handheld terminal, a vehicle-mounted terminal, and an intelligent maintenance system. The intelligent warehouse system generates warehouse entry instructions, in-warehouse instructions, and outbound instructions, and combines preset strategies to guide operations to complete the following tasks: intelligent stack allocation, inbound navigation, outbound navigation, planned inventory, and statistical analysis. The digital warehouse management system for raw materials includes a visual recognition module, a Mecanum wheel trolley drive module, and a human-computer interaction module. The visual recognition module is used to perform image processing and calibrate cargo coordinates. In image processing, after the camera collects image data, it performs the following processing: RGB to YCbCr format conversion, binarization, edge detection, and skin color recognition; The Mecanum wheel car drive module adopts chassis vector inverse kinematics algorithm and IMU sensor-based fusion processing algorithm; The human-computer interaction module is used for users to interact through voice commands, including: when sending a command to start temperature and humidity detection, obtaining storage environment data and broadcasting it in the form of sound through the voice synthesis module; and controlling the robotic arm to switch the working mode through voice.

2. The intelligent warehousing system with voice interaction function according to claim 1 is characterized in that: During the binarization process, the threshold is adjusted to make the image display part related to human skin color, wherein the yellow skin color recognition threshold is set to 77≤Cb≤127, 133≤Cr≤173.

3. The intelligent warehousing system with voice interaction function according to claim 1 is characterized in that: The intelligent maintenance system includes the following four functional modules: environmental perception, early warning control, maintenance operations, and data analysis.

4. The intelligent warehousing system with voice interaction function according to claim 1 is characterized in that: The calibration of cargo coordinates includes: Calculate the position of the pixel on the screen to obtain the coordinates of the pixel, and use the camera input enable as the counter synchronization switch to synchronize the counting with the frame counter. When the entire frame of pixel data is transmitted, the counter is cleared and the next frame counting begins. The x and y coordinate ranges are set to 1 < x < 480 and 1 < y < 272 based on the screen resolution. Convert the pixel coordinates to coordinates with the Mecanum wheel car drive module as the origin: ； in, and Indicates the offset of the Mecanum wheel car drive module relative to the pixel coordinate system; Through the above conversion, the position of the cargo in the Mecanum wheel trolley drive module is obtained.

5. The intelligent warehousing system with voice interaction function according to claim 1 is characterized in that: The Mecanum wheel car drive module establishes a velocity coordinate system with the chassis as the center, and derives the speed of each motor corresponding to the four wheels based on the relative relationship. The formula is as follows: ； Among them, ω is the rotation speed of the car around the vertical ground direction, r x With r y are the horizontal distances from the wheel to the x and y directions respectively; V wi is the speed of each motor; V tx With V ty is the horizontal speed of the car in the x and y directions, and the speed of the four motors is obtained by solving.

6. The intelligent warehousing system with voice interaction function according to claim 1 is characterized in that: The specific processing process of the voice module includes: Perform A / D conversion and pre-processing on the collected voice signals, including: frequency band enhancement, spectrum flattening, digitization and endpoint detection; Perform time-frequency, cepstral, and wavelet feature analysis on the processed speech signal, and extract features corresponding to timbre, language, and speech content; The speech recognition model is used to match the keywords in the keyword list, and the words with the highest matching degree are obtained as the recognition result output; synchronously, the keyword list is dynamically edited.

7. The intelligent warehousing system with voice interaction function according to claim 6 is characterized in that: The matching includes: comparing the characteristic parameters of the extracted speech information with the parameter models in the preset model library according to the set evaluation criteria to obtain the best matching result.

8. The intelligent warehousing system with voice interaction function according to any one of claims 1 to 9, characterized in that: The mechanical structure of the intelligent warehousing system includes a bottom-up Mecanum wheel trolley chassis, a power supply and drive, a temperature and humidity sensor module, a human-computer interaction module, a lifting platform equipped with a camera, and a six-degree-of-freedom robotic arm.

Images