Inspection robot commodity out-of-stock identification and reporting system based on multi-sensor fusion

Abstract

This invention proposes a multi-sensor fusion-based inspection robot system for identifying and reporting out-of-stock items. Relating to the field of out-of-stock identification technology, the system acquires initial environmental identification information, generates inspection path information from this information, and generates inspection task information based on the path. It then collects inspection task data from multiple sensors to obtain shelf-collected product data, performs data fusion processing on this data, and obtains fused data. Based on this fused data, it performs out-of-stock identification and status analysis, and reports and replenishes the stock based on the obtained status analysis data, thus obtaining replenishment information. This invention achieves intelligent operation that balances path optimization, multi-source data fusion, and accurate judgment.

Description

Technical Field

[0001] This invention proposes a product shortage identification and reporting system for inspection robots based on multi-sensor fusion, which relates to the field of product shortage identification technology, specifically to the field of product shortage identification technology for inspection robots based on multi-sensor fusion. Background Technology

[0002] In the field of retail merchandise management, the timeliness and accuracy of out-of-stock identification and replenishment response directly impact operational efficiency and customer experience. Current out-of-stock inspections largely rely on manual labor or single-sensor equipment. Manual inspections suffer from low efficiency, high rates of missed or false positives, and are ill-suited for the continuous operation of large retail stores, while also incurring high labor costs and workload. Single-sensor inspections, on the other hand, suffer from limited data dimensions, are susceptible to environmental interference, and cannot comprehensively capture the status of merchandise displays and spatial information. Furthermore, traditional inspection route planning is fixed, lacking a holistic consideration of out-of-stock risks and traffic efficiency, leading to unreasonable allocation of inspection resources. The isolated and uncoordinated nature of multi-source data, coupled with a lack of standardized integration and quantitative analysis mechanisms, further reduces the accuracy of out-of-stock identification, causing delayed replenishment responses and resulting in operational losses. Summary of the Invention

[0003] This invention provides a multi-sensor fusion-based system for identifying and reporting out-of-stock items in inspection robots, in order to solve the above-mentioned problems: The present invention proposes a multi-sensor fusion-based inspection robot product shortage identification and reporting system, the system comprising: The inspection task analysis module is used to obtain initial environment identification information, generate inspection path information based on the initial environment identification information, and generate inspection task information based on the inspection path information. The data acquisition, fusion, and analysis module is used to collect data on shelf inspection tasks through multiple sensors, obtain shelf merchandise data, and perform data fusion processing on the shelf merchandise data to obtain acquired and fused data. The out-of-stock identification and analysis module is used to perform out-of-stock identification and analysis and out-of-stock status analysis based on the collected and fused data, report out-of-stock status and replenish stock based on the obtained out-of-stock status analysis data, and obtain the reported replenishment information.

[0004] Furthermore, the method includes: S1. Obtain initial environment identification information, generate inspection path information based on the initial environment identification information, and generate inspection task information based on the inspection path information; S2. Collect data on the inspection task information of the shelves through multiple sensors, obtain the shelf merchandise collection data, and perform data fusion processing on the shelf merchandise collection data to obtain the collected fusion processing data; S3. Based on the collected and fused data, perform product out-of-stock identification and analysis and out-of-stock status analysis. Based on the obtained out-of-stock status analysis data, report the out-of-stock status and replenish the stock to obtain the reported replenishment information.

[0005] Further, S1 includes: Environmental data is collected from the product storage area to obtain initial environmental identification information; The initial environmental identification information is subjected to inspection feature processing to obtain environmental inspection feature processing data; Based on the environmental inspection feature processing data, inspection path information is generated; The inspection task information is determined based on the inspection route information and the product storage area.

[0006] Further, the step of performing inspection feature processing on the initial environmental identification information to obtain environmental inspection feature processing data includes: Based on the initial environmental identification information, the identification information is classified to obtain identification classification information; The identification and classification information is used to locate the information, thereby obtaining identification, location, and classification information; The identification and positioning classification information is matched with the preset inspection target to obtain identification matching information; Based on the identification and matching information, the initial environmental identification information is labeled with inspection features to obtain environmental inspection feature processing data.

[0007] Furthermore, the step of generating inspection path information based on the environmental inspection feature processing data includes: The starting and ending points of the inspection are determined based on the data processing characteristics of the environmental inspection. Based on the inspection start point and inspection end point, the environmental inspection feature data are concatenated to obtain the initial inspection information; Obtain the historical out-of-stock rate of goods from the initial inspection information, and sort the historical out-of-stock rates from largest to smallest to obtain the out-of-stock rate sequence; Determine the out-of-stock rate location and its weight based on the out-of-stock rate sequence, and obtain out-of-stock location weight information; Obtain the current number of obstacles in the initial inspection information, sort the current number of obstacles in ascending order, and obtain the obstacle sequence; The location and weight of obstacles are determined based on the obstacle sequence, and obstacle location weight information is obtained; The final positioning weight information is determined by combining the out-of-stock positioning weight information with the obstacle positioning weight information. The final positioning weight information is sorted from largest to smallest to obtain the inspection weight sequence; The initial inspection information is adjusted according to the inspection weight sequence to obtain the inspection path information.

[0008] Furthermore, determining the inspection task information based on the inspection path information and the product storage area includes: The inspection area nodes are determined based on the inspection route information and the product storage area. Pre-set inspection parameters for nodes in the inspection area to obtain preset inspection parameter data; Preset the area parameters for the product storage area and obtain the preset area parameter data; Based on the preset inspection parameter data, the preset area parameter data is allocated to obtain inspection task information.

[0009] Further, S2 includes: Based on the inspection task information, the inspection task area is determined, and regional data is collected from the inspection task area through multiple sensors to obtain shelf merchandise collection data. The data collected from the shelves is divided into multiple categories to obtain data for multiple categories. Based on data collected from multiple categories, the shelf merchandise data is analyzed to obtain multiple categories of collected and analyzed data. Based on the data collected and analyzed from multiple categories, the data collected from the shelf goods is labeled with data categories to obtain data labeling information for multiple categories; Multiple types of data annotation information are integrated and annotated for the inspection task area to obtain collected and fused data.

[0010] Furthermore, the step of performing category-based collection analysis on the shelf merchandise collection data based on multiple collection categories to obtain multiple categories of collection analysis data includes: Obtain the ratio of data for each collection category to the preset standard data for each category, and obtain the collection analysis coefficient for each category. The category collection and analysis coefficients are compared with the corresponding preset category collection and analysis thresholds to obtain the category analysis comparison results; Based on the comparison results of the category analysis, the category status is determined for each category collection analysis coefficient to obtain category collection status determination information; The category collection status determination information is the category collection analysis data.

[0011] Further, S3 includes: Based on the collected and fused data, determine the out-of-stock identification data; Based on the out-of-stock identification data, perform out-of-stock identification analysis to obtain out-of-stock identification analysis data; The out-of-stock identification and analysis data is compared with the preset out-of-stock identification threshold to obtain the out-of-stock identification comparison results. Based on the comparison results of out-of-stock identification, the out-of-stock identification analysis data is used to determine the out-of-stock status and obtain out-of-stock status determination information. Based on the out-of-stock status determination information, out-of-stock status is reported and replenishment is carried out, and the reported replenishment information is obtained.

[0012] Furthermore, the step of performing product out-of-stock identification analysis based on product out-of-stock identification data to obtain product out-of-stock identification analysis data includes: Determine the number of categories marked as out of stock and the number of categories marked as not out of stock based on the out-of-stock identification data; Obtain the ratio of the number of out-of-stock labels to the number of non-out-of-stock labels for each category to obtain the out-of-stock identification coefficient. The product shortage identification coefficient is the product shortage identification analysis data.

[0013] The beneficial effects of this invention are as follows: This system solves the technical problems of low efficiency, high rate of missed and false judgments, isolated and uncoordinated multi-sensor data, and delayed response to stockouts in traditional manual inspections. It also addresses the pain point of manual inspections being unsuitable for the all-day, all-coverage operations of large retail stores. It achieves intelligent and automated management of stockouts in retail stores, breaking down information barriers between data collection, analysis, reporting, and replenishment, forming a complete business loop. It improves the comprehensiveness of inspection coverage, the accuracy of stockout identification, and the timeliness of replenishment response, adapting to the complex display and operational scenarios of retail stores of different sizes, and enhancing the flexibility and adaptability of merchandise management. It reduces the labor costs and workload of manual inspections, avoids errors caused by subjective human judgment, strengthens the standardization and controllability of merchandise management, ensures the integrity of merchandise displays in the store, reduces operational losses due to stockouts, and improves the overall operational efficiency of the retail store. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of a multi-sensor fusion-based inspection robot system for identifying and reporting out-of-stock items. Detailed Implementation

[0015] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0016] In one embodiment of the present invention, the present invention proposes a multi-sensor fusion-based inspection robot product shortage identification and reporting system, the system comprising: The inspection task analysis module is used to obtain initial environment identification information, generate inspection path information based on the initial environment identification information, and generate inspection task information based on the inspection path information. The data acquisition, fusion, and analysis module is used to collect data on shelf inspection tasks through multiple sensors, obtain shelf merchandise data, and perform data fusion processing on the shelf merchandise data to obtain acquired and fused data. The out-of-stock identification and analysis module is used to perform out-of-stock identification and analysis and out-of-stock status analysis based on the collected and fused data, report out-of-stock status and replenish stock based on the obtained out-of-stock status analysis data, and obtain the reported replenishment information.

[0017] The working principle and technical effects of the above technical solution are as follows: This system accurately acquires initial environmental information such as the spatial layout, shelf distribution, and obstacle location of the merchandise storage area through environmental acquisition equipment. Based on this information, it plans an inspection path that covers the entire area and has no collision risk. Then, combined with the needs of store operation, the path is broken down into executable inspection task information. Multi-sensor collaborative acquisition is initiated. Visual cameras capture the appearance, display status, and empty areas of merchandise. LiDAR locates the spatial coordinates and distance of merchandise. Weight sensors collect the load-bearing data of shelf panels. Infrared sensors compensate for the shortcomings of visual blind spots. After collecting shelf merchandise data from multiple dimensions, data fusion technology is used to eliminate the detection limitations and errors of single sensors and integrate them to form comprehensive and reliable collected and processed data. The fused data is analyzed in layers. First, out-of-stock related characteristics are identified from the data. Then, the specific status of the out-of-stock is accurately determined. Based on the determination result, the reporting process is automatically triggered and a replenishment instruction is generated, realizing full-link automated control from inspection to replenishment. The accurate identification and response to merchandise out-of-stock can be completed without manual intervention.

[0018] This system solves the technical problems of traditional manual inspections, such as low efficiency, high rates of missed and false diagnoses, isolated and uncoordinated data from multiple sensors, and delayed response to stockouts. It also addresses the pain point of manual inspections being ill-suited for the all-day, all-coverage operations of large retail stores. It achieves intelligent and automated stockout management in retail stores, breaking down information barriers between data collection, analysis, reporting, and replenishment, forming a complete business loop. It improves the comprehensiveness of inspection coverage, the accuracy of stockout identification, and the timeliness of replenishment response, adapting to the complex display and operational scenarios of retail stores of different sizes, enhancing the flexibility and adaptability of merchandise management. It reduces the labor costs and workload of manual inspections, avoids errors caused by subjective human judgment, strengthens the standardization and controllability of merchandise management, ensures the integrity of merchandise displays in the store, reduces operational losses due to stockouts, and improves the overall operational efficiency of the retail store.

[0019] In one embodiment of the present invention, the method includes: S1. Obtain initial environment identification information, generate inspection path information based on the initial environment identification information, and generate inspection task information based on the inspection path information; S2. Collect data on the inspection task information of the shelves through multiple sensors to obtain the shelf merchandise collection data, and perform data fusion processing on the shelf merchandise collection data to obtain the collected fusion processing data; the multiple sensors include visual cameras, lidar, weight sensors, infrared sensors, etc. S3. Based on the collected and fused data, perform product out-of-stock identification and analysis, and out-of-stock status analysis. Based on the obtained out-of-stock status analysis data, report the out-of-stock status and replenish the stock, obtaining the reported replenishment information, such as... Figure 1 As shown.

[0020] In retail merchandise management, traditional manual and single-sensor inspections suffer from low efficiency, high false alarm rates, rigid paths, and isolated data, leading to delayed replenishment responses and impacting operational efficiency. This example provides an intelligent solution that uses multi-sensor fusion and weighted path optimization as its core to achieve closed-loop management of the entire process of out-of-stock identification and reporting, adaptable to various retail scenarios.

[0021] First, data on the store environment is collected using LiDAR and visual sensors. Elements such as shelves and obstacles are categorized and located, and inspection targets are matched to generate environmental feature data. Based on this data, an initial path is planned. A dual-weight system is constructed by combining historical out-of-stock rates and real-time obstacle counts to prioritize coverage of areas with high out-of-stock rates and low obstacles. After optimization, the path is broken down into standardized inspection tasks.

[0022] The system integrates visual, lidar, weight, and infrared sensors to collect data on product appearance, spatial coordinates, shelf load-bearing capacity, and blind spots. Analysis coefficients are calculated for each product type and compared to standard values. This data is then integrated and fused after comparing with threshold markings to form reliable data. Core data is filtered to count the number of out-of-stock and non-out-of-stock markings, and the ratio is calculated to obtain a recognition coefficient. This coefficient is then compared with thresholds to accurately determine the out-of-stock status.

[0023] The solution addresses the pain points of traditional technologies by quantitatively integrating multi-source data and optimizing dynamic paths, achieving full-process automation. It improves inspection efficiency, path rationality, and the accuracy of out-of-stock identification, while reducing labor costs and operational losses, and strengthening the standardization of merchandise management.

[0024] The working principle and technical effects of the above technical solution are as follows: This method accurately acquires initial environmental information such as the spatial layout, shelf distribution, and obstacle location of the merchandise storage area through environmental acquisition equipment. Based on this information, an inspection path covering the entire area and without collision risk is planned. Then, combined with the needs of store operation, the path is broken down into executable inspection task information. Multi-sensor collaborative acquisition is initiated. Visual cameras capture the appearance, display status, and empty areas of merchandise. LiDAR locates the spatial coordinates and distance of merchandise. Weight sensors collect the load-bearing data of shelf panels. Infrared sensors compensate for the shortcomings of visual blind spots. After collecting shelf merchandise data from multiple dimensions, the detection limitations and errors of single sensors are eliminated through data fusion technology, and comprehensive and reliable collected and processed data is integrated. The fused data is analyzed in layers. First, out-of-stock related characteristics are identified from the data. Then, the specific status of the out-of-stock is accurately determined. Based on the determination result, the reporting process is automatically triggered and a replenishment instruction is generated, realizing full-link automated control from inspection to replenishment. The accurate identification and response to merchandise out-of-stock can be completed without manual intervention.

[0025] This method addresses the technical problems of traditional manual inspections, such as low efficiency, high rates of missed and false diagnoses, isolated and uncoordinated multi-sensor data, and delayed out-of-stock response. It also overcomes the pain point of manual inspections being ill-suited for the all-day, all-coverage operations of large retail stores. It achieves intelligent and automated out-of-stock management in retail stores, breaking down information barriers between data collection, analysis, reporting, and replenishment, forming a complete business loop. It improves the comprehensiveness of inspection coverage, the accuracy of out-of-stock identification, and the timeliness of replenishment response, adapting to the complex display and operational scenarios of retail stores of different sizes, enhancing the flexibility and adaptability of merchandise management. It reduces the labor costs and workload of manual inspections, avoids errors caused by subjective human judgment, strengthens the standardization and controllability of merchandise management, ensures the integrity of merchandise displays in the store, reduces operational losses due to out-of-stock situations, and improves the overall operational efficiency of the retail store.

[0026] In one embodiment of the present invention, S1 includes: Environmental data is collected from the product storage area to obtain initial environmental identification information; The initial environmental identification information is subjected to inspection feature processing to obtain environmental inspection feature processing data; Based on the environmental inspection feature processing data, inspection path information is generated; The inspection task information is determined based on the inspection route information and the product storage area.

[0027] The working principle and technical effects of the above-mentioned technical solution are as follows: This method focuses on the environmental processing and planning stage in the early stage of inspection. The core is to transform the original environmental information into executable inspection tasks through a standardized process. Using equipment such as LiDAR and visual sensors, a comprehensive environmental scan and data collection is performed on the merchandise storage area to capture original data such as shelf location, aisle width, fixed obstacles, and boundaries of merchandise display areas. This data is then integrated to form initial environmental identification information, ensuring a comprehensive understanding of the store environment. Inspection feature processing is performed on the initial environmental identification information to filter out core features relevant to the inspection, eliminate irrelevant and redundant data, and strengthen key information such as shelf location and key inspection points, forming environmental inspection feature processing data. Based on the processed feature data, combined with the store spatial layout and the principle of no collisions, the starting and ending points of the inspection are determined, and a basic inspection path is planned that connects various shelf areas and avoids obstacles, ensuring that the path covers all merchandise storage areas. Considering the zoning management needs of the merchandise storage areas and the importance of the shelves, the inspection path is broken down into multiple inspection area nodes. Corresponding inspection content and requirements are assigned to each node, determining the inspection task information, making the inspection operation more targeted and operable.

[0028] This method solves the technical problems of traditional inspection route planning, such as high subjectivity, low adaptability to the actual store environment, and low inspection efficiency due to the disconnect between routes and tasks. It realizes the standardized transformation of environmental information into inspection tasks, ensuring that the inspection route conforms to the actual store layout and that task allocation accurately corresponds to the needs of the merchandise storage area. It improves the rationality of the inspection route and the feasibility of the inspection task, while enhancing environmental adaptability and being able to cope with scenarios such as store display adjustments and obstacle changes. It reduces the degree of human involvement and error rate in route planning, reduces inspection omissions or duplicate inspections caused by unreasonable routes, and improves the efficiency and accuracy of the initial planning.

[0029] In one embodiment of the present invention, the step of performing inspection feature processing on the initial environmental identification information to obtain environmental inspection feature processing data includes: Based on the initial environmental identification information, the identification information is classified to obtain identification classification information; The identification and classification information is used to locate the information, thereby obtaining identification, location, and classification information; The identification and positioning classification information is matched with the preset inspection target to obtain identification matching information; Based on the identification and matching information, the initial environmental identification information is labeled with inspection features to obtain environmental inspection feature processing data.

[0030] The working principle and technical effects of the above technical solution are as follows: This method focuses on the refined processing of initial environmental identification information. Through a four-step process of classification, positioning, matching, and annotation, core inspection features are extracted. The collected initial environmental identification information is classified by type, and different environmental elements such as shelves, aisles, obstacles, and pillars are categorized and organized to obtain identification classification information, avoiding processing errors caused by the mixing of different types of information. Spatial positioning technology is used to accurately locate the coordinates of each type of information, clarifying the specific location of each element in the store, forming identification positioning classification information, and ensuring the accuracy of feature location. The positioned classification information is compared and matched with preset inspection targets (such as designated shelves, key display areas, and mandatory inspection nodes), filtering out elements related to inspection and excluding environmental elements that do not need to be inspected, to obtain identification matching information. Based on the matching results, the initial environmental identification information is annotated with inspection features, marking key inspection points, shelf numbers, mandatory inspection area boundaries, and other core information, eliminating redundant data, and forming structured and accurate environmental inspection feature processing data.

[0031] This method addresses the technical challenges of cluttered initial environmental identification information, lack of prominent core inspection features, and low accuracy in subsequent path generation due to data redundancy. It achieves refined and structured processing of initial environmental information, accurately extracting core features relevant to inspection. This improves the accuracy and efficiency of environmental information processing, reduces interference from invalid data in subsequent processes, enhances the standardization of feature processing, and ensures consistency in feature extraction across different scenarios. It also reduces path planning errors caused by cluttered data, preventing missed or false inspections.

[0032] In one embodiment of the present invention, generating inspection path information based on the environmental inspection feature processing data includes: The starting and ending points of the inspection are determined based on the data processing characteristics of the environmental inspection. Based on the inspection start point and inspection end point, the environmental inspection feature data are concatenated to obtain the initial inspection information; Obtain the historical out-of-stock rate of goods from the initial inspection information, and sort the historical out-of-stock rates from largest to smallest to obtain the out-of-stock rate sequence; Determine the out-of-stock rate location and its weight based on the out-of-stock rate sequence, and obtain out-of-stock location weight information; Obtain the current number of obstacles in the initial inspection information, sort the current number of obstacles in ascending order, and obtain the obstacle sequence; The location and weight of obstacles are determined based on the obstacle sequence, and obstacle location weight information is obtained; The final positioning weight information is determined by combining the out-of-stock positioning weight information with the obstacle positioning weight information. The final positioning weight information is sorted from largest to smallest to obtain the inspection weight sequence; The initial inspection information is adjusted according to the inspection weight sequence to obtain the inspection path information.

[0033] The working principle and technical effects of the above technical solution are as follows: This method adopts a dual-weight adjustment mechanism, combining historical out-of-stock data with real-time environmental obstacle conditions to generate an optimized inspection path, balancing inspection priority and traffic efficiency. Based on environmental inspection feature processing data, combined with factors such as the location of store entrances and exits and shelf distribution density, the starting and ending points of the inspection are reasonably determined to ensure the rationality of the path planning; the starting and ending points are used as anchor points to connect various inspection-related feature elements, forming initial inspection information covering the entire area and building a basic path framework; historical out-of-stock rate data of each inspection area are retrieved, sorted from high to low to form an out-of-stock rate sequence, and high-out-of-stock rate areas are assigned higher weights, clarifying out-of-stock location weight information to ensure priority inspection of high-risk areas; simultaneously, real-time collection of the initial inspection information coverage area... The current number of obstacles is sorted in ascending order to form an obstacle sequence. Areas with fewer obstacles and smooth passage are assigned higher weights to obtain obstacle location weight information, ensuring inspection efficiency. The out-of-stock location weight is combined with the obstacle location weight through a weight fusion algorithm to calculate the final location weight information of each area, which is then sorted in descending order of weight to form an inspection weight sequence. Based on this sequence, the initial inspection information is adjusted to optimize the path order, prioritize coverage of high-weight areas (high out-of-stock rate, low obstacle), and adjust detour routes to ultimately form inspection path information that balances priority and efficiency.

[0034] This method solves the technical problems of traditional inspection routes being fixed, unable to balance stockout risk and traffic efficiency, and unreasonable allocation of inspection resources. It achieves intelligent and dynamic optimization of inspection routes, integrating historical stockout data and real-time obstacle information into route planning, making the routes more aligned with the needs of store operations. It improves the targeting and efficiency of inspections, prioritizing inspections of high stockout rate areas to promptly identify stockout issues, and prioritizing passage in low-obstacle areas to reduce inspection time, while also optimizing the allocation of inspection resources. It reduces problems such as missed stockouts and excessively long inspection times caused by unreasonable routes, reduces ineffective inspection operations, and improves the overall response efficiency of stockout management.

[0035] In one embodiment of the present invention, determining the inspection task information based on the inspection path information and the merchandise storage area includes: The inspection area nodes are determined based on the inspection route information and the product storage area. Pre-set inspection parameters for nodes in the inspection area to obtain preset inspection parameter data; Preset the area parameters for the product storage area and obtain the preset area parameter data; Based on the preset inspection parameter data, the preset area parameter data is allocated to obtain inspection task information.

[0036] The working principle and technical effect of the above technical solution are as follows: This method transforms the inspection path into specific and executable inspection tasks, and achieves precise task implementation through parameter preset and allocation. Combining the generated inspection path information with the zoning layout of the merchandise storage area, the inspection path is broken down into several independent inspection area nodes. Each node corresponds to a specific shelf area or display unit, clarifying the inspection scope of each node and ensuring that the task breakdown is complete and non-overlapping. Inspection parameters are preset for each inspection area node, including inspection duration, sensor acquisition frequency, key inspection areas, etc. Differentiated parameters are set according to the importance of different nodes to obtain preset inspection parameter data. At the same time, area parameters are preset for the merchandise storage area, including basic information such as the number of shelves in the area, merchandise categories, and display density, forming preset area parameter data. Based on the preset inspection parameter data, parameters such as inspection requirements and collection standards are accurately allocated to the preset area parameters of the corresponding area, clarifying the specific work content, standards, and requirements of each inspection area node. After integration, structured inspection task information is formed to ensure that the inspection operation is systematic and followable.

[0037] This method solves the technical problems of disjointed inspection paths and actual operations, unclear task requirements, and inconsistent inspection standards in different areas, which lead to chaotic operations. It achieves precise conversion of inspection paths into specific tasks, clarifies the parameters and requirements for inspection in each area, and standardizes and regulates inspection operations. It improves the executability and relevance of inspection tasks, and the differentiated parameter settings can adapt to the inspection needs of different areas, while facilitating precise execution by operators or robots. It reduces the chaos and error rate of inspection operations, reduces missed inspections and incorrect inspections caused by unclear tasks, and improves the quality and efficiency of inspection operations.

[0038] In one embodiment of the present invention, S2 includes: Based on the inspection task information, the inspection task area is determined, and regional data is collected from the inspection task area through multiple sensors to obtain shelf merchandise collection data. The data collected from the shelves is divided into multiple data categories to obtain multiple data categories; the data categories include collected weight, collected density, and collected distance, etc. Based on data collected from multiple categories, the shelf merchandise data is analyzed to obtain multiple categories of collected and analyzed data. Based on the data collected and analyzed from multiple categories, the data collected from the shelf goods is labeled with data categories to obtain data labeling information for multiple categories; Multiple types of data annotation information are integrated and annotated for the inspection task area to obtain collected and fused data.

[0039] The working principle and technical effects of the above technical solution are as follows: This method focuses on the collection and fusion processing of multi-source data. Through the processes of collection, classification, analysis, labeling, and integration, comprehensive and reliable fused data is formed. Based on the inspection task information, the specific inspection task area is located, and multi-sensor collaborative collection is initiated. Visual cameras collect the density and appearance of merchandise displays, weight sensors collect the load-bearing data of shelf shelves, lidar collects the distance and spatial coordinates between merchandise and sensors, and infrared sensors assist in collecting blind spot data, thus obtaining multi-dimensional merchandise data from the shelf. Data is classified according to the collection type, breaking it down into multiple collection types such as weight data, density data, and distance data to avoid mixing different types of data. Based on the characteristics of each type of data, targeted analysis is performed to extract core features related to the presence and display status of merchandise, obtaining multiple types of collected and analyzed data. Based on the analysis results, the original collected data is labeled, clarifying the corresponding merchandise areas and characteristic states for each type of data, forming multiple types of data labeling information. All labeled information is integrated and consolidated according to the inspection task area, associating multi-dimensional data from the same area to eliminate data redundancy and conflicts, forming comprehensive and accurate collected and fused processing data.

[0040] This method addresses the technical challenges of incomplete data acquisition from a single sensor, the inability to coordinate mixed data from multiple sources, and low data utilization leading to insufficient accuracy in out-of-stock identification. It achieves standardized acquisition and deep fusion of multi-sensor data, integrating multi-dimensional data to form a complete data system. This improves the comprehensiveness of data acquisition and the reliability of fused data. The complementarity of multi-dimensional data can compensate for the detection limitations of a single sensor, and the annotation and integration processing makes the data more targeted. It reduces out-of-stock identification errors caused by incomplete or mixed data, while also improving data utilization.

[0041] In one embodiment of the present invention, the step of performing category-based collection analysis on shelf merchandise collection data based on multiple categories of collected data to obtain multiple categories of collected and analyzed data includes: Obtain the ratio of data for each collection category to the preset standard data for each category, and obtain the collection analysis coefficient for each category. The category collection and analysis coefficients are compared with the corresponding preset category collection and analysis thresholds to obtain the category analysis comparison results; Based on the comparison results of the category analysis, the category status is determined for each category collection analysis coefficient to obtain category collection status determination information; The category collection status determination information is the category collection analysis data.

[0042] The working principle and technical effect of the above technical solution are as follows: This method achieves accurate status determination of each type of collected data through quantitative analysis and threshold comparison. For each type of collected data, the corresponding preset standard data (such as standard load-bearing capacity, standard display density, standard distance range, etc.) is retrieved, and the ratio of the actual collected data to the standard data is calculated to obtain the collection analysis coefficient for each type. Qualitative data is transformed into quantifiable coefficient indicators for unified analysis. The collection analysis coefficient for each type is compared with the preset corresponding threshold to determine whether the coefficient is within a reasonable range. If it exceeds or falls below the threshold, it is determined to be abnormal, and the category analysis comparison result is obtained. Based on the comparison result, the status of the collection analysis coefficient for each type is determined, clarifying the commodity status corresponding to each type of data (such as abnormal weight coefficient corresponding to possible stockout, abnormal density coefficient corresponding to disordered display), forming category collection status determination information, which is the category collection analysis data.

[0043] This method addresses the technical problems of low analytical accuracy caused by the lack of quantitative analysis of collected data, strong subjectivity in state determination, and the inability to uniformly evaluate different types of data. It achieves quantitative analysis and standardized state determination of collected data, transforming fuzzy data into clear state information. It improves the accuracy and standardization of data classification analysis, and the unified coefficient calculation and threshold comparison standards ensure the consistency of analysis results in different scenarios, while providing clear state basis for fusion processing. It reduces analytical errors caused by subjective judgment, avoids interference from single data anomalies on overall judgment, and improves the reliability of data processing.

[0044] In one embodiment of the present invention, S3 includes: Based on the collected and fused data, determine the out-of-stock identification data; Based on the out-of-stock identification data, perform out-of-stock identification analysis to obtain out-of-stock identification analysis data; The out-of-stock identification and analysis data is compared with the preset out-of-stock identification threshold to obtain the out-of-stock identification comparison results. Based on the comparison results of out-of-stock identification, the out-of-stock identification analysis data is used to determine the out-of-stock status and obtain out-of-stock status determination information. Based on the out-of-stock status determination information, out-of-stock status is reported and replenishment is carried out, and the reported replenishment information is obtained.

[0045] The working principle and technical effects of the above technical solution are as follows: This method focuses on the entire process of out-of-stock identification, reporting, and replenishment. Through hierarchical analysis and threshold comparison, it achieves accurate determination and rapid response to out-of-stock status. Core data related to product out-of-stock is selected from the collected and processed data, including characteristic data such as abnormal weight, insufficient density, and abnormal distance signals. This data is integrated to form product out-of-stock identification data, focusing on core identification dimensions. Based on this data, out-of-stock identification analysis is performed, extracting out-of-stock related features and eliminating interference factors such as display adjustments and temporary relocations to obtain product out-of-stock identification analysis data. The analyzed data is compared with preset product out-of-stock identification thresholds to determine whether the out-of-stock judgment criteria are met, obtaining the product out-of-stock identification comparison result. Based on the comparison result, the out-of-stock status is accurately determined, clarifying key information such as whether there is an out-of-stock situation and the scope of the out-of-stock situation, obtaining out-of-stock status judgment information. Based on the judgment information, the reporting process is automatically initiated, pushing out-of-stock details to relevant management terminals and replenishment personnel, while simultaneously generating replenishment instructions, specifying replenishment categories, locations, and other requirements, forming a reported replenishment information, and achieving an automated closed loop for out-of-stock response.

[0046] This method solves the technical problems of numerous interfering factors in stockout identification, vague judgment criteria, and delayed response due to the disconnect between reporting and replenishment. It achieves integrated management and control of accurate stockout status determination, automatic reporting, and replenishment instruction generation, forming a complete response chain. It improves the accuracy of stockout identification, the timeliness of reporting, and the targeting of replenishment, effectively eliminates interfering factors, avoids false alarms and missed alarms, and shortens the stockout response cycle. It reduces the cost of manual intervention in stockout identification and the operational losses caused by delayed response, strengthens the closed-loop management capability of stockout management, and improves the stability of merchandise display in the store.

[0047] In one embodiment of the present invention, the step of performing product out-of-stock identification analysis based on product out-of-stock identification data to obtain product out-of-stock identification analysis data includes: Determine the number of categories marked as out of stock and the number of categories marked as not out of stock based on the out-of-stock identification data; Obtain the ratio of the number of out-of-stock labels to the number of non-out-of-stock labels for each category to obtain the out-of-stock identification coefficient. The product shortage identification coefficient is the product shortage identification analysis data.

[0048] The working principle and technical effect of the above technical solution are as follows: This method generates a quantitative out-of-stock identification coefficient through counting comparison and ratio calculation. Based on the out-of-stock identification data, the labeling results of each collected category are statistically analyzed to distinguish and determine the number of out-of-stock labels (the number of labels judged to be out-of-stock related in each dimension of data) and the number of non-out-of-stock labels (the number of labels judged to be normal in each dimension of data), ensuring the accuracy of the statistical data; the ratio of the number of out-of-stock labels to the number of non-out-of-stock labels is calculated to obtain the out-of-stock identification coefficient, transforming the discrete labeling results into a unified quantitative indicator. This coefficient can intuitively reflect the proportion of out-of-stock characteristics; the higher the coefficient, the greater the possibility of out-of-stock; finally, this out-of-stock identification coefficient is used as the data for out-of-stock identification analysis, improving the accuracy and objectivity of the judgment.

[0049] This method addresses the technical challenges of stockout identification and analysis, such as the lack of quantitative indicators and reliance on single annotation results leading to strong subjectivity and insufficient accuracy. It achieves quantitative transformation of stockout identification and analysis, integrating discrete annotation results into unified coefficients to provide objective basis for status determination. This improves the accuracy and objectivity of stockout identification and analysis, avoiding the biased influence of single annotation results, while unifying analysis standards to ensure consistency of analysis results across different scenarios. Furthermore, it reduces the risk of false alarms and missed alarms caused by subjective judgment, further enhancing the overall accuracy of stockout management.

[0050] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-sensor fusion-based inspection robot product shortage identification and reporting system, characterized in that, The system includes: The inspection task analysis module is used to obtain initial environment identification information, generate inspection path information based on the initial environment identification information, and generate inspection task information based on the inspection path information. The data acquisition, fusion, and analysis module is used to collect data on shelf inspection tasks through multiple sensors, obtain shelf merchandise data, and perform data fusion processing on the shelf merchandise data to obtain acquired and fused data. The out-of-stock identification and analysis module is used to perform out-of-stock identification and analysis and out-of-stock status analysis based on the collected and fused data, report out-of-stock status and replenish stock based on the obtained out-of-stock status analysis data, and obtain the reported replenishment information.

2. A method for implementing the multi-sensor fusion-based inspection robot out-of-stock identification and reporting system as described in claim 1, characterized in that, The method includes: S1. Obtain initial environment identification information, generate inspection path information based on the initial environment identification information, and generate inspection task information based on the inspection path information; S2. Collect data on the inspection task information of the shelves through multiple sensors, obtain the shelf merchandise collection data, and perform data fusion processing on the shelf merchandise collection data to obtain the collected fusion processing data; S3. Based on the collected and fused data, perform product out-of-stock identification and analysis and out-of-stock status analysis. Based on the obtained out-of-stock status analysis data, report the out-of-stock status and replenish the stock to obtain the reported replenishment information.

3. The method for identifying and reporting out-of-stock goods in an inspection robot based on multi-sensor fusion according to claim 2, characterized in that, S1 includes: Environmental data is collected from the product storage area to obtain initial environmental identification information; The initial environmental identification information is subjected to inspection feature processing to obtain environmental inspection feature processing data; Based on the environmental inspection feature processing data, inspection path information is generated; The inspection task information is determined based on the inspection route information and the product storage area.

4. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 3, characterized in that, The step of performing inspection feature processing on the initial environmental identification information to obtain environmental inspection feature processing data includes: Based on the initial environmental identification information, the identification information is classified to obtain identification classification information; The identification and classification information is used to locate the information, thereby obtaining identification, location, and classification information; The identification and positioning classification information is matched with the preset inspection target to obtain identification matching information; Based on the identification and matching information, the initial environmental identification information is labeled with inspection features to obtain environmental inspection feature processing data.

5. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 3, characterized in that, The step of generating inspection path information based on the environmental inspection feature processing data includes: The starting and ending points of the inspection are determined based on the data processing characteristics of the environmental inspection. Based on the inspection start point and inspection end point, the environmental inspection feature data are concatenated to obtain the initial inspection information; Obtain the historical out-of-stock rate of goods from the initial inspection information, and sort the historical out-of-stock rates from largest to smallest to obtain the out-of-stock rate sequence; Determine the out-of-stock rate location and its weight based on the out-of-stock rate sequence, and obtain out-of-stock location weight information; Obtain the current number of obstacles in the initial inspection information, sort the current number of obstacles in ascending order, and obtain the obstacle sequence; The location and weight of obstacles are determined based on the obstacle sequence, and obstacle location weight information is obtained; The final positioning weight information is determined by combining the out-of-stock positioning weight information with the obstacle positioning weight information. The final positioning weight information is sorted from largest to smallest to obtain the inspection weight sequence; The initial inspection information is adjusted according to the inspection weight sequence to obtain the inspection path information.

6. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 3, characterized in that, The step of determining the inspection task information based on the inspection route information and the product storage area includes: The inspection area nodes are determined based on the inspection route information and the product storage area. Pre-set inspection parameters for nodes in the inspection area to obtain preset inspection parameter data; Preset the area parameters for the product storage area and obtain the preset area parameter data; Based on the preset inspection parameter data, the preset area parameter data is allocated to obtain inspection task information.

7. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 2, characterized in that, S2 includes: Based on the inspection task information, the inspection task area is determined, and regional data is collected from the inspection task area through multiple sensors to obtain shelf merchandise collection data. The data collected from the shelves is divided into multiple categories to obtain data for multiple categories. Based on data collected from multiple categories, the shelf merchandise data is analyzed to obtain multiple categories of collected and analyzed data. Based on the data collected and analyzed from multiple categories, the data collected from the shelf goods is labeled with data categories to obtain data labeling information for multiple categories; Multiple types of data annotation information are integrated and annotated for the inspection task area to obtain collected and fused data.

8. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 7, characterized in that, The process involves performing category-based data collection and analysis on the shelf merchandise data based on multiple data collection categories to obtain multiple categories of collected and analyzed data, including: Obtain the ratio of data for each collection category to the preset standard data for each category, and obtain the collection analysis coefficient for each category. The category collection and analysis coefficients are compared with the corresponding preset category collection and analysis thresholds to obtain the category analysis comparison results; Based on the comparison results of the category analysis, the category status is determined for each category collection analysis coefficient to obtain category collection status determination information; The category collection status determination information is the category collection analysis data.

9. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 2, characterized in that, S3 includes: Based on the collected and fused data, determine the out-of-stock identification data; Based on the out-of-stock identification data, perform out-of-stock identification analysis to obtain out-of-stock identification analysis data; The out-of-stock identification and analysis data is compared with the preset out-of-stock identification threshold to obtain the out-of-stock identification comparison results. Based on the comparison results of out-of-stock identification, the out-of-stock identification analysis data is used to determine the out-of-stock status and obtain out-of-stock status determination information. Based on the out-of-stock status determination information, out-of-stock status is reported and replenishment is carried out, and the reported replenishment information is obtained.

10. The method for a multi-sensor fusion-based inspection robot product shortage identification and reporting system according to claim 9, characterized in that, The step of performing out-of-stock identification and analysis based on out-of-stock identification data to obtain out-of-stock identification and analysis data includes: Determine the number of categories marked as out of stock and the number of categories marked as not out of stock based on the out-of-stock identification data; Obtain the ratio of the number of out-of-stock labels to the number of non-out-of-stock labels for each category to obtain the out-of-stock identification coefficient. The product shortage identification coefficient is the product shortage identification analysis data.

Images