A method, system, device, and medium for detecting loss of warehoused items

Abstract

This invention discloses a method, system, device, and medium for detecting lost stored goods, belonging to the field of intelligent agent task execution technology. The method includes: performing semantic detection and background subtraction processing on acquired video stream data to obtain item detection results; constructing a first foreground dwell time map of the stored area, and continuously updating the first foreground dwell time map based on the item detection results to obtain a second foreground dwell time map; calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the stored area based on the second foreground dwell time map; calculating the item loss fusion probability when the feature values ​​deviate from the historical distribution; and outputting an item loss alarm signal if the item loss fusion probability exceeds a preset adaptive threshold. Therefore, by implementing this invention, high-precision and low-false-alarm detection of lost stored goods can be achieved, providing intelligent early warning support for stored goods management.

Description

Technical Field

[0001] This invention relates to the field of intelligent security in warehouses, and in particular to a method, system, equipment and medium for detecting lost goods in warehouses. Background Technology

[0002] In the fields of warehousing and logistics and intelligent security, achieving real-time, automatic, and high-precision detection of lost items (such as theft or misplacement) is of paramount importance for ensuring asset security and reducing management costs. With the development of computer vision technology, contactless detection methods based on video surveillance have become a research hotspot due to their flexible deployment and controllable costs.

[0003] Existing warehouse item loss detection technologies suffer from the following main drawbacks: 1. Fragmented detection information lacks deep fusion. Existing methods either rely solely on background subtraction, failing to distinguish between item removal and personnel passing by, or rely solely on target detection, making them susceptible to occlusion and similar appearances, leading to missed detections. They also fail to achieve quantitative guidance and dynamic enhancement of semantic information for pixel-level state evolution, making it difficult to maintain stable judgments under complex interference. 2. Rigid state representation and decision mechanisms rely on fixed thresholds to determine foreground duration, failing to adapt to the turnover cycles and environmental noise of different shelf areas, resulting in missed detections in high-turnover areas and delayed alarms in low-turnover areas. 3. Single output information lacks interpretability, only outputting binary decision results without providing quantitative fusion probabilities and sub-item decision-making basis, leading to low system reliability and difficulties in operation and maintenance. Summary of the Invention

[0004] This invention provides a method, system, device, and medium for detecting lost stored goods, which can achieve high-precision and low-false-alarm detection of lost stored goods, and provide intelligent early warning support for the management of stored goods.

[0005] This invention provides a method for detecting lost stored goods, comprising: Acquire video stream data of the storage area, perform semantic detection and background subtraction processing on the video stream data, and obtain item detection results; A first foreground dwell time map of the storage area is constructed, and the first foreground dwell time map is continuously updated based on the item detection results to obtain a second foreground dwell time map; Based on the second foreground dwell time map, the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area are calculated. When the feature values ​​deviate from the historical distribution, the item loss fusion probability is calculated. If the probability of the item being lost does not exceed a preset adaptive threshold, then monitoring continues to acquire new video stream data; otherwise, an alarm signal for the lost item is output so that staff can check the corresponding storage area based on the alarm signal.

[0006] This invention achieves dual-channel parallel perception of the existence status of items by acquiring video stream data of the storage area, performing semantic detection and background subtraction processing, and obtaining item detection results. By continuously updating the first foreground dwell time map based on the item detection results, a second foreground dwell time map is obtained, effectively enhancing the stability and anti-interference capability of the state representation. By calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area, when the feature values ​​deviate from the historical distribution, the item loss fusion probability is calculated. This transforms the traditional binary threshold judgment into a probabilistic deviation measure based on historical statistical distribution, enabling detection decisions to adapt to the historical state patterns of each area and significantly improving detection accuracy. Compared to existing technologies that rely on fixed threshold judgments and single information sources, making it difficult to adapt to the diversity of storage scenarios and environmental noise interference, this application achieves a significant improvement in detection accuracy and robustness through dual-channel detection based on semantics and background, combined with continuously updated foreground dwell time maps to calculate the item loss fusion probability, providing highly reliable intelligent early warning support for storage item management.

[0007] Furthermore, semantic detection and background subtraction are performed on the video stream data to obtain the object detection results, specifically as follows: Semantic detection is performed on consecutive frames of the video stream data. If the number of frames in the video stream data that detect items exceeds a preset ratio, the corresponding storage area is determined to be in an item-containing state, and the semantic detection result of the storage area is obtained. Based on the semantic detection results, background subtraction processing is performed on the video stream data to obtain the background subtraction results of the warehouse area by enhancing the distinction between static items and the background. The final item detection result is output by combining the semantic detection result and the background difference processing result.

[0008] By performing semantic detection on consecutive frames of the video stream data, the spatiotemporal consistency of consecutive frames effectively filters out false detections in single frames, improving the reliability of object presence determination. By performing background subtraction processing on the video stream data based on the semantic detection results, targeted enhancement of static objects is achieved under the guidance of semantic information, avoiding the mis-absorption of objects by the background model due to their stillness. By combining the semantic detection results and the background subtraction processing results to output the final object detection result, the semantic-level category cognition and the spatial contour perception at the background subtraction level are complementary and fused, realizing dual-channel parallel detection and decision-level fusion, significantly improving the detection completeness of static objects, partially occluded objects, and objects in complex backgrounds.

[0009] Furthermore, a first foreground dwell time map of the storage area is constructed, and the first foreground dwell time map is continuously updated based on the item detection results to obtain a second foreground dwell time map, specifically as follows: A first foreground dwell time map of the storage area is constructed using a preset foreground dwell time counter; the first foreground dwell time map is used to represent the existence status and cumulative duration of items in the storage area; Based on the item detection results, the foreground dwell time count value of each pixel in the first foreground dwell time map is cumulatively updated or decayed through the foreground dwell time counter to obtain the updated foreground dwell time count value of each pixel. A second foreground dwell time map is obtained based on the updated foreground dwell time count.

[0010] By constructing a first foreground dwell time map of the storage area, the item detection results are mapped to pixel-level time accumulation representations. By accumulating and decaying the foreground dwell time count value of each pixel in the first foreground dwell time map, transient noise and state fluctuations caused by detection jitter are effectively suppressed. By obtaining a second foreground dwell time map based on the updated foreground dwell time count value, the continuity information of the time dimension is integrated into the spatial state representation, realizing an effective conversion from instantaneous detection results to stable state descriptions, enabling item state determination to have temporal consistency and adaptive adjustment capabilities.

[0011] Furthermore, the foreground dwell time counter is used to cumulatively update or decay the foreground dwell time count value of each pixel in the first foreground dwell time map, as shown in the following formula: in, This is the foreground dwell time count value of pixel x at time t; For semantically weighted increments; It is the attenuation factor; The background difference result at time t, Indicates that pixel x is the foreground. This indicates that pixel x represents the background and has not changed. The semantic detection result at time t is shown. This indicates that the target item has been detected. This indicates that the target item was not detected.

[0012] Further, based on the second foreground dwell time map, the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area are calculated, specifically as follows: Calculate the feature values ​​of the geometric state features and semantic confidence features of the storage area, and obtain the statistical distribution of the geometric state features and the semantic confidence features based on the feature values; the statistical distribution includes the steady-state mean of the geometric state features, the steady-state standard deviation of the geometric state features, the steady-state mean of the semantic confidence features, and the steady-state standard deviation of the semantic confidence features.

[0013] By calculating the feature values ​​and statistical distribution of the geometric state features and semantic confidence features of the storage area, the dynamically changing state features are transformed into quantifiable statistical parameters. This establishes a unique historical state benchmark for each storage area, providing an accurate statistical benchmark for subsequent calculation of the probability of item loss based on probabilistic deviation measurement, and significantly improving the accuracy of detection and the adaptability to different scenarios.

[0014] Furthermore, before calculating the probability of item loss fusion, the following is also included: Based on the second foreground dwell time plot, the steady-state standard deviation of the geometric state feature, and the steady-state standard deviation of the geometric state feature, combined with the standard normal distribution function, the geometric state deviation probability is calculated. Based on the second foreground dwell time plot and the cumulative distribution function, the probability of a geometric state downward trend is calculated. Based on the second foreground dwell time plot, the steady-state mean of the semantic confidence feature, and the steady-state standard deviation of the semantic confidence feature, combined with the standard normal distribution function and the mapping function, the semantic confidence decay probability is calculated.

[0015] By calculating the geometric state deviation probability, the current foreground dwell time state is probabilistically compared with the historical stable statistical distribution, quantifying the degree of deviation of the current state from the normal baseline, and providing a continuous probabilistic measure of the geometric dimension for item loss judgment. By calculating the geometric state decline trend probability, the cumulative distribution function is used to capture the continuous decline trend of the foreground dwell time in time series, transforming the process characteristics of state decay into a probabilistic evaluation index, enhancing the trend perception capability of item removal events. By calculating the semantic confidence decay probability, the decay of semantic detection confidence is mapped to a unified probability space, achieving alignment and fusion of the semantic and geometric dimensions at the probabilistic level, providing comprehensive and accurate quantitative input for the subsequent calculation of the item loss fusion probability, and significantly improving the detection sensitivity and false alarm suppression capability of item removal events.

[0016] Furthermore, when the feature value deviates from the historical distribution, the probability of item loss fusion is calculated, specifically as follows: When the feature value deviates from the historical distribution, the item loss fusion probability is calculated based on the geometric state deviation probability, the geometric state downward trend probability, and the semantic confidence decay probability, as shown in the following formula: in, Let be the probability of item loss and fusion in storage area R at time t; This represents the probability of deviation from the geometric state. The probability of a geometrically decreasing trend; This represents the semantic confidence decay probability. and The weights can be customized and dynamically adjusted according to the scenario. .

[0017] Another embodiment of the present invention provides a warehouse item loss detection system, including: an item detection module, a time map construction and update module, and a probability calculation and early warning module; The item detection module is used to acquire video stream data of the storage area, perform semantic detection and background subtraction processing on the video stream data, and obtain item detection results. The time map construction and update module is used to construct a first foreground residence time map of the storage area, and continuously update the first foreground residence time map according to the item detection results to obtain a second foreground residence time map. The probability calculation and early warning module is used to calculate the feature values ​​and statistical distribution of the geometric state features and semantic confidence features of the storage area based on the second foreground dwell time map. When the feature value deviates from the historical distribution, the module calculates the item loss fusion probability. If the item loss fusion probability does not exceed the preset adaptive threshold, the module continues to monitor and acquire new video stream data. Otherwise, it outputs an item loss alarm signal so that staff can check the corresponding storage area based on the alarm signal.

[0018] This invention achieves dual-channel parallel perception of the existence status of items by acquiring video stream data of the storage area, performing semantic detection and background subtraction processing, and obtaining item detection results. By continuously updating the first foreground dwell time map based on the item detection results, a second foreground dwell time map is obtained, effectively enhancing the stability and anti-interference capability of the state representation. By calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area, when the feature values ​​deviate from the historical distribution, the item loss fusion probability is calculated. This transforms the traditional binary threshold judgment into a probabilistic deviation measure based on historical statistical distribution, enabling detection decisions to adapt to the historical state patterns of each area and significantly improving detection accuracy. Compared to existing technologies that rely on fixed threshold judgments and single information sources, making it difficult to adapt to the diversity of storage scenarios and environmental noise interference, this application achieves a significant improvement in detection accuracy and robustness through dual-channel detection based on semantics and background, combined with continuously updated foreground dwell time maps to calculate the item loss fusion probability, providing highly reliable intelligent early warning support for storage item management.

[0019] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the warehouse item loss detection method of the present invention.

[0020] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the warehouse item loss detection method of the present invention. Attached Figure Description

[0021] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating one embodiment of the warehouse item loss detection method provided by the present invention; Figure 2 This is a schematic diagram of another embodiment of the warehouse item loss detection system provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0025] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0026] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0027] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0028] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0029] See Figure 1 To address the problem of detecting lost stored goods in existing technologies, an embodiment of the present invention provides a method for detecting lost stored goods, comprising steps S1 to S4, the specific steps of which are as follows: S1. Obtain video stream data of the storage area, perform semantic detection and background subtraction processing on the video stream data, and obtain item detection results; In this process, semantic detection is performed on multiple consecutive frames of the video stream data. If the number of frames in the video stream data that detect an item exceeds a preset ratio, the corresponding storage area is determined to be in an item state, and background subtraction processing is performed to obtain the final item detection result. S2. Construct a first foreground dwell time map of the storage area, and continuously update the first foreground dwell time map according to the item detection results to obtain a second foreground dwell time map; Specifically, a first foreground dwell time map of the storage area is constructed by using a preset foreground dwell time counter. Based on the item detection results, the foreground dwell time count value of each pixel in the first foreground dwell time map is cumulatively updated or decayed to obtain the updated foreground dwell time count value of each pixel. Based on the updated foreground dwell time count value, a second foreground dwell time map is obtained. S3. Based on the second foreground dwell time map, calculate the feature values ​​and statistical distribution of the geometric state features and semantic confidence features of the storage area. When the feature values ​​deviate from the historical distribution, calculate the item loss fusion probability. Specifically, based on the eigenvalues ​​and the statistical distribution, the geometric state deviation probability, the geometric state downward trend probability, and the semantic confidence decay probability are calculated respectively, and the item loss fusion probability is calculated based on the geometric state deviation probability, the geometric state downward trend probability, and the semantic confidence decay probability. S4. If the probability of the item being lost does not exceed the preset adaptive threshold, then continue monitoring to acquire new video stream data; otherwise, output an alarm signal for the lost item so that staff can check the corresponding storage area based on the alarm signal.

[0030] The alarm signal includes the overall probability of item loss and the individual probability of item loss, which serve as a basis for decision-making and explanatory information, allowing staff to understand the confidence level of the alarm and the source of its composition.

[0031] This invention achieves dual-channel parallel perception of the existence status of items by acquiring video stream data of the storage area, performing semantic detection and background subtraction processing, and obtaining item detection results. By continuously updating the first foreground dwell time map based on the item detection results, a second foreground dwell time map is obtained, effectively enhancing the stability and anti-interference capability of the state representation. By calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area, when the feature values ​​deviate from the historical distribution, the item loss fusion probability is calculated. This transforms the traditional binary threshold judgment into a probabilistic deviation measure based on historical statistical distribution, enabling detection decisions to adapt to the historical state patterns of each area and significantly improving detection accuracy. Compared to existing technologies that rely on fixed threshold judgments and single information sources, making it difficult to adapt to the diversity of storage scenarios and environmental noise interference, this application achieves a significant improvement in detection accuracy and robustness through dual-channel detection based on semantics and background, combined with continuously updated foreground dwell time maps to calculate the item loss fusion probability, providing highly reliable intelligent early warning support for storage item management.

[0032] In one embodiment, semantic detection and background subtraction processing are performed on the video stream data to obtain object detection results, including steps S201 to S203, each step of which is as follows: S201. Perform semantic detection on consecutive frames of the video stream data. If the number of frames in the video stream data that detect items exceeds a preset ratio, determine that the corresponding storage area is in a state of having items, and obtain the semantic detection result of the storage area. Specifically, the YOLO semantic detection channel is used to detect N consecutive frames (e.g., N=10). If a predefined shelf area R in the storage area is stably detected with items in more than a preset proportion (e.g., 90%) of the frames (confidence > threshold Th), then the initial state of the area is determined to be that there are items, and the semantic detection result of the storage area is obtained. S202. Based on the semantic detection results, perform background subtraction processing on the video stream data to obtain the background subtraction results of the warehouse area by enhancing the distinction between static items and the background. In this process, the initial background model of region R is set to a locked or delayed learning mode, and the background difference results of the storage area are obtained by enhancing the distinction between static items and the background. S203. Combining the semantic detection results and the background difference processing results, output the final item detection results.

[0033] This invention improves the reliability of object presence determination by performing semantic detection on consecutive frames of the video stream data and effectively filtering out false detections in single frames by utilizing the spatiotemporal consistency of consecutive frames. Furthermore, by performing background subtraction processing on the video stream data based on the semantic detection results, targeted enhancement of static objects is achieved under the guidance of semantic information, preventing the objects from being mistakenly absorbed by the background model due to their stillness. Finally, by combining the semantic detection results and the background subtraction processing results to output the final object detection result, the semantic-level category cognition and the background subtraction-level spatial contour perception are complementary and fused, achieving dual-channel parallel detection and decision-level fusion, significantly improving the detection completeness of static objects, partially occluded objects, and objects in complex backgrounds.

[0034] In one embodiment, a first foreground dwell time map of the storage area is constructed, and the first foreground dwell time map is continuously updated based on the item detection results to obtain a second foreground dwell time map, including steps S301 to S303, each step of which is as follows: S301. Construct a first foreground dwell time map of the storage area using a preset foreground dwell time counter; the first foreground dwell time map is used to represent the existence status and cumulative duration of items in the storage area; S302. Based on the item detection result, the foreground dwell time count value of each pixel in the first foreground dwell time map is cumulatively updated or decayed through the foreground dwell time counter to obtain the updated foreground dwell time count value of each pixel, as shown in the following formula: in, This is the foreground dwell time count value of pixel x at time t; For semantically weighted increments; It is the attenuation factor; The background difference result at time t, Indicates that pixel x is the foreground. This indicates that pixel x represents the background and has not changed. The semantic detection result at time t is shown. This indicates that the target item has been detected. This indicates that the target item was not detected. in, Through accumulation and decay mechanisms, the cumulative time that pixel x is continuously judged as "non-background" (i.e., a foreground moving area or a target judged by YOLO) is quantified. The larger the value, the longer it is in a state that may reflect the existence of an object. It is the output background difference result binary mask, which is a general pixel-level motion / change indicator; Used to slowly reduce the dwell time count in the absence of any positive evidence, reflecting a regression in the state; t is the time index of the video frame, representing the sequence number of the currently processed image frame; Semantic weighted increment The calculation formula is as follows: in, This represents the average confidence score of the detected items in region R when performing semantic detection through the YOLO semantic detection channel in the current frame. This is the gain coefficient; When YOLO detects an item with high confidence , making Accelerated accumulation strengthens the signal indicating the presence of an object, resulting in more stable detection results and better resistance to environmental noise; when an object is briefly obscured, causing the background differential channel to fail... At that time, as long as the YOLO channel can still detect , It will continue to accumulate rather than decay, effectively preventing misjudgments caused by temporary occlusion; the magnitude of the enhancement... Detection confidence compared to YOLO Proportional to this, enabling soft and quantitative guidance of semantic information for estimating the underlying state; S303. Obtain a second foreground dwell time map based on the updated foreground dwell time count value.

[0035] This invention constructs a first foreground dwell time map of the storage area, mapping the item detection results to a pixel-level time accumulation representation. By accumulating and decaying the foreground dwell time count value of each pixel in the first foreground dwell time map, transient noise and state fluctuations caused by detection jitter are effectively suppressed. By obtaining a second foreground dwell time map based on the updated foreground dwell time count value, the continuity information of the time dimension is integrated into the spatial state representation, realizing an effective conversion from instantaneous detection results to stable state descriptions, enabling item state determination to have temporal consistency and adaptive adjustment capabilities.

[0036] In one embodiment, based on the second foreground dwell time map, the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area are calculated, including step S401, as follows: S401. Calculate the feature values ​​of the geometric state features and semantic confidence features of the storage area, and obtain the statistical distribution of the geometric state features and the semantic confidence features based on the feature values; the statistical distribution includes the stable state mean of the geometric state features, the stable state standard deviation of the geometric state features, the stable state mean of the semantic confidence features, and the stable state standard deviation of the semantic confidence features.

[0037] The formula for calculating the regional average foreground dwell time is as follows: in, Geometric state characteristics (average foreground dwell time in region R); The foreground dwell time count value of pixel x at time t; The formula for calculating the peak confidence score of YOLO detection in a region is as follows: in, Semantic confidence features for region R; Let be the confidence score of the i-th detection box within region R, used to represent the confidence score of the determination that the item exists; When region R is in a stable, materialized stage, the statistical distributions of the geometric state features and the semantic confidence features are obtained through online rolling learning, including the statistical distribution of the geometric state features. and the statistical distribution of semantic confidence features ;in, The mean of the steady-state characteristics of the geometric state; The steady-state standard deviation represents the geometric state characteristics. The mean of the steady-state semantic confidence features; The steady-state standard deviation of semantic confidence features.

[0038] This invention converts dynamically changing state features into quantifiable statistical parameters by calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area. It establishes a unique historical state benchmark for each storage area, providing an accurate statistical benchmark for subsequent calculation of the item loss fusion probability based on probabilistic deviation measurement, and significantly improving the accuracy of detection and scene adaptability.

[0039] In one embodiment, when the feature value deviates from the historical distribution, calculating the item loss fusion probability further includes steps S501 to S504, each step being as follows: S501. Based on the second foreground dwell time map, the steady-state standard deviation of the geometric state feature, and the steady-state standard deviation of the geometric state feature, combined with the standard normal distribution function, calculate the geometric state deviation probability, as follows: in, Let be the probability of geometric deviation at time t; Let be the standard normal distribution function, representing the distribution under the current steady state, indicating that the observed difference compared to the current steady state is greater than that of the previous steady state. The possibility of a smaller value; The average foreground dwell time in the region; The mean of the steady-state characteristics of the geometric state; The steady-state standard deviation represents the geometric state characteristics. Among them, the geometric state deviation probability It is used to measure the significance of the deviation of the current geometric state from the historical stable state, and is calculated using hypothesis testing; S502. Based on the second foreground dwell time plot and the cumulative distribution function, calculate the probability of a geometric downward trend, as shown in the following formula: in, Let be the probability of a geometric state decline at time t; The cumulative distribution function of the distribution; The degree of freedom is usually equal to L⁻² (number of samples minus number of regression parameters). Among them, by using within the sliding window Perform linear regression on the sequence to obtain the slope K, and calculate the slope by testing whether K is significantly negative. The closer the probability value is to 1, the stronger the significance of the slope K being negative, that is, the more obvious the trend of continuous deterioration of the geometric state. S503. Based on the second foreground dwell time map, the steady-state mean of the semantic confidence feature, and the steady-state standard deviation of the semantic confidence feature, combined with the standard normal distribution function and the mapping function, calculate the semantic confidence decay probability, as follows: in, Let be the semantic confidence decay probability at time t; Let be the standard normal distribution function, representing the distribution under the current steady state, indicating that the observed difference compared to the current steady state is greater than that of the previous steady state. The possibility of a smaller value; Semantic confidence features for region R; The mean of the steady-state semantic confidence features; The steady-state standard deviation of semantic confidence features; The number of consecutive frames that region R has not been detected by YOLO. As a regulating factor; For the mapping function, Map to the interval (0,1); When YOLO fails to detect the item, It will non-linearly approach 1, effectively modeling the enhanced effect of detector failure on the judgment of possible item loss; S504. When the feature value deviates from the historical distribution, the item loss fusion probability is calculated based on the geometric state deviation probability, the geometric state downward trend probability, and the semantic confidence decay probability, as follows: in, Let be the probability of item loss and fusion in storage area R at time t; This represents the probability of deviation from the geometric state. The probability of a geometrically decreasing trend; This represents the semantic confidence decay probability. and The weights can be customized and dynamically adjusted according to the scenario. .

[0040] Among them, when At that time, the item was determined to be missing. It can be set according to the value of the item.

[0041] This invention, through calculating the geometric state deviation probability, probabilistically compares the current foreground dwell time state with historical stable statistical distributions, quantifying the degree of deviation of the current state from the normal baseline, and providing a continuous probabilistic measure of the geometric dimension for item loss judgment. By calculating the geometric state decline trend probability, the cumulative distribution function captures the continuous downward trend of the foreground dwell time over time, transforming the process characteristics of state decay into a probabilistic evaluation index, enhancing the trend perception capability of item removal events. Furthermore, by calculating the semantic confidence decay probability, the decay of semantic detection confidence is mapped to a unified probability space, achieving alignment and fusion of the semantic and geometric dimensions at the probabilistic level. This provides comprehensive and accurate quantitative input for calculating the item loss fusion probability, significantly improving the detection sensitivity and false alarm suppression capability for item removal events.

[0042] like Figure 2 As shown in the warehouse item loss detection, based on the above method implementation examples, a corresponding system implementation example is provided; This invention provides a warehouse item loss detection system, including: an item detection module 601, a time map construction and update module 602, and a probability calculation and early warning module 603; The item detection module 601 is used to acquire video stream data of the storage area, perform semantic detection and background subtraction processing on the video stream data, and obtain item detection results. The time map construction and update module 602 is used to construct a first foreground residence time map of the storage area, and continuously update the first foreground residence time map according to the item detection results to obtain a second foreground residence time map. The probability calculation and early warning module 603 is used to calculate the feature values ​​and statistical distribution of the geometric state features and semantic confidence features of the storage area based on the second foreground dwell time map. When the feature value deviates from the historical distribution, the probability of item loss is calculated. If the probability of item loss does not exceed the preset adaptive threshold, the monitoring continues to acquire new video stream data. Otherwise, an alarm signal for item loss is output so that staff can check the corresponding storage area based on the alarm signal.

[0043] This invention achieves dual-channel parallel perception of the existence status of items by acquiring video stream data of the storage area, performing semantic detection and background subtraction processing, and obtaining item detection results. By continuously updating the first foreground dwell time map based on the item detection results, a second foreground dwell time map is obtained, effectively enhancing the stability and anti-interference capability of the state representation. By calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area, when the feature values ​​deviate from the historical distribution, the item loss fusion probability is calculated. This transforms the traditional binary threshold judgment into a probabilistic deviation measure based on historical statistical distribution, enabling detection decisions to adapt to the historical state patterns of each area and significantly improving detection accuracy. Compared to existing technologies that rely on fixed threshold judgments and single information sources, making it difficult to adapt to the diversity of storage scenarios and environmental noise interference, this application achieves a significant improvement in detection accuracy and robustness through dual-channel detection based on semantics and background, combined with continuously updated foreground dwell time maps to calculate the item loss fusion probability, providing highly reliable intelligent early warning support for storage item management.

[0044] It is understood that the above system embodiments correspond to the method embodiments of the present invention, and can implement the warehouse item loss detection method provided by any of the above method embodiments of the present invention.

[0045] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0046] For ease of description and brevity, the embodiments of the system items of the present invention include all the implementation methods in the embodiments of the above-described storage item loss detection method, and will not be repeated here.

[0047] Based on the above embodiments of the method for detecting lost stored goods, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method for detecting lost stored goods according to any embodiment of the present invention.

[0048] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0049] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0050] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0051] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the warehouse item loss detection method described in any of the above-described method embodiments of the present invention.

[0052] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0053] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting lost stored goods, characterized in that, include: Acquire video stream data of the storage area, perform semantic detection and background subtraction processing on the video stream data, and obtain item detection results; A first foreground dwell time map of the storage area is constructed, and the first foreground dwell time map is continuously updated based on the item detection results to obtain a second foreground dwell time map; Based on the second foreground dwell time map, the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area are calculated. When the feature values ​​deviate from the historical distribution, the item loss fusion probability is calculated. If the probability of the item being lost does not exceed a preset adaptive threshold, then monitoring continues to acquire new video stream data; otherwise, an alarm signal for the lost item is output so that staff can check the corresponding storage area based on the alarm signal.

2. The warehouse item loss detection method of claim 1, wherein, The step of performing semantic detection and background subtraction on the video stream data to obtain the object detection result is as follows: Semantic detection is performed on consecutive frames of the video stream data. If the number of frames in the video stream data that detect items exceeds a preset ratio, the corresponding storage area is determined to be in an item-containing state, and the semantic detection result of the storage area is obtained. Based on the semantic detection results, background subtraction processing is performed on the video stream data to obtain the background subtraction results of the warehouse area by enhancing the distinction between static items and the background. The final item detection result is output by combining the semantic detection result and the background difference processing result.

3. The method of warehouse item loss detection of claim 1, wherein, The process of constructing a first foreground dwell time map of the storage area and continuously updating the first foreground dwell time map based on the item detection results to obtain a second foreground dwell time map specifically involves: A first foreground dwell time map of the storage area is constructed using a preset foreground dwell time counter; the first foreground dwell time map is used to represent the existence status and cumulative duration of items in the storage area; Based on the item detection results, the foreground dwell time count value of each pixel in the first foreground dwell time map is cumulatively updated or decayed through the foreground dwell time counter to obtain the updated foreground dwell time count value of each pixel. A second foreground dwell time map is obtained based on the updated foreground dwell time count.

4. The method of warehouse item loss detection of claim 3, wherein, The foreground dwell time counter is used to cumulatively update or decay the foreground dwell time count value of each pixel in the first foreground dwell time map, as follows: in, This is the foreground dwell time count value of pixel x at time t; For semantically weighted increments; It is the attenuation factor; The background difference result at time t, Indicates that pixel x is the foreground. This indicates that pixel x represents the background and has not changed. Here is the semantic detection result at time t. This indicates that the target item has been detected. This indicates that the target item was not detected.

5. The method for detecting lost stored goods as described in claim 1, characterized in that, The step of calculating the feature values ​​and statistical distributions of the geometric state features and semantic confidence features of the storage area based on the second foreground residence time map specifically involves: Calculate the feature values ​​of the geometric state features and semantic confidence features of the storage area, and obtain the statistical distribution of the geometric state features and the semantic confidence features based on the feature values; the statistical distribution includes the steady-state mean of the geometric state features, the steady-state standard deviation of the geometric state features, the steady-state mean of the semantic confidence features, and the steady-state standard deviation of the semantic confidence features.

6. The method for detecting lost stored goods as described in claim 5, characterized in that, Before calculating the probability of item loss fusion, the process also includes: Based on the second foreground dwell time plot, the steady-state standard deviation of the geometric state feature, and the steady-state standard deviation of the geometric state feature, combined with the standard normal distribution function, the geometric state deviation probability is calculated. Based on the second foreground dwell time plot and the cumulative distribution function, the probability of a geometric state downward trend is calculated. Based on the second foreground dwell time plot, the steady-state mean of the semantic confidence feature, and the steady-state standard deviation of the semantic confidence feature, combined with the standard normal distribution function and the mapping function, the semantic confidence decay probability is calculated.

7. The method of warehouse item loss detection of claim 6, wherein, When the feature value deviates from the historical distribution, the probability of item loss fusion is calculated, specifically as follows: When the feature value deviates from the historical distribution, the item loss fusion probability is calculated based on the geometric state deviation probability, the geometric state downward trend probability, and the semantic confidence decay probability, as shown in the following formula: in, Let be the probability of item loss and fusion in storage area R at time t; This represents the probability of deviation from the geometric state. The probability of a geometrically decreasing trend; This represents the semantic confidence decay probability. and The weights can be customized and dynamically adjusted according to the scenario. .

8. A system for detecting loss of stored items, characterized by include: Item detection module, time map construction and update module, and probability calculation and early warning module; The item detection module is used to acquire video stream data of the storage area, perform semantic detection and background subtraction processing on the video stream data, and obtain item detection results. The time map construction and update module is used to construct a first foreground residence time map of the storage area, and continuously update the first foreground residence time map according to the item detection results to obtain a second foreground residence time map. The probability calculation and early warning module is used to calculate the feature values ​​and statistical distribution of the geometric state features and semantic confidence features of the storage area based on the second foreground dwell time map, and to calculate the item loss fusion probability when the feature values ​​deviate from the historical distribution. If the probability of the item being lost does not exceed a preset adaptive threshold, then monitoring continues to acquire new video stream data; otherwise, an alarm signal for the lost item is output so that staff can check the corresponding storage area based on the alarm signal.

9. A terminal device, comprising: The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the warehouse item loss detection method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the warehouse item loss detection method as described in any one of claims 1-7.

Images