Article calibration method and system based on video identification

Abstract

The invention relates to the field of video recognition, in particular to an article calibration method and system based on video recognition. The method comprises the following steps: acquiring a multi-angle real-time monitoring video of the vending cabinet; performing inter-frame delay optimization and frame-by-frame linear brightness optimization on the multi-angle real-time monitoring video of the vending cabinet so as to construct a brightness optimization monitoring video; carrying out deep article visual inspection on the brightness optimization monitoring video, carrying out three-dimensional point cloud modeling, and constructing a real-time vending cabinet three-dimensional virtual model; performing dynamic behavior identification on the brightness optimization monitoring video, and performing article state change evolution to generate article real-time state change information; and on the basis of the real-time article state change information, real-time article state updating is performed on the real-time vending cabinet three-dimensional virtual model, article position deviation detection is performed, and articles with position deviation in the cabinet are marked. According to the invention, real-time and accurate user behavior identification is realized, so that intelligent calibration of goods in the vending cabinet is carried out.

Description

Technical Field

[0001] The present invention relates to the field of video recognition, and in particular to an object calibration method and system based on video recognition. Background Art

[0002] With the development of smart retail and automation technology, vending machines have gradually become important sales and storage equipment in modern commerce. Especially in unmanned retail and smart stores, vending machines, as the core component of automated sales, not only need to display and manage items, but also need to have efficient and accurate item calibration and inventory management functions. Traditional vending machine item calibration methods mostly rely on manual inspection and manual adjustment, which is not only inefficient, but also prone to human errors, and cannot meet the real-time, accuracy and intelligence requirements of modern retail environments.

[0003] With the continuous development of video surveillance technology, computer vision and deep learning technology, the method of vending machine item calibration based on video behavior analysis and recognition has gradually become a hot topic of research. By using a multi-angle video surveillance system combined with advanced image processing algorithms, the items in the vending machine can be monitored and identified in real time, thereby realizing automatic item calibration. This method can capture the dynamic changes of items in the vending machine based on real-time video streams, including the placement, placement, movement and other behaviors of items, and dynamically calculate the specific position and posture of each item through behavior analysis and recognition technology.

[0004] However, the existing traditional vending machine item calibration methods rely on simple sensors or manual inspection operations, which cannot effectively respond to changes in the placement of items and often ignore the relationship between the relative positions of items and the cabinet space. At the same time, the traditional method has a long calibration cycle and cannot automatically update and adjust the item status in real time, which will reduce the accuracy of item display and affect the customer's shopping experience. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method and system for object calibration based on video recognition to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides an object calibration method based on video recognition, comprising the following steps: Step S1: obtaining a multi-angle real-time monitoring video of a vending machine; optimizing the inter-frame delay and frame-by-frame linear brightness of the multi-angle real-time monitoring video of the vending machine, thereby constructing a brightness-optimized monitoring video; Step S2: Performing deep object visual inspection on the brightness optimization monitoring video, and performing stereo point cloud modeling to construct a real-time three-dimensional virtual model of the vending cabinet; Step S3: Perform dynamic behavior recognition on the brightness optimization monitoring video and perform object state change evolution to generate real-time object state change information; Step S4: based on the real-time status change information of the items, the real-time three-dimensional virtual model of the vending cabinet is updated with the item status in real time, and the item position deviation detection is performed to mark the items with position deviation in the cabinet; Step S5: Calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; Step S6: Make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the intelligent vending machine.

[0007] The present invention uses multiple cameras to monitor the items in the vending cabinet from different angles in real time, ensuring that all changes of the items are fully captured, including the items being taken away, placed or moved. Multi-angle combined monitoring can reduce the information loss caused by blind spots or occlusions in the viewing angle, and improve the monitoring accuracy of the system. The delay between video frames is optimized to reduce the fluctuation of the delay and ensure the real-time synchronization of each frame of video data. This optimization helps to avoid the problem of video frame asynchronization caused by delay, thereby reducing the error in item recognition and improving the accurate detection of item status. Through the frame-by-frame brightness optimization algorithm, the brightness of each frame is linearly adjusted to improve the image quality problem caused by uneven lighting or environmental factors (such as backlight, low light, etc.) in the video. This processing will enhance the clarity of the video, making the edges and details of the items more prominent, which is helpful for subsequent item recognition and status analysis. In the video after brightness optimization, the visual detection of the items is performed through deep learning technology (such as convolutional neural network, CNN). The method can accurately identify the position, shape and category of each item, not only can it process conventional items, but also can effectively identify items with complex shapes and irregular packaging, and improve the comprehensiveness of item recognition. Using multi-angle video data and combining deep learning technology to generate a three-dimensional point cloud model of the object, a real-time three-dimensional virtual model of the vending cabinet is established. This three-dimensional modeling process can accurately capture the position, direction and spatial layout of the object in three-dimensional space. The three-dimensional point cloud not only improves the recognition accuracy of the object, but also makes the spatial information of the object more intuitive, providing accurate data support for subsequent calibration and analysis. The real-time three-dimensional virtual model provides a reliable basis for the subsequent update of the item status and position calibration. Through the virtual model, the operator can more intuitively view the internal status of the vending cabinet and conduct remote monitoring and operation. Through the dynamic behavior recognition technology in the video data, the system can capture the interaction between customers and items, such as picking up, placing, and viewing. This behavior recognition technology can not only understand customer needs in real time, but also help analyze the change trend of items, such as items being purchased or replenished. Through the analysis of dynamic behavior, the system can track the evolution of the item status and record the change information such as the quantity and location of the item. Combined with real-time behavior data, the system can accurately judge the status of the item (for example: sold, removed, position adjusted, etc.), providing a basis for the status update and position calibration of the virtual model. Generating real-time status change information of items will help with subsequent analysis and decision-making. By further mining these data, it is possible to identify which items are frequently sold, which items need to be replenished, and which items need to be adjusted, making the management of the vending cabinet more intelligent and automated. Based on the information on item status changes, the item status in the virtual model will be updated in real time to ensure the consistency between the virtual model and the actual vending cabinet. Real-time updating of the virtual model can provide real-time and accurate item data support and provide timely information feedback for item management.The system can accurately detect the deviation between the virtual model and the actual object, especially when the spatial position of the object in the cabinet is abnormally offset, and can mark the position deviation object in time. This detection can identify the position calibration problem in time and provide data support for subsequent compensation processing. Through position deviation detection, abnormal conditions in the vending cabinet can be discovered in time, and targeted calibration can be carried out to ensure that the object is always in the correct position and reduce the recognition error caused by position error. For the marked deviation objects, the system will calculate the posture deviation and analyze the deviation of the object relative to the preset standard position. This calculation helps to understand the specific degree of the position deviation of the object, thereby providing accurate data for the formulation of compensation strategy. According to the calculated posture deviation, the system will perform dynamic compensation processing. The posture of the object is corrected by the algorithm, and the three-dimensional spatial coordinates of the object are automatically adjusted. This compensation processing is adaptive and can be flexibly adjusted according to different environments and different characteristics of objects, thereby improving the adaptability of the overall system. According to the deviation compensation processing results, the system generates an adaptive posture compensation strategy to ensure the correct placement and accurate identification of objects in the cabinet. This strategy can self-optimize with changes in the environment and the interactive behavior of objects. Based on the inventory information of items in the real-time virtual model, the system can make intelligent replenishment decisions to ensure that the items in the vending machine always meet customer needs. This decision is not only based on real-time inventory data, but also combined with user purchasing behavior to predict the consumption trend of items. Combined with the adaptive posture compensation strategy, the system can dynamically coordinate the replenishment and calibration tasks of items. By coordinating the work of replenishment and item calibration, it is ensured that the vending machine can operate efficiently at every moment to avoid customer dissatisfaction caused by position errors or out-of-stock. The entire process is controlled by intelligent dynamic collaboration to form an intelligent vending machine collaborative calibration model to improve the management efficiency and customer experience of the vending machine. The system can automatically detect and optimize the placement of items, intelligently replenish, and self-adjust and calibrate according to real-time interactive data.

[0008] In this specification, a video recognition-based object calibration system is provided, which is used to perform the video recognition-based object calibration method as described above, including: The video optimization module is used to obtain multi-angle real-time monitoring videos of the vending cabinets; the inter-frame delay optimization and frame-by-frame linear brightness optimization of the multi-angle real-time monitoring videos of the vending cabinets are performed to construct brightness-optimized monitoring videos; The object visual inspection module is used to perform in-depth object visual inspection on the brightness optimization monitoring video, and to perform stereo point cloud modeling to build a real-time three-dimensional virtual model of the vending cabinet; Dynamic behavior analysis module, used to identify dynamic behaviors of brightness optimization monitoring videos and perform object state change evolution to generate real-time object state change information; The real-time item update module is used to update the real-time item status of the three-dimensional virtual model of the real-time vending cabinet based on the real-time item status change information, and to detect the position deviation of the item and mark the position deviation items in the cabinet; Deviation compensation module, used to calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; The collaborative control module is used to make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and to perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the smart vending machine.

[0009] The present invention can improve the visibility of the video by optimizing the brightness of the video frame, especially in low light or complex lighting environments. The optimized brightness enhances the recognition effect of the object and improves the system's ability to extract the features of the object. Inter-frame delay optimization ensures real-time synchronization of video data, which is crucial for object position detection and behavior analysis. Delay optimization can reduce errors in object status monitoring and ensure that changes in objects are reflected in real time. Brightness optimization makes the shape, details and other information of the object more obvious under different lighting conditions, thereby enhancing the recognition effect of the subsequent object visual detection module. Deep visual detection can efficiently and accurately identify various objects and capture changes in objects from multiple angles through advanced deep learning technologies (such as convolutional neural networks). Even complex, obscured or irregular objects can be effectively identified. Through stereo point cloud modeling, the position, direction and layout of objects in three-dimensional space can be accurately restored. The three-dimensional model not only improves the spatial perception ability of the object, but also provides high-precision data support for subsequent object position calibration and compensation processing. As the position and state of the object change, the three-dimensional virtual model will continue to update. This real-time dynamic virtual model enables administrators to intuitively monitor and adjust the placement status of objects. By identifying the interaction between customers and items, the system can capture customer needs and purchasing behavior, so as to understand which items are popular and which items are slow to sell. It can provide real-time customer behavior data, which is helpful for analyzing customer preferences. The evolution of item status changes helps to track the quantity and position changes of items in real time. For example, if an item is taken away or rearranged, the system can record this change to ensure that the state of the virtual model is always consistent with the actual state. The item status change information generated by this module will directly affect the subsequent decision-making of item replenishment, calibration and compensation strategies, and provide a dynamic operation basis for the system. The item status update module ensures the synchronization between the virtual model and the actual item, reflects the status of the item in real time, and avoids management errors caused by delays or deviations. By detecting the difference between the physical position of the item and the position in the virtual model in real time, the system can quickly identify the position deviation, mark the items that need to be calibrated, and provide data support for subsequent calibration and compensation. Real-time status updates can help the system detect abnormal conditions of items in a timely manner, make adjustments in a timely manner, and ensure that the items in the vending machine are placed in accordance with the requirements. Calculating and correcting the posture deviation of the item can effectively solve the deviation caused by improper placement, external force and other factors. The compensation strategy dynamically adjusts the position and angle of the object through an algorithm to return the object to the ideal state. The module is highly adaptive and can flexibly adjust the compensation strategy according to the type, size, weight of the object and the environmental changes of the vending cabinet. Regardless of how the environment changes, the system can ensure the accurate calibration of the position and posture of the object. Through automated compensation processing, the need for manual intervention is reduced, management efficiency is improved, and errors caused by improper human operation are reduced.Based on real-time data (such as item sales speed, inventory status, etc.), the system can intelligently generate replenishment decisions to ensure that the vending cabinet always has sufficient items to avoid out-of-stock or overstock situations. Through collaborative control, replenishment decisions and posture compensation are effectively linked to ensure that replenishment does not cause deviations in item positions. In this way, the replenishment and position calibration processes can be efficiently coordinated, reducing the need for manual intervention. The collaborative control module enables the vending cabinet to automatically complete replenishment and calibration tasks without manual intervention, optimizing management efficiency and improving customer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic flow chart of the steps of a method for calibrating an object based on video recognition according to the present invention; Figure 2 Detailed implementation flow chart of step S1; Figure 3 Detailed implementation flow chart of step S2; Figure 4 Detailed implementation flow chart of step S3. DETAILED DESCRIPTION

[0011] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0012] The present application example provides a method and system for object calibration based on video recognition. The execution subject of the method and system for object calibration based on video recognition includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of the present application, and the data processing platform includes but is not limited to: at least one of an audio image management system, an information management system, and a cloud data management system.

[0013] See also Figures 1 to 4 The present invention provides an object calibration method based on video recognition, and the object calibration method based on video recognition comprises the following steps: Step S1: obtaining a multi-angle real-time monitoring video of a vending machine; optimizing the inter-frame delay and frame-by-frame linear brightness of the multi-angle real-time monitoring video of the vending machine, thereby constructing a brightness-optimized monitoring video; Step S2: Performing deep object visual inspection on the brightness optimization monitoring video, and performing stereo point cloud modeling to construct a real-time three-dimensional virtual model of the vending cabinet; Step S3: Perform dynamic behavior recognition on the brightness optimization monitoring video and perform object state change evolution to generate real-time object state change information; Step S4: based on the real-time status change information of the items, the real-time three-dimensional virtual model of the vending cabinet is updated with the item status in real time, and the item position deviation detection is performed to mark the items with position deviation in the cabinet; Step S5: Calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; Step S6: Make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the intelligent vending machine.

[0014] The present invention uses multiple cameras to monitor the items in the vending cabinet from different angles in real time, ensuring that all changes of the items are fully captured, including the items being taken away, placed or moved. Multi-angle combined monitoring can reduce the information loss caused by blind spots or occlusions in the viewing angle, and improve the monitoring accuracy of the system. The delay between video frames is optimized to reduce the fluctuation of the delay and ensure the real-time synchronization of each frame of video data. This optimization helps to avoid the problem of video frame asynchronization caused by delay, thereby reducing the error in item recognition and improving the accurate detection of item status. Through the frame-by-frame brightness optimization algorithm, the brightness of each frame is linearly adjusted to improve the image quality problem caused by uneven lighting or environmental factors (such as backlight, low light, etc.) in the video. This processing will enhance the clarity of the video, making the edges and details of the items more prominent, which is helpful for subsequent item recognition and status analysis. In the video after brightness optimization, the visual detection of the items is performed through deep learning technology (such as convolutional neural network, CNN). The method can accurately identify the position, shape and category of each item, not only can it process conventional items, but also can effectively identify items with complex shapes and irregular packaging, and improve the comprehensiveness of item recognition. Using multi-angle video data and combining deep learning technology to generate a three-dimensional point cloud model of the object, a real-time three-dimensional virtual model of the vending cabinet is established. This three-dimensional modeling process can accurately capture the position, direction and spatial layout of the object in three-dimensional space. The three-dimensional point cloud not only improves the recognition accuracy of the object, but also makes the spatial information of the object more intuitive, providing accurate data support for subsequent calibration and analysis. The real-time three-dimensional virtual model provides a reliable basis for the subsequent update of the item status and position calibration. Through the virtual model, the operator can more intuitively view the internal status of the vending cabinet and conduct remote monitoring and operation. Through the dynamic behavior recognition technology in the video data, the system can capture the interaction between customers and items, such as picking up, placing, and viewing. This behavior recognition technology can not only understand customer needs in real time, but also help analyze the change trend of items, such as items being purchased or replenished. Through the analysis of dynamic behavior, the system can track the evolution of the item status and record the change information such as the quantity and location of the item. Combined with real-time behavior data, the system can accurately judge the status of the item (for example: sold, removed, position adjusted, etc.), providing a basis for the status update and position calibration of the virtual model. Generating real-time status change information of items will help with subsequent analysis and decision-making. By further mining these data, it is possible to identify which items are frequently sold, which items need to be replenished, and which items need to be adjusted, making the management of the vending cabinet more intelligent and automated. Based on the information on item status changes, the item status in the virtual model will be updated in real time to ensure the consistency between the virtual model and the actual vending cabinet. Real-time updating of the virtual model can provide real-time and accurate item data support and provide timely information feedback for item management.The system can accurately detect the deviation between the virtual model and the actual object, especially when the spatial position of the object in the cabinet is abnormally offset, and can mark the position deviation object in time. This detection can identify the position calibration problem in time and provide data support for subsequent compensation processing. Through position deviation detection, abnormal conditions in the vending cabinet can be discovered in time, and targeted calibration can be carried out to ensure that the object is always in the correct position and reduce the recognition error caused by position error. For the marked deviation objects, the system will calculate the posture deviation and analyze the deviation of the object relative to the preset standard position. This calculation helps to understand the specific degree of the position deviation of the object, thereby providing accurate data for the formulation of compensation strategy. According to the calculated posture deviation, the system will perform dynamic compensation processing. The posture of the object is corrected by the algorithm, and the three-dimensional spatial coordinates of the object are automatically adjusted. This compensation processing is adaptive and can be flexibly adjusted according to different environments and different characteristics of objects, thereby improving the adaptability of the overall system. According to the deviation compensation processing results, the system generates an adaptive posture compensation strategy to ensure the correct placement and accurate identification of objects in the cabinet. This strategy can self-optimize with changes in the environment and the interactive behavior of objects. Based on the inventory information of items in the real-time virtual model, the system can make intelligent replenishment decisions to ensure that the items in the vending machine always meet customer needs. This decision is not only based on real-time inventory data, but also combined with user purchasing behavior to predict the consumption trend of items. Combined with the adaptive posture compensation strategy, the system can dynamically coordinate the replenishment and calibration tasks of items. By coordinating the work of replenishment and item calibration, it is ensured that the vending machine can operate efficiently at every moment to avoid customer dissatisfaction caused by position errors or out-of-stock. The entire process is controlled by intelligent dynamic collaboration to form an intelligent vending machine collaborative calibration model to improve the management efficiency and customer experience of the vending machine. The system can automatically detect and optimize the placement of items, intelligently replenish, and self-adjust and calibrate according to real-time interactive data.

[0015] In the embodiment of the present invention, refer to Figure 1 , is a schematic flow chart of the steps of a method for calibrating an object based on video recognition of the present invention. In this example, the steps of the method for calibrating an object based on video recognition include: Step S1: obtaining a multi-angle real-time monitoring video of a vending machine; optimizing the inter-frame delay and frame-by-frame linear brightness of the multi-angle real-time monitoring video of the vending machine, thereby constructing a brightness-optimized monitoring video; In this embodiment, multiple high-definition cameras are installed around the vending cabinet to ensure that all items in the cabinet can be covered from different angles. These cameras should have good low-light performance and wide dynamic range to adapt to different ambient light conditions. Set the camera resolution (such as 1080p or 4K) and frame rate (usually 30 frames per second) to ensure that the captured video quality meets the analysis requirements. Start the camera system and transmit the real-time video stream to the central processing unit through the network (such as Wi-Fi or wired network). Use a suitable protocol (such as RTSP or RTMP) to ensure the stability and low latency of the video stream. During the video capture process, record the timestamp of each frame for subsequent analysis and processing. Ensure the continuity of the video stream, and the acquisition time can be set to the daily working hours to capture user interaction behaviors. Store the real-time monitoring video stream on the server or cloud for subsequent processing and analysis. Compressed formats (such as H.264 or H.265) can be used to save storage space while maintaining video quality. During the video storage process, check the video quality regularly to ensure that there is no video loss or damage caused by equipment failure or network problems. In the video processing stage, inter-frame delay optimization is first performed to ensure the continuity and smoothness of the video stream. Advanced algorithms (such as motion compensation technology) are used to analyze the motion information between adjacent frames and reduce inter-frame delay. Optimization parameters, such as delay thresholds (such as no more than 100 milliseconds), are set to ensure that the video can reflect the status and changes of objects in real time when users interact. At the same time, the processing delay is monitored to ensure that it is within an acceptable range. The optimized monitoring video is subjected to frame-by-frame linear brightness optimization to improve the visibility and details of the image. First, each frame of the image is converted into a grayscale image, the grayscale histogram of each frame is calculated, and its brightness distribution is analyzed. According to the grayscale histogram, linear transformation parameters (such as brightness gain and offset) are set, and each pixel is linearly stretched. The goal is to expand the brightness range of the image to the full range (0-255) and improve the contrast. The optimization parameters and results of each frame are recorded to ensure the traceability of the optimization process. For the brightness optimization of each object, ensure that it is clearly visible under different lighting conditions. After the brightness optimization is completed, the optimization effect of each frame is verified, and the processing effect is confirmed by visual inspection and quantitative indicators (such as the contrast improvement). Make sure your optimized video accurately reflects the color and details of your items.

[0016] Step S2: Performing deep object visual inspection on the brightness optimization monitoring video, and performing stereo point cloud modeling to construct a real-time three-dimensional virtual model of the vending cabinet; In this embodiment, a suitable deep learning model is selected for visual inspection of objects, and commonly used ones include YOLO (YouOnly Look Once) or Faster R-CNN. These models can efficiently identify and locate objects in images and are suitable for real-time monitoring applications. The model is trained using a data set containing rich object images to ensure that the model can recognize different types of objects. During the training process, data enhancement techniques (such as rotation, scaling, and cropping) are used to improve the generalization ability of the model. Each frame is extracted from the brightness optimization monitoring video and input into the trained deep learning model for object detection. A suitable input size (such as 416x416) is set to ensure that the model can process the image and output the detection results. The model output includes the category, bounding box coordinates, and confidence score of each detected object. A confidence threshold (such as 0.5) is set to filter out low-confidence detection results to ensure the accuracy of recognition. The object detection results in each frame are marked and the bounding box is drawn on the image for subsequent analysis and visualization. The location information and category of each object are recorded, and the detection data is stored in the database. Ensure that each record contains the item ID, bounding box coordinates, category, and confidence for subsequent 3D modeling and analysis. Use stereo vision technology for point cloud modeling, and combine multi-angle monitoring videos to generate 3D spatial information of the item. Select a suitable stereo matching algorithm, such as block matching or image segmentation technology, to improve the accuracy of the point cloud. Determine the internal and external parameters of the camera (such as focal length, principal point position), and obtain these parameters through the calibration process to ensure the accuracy of point cloud generation. Calculate the depth information of each item by processing the extracted multiple frames of images. Use the stereo matching algorithm to extract feature points from images of different perspectives, calculate the disparity of each point, and generate a depth map. The depth map maps the depth information of each pixel, combines it with the 2D position of the item, and constructs a 3D coordinate system for subsequent point cloud generation. Combine the generated depth information with the 2D detection results of the item to construct a 3D point cloud. The coordinates of each point are calculated based on its depth information and bounding box position to form a complete 3D model. Filter and optimize the generated point cloud to remove noise and unnecessary points to ensure the quality and density of the point cloud. Statistical filtering or voxel grid method can be used to simplify the point cloud and improve computational efficiency. Generate a 3D virtual model of the real-time vending cabinet based on the optimized point cloud data. You can use 3D modeling tools (such as Meshlab or CloudCompare) to convert the point cloud into a mesh model. Set the detailed parameters of the model, such as the resolution and accuracy of the mesh, to ensure that the generated 3D model can truly reflect the layout and status of the items in the cabinet.

[0017] Step S3: Perform dynamic behavior recognition on the brightness optimization monitoring video and perform object state change evolution to generate real-time object state change information; In this embodiment, a suitable dynamic behavior recognition model is selected, and an architecture based on a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM) is usually used to effectively capture the spatiotemporal features in the video. The advantage of this model is that it can process the relationship between consecutive frames and identify various interactive behaviors between users and items. The model is trained using a video data set containing a variety of user interactive behaviors to ensure that the model can recognize common dynamic behaviors, such as taking away items, putting items back, and viewing items. During the training process, data enhancement techniques (such as temporal upsampling and spatial transformation) are used to improve the robustness of the model. Each frame is extracted from the brightness optimization monitoring video, and a suitable input size (such as 64x64 or 128x128) is set to meet the requirements of the model. Generally, 30 frames are selected per second to ensure the continuity of the video stream and the capture of dynamic changes. Preprocessing steps such as normalization and data enhancement are performed on each frame to improve the effect of the model. The processed frame is input into the trained dynamic behavior recognition model for inference, and the category, confidence score and timestamp of each behavior are output. For each frame, record the recognized user behavior, including the behavior category, confidence, and corresponding timestamp to ensure the integrity and accuracy of the data. This information will be stored in the database for subsequent analysis and query. Set an appropriate threshold (e.g., a confidence of 0.6) to filter low-confidence recognition results to ensure that real and effective behaviors are captured and stored. Design an item state change evolution model, combine the dynamic behavior recognition results, analyze the interaction between users and items, and generate real-time state change information of items. The model should be able to dynamically adjust the state of items (such as available, out of stock, hot selling, etc.) according to the type and frequency of user behavior. Set state change rules, such as when a user takes an item, the inventory quantity decreases; when a user views an item, record the viewing frequency of the item for subsequent analysis. According to the results of dynamic behavior recognition, analyze the state changes of items frame by frame. For example, when the behavior of a user taking an item is recognized, immediately update the state of the item to "taken away" and record the timestamp of the event. Record the state changes of each item, including the previous state, new state, reason for change, and timestamp, for subsequent analysis and decision-making. The generated real-time status change information of items is stored in the database, ensuring that each record contains information such as item ID, current status, status change time and change reason for subsequent query and analysis. The data structure is designed to facilitate rapid retrieval and analysis, ensuring that the dynamic changes of item status can be reflected in the system in real time.

[0018] Step S4: based on the real-time status change information of the items, the real-time three-dimensional virtual model of the vending cabinet is updated with the item status in real time, and the item position deviation detection is performed to mark the items with position deviation in the cabinet; In this embodiment, Step S5: Calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; In this embodiment, the status change information of the item is obtained from the real-time monitoring system, including the item ID, current status (such as available, sold out, to be replenished) and real-time inventory. This information should be extracted through an API or directly from the database to ensure the real-time and accuracy of the data. Set the frequency of data update, such as updating once every 5 seconds, so as to timely reflect the status changes of the items in the cabinet. Ensure that the timestamp of each update is recorded during the update process for subsequent tracking. Apply the acquired item status information to the virtual model by using 3D modeling software (such as Unity or Unreal Engine). The performance of each item in the virtual model should be consistent with the status in the actual cabinet, such as displaying out-of-stock items as translucent or adding an "out of stock" label. Update the location information of each item to ensure that the location of the item in the 3D model is consistent with the location of the item in the actual cabinet. The coordinate conversion method can be used to combine the coordinates of the item in the virtual model with the real-time status information. Dynamically display the status changes of the item in the 3D virtual model, for example, when an item is purchased, immediately remove the item from the model or mark it as "sold out". This instant feedback mechanism can enhance the user's immersion and interactive experience. Ensure that all status updates are reflected in the virtual model in real time, and perform regular visual checks to ensure the accuracy and consistency of the update process. Set the reference position of each item in the 3D virtual model, which should match the fixed position of the items in the actual cabinet. The initial coordinates and rotation angle of each item can be recorded according to the arrangement of the items and the structure of the cabinet. Set the allowed deviation threshold, such as ±5 cm spatial deviation and ±5 degrees angular deviation, to evaluate the actual position of the items. Use depth cameras or sensors (such as laser rangefinders) to monitor the position of items in the cabinet in real time. Use these devices to obtain the actual coordinates of the items and compare them with the preset reference position. Obtain the actual position data of the items regularly and set the detection frequency (such as once every 10 seconds) to ensure real-time monitoring of the deviation of the items. Compare the actual measured position with the reference position and calculate the deviation value of each item. Use the Euclidean distance formula to calculate the spatial deviation and record any items that exceed the set threshold. Mark the items with detected position deviation. Color coding or warning signs (such as flashing borders) can be used in the 3D virtual model to highlight these items for managers to quickly identify. Store the real-time updated item status and position deviation information in the database, ensuring that each record contains information such as item ID, current status, actual location, reference location, and deviation value. This data will be used for subsequent analysis and decision-making. Design a reasonable data structure to facilitate fast retrieval and update. For example, you can use the item ID as the primary key to quickly find and update related information.

[0019] Step S6: Make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the intelligent vending machine.

[0020] In this embodiment, the status information of the items is extracted from the three-dimensional virtual model of the real-time vending cabinet, including the current inventory, sales frequency, and status change records. These data should be updated in a timely manner to reflect the actual situation of the items in the cabinet. Combined with the historical sales data of the items, the consumption rate of each item is analyzed. Set a suitable time window (such as sales data for the past 30 days) to more accurately predict future replenishment needs. Calculate the replenishment demand for each item based on the real-time inventory and the expected sales rate. A simple calculation formula can be used, such as: replenishment quantity = safety stock - current inventory + expected sales. Set the safety stock level to prevent unexpected demand surges, which usually depends on the sales fluctuations of the items and the replenishment time. For example, for fast-selling items, the safety stock can be set to 1.5 times the sales rate. Generate a replenishment decision list based on the calculated replenishment demand. This list should include the item ID that needs to be replenished, the recommended replenishment quantity, and the replenishment priority (such as sorting according to sales frequency or slow sales). Record the timestamp and decision basis of each replenishment decision for subsequent analysis and optimization of the replenishment strategy. Use depth cameras or sensors to monitor the actual posture of the item, including position, direction, and tilt. Through real-time data analysis, identify whether the item deviates from the preset posture standard. Set reasonable posture deviation thresholds, such as position deviation not exceeding 5 cm and tilt angle not exceeding 5 degrees. These thresholds can be adjusted according to the characteristics of the item and the layout of the cabinet. According to the monitored posture deviation, apply an adaptive compensation algorithm (such as a PID control algorithm) to adjust the posture. The algorithm will dynamically adjust the compensation strategy according to the deviation value to ensure that the item returns to the preset posture. Record the specific parameters and effect evaluation of each compensation to ensure the adaptability and accuracy of the algorithm. For example, when the item tilts beyond the set angle, the system can automatically adjust the angle of the item to restore it to its normal state. Design a dynamic collaborative control mechanism to combine replenishment decisions with posture compensation strategies. Ensure that during the replenishment process, the posture adjustment of the item can be coordinated with the replenishment operation. For example, when replenishing, automatically check the posture of the item to be replenished to ensure its stability during the replenishment process. Set a feedback mechanism for the control algorithm to ensure that each replenishment and posture adjustment can affect the system status update in real time. This feedback mechanism can improve the overall response speed and accuracy of the system. Build a collaborative calibration model for smart vending machines based on replenishment decisions and adaptive posture compensation strategies. The model should be able to integrate replenishment data, posture data, and user behavior data in real time to form a dynamic management system. Design the input and output structure of the model to ensure that all types of data can flow effectively and achieve real-time data update and processing. Regularly evaluate the performance of the collaborative calibration model to analyze the accuracy of replenishment decisions and the effectiveness of posture compensation. Identify the shortcomings of the model by comparing actual sales data with model prediction results. Based on the evaluation results, continuously optimize model parameters, such as replenishment thresholds, adjustment factors for posture compensation algorithms, etc., to improve the adaptability and accuracy of the model.

[0021] In this embodiment, refer to Figure 2 , is a flowchart of detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Step S11: acquiring multi-angle real-time monitoring videos of the vending cabinet based on a multi-camera array; Step S12: monitoring the inter-frame delay of the multi-angle real-time monitoring video of the vending machine, and extracting adjacent image frames with delay mutations; Step S13: performing image frame bilinear interpolation calculation on adjacent image frames of delayed mutation, thereby obtaining delayed interpolation optimized frames; Step S14: interpolating and reconstructing the multi-angle real-time monitoring video of the vending machine based on the delayed interpolation optimization frame, and performing smooth transition processing on the interpolation frame, so as to construct an interpolated and reconstructed monitoring video; Step S15: performing frame-by-frame linear brightness optimization on the interpolated and reconstructed monitoring video, thereby constructing a brightness-optimized monitoring video.

[0022] In this embodiment, multiple cameras are arranged around the vending machine to achieve 360-degree all-round monitoring. Each camera should have a high-definition resolution (such as 1080p or higher) to ensure image quality. In addition, the camera should have an appropriate viewing angle to ensure that every corner of the vending machine can be captured. Ensure synchronization between cameras, and use the network time protocol (NTP) to accurately synchronize the clock to ensure that the image frame time captured by each camera is consistent. This step is crucial for subsequent inter-frame delay monitoring and interpolation processing. Transmit the real-time video stream of each camera to the central processing unit, and use a streaming protocol (such as RTSP or WebRTC) for real-time transmission. Ensure that the system can process and store high-traffic data from multiple cameras. Encode the video stream (such as H.264 or H.265) to compress the data and reduce storage requirements while maintaining image quality. The appropriate bit rate can be set to balance image quality and storage requirements. Integrate real-time video streams from different cameras to build a multi-angle monitoring system. Video management software (such as VMS) can be used to manage video streams from different sources and achieve unified monitoring. Implement real-time monitoring to ensure that the system can display and record the video content captured by all cameras in real time for subsequent analysis and processing. Use an inter-frame delay monitoring algorithm to monitor the delay in each camera video stream. Delay mutations can be detected by calculating the time difference between adjacent frames, and a threshold (such as 100 milliseconds) is set to identify frames with significant delay changes. By analyzing the timestamps of each camera, after detecting a delay mutation, the index and time information of the relevant frame are recorded for subsequent processing. Once a delay mutation is detected, the adjacent image frames are immediately extracted for analysis. These frames will be used for subsequent interpolation calculations to optimize the quality of the video stream. Record the properties of the delay mutation frame, including frame number, timestamp, and delay value, for subsequent interpolation and reconstruction processing. Select bilinear interpolation as the interpolation method between image frames. Bilinear interpolation can effectively smooth images and reduce visual discomfort caused by delay mutations. When interpolating, set an interpolation factor (such as 0.5) to generate a new optimized frame between two adjacent frames. For the extracted adjacent image frames with delay mutations, apply the bilinear interpolation algorithm to calculate the new frame. The algorithm generates new image data by weighted averaging adjacent pixels to ensure a smooth visual transition of the image. Record the generation time and parameters of each optimized frame for subsequent performance evaluation and optimization adjustments. Verify the quality of the interpolation results and evaluate the interpolation effect through visual inspection and image quality indicators (such as peak signal-to-noise ratio PSNR and structural similarity SSIM). If necessary, adjust the interpolation factor or algorithm parameters to improve the interpolation quality. Design an interpolation and reconstruction process to combine the delayed interpolation optimized frames with the original frames to construct a continuous sequence of video frames. Insert optimized frames between each delayed mutation frame to achieve a smooth transition of the video stream. Determine the frame rate of interpolation and reconstruction (such as 30 frames / second) to ensure the smoothness and coherence of the video.During the interpolation and reconstruction process, smooth transition processing is implemented. The weighted average method can be used to blend adjacent frames to reduce the sudden changes and jumps between frames. Set the blending ratio (such as 0.7:0.3) to ensure a smooth transition between the new frame and the original frame. Record the properties of each reconstructed frame, including the frame number and generation time, for subsequent quality assessment. Save the interpolated and reconstructed video as a new file, using an efficient encoding format (such as H.264) to ensure a balance between video quality and storage efficiency. Ensure that the reconstructed video can be played in real time in the monitoring system and has good playback performance. Select the linear brightness optimization algorithm to process the interpolated and reconstructed monitoring video frame by frame. Linear brightness adjustment can effectively improve the visual effect of the video and ensure that the brightness distribution of the image is uniform. Set the brightness adjustment parameters (such as the gain factor) to make appropriate adjustments based on the overall brightness level of the video. Traverse each frame of the interpolated and reconstructed monitoring video and apply the linear brightness optimization algorithm. According to the set gain factor, adjust the brightness value of each pixel to ensure that the image is not overexposed or underexposed. Record the brightness adjustment parameters of each frame for subsequent analysis and optimization. Verify the video quality after brightness optimization and confirm the adjustment effect through visual inspection and brightness histogram analysis. If problems are found, adjust the gain factor or optimize the algorithm parameters. Save the surveillance video after brightness optimization and ensure it is available in the monitoring system for subsequent monitoring and analysis.

[0023] In this embodiment, the specific steps of step S12 are: Decompose the multi-angle real-time monitoring video of the vending machine frame by frame and extract all image frames; Extracting a timestamp of the image frame; Performing traversal calculation of the timestamp differences of adjacent frames on the timestamps to obtain the delay time between each pair of frames; The average delay time is calculated based on the delay time between each pair of frames, so as to obtain the average delay time of the video frame; Perform delay interval serialization fitting on each pair of inter-frame delay times, thereby generating an inter-frame delay time series; According to the average delay time of the video frames, the inter-frame delay mutation detection is performed on the inter-frame delay time sequence, and the adjacent image frames with delay mutation are extracted.

[0024] In this embodiment, a data stream is read from a multi-angle monitoring video. This can be achieved by using a video processing library (such as OpenCV) to decode the video frame by frame. Set an appropriate frame rate (such as 30 frames per second) to ensure that all key frames can be captured. Store each frame and save the extracted image frames in an image file format (such as JPEG or PNG) to a specified directory. In order to facilitate subsequent processing, each frame can be named "frame_001.jpg", "frame_002.jpg", etc. to maintain the order. During the extraction process, the number of extracted image frames is recorded for subsequent analysis. A statistical variable can be set to update the total number of extracted frames in real time to ensure that each frame in the video is processed. Record a timestamp for each extracted image frame, which should contain the time difference between the start time of the video and the current frame to ensure accuracy. The system time function can be used to obtain the current time and calculate the time stamp of each frame. Traverse the extracted timestamps and calculate the time difference between adjacent frames. Set a loop to traverse each element in the timestamp array and calculate the difference with the next element. The unit of time difference is usually milliseconds. The calculation results are stored in a new array, with each element corresponding to the delay time between adjacent frames. This process ensures that the delay of each pair of frames can be captured. The calculated delay time is stored in a structured data table, including "frame pair number" and "delay time", to provide basic data for subsequent average delay and mutation detection. The average delay time between all adjacent frames is calculated, with the formula: average delay = sum of all delay times / total number of frames. An accumulator variable can be used to store the sum of delay times and record the total number of frame pairs. Ensure that outliers (such as extreme delay values) are excluded during the calculation process to improve the accuracy of the average value. The calculated average delay time is recorded in the database, including the video ID, calculation time and average delay value, for subsequent analysis and comparison. The delay time data between adjacent frames is serialized to form a delay time series. This can be achieved through simple array operations to ensure the order and integrity of the data. Statistical analysis is performed on the delay series to identify data distribution characteristics, such as mean, standard deviation, etc., to provide reference for subsequent mutation detection. Select a suitable fitting method (such as linear regression or spline interpolation) to fit the delay time series. This process can analyze the trend of delay time changing with the number of frames through algorithms and identify potential mutation points. In the fitting process, reasonable fitting parameters are set to ensure the accuracy and robustness of the model. According to the calculated average delay time, mutation detection is performed on the delay time series. A threshold is set (such as 1.5 times the average delay) and delay times outside this range are identified as mutations. For each pair of delay times, determine whether it exceeds the set mutation threshold and record the corresponding frame pair number to identify the mutation area. Once a mutation is detected, the corresponding adjacent image frames are immediately extracted and stored for subsequent analysis.Ensure that the timestamp and frame number of the mutation frame are recorded for subsequent quality assessment and processing. Store the extracted mutation frame information in the database to ensure the integrity and traceability of the data, providing a basis for subsequent monitoring and analysis.

[0025] In this embodiment, the specific steps of step S15 are: The gray value histogram distribution of the interpolated and reconstructed monitoring video is calculated frame by frame to obtain the gray value histogram of each frame; Perform peak distribution analysis on the gray value histogram of each frame to obtain the peak distribution of the histogram of each frame; Perform pixel value range statistics according to the gray value histogram of each frame to generate the pixel value range of each frame; Adaptive pixel value stretching processing is performed according to the peak distribution of the histogram of each frame and the pixel value range of each frame to obtain the pixel stretching value of each frame; The interpolated and reconstructed monitoring video is linearly optimized frame by frame according to the pixel stretching value of each frame, so as to construct a brightness optimized monitoring video.

[0026] In this embodiment, each frame of the image is extracted from the interpolated reconstructed monitoring video. Ensure that the extracted frame has sufficient quality and clarity for subsequent grayscale analysis. The extraction interval (such as extracting 30 frames per second) can be set to ensure the representativeness of the video content. During the extraction process, the timestamp of each frame is recorded for subsequent analysis and result comparison. The grayscale value of each frame is calculated. The RGB value of each pixel is converted to a grayscale value, usually using a weighted average method (such as Y = 0.299R + 0.587G + 0.114B) to ensure the accuracy of the grayscale value. Generate a grayscale image for each frame for subsequent analysis. Ensure that the size of the grayscale image is consistent with the original image for frame-by-frame comparison and feature extraction. Calculate the grayscale value histogram for each frame of the grayscale image. The histogram represents the frequency of occurrence of each grayscale level (usually 0 to 255). 256 grayscale levels can be used for statistics to generate an array containing 256 values. Select a suitable peak recognition algorithm, such as a simple threshold method or a derivative-based peak detection method. Set threshold parameters to determine which frequency values ​​in the histogram are considered peaks. A relative peak threshold (such as greater than 60% of the histogram mean) can be set to ensure that only significant peaks are detected. Traverse the grayscale value histogram of each frame, identify all grayscale levels that meet the peak conditions, and record the corresponding frequency values. For each peak, record its position (grayscale level) and height (frequency). Analyze the peak distribution of each frame and generate a list of peaks, including the grayscale level of each peak and the corresponding frequency value. Store the peak information of each frame in a database for subsequent analysis and comparison. Ensure that each record includes the frame number, peak grayscale level, and corresponding frequency for visualization and statistical analysis. Analyze the grayscale value histogram of each frame and calculate its pixel value range. The pixel value range can be determined by finding the minimum and maximum grayscale levels of non-zero frequencies in the histogram. Record the pixel value range of each frame, including the minimum and maximum grayscale values, to generate a range data set. Store the pixel value range information of each frame in a database, ensuring that each record includes the frame number, minimum value, and maximum value for subsequent processing and analysis. Additional calculated data, such as the total number of pixels and the effective pixel ratio, can be added when the data is stored for subsequent quality assessment. Visualize the pixel value range of each frame and generate a range map to intuitively understand the brightness distribution of the video content. The pixel value range of each frame can be displayed using a histogram or a box plot. Analyze the pixel value range data to identify frames with uneven brightness in the video and provide a basis for subsequent pixel value stretching processing. Design an adaptive pixel value stretching algorithm that adjusts according to the peak distribution and pixel value range of each frame. Set stretching parameters, such as the stretching factor and the target grayscale range (usually 0 to 255). The stretching algorithm should consider the location and number of peaks to determine the strength and direction of the stretch.Linear interpolation can be used to increase the pixel values ​​in low-brightness areas while suppressing overly bright areas. Perform adaptive stretching on each frame. Adjust the values ​​of each grayscale level based on the calculated minimum, maximum, and peak information to ensure enhanced image contrast and detail. Record the stretch values ​​for each frame, including the new grayscale value of each pixel, for subsequent analysis and optimization. Verify the effectiveness of the stretching process by visual inspection and image quality indicators such as contrast and brightness histogram changes. If problems are found, adjust the stretching parameters to optimize the processing effect. Select the linear brightness optimization algorithm to process the video frame by frame based on the pixel stretch value of each frame. Set the gain factor and offset to adjust the brightness of each frame based on the stretch value. The gain factor should be adjusted based on the overall brightness level of each frame to ensure that the visual effect of the video is not distorted. Iterate through each frame of the brightness optimization monitoring video and apply the linear brightness optimization algorithm. Adjust the brightness value of each pixel based on the stored pixel stretch value to ensure uniform brightness across the video. Record the brightness adjustment parameters for each frame for subsequent analysis and optimization. Verify the video quality after brightness optimization and confirm the adjustment effect through visual inspection and brightness histogram analysis. If problems are found, adjust the gain factor or optimize the algorithm parameters. Save the surveillance video after brightness optimization to ensure that it is available in the monitoring system and has good playback performance and visualization effect.

[0027] In this embodiment, refer to Figure 3 , is a flowchart of detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Step S21: Perform deep object visual inspection on the brightness optimization monitoring video and mark each item in the cabinet; Step S22: calibrating the camera position of the multi-camera array to generate camera position and posture information; Step S23: positioning each item in the cabinet in three dimensions based on the camera position and posture information to obtain the item's three-dimensional spatial coordinates; Step S24: performing spatial placement analysis in the cabinet based on the three-dimensional spatial coordinates of the items to obtain spatial distribution data in the cabinet; Step S25: performing a three-dimensional structural analysis of the vending cabinet based on the brightness optimization monitoring video to generate three-dimensional structural data of the vending cabinet; Step S26: Perform stereo point cloud modeling based on the three-dimensional structure data of the vending cabinet and the spatial distribution data in the item cabinet to construct a real-time three-dimensional virtual model of the vending cabinet.

[0028] In this embodiment, a suitable deep learning model is selected for object visual detection, such as YOLO (You Only Look Once) or Faster R-CNN. These models perform well in target detection tasks and can quickly and accurately identify objects. Ensure that the model is fully trained and use a large labeled data set containing images of objects in the cabinet to improve detection accuracy. Input each frame of the brightness optimization monitoring video into the trained deep learning model for object detection. Set a suitable input size (such as 416x416 or 300x300) to ensure the detection effect. In each frame, the model will output the detected object category, location information (bounding box coordinates) and confidence score. Mark the detected objects and draw the bounding box and object information (such as category, confidence) on the video frame. Visualization tools can be used to enhance the user experience and ensure that the markings are clear and visible. The object detection results of each frame, including object ID, location and confidence, are stored in the database for subsequent analysis and reference. Select a suitable camera calibration method, such as Zhang's calibration method or checkerboard method. These methods can obtain the intrinsic and extrinsic parameters of the camera through a series of images, ensuring high-precision calibration results. Prepare a calibration board (such as a chessboard) and take multiple images in the camera's field of view, covering different angles and positions. Use calibration tools (such as OpenCV) to process the captured images, extract corner points and calculate the camera's intrinsic parameters (focal length, principal point position, distortion coefficient) and extrinsic parameters (rotation matrix and translation vector). Record the pose information of each camera, including its position and orientation relative to the world coordinate system, to provide a basis for subsequent 3D positioning. Store the camera position and pose information in a database for subsequent query and application. Ensure that the data structure is clear and convenient for multi-camera data integration. Use multi-view 3D reconstruction methods, such as triangulation, to calculate the 3D spatial coordinates of the object using the 2D coordinates obtained by different cameras. Use the camera's extrinsic parameters and the geometric properties of triangles to perform coordinate transformation to ensure the accuracy of the calculation. For each object, match the positions captured by different cameras and calculate the 3D coordinates using its bounding box coordinates and confidence information. Through linear algebra calculations, using the known camera position and posture information, the position of the object in three-dimensional space is inferred, and the three-dimensional coordinates of each object are recorded. The calculated three-dimensional spatial coordinates of each object are stored in the database, ensuring that each record includes the item ID and the corresponding three-dimensional coordinates (X, Y, Z) for subsequent analysis and application. Use spatial analysis algorithms to analyze the placement and distribution of items in the cabinet. Cluster analysis methods such as K-means or DBSCAN can be used to identify the spatial distribution pattern of items. Define spatial distribution metrics, such as item density, distance between adjacent items, etc., to provide a quantitative basis for subsequent analysis. By calculating the spatial coordinates of each item and analyzing its relative position, the overall layout of the items in the cabinet can be obtained.Identify high-density and low-density areas to provide a basis for optimizing the placement of items. Record the spatial distribution data of each item, including its location, category, and information about neighboring items, for subsequent optimization decisions. Select appropriate 3D reconstruction algorithms, such as Structure from Motion (SfM) or Multi-view Stereo (MVS). These methods can reconstruct the 3D structure of an object from multi-view images. Ensure that the algorithm used can process the optimized monitoring video to ensure the accuracy of the output 3D model. Perform feature extraction on each frame of the brightness optimization monitoring video, use feature descriptors such as SIFT or ORB for image matching, and construct a matching point set. Use the matching point set and camera position data to reconstruct the 3D structure of the vending cabinet and generate complete 3D model data, including the shape, size, and position of the cabinet. Store the generated 3D structure data (such as point cloud, mesh model) in a database, and ensure that the data format is compatible (such as OBJ or PLY) for subsequent use and calling. Select appropriate point cloud processing techniques, such as point cloud registration and fusion algorithms, to ensure that point cloud data from different perspectives can be effectively merged. The Iterative Closest Point (ICP) algorithm can be used for point cloud registration. Ensure the density and accuracy of the point cloud data and set appropriate thresholds to obtain a high-quality 3D model. Build a real-time 3D virtual model based on the 3D structural data of the vending cabinet and the spatial distribution data of the items. Convert the point cloud data into a visual model and process it using 3D modeling tools such as Meshlab or Blender. Ensure that the model is rich in details and can truly reflect the spatial layout of the items and the cabinet structure, which is convenient for subsequent visual display. Integrate the constructed 3D virtual model into the real-time rendering system to ensure that the model can be dynamically updated and displayed in a real-time monitoring environment. Record the version information and update log of the model for subsequent maintenance and optimization to ensure the accuracy and timeliness of the model.

[0029] In this embodiment, refer to Figure 4 , is a flowchart of detailed implementation steps of step S3. In this embodiment, the detailed implementation steps of step S3 include: Step S31: performing dynamic behavior recognition on the brightness optimization monitoring video to extract user dynamic interaction behavior data; Step S32: Analyze the quantity of items in the cabinet based on the user's dynamic interaction behavior data to obtain real-time data on the change in the quantity of items in the cabinet; Step S33: Tracking the position change of the item based on the user's dynamic interaction behavior data to generate an item movement trajectory; Step S34: Evolving the item status changes on the item movement trajectory and the real-time item quantity change data in the cabinet to generate real-time item status change information.

[0030] In this embodiment, a suitable dynamic behavior recognition model is selected, such as a model based on a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM) to capture the spatiotemporal features in the video. This model can effectively analyze the changes between consecutive frames and identify the dynamic behavior of the user. Ensure that the model is fully trained and use a video data set containing different user interaction behaviors to improve the accuracy and robustness of recognition. Input each frame in the brightness optimization monitoring video into the trained model for dynamic behavior recognition. Set a suitable input size (such as 224x224) to ensure the effectiveness of feature extraction. During the processing, the model will output the user's dynamic behavior category, confidence score and corresponding timestamp, and record the duration and frequency of each behavior. The extracted user dynamic interaction behavior data is stored in a database, including information such as user ID, behavior category, timestamp and duration, for subsequent analysis and query. The data storage structure can be designed to ensure the clarity and traceability of the information, and facilitate subsequent data statistics and visualization. Select a suitable item quantity analysis algorithm to identify the user's addition or removal behavior of items based on the user's dynamic interaction behavior data. A rule-based analysis method or a machine learning model can be used to predict quantity changes. Set analysis thresholds, such as the time or frequency of contact between the user and the item, to determine whether the item has been taken away or added. Update the number of items in the cabinet in real time by analyzing the user's interactive behavior. For example, if the user takes an item, the number is reduced by 1; if the user puts the item back, the number is increased by 1. Record the timestamp of each quantity change for subsequent analysis of the trend of item flow. Store the real-time data on the change in the number of items in the cabinet in the database, ensuring that each record includes information such as item ID, quantity, timestamp, etc. for subsequent statistics and analysis. Generate visual charts, such as bar charts or line charts, to show the trend of the change in the number of items in the cabinet, helping decision makers to quickly understand the flow of items. Select appropriate object position tracking algorithms, such as tracking technologies based on Kalman filters or optical flow methods. These algorithms can effectively handle the movement of objects in dynamic scenes and ensure the accuracy of tracking. Set tracking parameters, such as tracking window size and thresholds, to improve the stability and robustness of the algorithm. Use user dynamic interactive behavior data to extract information about the position changes of items in the monitoring video. Whenever a user interacts with an item (such as taking it out or putting it back), the initial and new positions of the item are recorded to generate a movement trajectory. Through timestamp and coordinate information, the movement trajectory of the item is drawn to analyze its flow direction and path in the cabinet. Design an item state change evolution model, combine the movement trajectory of the item with the real-time quantity change data, and analyze the status of the item (such as available, out of stock, hot selling, etc.). Set the rules for state changes. For example, if the number of items decreases and they are frequently taken out, they can be marked as "hot selling"; if the number of items is zero, they are marked as "out of stock". Update the status information of each item based on the user's dynamic interaction behavior and the item movement trajectory.Through real-time data analysis, the status of items is dynamically adjusted to ensure that it reflects the current actual situation. The timestamp of each status change is recorded to facilitate subsequent analysis of the trend of item status evolution. The generated real-time status change information of items is stored in the database, ensuring that each record includes information such as item ID, status, timestamp, etc. for subsequent query and statistics. Visualize status changes and use dashboards or charts to show the dynamic changes of item status, helping managers monitor item status in real time and provide support for decision-making.

[0031] In this embodiment, step S4 includes the following steps: Step S41: updating the status of items in the real-time vending cabinet three-dimensional virtual model based on the real-time status change information of the items, and constructing a real-time updated virtual model; Step S42: positioning the current position of each object on the real-time updated virtual model to generate the latest position status of each object; Step S43: Perform position deviation detection on the latest position status of each item, and mark items with position deviation in the cabinet.

[0032] In this embodiment, real-time status change information of items is collected, including item quantity, status (such as available, out of stock, hot sale, etc.) and location change data. This information should be extracted from the database, and the real-time and accuracy of the data should be ensured. Use appropriate data structures (such as dictionaries or data frames) to store item status information to ensure fast access and update. Select a suitable 3D modeling tool or engine (such as Unity or Unreal Engine) to support the function of updating the virtual model in real time. Ensure that the tool used can handle a large number of 3D objects and support dynamic updates. Set an update mechanism, such as checking the item status change every certain time (such as every second), or updating immediately when receiving status change information. Traverse the status information of each item and update the corresponding item in the virtual model according to the real-time status change. For each item, modify its appearance (such as color, label) and position to reflect the current status. For example, when the item status becomes "out of stock", it can be set to be translucent or add a warning mark in the virtual model to facilitate intuitive identification by users. Select a suitable algorithm for item location positioning, and use a coordinate-based tracking method to ensure that the accurate position of each item in the virtual model is obtained. A spatial coordinate system (such as Cartesian coordinates) can be used to represent the position. Determine the initial position and relative offset of the item, and update it according to the user interaction behavior and the movement trajectory of the item. Traverse the virtual model that is updated in real time and extract the current spatial coordinates of each item one by one. Obtain the latest position status of each item through the constructed three-dimensional spatial coordinate system. Record the latest position of each item, including coordinates (X, Y, Z) and status information (such as availability) to ensure data integrity. Store the latest position status of each item in the database, ensuring that each record includes the item ID, latest coordinates and status for subsequent analysis and query. Select a suitable deviation detection algorithm and set an allowable deviation threshold (such as 5 cm) to determine whether the position of the item deviates from the predetermined position. The mean square error or absolute error calculation method can be used for positioning accuracy analysis. Compare the actual position of the item with the preset position and determine whether there is a deviation by calculating the distance between the two. Traverse the latest position status of each item and calculate the distance between its actual position and the preset position. If the distance exceeds the set deviation threshold, mark the item as a position deviation item. Record the information of each position deviation item, including item ID, actual location, preset location and deviation value, for subsequent analysis and processing. Store the information of marked position deviation items in the cabinet in the database, and ensure that each record includes information such as item ID, status, deviation value, etc. for subsequent query. Use visualization tools to display the status of position deviation items in the cabinet, use identification or color coding to intuitively display the deviation situation, and help managers quickly identify and handle abnormal item locations.

[0033] In this embodiment, step S5 includes the following steps: Step S51: Calculating the original posture deviation of the position deviation items in the cabinet to obtain the original posture deviation value of the items; Step S52: performing adaptive deviation compensation calculation on the original posture deviation value of the object to obtain an adaptive deviation compensation value; Step S53: Perform dynamic deviation compensation processing according to the adaptive deviation compensation value to generate an adaptive posture compensation strategy.

[0034] In this embodiment, a suitable calculation method is selected, and the Euclidean distance or Manhattan distance is used to quantify the deviation between the actual posture and the original posture of the object to ensure the accuracy of the calculation. The latest position information of each object with position deviation is extracted from the database, including its current coordinates (X, Y, Z) and posture information (such as rotation angle). The difference between the original posture and the current posture is calculated. A deviation threshold (such as 5 degrees or 5 cm) can be set to determine whether compensation is required. According to the extracted actual position and posture data, the original posture deviation value of each object is calculated. The deviation information of each object is recorded, including the type of deviation (position or direction), the specific value of the deviation and its impact in the cabinet. A suitable adaptive compensation algorithm can be selected, and a PID control algorithm or a fuzzy logic controller can be used. These algorithms can dynamically adjust the compensation strategy according to the deviation value to achieve a more accurate compensation effect. Algorithm parameters, such as proportional factors, integral factors, and differential factors, are set so as to adjust and optimize according to actual conditions. According to the original posture deviation value of each object, the corresponding compensation value is calculated by the adaptive compensation algorithm. The algorithm will take into account the current deviation value, historical deviation conditions, and other relevant factors (such as environmental changes, object characteristics, etc.). Record the compensation calculation process, including the input deviation value, the calculated compensation value and its adjusted posture parameters, which provides a basis for the subsequent formulation of compensation strategies. Store the adaptive deviation compensation values ​​in the database, ensuring that each record contains the item ID, the original deviation value, the adaptive compensation value and its calculation method for subsequent query and use. Design a dynamic compensation processing scheme based on the adaptive deviation compensation value. You can set the execution conditions of the compensation strategy, such as automatically starting compensation when the item deviation exceeds a certain threshold. Set the goal of the compensation processing, such as restoring the item to its original position, adjusting its angle or achieving better stability. Implement dynamic deviation compensation processing to adjust the item according to the calculated compensation value. This process can be manual adjustment, automatic mechanical device or software control, depending on the characteristics of the items in the cabinet and the operating environment. Record the results of each compensation processing, including the state change before and after compensation, the specific parameters of the adjustment (such as position, angle change) and the execution time, for subsequent analysis and optimization. Store the generated adaptive posture compensation strategy in the database, ensuring that each record includes the item ID, the specific description of the compensation strategy and the implementation effect, for subsequent tracking and evaluation. The compensation strategy is evaluated and updated regularly to ensure its effectiveness and adaptability in practical applications, and optimized based on user feedback and environmental changes.

[0035] In this embodiment, step S6 includes the following steps: Step S61: Perform real-time inventory statistics of the real-time updated virtual model in the item cabinet to obtain inventory information of each item; Step S62: Calculate the purchase frequency of the item within a period based on the real-time status change information of the item to obtain the period purchase frequency of multiple items; Step S63: forecasting item consumption according to the periodic purchase frequencies of multiple items, and generating multiple item consumption forecast values; Step S64: making item replenishment decisions based on the multiple item consumption prediction values ​​and the inventory information of each item to generate an intelligent item replenishment strategy; Step S65: Dynamically coordinate and control the intelligent item replenishment strategy and the adaptive posture compensation strategy to build a collaborative calibration model for the intelligent vending machine.

[0036] In this embodiment, a suitable inventory statistics method is selected, and the latest status information of each item in the virtual model is used for statistics. Ensure that the real-time updated virtual model can accurately reflect the actual inventory of the item. Set the time interval for inventory statistics, such as updating the inventory once every minute, to ensure the timeliness and accuracy of the data. Traverse the real-time updated virtual model to extract the inventory information of each item. Use a data structure (such as an array or dictionary) to store the ID and current inventory of the item for quick access and update. For each item, calculate its current inventory and consider possible factors of change, such as user purchase behavior and replenishment, to ensure the accuracy of the statistical results. Store the inventory information of each item obtained in the database, and ensure that each record contains the item ID, current inventory and recording time for subsequent analysis and query. Generate a real-time inventory monitoring dashboard through a visualization tool to intuitively display the inventory status of each item, helping managers quickly understand the status of the item. Select a suitable purchase frequency calculation method and set a period (such as day, week, month) for analysis. The time series analysis method can be used to calculate the number of purchases of each item within a set period. To ensure data integrity and avoid missing purchase records, you can set a reasonable time window (such as transaction data within the past 30 days). Extract the purchase records of items from the database and record the number of purchases of each item in the selected period. Use counting methods (such as SQL's COUNT function) to count the total purchases of each item in the period. For each item, calculate its purchase frequency, that is, the number of purchases divided by the number of days in the period to obtain the average daily purchase frequency. Store the calculated period purchase frequencies of multiple items in the database, ensuring that each record includes the item ID and purchase frequency for subsequent analysis and decision-making. Select a suitable item consumption prediction model, such as linear regression, ARIMA model, or machine learning algorithm (such as random forest or LSTM). These models can predict future item consumption based on historical purchase frequency patterns. According to the characteristics of the item and historical data, set the input parameters of the model, such as period purchase frequency, historical inventory, and external factors (such as seasonal changes). Use the collected period purchase frequency data and input it into the selected consumption prediction model for calculation. The model will generate future consumption forecast values ​​based on historical data trends. For each item, record the forecast results, including the predicted consumption and the corresponding confidence interval, to evaluate the accuracy of the forecast. Store the generated multiple item consumption forecasts in the database, ensuring that each record includes the item ID and the forecast value for subsequent replenishment decisions. Perform a visual analysis of the forecast results and generate a trend chart to help managers quickly understand the trend of future item consumption. Select a suitable replenishment decision model, combine the consumption forecast value and current inventory information, and set the replenishment threshold and replenishment quantity calculation formula. You can use the economic order quantity (EOQ) model or other inventory management models.Determine replenishment parameters, such as safety stock level, minimum inventory, and replenishment cycle, to make effective replenishment decisions. Perform replenishment calculations for each item and compare the current inventory with the predicted consumption. If the current inventory is lower than the set safety stock level, a replenishment decision is triggered and the required replenishment quantity is calculated. Record the results of the replenishment decision, including the triggered item ID, recommended replenishment quantity, and estimated replenishment time, for subsequent execution. Store the generated intelligent item replenishment strategy in the database, ensuring that each record includes the item ID, replenishment decision, and corresponding parameters for subsequent query and execution. Display the current replenishment strategy through visualization tools and generate a replenishment recommendation report to help management make quick decisions. Design a dynamic collaborative control model that combines the intelligent item replenishment strategy with the adaptive posture compensation strategy to optimize the overall operating efficiency of the vending cabinet. Determine the goals of collaborative control, such as maintaining the accuracy and neatness of item placement while ensuring item availability. Implement dynamic collaborative control based on real-time data and algorithms. For example, when the item deviation exceeds the set threshold, the posture compensation strategy is triggered, and the replenishment demand is evaluated at the same time to ensure the stability and availability of items in the cabinet. Record the implementation effect of collaborative control, including the evaluation of the effect of replenishment and posture compensation and the corresponding adjustment measures for subsequent optimization. Evaluate the constructed collaborative calibration model of smart vending machines and analyze its effect in actual application. Evaluate the accuracy and adaptability of the model through historical data comparison to ensure its effectiveness in different scenarios. Based on the evaluation results, continuously optimize the parameters and strategies of the collaborative control model to ensure that it can adapt to the ever-changing market demand and user behavior.

[0037] In this embodiment, a video recognition-based object calibration system is provided, which is used to execute the video recognition-based object calibration method as described above, including: The video optimization module is used to obtain multi-angle real-time monitoring videos of the vending cabinets; the inter-frame delay optimization and frame-by-frame linear brightness optimization of the multi-angle real-time monitoring videos of the vending cabinets are performed to construct brightness-optimized monitoring videos; The object visual inspection module is used to perform in-depth object visual inspection on the brightness optimization monitoring video, and to perform stereo point cloud modeling to build a real-time three-dimensional virtual model of the vending cabinet; Dynamic behavior analysis module, used to identify dynamic behaviors of brightness optimization monitoring videos and perform object state change evolution to generate real-time object state change information; The real-time item update module is used to update the real-time item status of the three-dimensional virtual model of the real-time vending cabinet based on the real-time item status change information, and to detect the position deviation of the item and mark the position deviation items in the cabinet; Deviation compensation module, used to calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; The collaborative control module is used to make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and to perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the smart vending machine.

[0038] The present invention can improve the visibility of the video by optimizing the brightness of the video frame, especially in low light or complex lighting environments. The optimized brightness enhances the recognition effect of the object and improves the system's ability to extract the features of the object. Inter-frame delay optimization ensures real-time synchronization of video data, which is crucial for object position detection and behavior analysis. Delay optimization can reduce errors in object status monitoring and ensure that changes in objects are reflected in real time. Brightness optimization makes the shape, details and other information of the object more obvious under different lighting conditions, thereby enhancing the recognition effect of the subsequent object visual detection module. Deep visual detection can efficiently and accurately identify various objects and capture changes in objects from multiple angles through advanced deep learning technologies (such as convolutional neural networks). Even complex, obscured or irregular objects can be effectively identified. Through stereo point cloud modeling, the position, direction and layout of objects in three-dimensional space can be accurately restored. The three-dimensional model not only improves the spatial perception ability of the object, but also provides high-precision data support for subsequent object position calibration and compensation processing. As the position and state of the object change, the three-dimensional virtual model will continue to update. This real-time dynamic virtual model enables administrators to intuitively monitor and adjust the placement status of objects. By identifying the interaction between customers and items, the system can capture customer needs and purchasing behavior, so as to understand which items are popular and which items are slow to sell. It can provide real-time customer behavior data, which is helpful for analyzing customer preferences. The evolution of item status changes helps to track the quantity and position changes of items in real time. For example, if an item is taken away or rearranged, the system can record this change to ensure that the state of the virtual model is always consistent with the actual state. The item status change information generated by this module will directly affect the subsequent decision-making of item replenishment, calibration and compensation strategies, and provide a dynamic operation basis for the system. The item status update module ensures the synchronization between the virtual model and the actual item, reflects the status of the item in real time, and avoids management errors caused by delays or deviations. By detecting the difference between the physical position of the item and the position in the virtual model in real time, the system can quickly identify the position deviation, mark the items that need to be calibrated, and provide data support for subsequent calibration and compensation. Real-time status updates can help the system detect abnormal conditions of items in a timely manner, make adjustments in a timely manner, and ensure that the items in the vending machine are placed in accordance with the requirements. Calculating and correcting the posture deviation of the item can effectively solve the deviation caused by improper placement, external force and other factors. The compensation strategy dynamically adjusts the position and angle of the object through an algorithm to return the object to the ideal state. The module is highly adaptive and can flexibly adjust the compensation strategy according to the type, size, weight of the object and the environmental changes of the vending cabinet. Regardless of how the environment changes, the system can ensure the accurate calibration of the position and posture of the object. Through automated compensation processing, the need for manual intervention is reduced, management efficiency is improved, and errors caused by improper human operation are reduced.Based on real-time data (such as item sales speed, inventory status, etc.), the system can intelligently generate replenishment decisions to ensure that the vending cabinet always has sufficient items to avoid out-of-stock or overstock situations. Through collaborative control, replenishment decisions and posture compensation are effectively linked to ensure that replenishment does not cause deviations in item positions. In this way, the replenishment and position calibration processes can be efficiently coordinated, reducing the need for manual intervention. The collaborative control module enables the vending cabinet to automatically complete replenishment and calibration tasks without manual intervention, optimizing management efficiency and improving customer experience.

[0039] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0040] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for object calibration based on video recognition, characterized in that: The following steps are involved: Step S1: obtaining a multi-angle real-time monitoring video of a vending machine; optimizing the inter-frame delay and frame-by-frame linear brightness of the multi-angle real-time monitoring video of the vending machine, thereby constructing a brightness-optimized monitoring video; Step S2: Performing deep object visual inspection on the brightness optimization monitoring video, and performing stereo point cloud modeling to construct a real-time three-dimensional virtual model of the vending cabinet; Step S3: Perform dynamic behavior recognition on the brightness optimization monitoring video and perform object state change evolution to generate real-time object state change information; Step S4: based on the real-time status change information of the items, the real-time three-dimensional virtual model of the vending cabinet is updated with the item status in real time, and the item position deviation detection is performed to mark the items with position deviation in the cabinet; Step S5: Calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; Step S6: Make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the intelligent vending machine.

2. The object calibration method based on video recognition according to claim 1, characterized in that: The specific steps of step S1 are: Step S11: acquiring multi-angle real-time monitoring videos of the vending cabinet based on a multi-camera array; Step S12: monitoring the inter-frame delay of the multi-angle real-time monitoring video of the vending machine, and extracting adjacent image frames with delay mutations; Step S13: performing image frame bilinear interpolation calculation on the delayed mutation adjacent image frames, thereby obtaining a delayed interpolation optimized frame; Step S14: interpolating and reconstructing the multi-angle real-time monitoring video of the vending machine based on the delayed interpolation optimization frame, and performing smooth transition processing on the interpolation frame, thereby constructing an interpolated and reconstructed monitoring video; Step S15: performing frame-by-frame linear brightness optimization on the interpolated and reconstructed monitoring video, thereby constructing a brightness-optimized monitoring video.

3. The object calibration method based on video recognition according to claim 2, characterized in that: The specific steps of step S12 are: Decompose the multi-angle real-time monitoring video of the vending machine frame by frame and extract all image frames; Extracting a timestamp of the image frame; Performing traversal calculation of the timestamp differences of adjacent frames on the timestamps to obtain the delay time between each pair of frames; The average delay time is calculated based on the delay time between each pair of frames, so as to obtain the average delay time of the video frame; Perform delay interval serialization fitting on each pair of inter-frame delay times, thereby generating an inter-frame delay time series; According to the average delay time of the video frames, the inter-frame delay mutation detection is performed on the inter-frame delay time sequence, and the adjacent image frames with delay mutation are extracted.

4. The object calibration method based on video recognition according to claim 2, characterized in that: The specific steps of step S15 are: The gray value histogram distribution of the interpolated and reconstructed monitoring video is calculated frame by frame to obtain the gray value histogram of each frame; Perform peak distribution analysis on the gray value histogram of each frame to obtain the peak distribution of the histogram of each frame; Perform pixel value range statistics according to the gray value histogram of each frame to generate the pixel value range of each frame; Adaptive pixel value stretching processing is performed according to the peak distribution of the histogram of each frame and the pixel value range of each frame to obtain the pixel stretching value of each frame; The interpolated and reconstructed monitoring video is linearly optimized frame by frame according to the pixel stretching value of each frame, so as to construct a brightness optimized monitoring video.

5. The object calibration method based on video recognition according to claim 1, characterized in that: The specific steps of step S2 are: Step S21: Perform deep object visual inspection on the brightness optimization monitoring video and mark each item in the cabinet; Step S22: calibrating the camera position of the multi-camera array to generate camera position and posture information; Step S23: positioning each item in the cabinet in three dimensions based on the camera position and posture information to obtain the item's three-dimensional spatial coordinates; Step S24: performing spatial placement analysis in the cabinet based on the three-dimensional spatial coordinates of the items to obtain spatial distribution data in the cabinet; Step S25: performing a three-dimensional structural analysis of the vending cabinet based on the brightness optimization monitoring video to generate three-dimensional structural data of the vending cabinet; Step S26: Perform stereo point cloud modeling based on the three-dimensional structure data of the vending cabinet and the spatial distribution data in the item cabinet to construct a real-time three-dimensional virtual model of the vending cabinet.

6. The object calibration method based on video recognition according to claim 1, characterized in that: The specific steps of step S3 are: Step S31: performing dynamic behavior recognition on the brightness optimization monitoring video to extract user dynamic interaction behavior data; Step S32: Analyze the quantity of items in the cabinet based on the user's dynamic interaction behavior data to obtain real-time data on the change in the quantity of items in the cabinet; Step S33: Tracking the position change of the item based on the user's dynamic interaction behavior data to generate an item movement trajectory; Step S34: Evolving the item status changes on the item movement trajectory and the real-time item quantity change data in the cabinet to generate real-time item status change information.

7. The object calibration method based on video recognition according to claim 1, characterized in that: The specific steps of step S4 are: Step S41: updating the status of items in the real-time vending cabinet three-dimensional virtual model based on the real-time status change information of the items, and constructing a real-time updated virtual model; Step S42: positioning the current position of each object on the real-time updated virtual model to generate the latest position status of each object; Step S43: Perform position deviation detection on the latest position status of each item, and mark items with position deviation in the cabinet.

8. The object calibration method based on video recognition according to claim 1, characterized in that: The specific steps of step S5 are: Step S51: Calculating the original posture deviation of the position deviation items in the cabinet to obtain the original posture deviation value of the items; Step S52: performing adaptive deviation compensation calculation on the original posture deviation value of the object to obtain an adaptive deviation compensation value; Step S53: Perform dynamic deviation compensation processing according to the adaptive deviation compensation value to generate an adaptive posture compensation strategy.

9. The object calibration method based on video recognition according to claim 1, characterized in that: The specific steps of step S6 are: Step S61: Perform real-time inventory statistics of the real-time updated virtual model in the item cabinet to obtain inventory information of each item; Step S62: Calculate the purchase frequency of the item within a period based on the real-time status change information of the item to obtain the period purchase frequency of multiple items; Step S63: forecasting item consumption according to the periodic purchase frequencies of multiple items, and generating multiple item consumption forecast values; Step S64: making item replenishment decisions based on the multiple item consumption prediction values ​​and the inventory information of each item to generate an intelligent item replenishment strategy; Step S65: Dynamically coordinate and control the intelligent item replenishment strategy and the adaptive posture compensation strategy to build a collaborative calibration model for the intelligent vending machine.

10. An object calibration system based on video recognition, characterized in that: The method for performing the object calibration method based on video recognition as claimed in claim 1 comprises: The video optimization module is used to obtain multi-angle real-time monitoring videos of the vending cabinets; the inter-frame delay optimization and frame-by-frame linear brightness optimization of the multi-angle real-time monitoring videos of the vending cabinets are performed to construct brightness-optimized monitoring videos; The object visual inspection module is used to perform in-depth object visual inspection on the brightness optimization monitoring video, and to perform stereo point cloud modeling to build a real-time three-dimensional virtual model of the vending cabinet; Dynamic behavior analysis module, used to identify dynamic behaviors of brightness optimization monitoring videos and perform object state change evolution to generate real-time object state change information; The real-time item update module is used to update the real-time item status of the three-dimensional virtual model of the real-time vending cabinet based on the real-time item status change information, and to detect the position deviation of the item and mark the position deviation items in the cabinet; Deviation compensation module, used to calculate the original posture deviation of the position deviation items in the cabinet, and perform dynamic deviation compensation processing to generate an adaptive posture compensation strategy; The collaborative control module is used to make item replenishment decisions based on the real-time three-dimensional virtual model of the vending machine, and to perform dynamic collaborative control based on the adaptive posture compensation strategy to build a collaborative calibration model for the smart vending machine.

Images