A self-service vending machine commodity identification method fusing a light curtain and AI image recognition

Abstract

The application relates to the technical field of commodity identification, and particularly discloses a self-service vending machine commodity identification method fusing a light curtain and AI image identification, which comprises the following steps: when an automatic vending machine starts a working state, a light curtain installed above and on the inner side of a door of the automatic vending machine is started; when the door of the automatic vending machine is opened, at least one presumed user taking and placing shelf vertical position is determined by using real-time receiving results of all receivers in the light curtain and a plane matrix analysis method; a camera installed on a shelf of the automatic vending machine is used to acquire commodity taking and placing monitoring video; AI image identification is performed on commodities taken and placed this time based on all presumed user taking and placing shelf vertical positions and the commodity taking and placing monitoring video when the door of the automatic vending machine is opened each time, and a commodity taking and placing result this time is obtained; the taking and placing situation of commodities is accurately and effectively identified, the commodity management efficiency and accuracy of the self-service vending machine are significantly improved, and a more convenient and high-quality shopping experience is brought to users.

Description

Technical Field

[0001] This invention relates to the field of technology, and in particular to a product recognition method for vending machines that integrates light curtain and AI image recognition. Background Technology

[0002] Currently, vending machines are increasingly appearing in people's lives as a convenient retail method in today's consumption landscape. To achieve more efficient and accurate product management and transactions, the technology for recognizing products within vending machines is constantly evolving. Light curtain technology and AI image recognition technology each have their applications in their respective fields, but their integration for product recognition in vending machines is still in the exploratory stage. Light curtain technology detects the position and movement of objects by blocking and receiving light, featuring fast response speed and high accuracy. AI image recognition technology, with its powerful learning and analysis capabilities, can process and understand complex images. Installing cameras on the vending machine shelves allows for the acquisition of real-time product status and user interaction footage.

[0003] However, the current application of AI image recognition technology alone in product identification in vending machines has certain limitations. Relying solely on AI image recognition can be affected by factors such as lighting and angle, leading to inaccurate results. When handling rapid product handling operations, it may not be able to obtain product information in a timely and accurate manner. Furthermore, existing technologies perform poorly in recognizing complex user actions and special situations.

[0004] Therefore, this invention proposes a product recognition method for vending machines that integrates light curtain and AI image recognition. Summary of the Invention

[0005] This invention provides a product identification method for vending machines that integrates light curtain and AI image recognition. In step S1, the activated light curtain, through a matrix arrangement of emitters and receivers, and the propagation of parallel or perpendicular light rays, establishes a complete infrastructure for subsequent monitoring, ensuring the comprehensiveness and accuracy of the monitoring. Step S2, based on the real-time reception results of the light curtain receiver, uses a planar matrix analysis method to accurately infer the vertical location of the shelf where the user picks up or places the product, providing crucial location information for subsequent product identification. Step S3 acquires monitoring video when the vending machine door opens, preparing intuitive and rich image data for product identification. Step S4 combines the inferred vertical location and monitoring video for AI image recognition, greatly improving the accuracy and reliability of product identification results. Overall, this process integrates light curtain and AI image recognition technologies, enabling accurate and effective identification of product placement and retrieval, significantly improving the efficiency and accuracy of product management in vending machines, and providing users with a more convenient and high-quality shopping experience.

[0006] This invention provides a product recognition method for vending machines that integrates light curtain and AI image recognition, including:

[0007] S1: When the vending machine is in operation, the light curtain installed above and inside the vending machine door is activated. The light emitters of the light curtain are arranged in a matrix, and a receiver is installed on the opposite side of each light emitter. When the light curtain is working normally, the light rays propagating from all groups of light emitters and receivers are parallel or perpendicular to each other.

[0008] S2: When the vending machine door is opened, at least one vertical location of the user's pick-up and drop-off shelf is determined by using the real-time reception results of all receivers in the light curtain and the planar matrix analysis method.

[0009] S3: When the vending machine door is opened, the camera installed on the vending machine shelf will be used to obtain video of the goods being picked up and put down.

[0010] S4: Based on all the inferred vertical locations of the shelves and the monitoring video of the product retrieval each time the vending machine door is opened, AI image recognition is performed on the product to be retrieved and retrieved to obtain the product retrieval result.

[0011] Preferably, in the vending machine product recognition method integrating light curtain and AI image recognition, S2: when the vending machine door is opened, at least one hypothetical vertical location of the user's pick-up / placement shelf is determined using the real-time reception results of all receivers in the light curtain and a planar matrix analysis method, including:

[0012] S201: Generate a vertical plane matrix based on the position and number of all emitters in the light curtain;

[0013] S202: When the vending machine door is opened, the occlusion area is located in the vertical plane matrix based on the real-time reception results of all receivers in the light curtain, and the vertical plane occlusion area of ​​the shelf is obtained at each moment after the vending machine door is opened.

[0014] S203: Determine at least one hypothetical vertical location of the shelf for a user to pick up or place items based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened.

[0015] Preferably, the vending machine product recognition method integrating light curtain and AI image recognition, S203: determines at least one inferred user shelf vertical location based on the shelf vertical plane obstruction area at all times after the vending machine door is opened, including:

[0016] A dynamic sequence of obstruction areas is generated based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened.

[0017] Based on the geometric dynamic characteristics of the dynamic sequence of the occluded area, at least one inferred vertical location of the user picking up or placing items on the shelf is analyzed.

[0018] Preferably, the product recognition method for vending machines that integrates light curtains and AI image recognition analyzes at least one inferred vertical location of the user's pick-up / placement on the shelf based on the geometric dynamic features of the dynamic sequence of the occluded area, including:

[0019] The geometric area of ​​each occluded region in the dynamic sequence of occluded regions is calculated, and the physical center of each occluded region in the dynamic sequence of occluded regions is determined.

[0020] In the dynamic sequence of occluded areas, select the continuous sequence of occluded areas with a geometric area deviation not exceeding the area deviation threshold, a physical center jitter amplitude not exceeding the jitter amplitude threshold, and a physical center jitter frequency not exceeding the jitter frequency threshold. The number of consecutive frames is not less than a preset number of frames. This sequence is used as the sequence of occluded areas where the user stays.

[0021] Each predicted user stay occlusion area sequence is represented by the corresponding shelf location on the vertical plane of the shelf as a predicted user shelf location.

[0022] Preferably, the product recognition method for vending machines that integrates light curtain and AI image recognition selects a continuous sequence of occluded areas from the dynamic sequence of occluded areas, where the geometric area deviation does not exceed an area deviation threshold, the physical center jitter amplitude does not exceed a jitter amplitude threshold, and the physical center jitter frequency does not exceed a jitter frequency threshold. This continuous sequence of occluded areas with a number of consecutive frames not less than a preset number of frames is used as the sequence for inferring the user's occluded area. This includes:

[0023] The geometric area deviation between each group of adjacent occluded regions in the dynamic sequence of occluded regions is calculated. Adjacent occluded regions whose geometric area deviation does not exceed the area deviation threshold are regarded as the first qualified occluded region group. The occluded regions adjacent to any occluded region in each first qualified occluded region group in the dynamic sequence of occluded regions are merged with the corresponding first qualified occluded region group to obtain multiple first merged occluded region groups. The average geometric area deviation of each pair of occluded regions in each first merged occluded region group is regarded as the geometric area deviation of each first merged occluded region group. All first merged occluded region groups whose geometric area deviation does not exceed the area deviation threshold are regarded as all second qualified occluded region groups. Each second qualified occluded region group continues to be merged and compared with the geometric deviation threshold until there is no new merged occluded region group whose geometric area deviation does not exceed the area deviation threshold. Then, all qualified occluded region groups with a continuous frame count not less than a preset frame count are regarded as the first filtered occluded region sequence.

[0024] Calculate the physical center jitter amplitude and physical center jitter frequency for each first-screened occlusion region sequence;

[0025] The first filtered occlusion region sequence, in which the physical center jitter amplitude does not exceed the jitter amplitude threshold and the physical center jitter frequency does not exceed the jitter frequency threshold, is used as the inferred user dwell occlusion region sequence.

[0026] Preferably, the product recognition method for vending machines that integrates light curtain and AI image recognition calculates the physical center jitter amplitude and physical center jitter frequency of each first-selection occlusion region sequence, including:

[0027] Based on the two-dimensional coordinates of the physical center of each occluded region in each first-selection occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter amplitude of the physical center of each first-selection occluded region sequence is calculated:

[0028]

[0029] In the formula, J represents the physical center jitter amplitude of the first filtered occlusion region sequence currently calculated, max() is the maximum value among the multiple elements within the parentheses, and x ij Let y be the x-coordinate of the physical center of the i-th occluded region in the currently calculated first filtered occluded region sequence in the j-th preset two-dimensional coordinate system, and n be the total number of occluded regions in the currently calculated first filtered occluded region sequence. ij The ordinate value of the physical center of the i-th occlusion region in the currently calculated first filtered occlusion region sequence in the j-th preset two-dimensional coordinate system;

[0030] Based on the two-dimensional coordinates of the physical center of each occluded region in each first screening occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter frequency of the physical center of each first screening occluded region sequence is calculated.

[0031] Preferably, the product recognition method for vending machines that integrates light curtain and AI image recognition calculates the physical center jitter frequency of each first-selection occlusion region sequence based on the two-dimensional coordinates of the physical center of each occlusion region in each preset two-dimensional coordinate system on the vertical plane of the shelf, including:

[0032] Determine the difference in the horizontal and vertical coordinates of the physical centers of all adjacent groups of occluded regions in each first screening occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, and generate the difference sequence of the horizontal and vertical coordinates of each first screening occluded region sequence in each preset two-dimensional coordinate system;

[0033] Based on the difference sequences of the horizontal and vertical coordinates of each first-selected occlusion region sequence in each preset two-dimensional coordinate system, the horizontal and vertical autocorrelation functions of the physical centers of all occlusion regions in each first-selected occlusion region sequence relative to the integer period k in each preset two-dimensional coordinate system are generated:

[0034]

[0035] In the formula, R xxj (k) is the transverse autocorrelation function of the physical centers of all occluded regions in the currently calculated first filtered occluded region sequence relative to period k in the j-th preset two-dimensional coordinate system, Δx. ij Let Δx be the projected displacement of the displacement between the physical centers of the i-th and (i+1)-th occlusion regions in the currently calculated first filtered occlusion region sequence, along the x-axis of the j-th preset two-dimensional coordinate system. (i+k)j Let Δy be the projected displacement of the displacement between the physical center of the (i+k)th occlusion region and the physical center of the (i+k+1)th occlusion region in the currently calculated first filtered occlusion region sequence, along the x-axis of the j-th preset two-dimensional coordinate system. ij Let Δy be the projected displacement of the displacement between the physical centers of the i-th and (i+1)-th occlusion regions in the currently calculated first filtered occlusion region sequence, along the ordinate axis of the j-th preset two-dimensional coordinate system. (i+k)j The displacement between the physical center of the (i+k)th occlusion region and the physical center of the (i+k+1)th occlusion region in the currently calculated first filtered occlusion region sequence is the projected displacement along the vertical axis of the j-th preset two-dimensional coordinate system.

[0036] Based on the horizontal and vertical autocorrelation functions of the physical centers of all occluded regions in each first-selection occluded region sequence relative to integer period k in each preset two-dimensional coordinate system, the jitter frequency of the physical center of each first-selection occluded region sequence is determined.

[0037] Preferably, the product recognition method for vending machines that integrates light curtain and AI image recognition determines the jitter frequency of the physical center of each first-selection occlusion region sequence based on the horizontal and vertical autocorrelation functions of the physical centers of all occlusion regions in each preset two-dimensional coordinate system relative to an integer period k, including:

[0038] The reciprocal of the integer period k corresponding to the horizontal autocorrelation function at which the physical center of all occluded regions in each first-selection occluded region sequence reaches a peak in each preset two-dimensional coordinate system is taken as the horizontal jitter frequency of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system.

[0039] The reciprocal of the integer period k corresponding to the longitudinal autocorrelation function at which the physical center of all occluded regions in each first-selection occluded region sequence reaches a peak in the longitudinal autocorrelation function relative to the integer period k in each preset two-dimensional coordinate system is taken as the longitudinal jitter frequency of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system.

[0040] Based on the horizontal and vertical jitter frequencies of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system, the jitter frequency of the physical center of each first-selection occluded region sequence is determined.

[0041] Preferably, the product recognition method for vending machines that integrates light curtain and AI image recognition determines the physical center jitter frequency of each first-selection occlusion region sequence based on the horizontal and vertical jitter frequencies of the physical center of all occlusion regions in each preset two-dimensional coordinate system, including:

[0042] The square root of the sum of the squares of the horizontal and vertical jitter frequencies of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system is taken as the comprehensive jitter frequency of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system.

[0043] The average of the combined jitter frequencies of the physical centers of all occluded regions in all preset two-dimensional coordinate systems of each first-selection occluded region sequence is taken as the jitter frequency of the physical center of each first-selection occluded region sequence.

[0044] Preferably, the vending machine product recognition method integrating light curtain and AI image recognition, S4: Based on all the inferred vertical locations of the user's pick-up and drop-off shelves and the product pick-up and drop-off monitoring video each time the vending machine door is opened, AI image recognition is performed on the product picked up and dropped this time to obtain the product pick-up and drop-off result, including:

[0045] S401: Perform AI image recognition on the monitoring video of goods picking and placing to determine the continuous time period of all actual users and the type of picking and placing operation for each actual user during the continuous time period;

[0046] S402: Based on all actual user duration periods and the pick-up and put-down operation types for each actual user duration period, identify all actual user pick-up and put-down shelf vertical locations and the pick-up and put-down operation types for each actual user pick-up and put-down shelf vertical location in all presumed user pick-up and put-down shelf vertical locations each time the vending machine door is opened.

[0047] S403: Based on the vertical locations of all actual users picking up and placing items on the shelves and the picking and placing operation types of each actual user picking up and placing items on the shelves, determine all actual users picking up and placing items and the corresponding picking and placing operation types, and use this as the result of picking up and placing items.

[0048] The beneficial effects of this invention compared to existing technologies are as follows: In step S1, the activated light curtain, through a matrix arrangement of emitters and receivers, and the propagation of parallel or perpendicular light rays, constructs a complete infrastructure for subsequent monitoring, ensuring the comprehensiveness and accuracy of the monitoring. Step S2, based on the real-time reception results of the light curtain receiver, uses a planar matrix analysis method to accurately infer the vertical location of the user's product placement on the shelf, providing crucial location information for subsequent product identification. Step S3 acquires monitoring video when the vending machine door opens, preparing intuitive and rich image data for product placement identification. Step S4 combines the inferred vertical location and monitoring video for AI image recognition, greatly improving the accuracy and reliability of product placement identification. Overall, this process integrates light curtain and AI image recognition technologies, enabling accurate and effective identification of product placement, significantly improving the efficiency and accuracy of vending machine product management, and providing users with a more convenient and high-quality shopping experience.

[0049] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a flowchart of the product recognition method for a self-service vending machine that integrates light curtain and AI image recognition in an embodiment of the present invention;

[0053] Figure 2 This is a flowchart illustrating the specific execution method of step S2 in this embodiment of the invention.

[0054] Figure 3 This is a flowchart illustrating the specific execution method of step S4 in an embodiment of the present invention. Detailed Implementation

[0055] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0056] Example 1:

[0057] This invention provides a product recognition method for vending machines that integrates light curtain and AI image recognition, with reference to... Figure 1 ,include:

[0058] S1: When the vending machine is in operation, the light curtain installed above and inside the vending machine door is activated. The light emitters of the light curtain are arranged in a matrix, and a receiver is installed on the opposite side of each light emitter. When the light curtain is working normally, the light rays propagating from all groups of light emitters and receivers are parallel or perpendicular to each other.

[0059] S2: When the vending machine door is opened, at least one vertical location of the user's pick-up and drop-off shelf is determined by using the real-time reception results of all receivers in the light curtain and the planar matrix analysis method.

[0060] S3: When the vending machine door is opened, the camera installed on the vending machine shelf will be used to obtain video of the goods being picked up and put down.

[0061] S4: Based on all the inferred vertical locations of the shelves and the monitoring video of the product retrieval each time the vending machine door is opened, AI image recognition is performed on the product to be retrieved and retrieved to obtain the product retrieval result.

[0062] In this embodiment, the real-time reception result of the receiver refers to the light state information received by the receiver in the light curtain at each moment when the vending machine is working, such as which light rays are blocked and which light rays are received normally.

[0063] In this embodiment, the inferred vertical location of the shelf where the user might pick up or put down the goods refers to the vertical location area of ​​the shelf where the user might pick up or put back the goods, inferred from the real-time reception results of the light curtain receiver and related analysis methods.

[0064] In this embodiment, since the types of goods are relatively fixed and they are usually arranged according to the layers and areas of the shelves inside the display case, the replenishment staff will organize the goods during regular or irregular replenishment to ensure the regularity of the goods placement. Therefore, planar matrix technology is used to accurately sense and collect the location information of customers picking up goods (i.e., the layer number and approximate location of the goods).

[0065] In this embodiment, the product retrieval and placement monitoring video refers to the video captured by the camera installed on the shelf when the vending machine door is opened, recording the user's actions of taking or putting back products.

[0066] In this embodiment, the product retrieval and placement result refers to the specific information about the user taking or putting back products this time, which is finally determined based on various data and analysis when the vending machine door is opened this time, including which products were taken or put back and the corresponding operation type.

[0067] The beneficial effects of the above technologies are as follows: In step S1, the activated light curtain, through a matrix arrangement of emitters and receivers, and the propagation of parallel or perpendicular light rays, constructs a complete infrastructure for subsequent monitoring, ensuring the comprehensiveness and accuracy of the monitoring. Step S2, based on the real-time reception results of the light curtain receiver, uses a planar matrix analysis method to accurately infer the vertical location of the user's product placement on the shelf, providing crucial location information for subsequent product identification. Step S3 acquires monitoring video when the vending machine door opens, preparing intuitive and rich image data for product placement identification. Step S4 combines the inferred vertical location and monitoring video for AI image recognition, greatly improving the accuracy and reliability of product placement identification. Overall, this process integrates light curtain and AI image recognition technologies, enabling accurate and effective identification of product placement, significantly improving the efficiency and accuracy of vending machine product management, and providing users with a more convenient and high-quality shopping experience.

[0068] Example 2:

[0069] Based on Example 1, a vending machine product recognition method integrating light curtain and AI image recognition is proposed, S2: When the vending machine door is opened, at least one hypothetical vertical location of the user's pick-up / placement shelf is determined using the real-time reception results of all receivers in the light curtain and a planar matrix analysis method, with reference to... Figure 2 ,include:

[0070] S201: Generate a vertical plane matrix based on the position and number of all emitters in the light curtain;

[0071] S202: When the vending machine door is opened, the occlusion area is located in the vertical plane matrix based on the real-time reception results of all receivers in the light curtain, and the vertical plane occlusion area of ​​the shelf is obtained at each moment after the vending machine door is opened.

[0072] S203: Determine at least one hypothetical vertical location of the shelf for a user to pick up or place items based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened.

[0073] In this embodiment, the location and number of the light emitters refer to the specific installation location of each light emitter inside the vending machine and the identification number assigned to each light emitter.

[0074] In this embodiment, generating a vertical plane matrix based on the positions and numbers of all emitters in the light curtain means constructing a matrix-like data structure that can describe their distribution on the vertical plane based on the specific positions and numbers of all emitters in the light curtain.

[0075] In this embodiment, the occlusion area is located in the vertical plane matrix based on the real-time reception results of all receivers in the light curtain. The occlusion area of ​​the shelf vertical plane at each moment after the vending machine door is opened means that the area where the light is blocked is determined in the constructed vertical plane matrix using the real-time data received by the receivers in the light curtain at each moment, thereby obtaining the range of the occluded area on the shelf vertical plane at each moment after the vending machine door is opened.

[0076] In this embodiment, the shelf vertical plane obstruction area refers to the area on the vertical plane of the vending machine shelf that cannot be illuminated by light due to objects (such as the user's hand or body) blocking the light from the light curtain.

[0077] The beneficial effects of the above technology are as follows: In step S201, a vertical plane matrix is ​​generated based on the position and number of the emitters, providing an accurate reference frame for subsequent occlusion area positioning. Step S202 locates the occlusion area in the vertical plane matrix using the real-time reception results of the receiver, enabling real-time and accurate capture of shelf occupancy when the vending machine door is open. Step S203 determines the inferred vertical location of the user's shelf based on the occlusion area at all times, improving the accuracy and reliability of location estimation. Overall, this step-by-step processing method effectively utilizes light curtain information to accurately infer the vertical location of the user's shelf, providing an important foundation for subsequent product recognition and helping to improve the accuracy and efficiency of product recognition in vending machines.

[0078] Example 3:

[0079] Based on Example 2, a vending machine product recognition method integrating light curtain and AI image recognition is described in step S203: At least one hypothetical vertical location of the user's pick-up / placement shelf is determined based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened, including:

[0080] A dynamic sequence of obstruction areas is generated based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened.

[0081] Based on the geometric dynamic characteristics of the dynamic sequence of the occluded area, at least one inferred vertical location of the user picking up or placing items on the shelf is analyzed.

[0082] In this embodiment, the dynamic sequence of the obstruction area refers to a series of continuous state records formed by the change of the obstruction area on the vertical plane of the vending machine shelf over a period of time.

[0083] In this embodiment, geometric dynamic features refer to the dynamic characteristics and patterns exhibited by the occluded area in terms of geometric attributes such as shape, size, and position over time.

[0084] The beneficial effects of the above technology are as follows: By generating a dynamic sequence of occlusion areas, the changes in the vertical plane occlusion area of ​​the shelves after the vending machine door is opened can be presented comprehensively and dynamically. Analysis based on the geometric dynamic characteristics of the dynamic sequence of occlusion areas allows for a deeper and more accurate prediction of the vertical location of the shelves where users pick up or place items. This method fully utilizes the dynamic information of the occlusion areas, improving the accuracy and reliability of location prediction. It also helps to more accurately identify products, improving the efficiency and accuracy of product management in vending machines.

[0085] Example 4:

[0086] Building upon Example 3, a product recognition method for vending machines that integrates light curtains and AI image recognition analyzes at least one hypothetical vertical location of the user's pick-up / placement shelf based on the geometric dynamic features of the dynamic sequence of the occluded area, including:

[0087] The geometric area of ​​each occluded region in the dynamic sequence of occluded regions is calculated, and the physical center of each occluded region in the dynamic sequence of occluded regions is determined.

[0088] In the dynamic sequence of occluded areas, select the continuous sequence of occluded areas with a geometric area deviation not exceeding the area deviation threshold, a physical center jitter amplitude not exceeding the jitter amplitude threshold, and a physical center jitter frequency not exceeding the jitter frequency threshold. The number of consecutive frames is not less than a preset number of frames. This sequence is used as the sequence of occluded areas where the user stays.

[0089] Each predicted user stay occlusion area sequence is represented by the corresponding shelf location on the vertical plane of the shelf as a predicted user shelf location.

[0090] In this embodiment, the geometric area of ​​the obstructed area refers to the area on a two-dimensional plane occupied by a specific obstructed area on the vertical plane of the vending machine shelf.

[0091] In this embodiment, the geometric area deviation is an indicator used to measure the degree of difference between the geometric areas of adjacent or different occluded areas.

[0092] In this embodiment, the area deviation threshold is a pre-set critical value used to determine whether the geometric area deviation is acceptable.

[0093] In this embodiment, the physical center jitter amplitude represents the jitter amplitude of the physical center position of the occluded area within a certain period of time or between different occluded areas.

[0094] In this embodiment, the jitter amplitude threshold is the upper limit of the jitter amplitude at the physical center.

[0095] In this embodiment, the physical center jitter frequency refers to the frequency at which the physical center position of the occluded area jitters periodically over a certain period of time.

[0096] In this embodiment, the jitter frequency threshold is the upper limit of the set physical center jitter frequency.

[0097] In this embodiment, the preset frame number is a pre-set number of frames used to determine the continuity of the occluded area.

[0098] In this embodiment, the continuous sequence of occluded regions refers to a series of continuous occluded regions that have some correlation or similarity.

[0099] In this embodiment, the shelf location corresponding to the sequence of user stay obstruction areas on the vertical plane of the shelf refers to the specific shelf location corresponding to the sequence of obstruction areas corresponding to the user stay on the vertical plane of the shelf, which is inferred from the analysis.

[0100] The beneficial effects of the above technologies are as follows: By statistically analyzing the geometric area of ​​the obstructed region and determining its physical center, crucial quantitative data is provided for subsequent analysis. Selecting continuous sequences of obstructed regions that meet specific conditions as the inferred user dwell time sequence improves the accuracy and reliability of the inference. Using the shelf locations corresponding to the inferred user dwell time sequence as the inferred vertical locations of the shelves allows for more precise positioning of the user's potential operating positions. This analysis method based on detailed geometric dynamic features effectively eliminates interference factors and improves the accuracy of location inference. It also contributes to more accurate product identification, improving the service quality and management efficiency of vending machines.

[0101] Example 5:

[0102] Based on Example 4, the self-service vending machine product recognition method integrating light curtain and AI image recognition selects a continuous sequence of occluded areas from the dynamic sequence of occluded areas where the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold. This sequence is used as the inferred user dwell occluded area sequence, including:

[0103] The geometric area deviation between each group of adjacent occluded regions in the dynamic sequence of occluded regions is calculated. Adjacent occluded regions whose geometric area deviation does not exceed the area deviation threshold are regarded as the first qualified occluded region group. The occluded regions adjacent to any occluded region in each first qualified occluded region group in the dynamic sequence of occluded regions are merged with the corresponding first qualified occluded region group to obtain multiple first merged occluded region groups. The average geometric area deviation of each pair of occluded regions in each first merged occluded region group is regarded as the geometric area deviation of each first merged occluded region group. All first merged occluded region groups whose geometric area deviation does not exceed the area deviation threshold are regarded as all second qualified occluded region groups. Each second qualified occluded region group continues to be merged and compared with the geometric deviation threshold until there is no new merged occluded region group whose geometric area deviation does not exceed the area deviation threshold. Then, all qualified occluded region groups with a continuous frame count not less than a preset frame count are regarded as the first filtered occluded region sequence.

[0104] Calculate the physical center jitter amplitude and physical center jitter frequency for each first-screened occlusion region sequence;

[0105] The first filtered occlusion region sequence, in which the physical center jitter amplitude does not exceed the jitter amplitude threshold and the physical center jitter frequency does not exceed the jitter frequency threshold, is used as the inferred user dwell occlusion region sequence.

[0106] In this embodiment, the first merged occlusion area group includes three occlusion areas.

[0107] In this embodiment, the method for calculating the geometric area deviation between two occluded areas may be as follows: first, obtain the geometric area of ​​each of the two occluded areas, then calculate the difference between the two areas, and then divide the difference by the area of ​​one of the occluded areas (or the average of the two areas). The result is the geometric area deviation between the two occluded areas.

[0108] In this embodiment, the process of neighbor merging and geometric deviation threshold comparison is the same as the process of obtaining a first merged occlusion region group, calculating the geometric area deviation of the first merged occlusion region group, and comparing the first merged occlusion region group with the area deviation threshold to obtain a second qualified occlusion region group.

[0109] The beneficial effects of the above technologies are as follows: By progressively merging and filtering occluded region groups, areas meeting the geometric area deviation requirements can be more accurately identified, improving the accuracy of the filtering. The amplitude and frequency of the physical center jitter of each filtered region sequence are calculated, evaluating the stability of the occluded regions from multiple dimensions. The final sequence of inferred user dwell occluded regions has higher reliability and accuracy. This refined filtering and analysis method effectively eliminates interference and unstable regions, more accurately predicting the user's dwell location. This helps improve the accuracy of product recognition and the management efficiency of vending machines, providing users with a better service experience.

[0110] Example 6:

[0111] Based on Example 5, the self-service vending machine product recognition method, which integrates light curtain and AI image recognition, calculates the physical center jitter amplitude and physical center jitter frequency of each first screening occlusion region sequence, including:

[0112] Based on the two-dimensional coordinates of the physical center of each occluded region in each first-selection occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter amplitude of the physical center of each first-selection occluded region sequence is calculated:

[0113]

[0114] In the formula, J represents the physical center jitter amplitude of the first filtered occlusion region sequence currently calculated, max() is the maximum value among the multiple elements within the parentheses, and x ij Let y be the x-coordinate of the physical center of the i-th occluded region in the currently calculated first filtered occluded region sequence in the j-th preset two-dimensional coordinate system, and n be the total number of occluded regions in the currently calculated first filtered occluded region sequence. ij The ordinate value of the physical center of the i-th occlusion region in the currently calculated first filtered occlusion region sequence in the j-th preset two-dimensional coordinate system;

[0115] Based on the two-dimensional coordinates of the physical center of each occluded region in each first screening occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter frequency of the physical center of each first screening occluded region sequence is calculated.

[0116] In this embodiment, the origins of all preset two-dimensional coordinate systems coincide, and the horizontal axes of all preset two-dimensional coordinate systems are evenly distributed. For example, there are three preset two-dimensional coordinate systems, and the horizontal axes of each pair of these three preset two-dimensional coordinate systems form an angle of 120 degrees.

[0117] The beneficial effects of the above technology are as follows: The formulas described above can accurately calculate the physical center jitter amplitude of each first-selection occlusion region sequence, quantifying the degree of positional change of the physical center. The coordinate values ​​of each occlusion region in different preset two-dimensional coordinate systems are considered, making the jitter amplitude calculation more comprehensive and accurate. Simultaneously, it provides a basis for calculating the physical center jitter frequency. This helps to more accurately assess the stability and reliability of the occlusion region sequence. Therefore, it can more accurately filter out the sequence of inferred user-occluded occlusion regions, improving the accuracy of product recognition and the operating efficiency of vending machines.

[0118] Example 7:

[0119] Based on Example 6, the self-service vending machine product recognition method integrating light curtain and AI image recognition calculates the physical center jitter frequency of each first screening occlusion region sequence based on the two-dimensional coordinate values ​​of the physical center of each occlusion region in each preset two-dimensional coordinate system on the vertical plane of the shelf, including:

[0120] Determine the difference in the horizontal and vertical coordinates of the physical centers of all adjacent groups of occluded regions in each first screening occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, and generate the difference sequence of the horizontal and vertical coordinates of each first screening occluded region sequence in each preset two-dimensional coordinate system;

[0121] Based on the difference sequences of the horizontal and vertical coordinates of each first-selected occlusion region sequence in each preset two-dimensional coordinate system, the horizontal and vertical autocorrelation functions of the physical centers of all occlusion regions in each first-selected occlusion region sequence relative to the integer period k in each preset two-dimensional coordinate system are generated:

[0122]

[0123] In the formula, R xxj (k) is the transverse autocorrelation function of the physical centers of all occluded regions in the currently calculated first filtered occluded region sequence relative to period k in the j-th preset two-dimensional coordinate system, Δx. ij Let Δx be the projected displacement of the displacement between the physical centers of the i-th and (i+1)-th occlusion regions in the currently calculated first filtered occlusion region sequence, along the x-axis of the j-th preset two-dimensional coordinate system. (i+k)j Let Δy be the projected displacement of the displacement between the physical center of the (i+k)th occlusion region and the physical center of the (i+k+1)th occlusion region in the currently calculated first filtered occlusion region sequence, along the x-axis of the j-th preset two-dimensional coordinate system. ijLet Δy be the projected displacement of the displacement between the physical centers of the i-th and (i+1)-th occlusion regions in the currently calculated first filtered occlusion region sequence, along the ordinate axis of the j-th preset two-dimensional coordinate system. (i+k)j The displacement between the physical center of the (i+k)th occlusion region and the physical center of the (i+k+1)th occlusion region in the currently calculated first filtered occlusion region sequence is the projected displacement along the vertical axis of the j-th preset two-dimensional coordinate system.

[0124] Based on the horizontal and vertical autocorrelation functions of the physical centers of all occluded regions in each first-selection occluded region sequence relative to integer period k in each preset two-dimensional coordinate system, the jitter frequency of the physical center of each first-selection occluded region sequence is determined.

[0125] In this embodiment, the horizontal coordinate difference sequence refers to the difference in coordinate values ​​of the physical centers of adjacent occluded regions on the horizontal axis of a preset two-dimensional coordinate system in each first screening occluded region sequence, arranged in chronological order or the order of occluded regions.

[0126] In this embodiment, the ordinate difference sequence refers to the difference in coordinate values ​​of the physical centers of adjacent occluded regions on the ordinate axis of a preset two-dimensional coordinate system in each first screening occluded region sequence, arranged in chronological order or occluded region order.

[0127] In this embodiment, the horizontal autocorrelation function (vertical autocorrelation function) is a function used to describe the similarity or correlation between the physical centers of two sets of partially continuous occluded regions formed by the interval period k in each first screening occluded region sequence in the direction of the horizontal axis (or the vertical axis) of a preset two-dimensional coordinate system.

[0128] The beneficial effects of the above techniques are as follows: By determining the differences in the horizontal and vertical coordinates, a difference sequence is generated, providing fundamental data for subsequent calculation of the autocorrelation function. Utilizing complex formulas to generate horizontal and vertical autocorrelation functions enables a comprehensive and accurate description of the correlation of the physical center. Determining the physical center jitter frequency based on the autocorrelation function improves the accuracy and reliability of frequency calculation. This refined calculation method helps to more accurately evaluate the dynamic characteristics of the physical center of the occlusion region sequence. This allows for more effective screening of sequences of inferred user-occluded areas, improving the accuracy of product recognition and the management effectiveness of vending machines.

[0129] Example 8:

[0130] Based on Example 7, the self-service vending machine product recognition method integrating light curtain and AI image recognition determines the physical center jitter frequency of each first-selection occlusion region sequence based on the horizontal and vertical autocorrelation functions of the physical centers of all occlusion regions in each preset two-dimensional coordinate system relative to an integer period k, including:

[0131] The reciprocal of the integer period k corresponding to the horizontal autocorrelation function at which the physical center of all occluded regions in each first-selection occluded region sequence reaches a peak in each preset two-dimensional coordinate system is taken as the horizontal jitter frequency of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system.

[0132] The reciprocal of the integer period k corresponding to the longitudinal autocorrelation function at which the physical center of all occluded regions in each first-selection occluded region sequence reaches a peak in the longitudinal autocorrelation function relative to the integer period k in each preset two-dimensional coordinate system is taken as the longitudinal jitter frequency of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system.

[0133] Based on the horizontal and vertical jitter frequencies of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system, the jitter frequency of the physical center of each first-selection occluded region sequence is determined.

[0134] In this embodiment, the horizontal autocorrelation function (or vertical autocorrelation function) at which the peak occurs is the horizontal autocorrelation function (or vertical autocorrelation function) for the horizontal autocorrelation function (or vertical autocorrelation function) of the discrete independent variable with an integer period k, which is the horizontal autocorrelation function (or vertical autocorrelation function) formed by the integer period k corresponding to the maximum value of its function value.

[0135] The beneficial effects of the above technology are as follows: Using the reciprocal of the integer period corresponding to the horizontal autocorrelation function at the peak as the horizontal jitter frequency, and the reciprocal of the peak value of the vertical autocorrelation function as the vertical jitter frequency, this determination method is accurate and intuitive. Considering both horizontal and vertical jitter frequencies separately makes the jitter frequency evaluation more comprehensive and detailed. Determining the physical center jitter frequency based on a comprehensive analysis of both horizontal and vertical jitter frequencies improves the accuracy and reliability of frequency determination. It helps to more accurately filter out stable occlusion region sequences, thereby more effectively inferring the user's location. It improves the accuracy of product recognition and the vending machine's ability to judge user operations, optimizing user experience and management efficiency.

[0136] Example 9:

[0137] Based on Example 8, the self-service vending machine product recognition method integrating light curtain and AI image recognition determines the physical center jitter frequency of each first-selection occlusion region sequence based on the horizontal and vertical jitter frequencies of the physical center of all occlusion regions in each preset two-dimensional coordinate system, including:

[0138] The square root of the sum of the squares of the horizontal and vertical jitter frequencies of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system is taken as the comprehensive jitter frequency of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system.

[0139] The average of the combined jitter frequencies of the physical centers of all occluded regions in all preset two-dimensional coordinate systems of each first-selection occluded region sequence is taken as the jitter frequency of the physical center of each first-selection occluded region sequence.

[0140] In this embodiment, the comprehensive jitter frequency refers to the value obtained by first squaring the horizontal jitter frequency and the vertical jitter frequency of the physical center of all the occluded regions in each first screening occluded region sequence in each preset two-dimensional coordinate system, and then taking the square root of the result.

[0141] The beneficial effects of the above technology are as follows: By calculating the square root of the sum of the squares of the horizontal and vertical jitter frequencies as the comprehensive jitter frequency, the jitter situation in both directions is comprehensively considered, making the evaluation more comprehensive and balanced. Taking the average of the comprehensive jitter frequencies in all preset two-dimensional coordinate systems as the physical center jitter frequency improves the stability and reliability of jitter frequency determination. This precise calculation method can more accurately reflect the jitter characteristics of the physical center of each first-selection occlusion region sequence. This helps to more rigorously and accurately select occlusion region sequences that meet the requirements, thereby more accurately inferring the vertical location of the user's pick-up and drop-off shelves. It improves the accuracy and efficiency of product recognition in vending machines, providing users with a better service and shopping experience.

[0142] Example 10:

[0143] Based on Example 1, a product recognition method for vending machines that integrates light curtain and AI image recognition is described in step S4: AI image recognition is performed on the product being retrieved each time the vending machine door is opened, based on all predicted vertical locations of the user's shelf and the product retrieval monitoring video, to obtain the product retrieval result. (Refer to...) Figure 3 ,include:

[0144] S401: Perform AI image recognition on the monitoring video of goods picking and placing to determine the continuous time period of all actual users and the type of picking and placing operation for each actual user during the continuous time period;

[0145] S402: Based on all actual user duration periods and the pick-up and put-down operation types for each actual user duration period, identify all actual user pick-up and put-down shelf vertical locations and the pick-up and put-down operation types for each actual user pick-up and put-down shelf vertical location in all presumed user pick-up and put-down shelf vertical locations each time the vending machine door is opened.

[0146] S403: Based on the vertical locations of all actual users picking up and placing items on the shelves and the picking and placing operation types of each actual user picking up and placing items on the shelves, determine all actual users picking up and placing items and the corresponding picking and placing operation types, and use this as the result of picking up and placing items.

[0147] In this embodiment, AI image recognition is used to analyze the monitoring video of goods retrieval and placement to determine the continuous time periods of all actual users and the type of retrieval and placement operations for each actual user during the continuous time period. This means that by applying AI image recognition technology to the monitoring video of goods retrieval and placement, the time periods (i.e., continuous time periods) of all actual users' operations are determined, as well as the type of operation performed by the user in each such time period, whether it is retrieving or placing goods.

[0148] In this embodiment, the actual user duration refers to the time range during which the user actually takes or puts away goods in front of the vending machine.

[0149] In this embodiment, the type of pick-up and put-down operation during the actual user's continuous period is whether it is a pick-up operation or a put-down operation during the period when the actual user is continuously performing the operation.

[0150] In this embodiment, based on the duration of all actual users and the type of pick-up and put-down operations for each actual user during the duration of all actual users, identifying the vertical locations of all actual user pick-up and put-down shelves and the type of pick-up and put-down operations for each actual user pick-up and put-down shelf vertical location from all the estimated user pick-up and put-down shelf vertical locations each time the vending machine door is opened means: based on the duration of the actual users and the type of pick-up and put-down operations determined earlier, accurately identifying the vertical position of the actual user pick-up and put-down shelf and the type of pick-up and put-down operations at that position from the previously estimated user pick-up and put-down shelf vertical locations.

[0151] In this embodiment, the actual user's vertical shelf location is the vertical position area of ​​the shelf corresponding to the actual user's operation of picking up and placing goods (i.e., the shelf layer and approximate orientation, such as left or right).

[0152] In this embodiment, the type of actual user's pick-up and put-down operation in the vertical area of ​​the shelf is the specific operation category of whether the user is picking up or putting down goods in the vertical area of ​​the shelf.

[0153] The beneficial effects of the above technology are as follows: In step S401, AI image recognition is performed on the monitoring video of goods retrieval and placement to determine the actual user's continuous time period and retrieval and placement operation type, providing detailed basic information for subsequent analysis. In step S402, based on the actual user's continuous time period and retrieval and placement operation type, relevant information of the actual user is identified from the inferred vertical location of the user's retrieval and placement on the shelf, improving the accuracy and targeting of the identification. In step S403, based on the above identification results, the actual user's goods retrieval and placement and the corresponding operation type are determined, obtaining comprehensive and accurate goods retrieval and placement results. Overall, this step-by-step processing method can make full use of the monitoring video and the inferred location information to accurately determine the goods retrieval and placement situation, improve the accuracy and reliability of goods recognition in vending machines, and help optimize goods management and improve user experience.

[0154] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for product recognition in a self-service vending machine that integrates light curtain and AI image recognition, characterized in that, include: S1: When the vending machine is in operation, the light curtain installed above and inside the vending machine door is activated. The light emitters of the light curtain are arranged in a matrix, and a receiver is installed on the opposite side of each light emitter. When the light curtain is working normally, the light rays propagating from all groups of light emitters and receivers are parallel or perpendicular to each other. S2: When the vending machine door is opened, at least one vertical location of the user's pick-up and drop-off shelf is determined by using the real-time reception results of all receivers in the light curtain and the planar matrix analysis method. S3: When the vending machine door is opened, the camera installed on the vending machine shelf will be used to obtain video of the goods being picked up and put down. S4: Based on all the predicted vertical locations of the user's pick-up and drop-off shelves and the monitoring video of the product pick-up and drop-off each time the vending machine door is opened, AI image recognition is performed on the product picked up and dropped this time to obtain the product pick-up and drop-off result; Step S2 includes: A dynamic sequence of obstruction areas is generated based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened. The geometric area deviation between each group of adjacent occluded regions in the dynamic sequence of occluded regions is calculated. Adjacent occluded regions whose geometric area deviation does not exceed the area deviation threshold are regarded as the first qualified occluded region group. The occluded regions adjacent to any occluded region in each first qualified occluded region group in the dynamic sequence of occluded regions are merged with the corresponding first qualified occluded region group to obtain multiple first merged occluded region groups. The average geometric area deviation of each pair of occluded regions in each first merged occluded region group is regarded as the geometric area deviation of each first merged occluded region group. All first merged occluded region groups whose geometric area deviation does not exceed the area deviation threshold are regarded as all second qualified occluded region groups. Each second qualified occluded region group continues to be merged and compared with the geometric deviation threshold until there is no new merged occluded region group whose geometric area deviation does not exceed the area deviation threshold. Then, all qualified occluded region groups with a continuous frame count not less than a preset frame count are regarded as the first filtered occluded region sequence. Calculate the physical center jitter amplitude and physical center jitter frequency for each first-screened occlusion region sequence; The first filtered occlusion region sequence, in which the physical center jitter amplitude does not exceed the jitter amplitude threshold and the physical center jitter frequency does not exceed the jitter frequency threshold, is used as the inferred user dwell occlusion region sequence. Each predicted user stays in an occluded area sequence, and the corresponding shelf location on the vertical plane of the shelf is taken as a predicted user's shelf vertical location.

2. The self-service vending machine product recognition method integrating light curtain and AI image recognition according to claim 1, characterized in that, S2: When the vending machine door is opened, at least one presumed vertical location of the user's pick-up / placement shelf is determined using the real-time reception results of all receivers in the light curtain and a planar matrix analysis method, including: S201: Generate a vertical plane matrix based on the position and number of all emitters in the light curtain; S202: When the vending machine door is opened, the occlusion area is located in the vertical plane matrix based on the real-time reception results of all receivers in the light curtain, and the vertical plane occlusion area of ​​the shelf is obtained at each moment after the vending machine door is opened. S203: Determine at least one hypothetical vertical location of the shelf for a user to pick up or place items based on the vertical plane obstruction area of ​​the shelf at all times after the vending machine door is opened.

3. The self-service vending machine product recognition method integrating light curtain and AI image recognition according to claim 1, characterized in that, Calculate the physical center jitter amplitude and physical center jitter frequency for each first-selection occlusion region sequence, including: Based on the two-dimensional coordinates of the physical center of each occluded region in each first-selection occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter amplitude of the physical center of each first-selection occluded region sequence is calculated: ； In the formula, This represents the jitter amplitude of the physical center of the currently calculated first filtered occlusion region sequence. It takes the maximum value among the multiple elements within the parentheses. The first filtered occlusion region sequence currently being calculated The physical center of the obstruction area is at the The x-coordinate value in a preset two-dimensional coordinate system This represents the total number of occluded regions in the first filtered occluded region sequence currently calculated. The first filtered occlusion region sequence currently being calculated The physical center of the obstruction area is at the The ordinate value in a preset two-dimensional coordinate system; Based on the two-dimensional coordinates of the physical center of each occluded region in each first screening occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter frequency of the physical center of each first screening occluded region sequence is calculated.

4. The product recognition method for a vending machine integrating light curtain and AI image recognition according to claim 3, characterized in that, Based on the two-dimensional coordinates of the physical center of each occluded region in each first-selection occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the jitter frequency of the physical center of each first-selection occluded region sequence is calculated, including: Determine the difference in the horizontal and vertical coordinates of the physical centers of all adjacent groups of occluded regions in each first screening occluded region sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, and generate the difference sequence of the horizontal and vertical coordinates of each first screening occluded region sequence in each preset two-dimensional coordinate system; Based on the difference sequences of the horizontal and vertical coordinates of each first-selected occlusion region sequence in each preset two-dimensional coordinate system, the physical center of all occlusion regions in each first-selected occlusion region sequence is generated relative to an integer period in each preset two-dimensional coordinate system. The horizontal autocorrelation function and the vertical autocorrelation function: ； ； In the formula, The physical center of all occluded regions in the currently calculated first filtered occluded region sequence is at the 1st... In a preset two-dimensional coordinate system, relative to the period The horizontal autocorrelation function, The first filtered occlusion region sequence currently being calculated The physical center of the first obstruction area and the first The displacement between the physical centers of the occlusion regions in the __ Projected displacement along the horizontal axis of a preset two-dimensional coordinate system The first filtered occlusion region sequence currently being calculated The physical center of the first obstruction area and the first The displacement between the physical centers of the occlusion regions in the __ Projected displacement along the horizontal axis of a preset two-dimensional coordinate system The first filtered occlusion region sequence currently being calculated The physical center of the first obstruction area and the first The displacement between the physical centers of the occlusion regions in the __ Projected displacement along the vertical axis of a preset two-dimensional coordinate system The first filtered occlusion region sequence currently being calculated The physical center of the first obstruction area and the first The displacement between the physical centers of the occlusion regions in the __ The projected displacement along the vertical axis of a preset two-dimensional coordinate system; The physical center of all occluded regions in each first-selected occluded region sequence is relative to an integer period in each preset two-dimensional coordinate system. The horizontal and vertical autocorrelation functions are used to determine the physical center jitter frequency of each first-screened occlusion region sequence.

5. The self-service vending machine product recognition method integrating light curtain and AI image recognition according to claim 4, characterized in that, The physical center of all occluded regions in each first-selected occluded region sequence is relative to an integer period in each preset two-dimensional coordinate system. The horizontal and vertical autocorrelation functions are used to determine the physical center jitter frequency of each first-selection occlusion region sequence, including: The physical center of all occluded regions in each first-selection occluded region sequence is relative to an integer period in each preset two-dimensional coordinate system. In the transverse autocorrelation function, the integer period corresponding to the peak value of the transverse autocorrelation function. The reciprocal of the first filtering occlusion region sequence is used as the lateral jitter frequency of the physical center of all occlusion regions in each preset two-dimensional coordinate system; The physical center of all occluded regions in each first-selection occluded region sequence is relative to an integer period in each preset two-dimensional coordinate system. In the longitudinal autocorrelation function, the integer period corresponding to the longitudinal autocorrelation function at the peak value. The reciprocal of the first filtering occlusion region sequence is used as the vertical jitter frequency of the physical center of all occlusion regions in each preset two-dimensional coordinate system; Based on the horizontal and vertical jitter frequencies of the physical center of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system, the jitter frequency of the physical center of each first-selection occluded region sequence is determined.

6. The self-service vending machine product recognition method integrating light curtain and AI image recognition according to claim 5, characterized in that, Based on the horizontal and vertical jitter frequencies of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system, the jitter frequency of the physical center of each first-selection occluded region sequence is determined, including: The square root of the sum of the squares of the horizontal and vertical jitter frequencies of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system is taken as the comprehensive jitter frequency of the physical centers of all occluded regions in each first-selection occluded region sequence in each preset two-dimensional coordinate system. The average of the combined jitter frequencies of the physical centers of all occluded regions in all preset two-dimensional coordinate systems of each first-selection occluded region sequence is taken as the jitter frequency of the physical center of each first-selection occluded region sequence.

7. The self-service vending machine product recognition method integrating light curtain and AI image recognition according to claim 1, characterized in that, S4: Based on all the inferred vertical locations of the shelves and the monitoring video of product retrieval each time the vending machine door is opened, AI image recognition is performed on the products retrieved in this instance to obtain the product retrieval results, including: S401: Perform AI image recognition on the monitoring video of goods picking and placing to determine the continuous time period of all actual users and the type of picking and placing operation for each actual user during the continuous time period; S402: Based on all actual user duration periods and the pick-up and put-down operation types for each actual user duration period, identify all actual user pick-up and put-down shelf vertical locations and the pick-up and put-down operation types for each actual user pick-up and put-down shelf vertical location in all presumed user pick-up and put-down shelf vertical locations each time the vending machine door is opened. S403: Based on the vertical locations of all actual users picking up and placing items on the shelves and the picking and placing operation types of each actual user picking up and placing items on the shelves, determine all actual users picking up and placing items and the corresponding picking and placing operation types, and use this as the result of picking up and placing items.

Images