Self-service vending machine commodity identification method fusing light curtain and AI image identification

Abstract

The invention relates to the technical field of commodity recognition, and particularly discloses a self-service vending machine commodity recognition method fusing light curtains and AI image recognition, and the method comprises the steps: starting the light curtains disposed at the upper part and the inner side of a door of a vending machine when the vending machine starts a working state; when the door of the vending machine is opened, determining at least one vertical location for speculating the user to pick and place the goods shelf by using real-time receiving results of all receivers in the light curtain and a plane matrix analysis method; acquiring a commodity pick-and-place monitoring video by using a camera mounted on a goods shelf of the vending machine; carrying out AI image recognition on the goods which are taken and placed this time based on all the speculated vertical positions of the goods shelf taken and placed by the user and the goods taking and placing monitoring video when the door of the vending machine is opened each time, and obtaining a goods taking and placing result this time; the method and the device are used for accurately and effectively identifying taking and placing conditions of commodities, remarkably improving commodity management efficiency and accuracy of the self-service vending machine and bringing more convenient and high-quality shopping experience to users.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a commodity recognition method for a self-service vending machine integrating light curtain and AI image recognition. Background Art

[0002] At present, in the current consumption scenario, self-service vending machines, as a convenient retail method, are increasingly appearing in people's lives. In order to achieve more efficient and accurate commodity management and transactions, the identification technology of commodities in self-service vending machines is constantly developing. Light curtain technology and AI image recognition technology have certain applications in their respective fields, but the integration of the two for self-service vending machine commodity identification is still in the exploratory stage. Light curtain technology detects the position and movement of objects by blocking and receiving light, and has the characteristics of fast response speed and high accuracy. AI image recognition technology, with its powerful learning and analysis capabilities, can process and understand complex images. By installing the camera on the shelf of the self-service vending machine, the real-time status of the commodity and the user's operation screen can be obtained.

[0003] However, there are certain limitations in the use of AI image recognition technology alone in the identification of goods in self-service vending machines. Relying solely on AI image recognition may be affected by factors such as light and angle, resulting in inaccurate recognition results. When processing fast product pick-up and placement operations, it may not be possible to obtain product information in a timely and accurate manner. For some complex pick-up and placement actions and special situations of users, the recognition effect of existing technologies is not good.

[0004] Therefore, the present invention proposes a self-service vending machine commodity recognition method that integrates light curtain and AI image recognition. Summary of the invention

[0005] The present invention provides a self-service vending machine commodity identification method integrating light curtain and AI image recognition. In step S1, the activated light curtain constructs a complete infrastructure for subsequent monitoring through the light emitters and receivers arranged in a matrix, and the parallel or vertical light propagation, thereby ensuring the comprehensiveness and accuracy of the monitoring. Step S2 can accurately infer the vertical position of the user's pick-up and placement of the shelf based on the real-time reception result of the light curtain receiver, using the plane matrix analysis method, and provide key location information for subsequent commodity identification. Step S3 obtains the monitoring video when the vending machine door is opened, and prepares intuitive and rich image data for the identification of commodity placement. Step S4 combines the inferred vertical position and the monitoring video for AI image recognition, which greatly improves the recognition accuracy and reliability of the commodity placement results. Overall, this set of processes integrates light curtain and AI image recognition technology, can accurately and effectively identify the placement of commodities, significantly improves the commodity management efficiency and accuracy of the self-service vending machine, and brings users a more convenient and high-quality shopping experience.

[0006] The present invention provides a self-service vending machine commodity recognition method integrating light curtain and AI image recognition, comprising:

[0007] S1: When the vending machine starts working, the light curtain installed on the upper and inner sides of the door of the vending machine is turned on, wherein the light emitters of the light curtain are arranged in a matrix form, and a receiver is installed on the opposite side of each light emitter. When the light curtain works normally, the light rays transmitted from all groups of light emitters and receivers are parallel or perpendicular to each other;

[0008] S2: When the vending machine door is opened, the real-time receiving results of all receivers in the light curtain and the plane matrix analysis method are used to determine at least one estimated vertical position of the user to pick up and place the shelf;

[0009] S3: When the door of the vending machine is opened, the camera installed on the shelf of the vending machine is used to obtain the monitoring video of the product placement;

[0010] S4: Based on all the inferred vertical positions of the shelves where users pick up and place goods each time the vending machine door is opened and the monitoring videos of the goods being picked up and placed, AI image recognition is performed on the goods being picked up and placed to obtain the results of the goods being picked up and placed.

[0011] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition, S2: when the door of the vending machine is opened, at least one inferred vertical position of the user's shelf is determined by using the real-time receiving results of all receivers in the light curtain and the plane matrix analysis method, including:

[0012] S201: Generate a vertical plane matrix based on the setting positions and numbers of all light emitters in the light curtain;

[0013] S202: When the vending machine door is opened, the blocked area is located in the vertical plane matrix based on the real-time receiving results of all receivers in the light curtain, and the vertical plane blocked area of ​​the shelf at each moment after the vending machine door is opened is obtained;

[0014] S203: Determine at least one estimated vertical location of the shelf where the user picks up or places the product based on the vertical plane occlusion area of ​​the shelf at all times after the vending machine door is opened.

[0015] Preferably, the method for identifying goods for a self-service vending machine that integrates light curtain and AI image recognition, S203: determining at least one estimated vertical position of a shelf where a user picks up or places goods based on the vertical plane occlusion area of ​​the shelf at all times after the vending machine door is opened, comprises:

[0016] Generate a dynamic sequence of occlusion areas based on the vertical plane occlusion areas of the shelves at all times after the vending machine door is opened;

[0017] Based on the geometric dynamic features of the dynamic sequence of the occluded area, at least one inferred vertical position of the user picking up and placing the shelf is analyzed.

[0018] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition analyzes at least one inferred vertical position of the user's shelf pick-up and placement based on the geometric dynamic features of the dynamic sequence of the occluded area, including:

[0019] Counting the geometric area of ​​each occlusion area in the occlusion area dynamic sequence, and determining the physical center of each occlusion area in the occlusion area dynamic sequence;

[0020] In the dynamic sequence of the occluded area, a continuous sequence of occluded areas is selected, in which the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold, and the number of continuous frames is not less than the preset number of frames, and the sequence is used as the sequence of the occluded area where the user is estimated to stay;

[0021] The shelf position corresponding to each sequence of occluded areas where the user is estimated to stay on the shelf vertical plane is regarded as a vertical position of the shelf where the user is estimated to pick up and place.

[0022] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition selects a continuous sequence of occlusion areas in the dynamic sequence of occlusion areas, in which the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold, and the number of continuous frames is not less than a preset number of frames, as the sequence of occlusion areas where the user is estimated to stay, including:

[0023] The geometric area deviation between each group of adjacent occlusion areas in the occlusion area dynamic sequence is calculated, and the adjacent occlusion areas whose geometric area deviation does not exceed the area deviation threshold are regarded as the first qualified occlusion area group, and the occlusion areas adjacent to any occlusion area in each first qualified occlusion area group in the occlusion area dynamic sequence are merged with the corresponding first qualified occlusion area group to obtain multiple first merged occlusion area groups, and the average of the geometric area deviations of the two occlusion areas in each first merged occlusion area group is regarded as the geometric area deviation of each first merged occlusion area group, and all first merged occlusion area groups whose geometric area deviation does not exceed the area deviation threshold are regarded as all second qualified occlusion area groups, and each second qualified occlusion area group is continuously merged and compared with the geometric deviation threshold until there is no updated merged occlusion area group whose geometric area deviation does not exceed the area deviation threshold, then all currently obtained qualified occlusion area groups whose continuous frame number is not less than the preset frame number are regarded as the first screened occlusion area sequence;

[0024] Calculating the physical center jitter amplitude and physical center jitter frequency of each first screened occlusion region sequence;

[0025] The first screened occlusion region sequence in which all physical center jitter amplitudes do not exceed the jitter amplitude threshold and the physical center jitter frequency does not exceed the jitter frequency threshold is regarded as the occlusion region sequence in which the user is estimated to stay.

[0026] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition calculates the physical center jitter amplitude and physical center jitter frequency of each first screening occlusion area sequence, including:

[0027] Based on the two-dimensional coordinate value of the physical center of each occlusion area in each first screening occlusion area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter amplitude of each first screening occlusion area sequence is calculated:

[0028]

[0029] Where J is the physical center jitter amplitude of the first filtered occlusion area sequence currently calculated, max() is the maximum value of multiple elements in brackets, and x ij is the horizontal coordinate value of the physical center of the i-th occlusion area in the first screening occlusion area sequence currently calculated in the j-th preset two-dimensional coordinate system, n is the total number of occlusion areas in the first screening occlusion area sequence currently calculated, y ij is the vertical coordinate value of the physical center of the i-th occlusion area in the first screened occlusion area sequence currently calculated in the j-th preset two-dimensional coordinate system;

[0030] Based on the two-dimensional coordinate value of the physical center of each blocking area in each first screening blocking area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter frequency of each first screening blocking area sequence is calculated.

[0031] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition calculates the physical center jitter frequency of each first screening occlusion area sequence based on the two-dimensional coordinate value of the physical center of each occlusion area in each first screening occlusion area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, including:

[0032] Determine the horizontal coordinate difference and the vertical coordinate difference of the physical centers of all groups of adjacent blocking areas in each first screening blocking area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, and generate a horizontal coordinate difference sequence and a vertical coordinate difference sequence of each first screening blocking area sequence in each preset two-dimensional coordinate system;

[0033] Based on the horizontal coordinate difference sequence and the vertical coordinate difference sequence of each first screened occlusion area sequence in each preset two-dimensional coordinate system, the horizontal autocorrelation function and the vertical autocorrelation function of the physical center of all occlusion areas of each first screened occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system are generated:

[0034]

[0035] In the formula, R xxj (k) is the lateral autocorrelation function of the physical center of all the occluded areas of the first screened occluded area sequence currently calculated in the jth preset two-dimensional coordinate system relative to the period k, Δx ij is the projection displacement of the displacement between the physical center of the i-th occlusion area and the physical center of the i+1-th occlusion area in the first screening occlusion area sequence currently calculated in the abscissa axis direction of the j-th preset two-dimensional coordinate system, Δx (i+k)j is the projection displacement of the displacement between the physical center of the i+kth occlusion area and the physical center of the i+k+1th occlusion area in the first screening occlusion area sequence currently calculated in the abscissa axis direction of the jth preset two-dimensional coordinate system, Δy ij is the projection displacement of the displacement between the physical center of the i-th occlusion area and the physical center of the i+1-th occlusion area in the first screening occlusion area sequence currently calculated in the ordinate axis direction of the j-th preset two-dimensional coordinate system, Δy (i+k)j is the projection displacement of the displacement between the physical center of the i+kth occlusion area and the physical center of the i+k+1th occlusion area in the first screened occlusion area sequence currently calculated in the vertical coordinate axis direction of the jth preset two-dimensional coordinate system;

[0036] Based on the transverse autocorrelation function and the longitudinal autocorrelation function of the physical centers of all occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system, the physical center jitter frequency of each first screening occlusion area sequence is determined.

[0037] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition determines the physical center jitter frequency of each first screening occlusion area sequence based on the horizontal autocorrelation function and the vertical autocorrelation function of the physical center of all occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system, including:

[0038] The reciprocal of the integer period k corresponding to the lateral autocorrelation function when the peak value appears in the lateral autocorrelation function of the physical center of all the occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system is regarded as the lateral jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system;

[0039] The reciprocal of the integer period k corresponding to the longitudinal autocorrelation function when the peak value appears in the longitudinal autocorrelation function of the physical center of all the occluded areas of each first screened occluded area sequence relative to the integer period k in each preset two-dimensional coordinate system is regarded as the longitudinal jitter frequency of the physical center of all the occluded areas of each first screened occluded area sequence in each preset two-dimensional coordinate system;

[0040] Based on the lateral jitter frequency and the longitudinal jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system, the physical center jitter frequency of each first screening occlusion area sequence is determined.

[0041] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition determines the physical center jitter frequency of each first screening occlusion area sequence based on the lateral jitter frequency and the longitudinal jitter frequency of the physical center of all occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system, including:

[0042] The square root of the sum of the squares of the horizontal jitter frequency and the vertical jitter frequency of the physical centers of all the occlusion areas of each first screened occlusion area sequence in each preset two-dimensional coordinate system is used as the comprehensive jitter frequency of the physical centers of all the occlusion areas of each first screened occlusion area sequence in each preset two-dimensional coordinate system;

[0043] The average value of the comprehensive jitter frequencies of the physical centers of all the occlusion regions of each first screening occlusion region sequence in all the preset two-dimensional coordinate systems is taken as the physical center jitter frequency of each first screening occlusion region sequence.

[0044] Preferably, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition, S4: performing AI image recognition on the commodity to be taken and placed this time based on all the inferred vertical positions of the shelf and commodity taking and placing monitoring videos when the vending machine door is opened each time, and obtaining the commodity taking and placing results this time, including:

[0045] S401: Perform AI image recognition on the surveillance video of product placement and take-out to determine all actual user durations and the type of placement and take-out operation in each actual user duration;

[0046] S402: Based on all actual user durations and the pick-and-place operation type of each actual user duration, identifying all actual user vertical positions for picking up and placing the shelf and the pick-and-place operation type of each actual user vertical position for picking up and placing the shelf in all the inferred user vertical positions for picking up and placing the shelf each time the vending machine door is opened;

[0047] S403: Based on the vertical positions of all actual users picking up and placing goods and the picking and placing operation types of each actual user picking up and placing goods at the vertical positions of the shelf, all actual users picking up and placing goods and the corresponding picking and placing operation types are determined as the picking and placing result of this time.

[0048] The beneficial effects of the present invention compared with the prior art are as follows: in step S1, the activated light curtain builds a complete infrastructure for subsequent monitoring through the matrix-arranged light emitters and receivers, as well as parallel or vertical light propagation, to ensure the comprehensiveness and accuracy of the monitoring. Step S2 can accurately infer the vertical position of the user's pick-up and placement of the shelf based on the real-time reception results of the light curtain receiver, using the plane matrix analysis method, to provide key location information for subsequent commodity identification. Step S3 obtains the monitoring video when the vending machine door is opened, and prepares intuitive and rich image data for the identification of commodity placement. Step S4 combines the inferred vertical position and monitoring video for AI image recognition, which greatly improves the recognition accuracy and reliability of the commodity placement results. Overall, this set of processes integrates light curtains and AI image recognition technology, can accurately and effectively identify the placement of commodities, significantly improves the commodity management efficiency and accuracy of self-service vending machines, and brings users a more convenient and high-quality shopping experience.

[0049] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 This is a flow chart of a method for identifying goods in a self-service vending machine by integrating light curtain and AI image recognition in an embodiment of the present invention;

[0053] Figure 2 is a flowchart of a specific execution method of step S2 in an embodiment of the present invention;

[0054] Figure 3 4 is a flowchart of a specific execution method of step S4 in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0056] Embodiment 1:

[0057] The present invention provides a self-service vending machine commodity recognition method integrating light curtain and AI image recognition, referring to Figure 1 ,include:

[0058] S1: When the vending machine starts working, the light curtain installed on the upper and inner sides of the door of the vending machine is turned on, wherein the light emitters of the light curtain are arranged in a matrix form, and a receiver is installed on the opposite side of each light emitter. When the light curtain works normally, the light rays transmitted from all groups of light emitters and receivers are parallel or perpendicular to each other;

[0059] S2: When the vending machine door is opened, the real-time receiving results of all receivers in the light curtain and the plane matrix analysis method are used to determine at least one estimated vertical position of the user to pick up and place the shelf;

[0060] S3: When the door of the vending machine is opened, the camera installed on the shelf of the vending machine is used to obtain the monitoring video of the product placement;

[0061] S4: Based on all the inferred vertical positions of the shelves where users pick up and place goods each time the vending machine door is opened and the monitoring videos of the goods being picked up and placed, AI image recognition is performed on the goods being picked up and placed to obtain the results of the goods being picked up and placed.

[0062] In this embodiment, the real-time receiving result of the receiver refers to the light state information received by the receiver in the light curtain at each moment when the vending machine is working, such as which light is blocked and which light is received normally.

[0063] In this embodiment, inferring the vertical position of the shelf where the user takes or puts the goods refers to the vertical position area of ​​the shelf where the user may take or put back the goods, inferred by the real-time reception results of the light curtain receiver and related analysis methods.

[0064] In this embodiment, since the types of goods are relatively fixed and are usually placed according to the layers and areas of the shelves in the container, the replenisher will sort the goods during regular or irregular replenishment to ensure the regularity of the placement of the goods. Therefore, the plane matrix technology is used to accurately sense and collect the location information of the customer taking the goods (that is, the layer number and approximate location of the goods).

[0065] In this embodiment, the commodity taking and placing monitoring video refers to the video recorded by the camera installed on the shelf when the door of the vending machine is opened, recording the user's action of taking or putting back the commodity.

[0066] In this embodiment, the result of taking or putting goods this time refers to the specific information about the user taking or putting back the goods this time, which is finally determined based on various data and analysis when the vending machine door is opened this time, including which goods are taken or put back and the corresponding operation type.

[0067] The beneficial effects of the above technology are as follows: in step S1, the activated light curtain builds a complete infrastructure for subsequent monitoring through the matrix-arranged light emitters and receivers, as well as parallel or vertical light propagation, ensuring the comprehensiveness and accuracy of the monitoring. Step S2 can accurately infer the vertical position of the user's pick-up and placement of shelves based on the real-time reception results of the light curtain receiver, using the plane matrix analysis method, and provide key location information for subsequent product identification. Step S3 obtains the monitoring video when the vending machine door is opened, and prepares intuitive and rich image data for the identification of product placement. Step S4 combines the inferred vertical position and monitoring video for AI image recognition, which greatly improves the recognition accuracy and reliability of the product placement results. Overall, this set of processes integrates light curtains and AI image recognition technology, which can accurately and effectively identify the placement of goods, significantly improve the efficiency and accuracy of product management of self-service vending machines, and bring users a more convenient and high-quality shopping experience.

[0068] Embodiment 2:

[0069] Based on Example 1, a self-service vending machine commodity recognition method integrating light curtain and AI image recognition, S2: When the door of the vending machine is opened, the real-time reception results of all receivers in the light curtain and the plane matrix analysis method are used to determine at least one inferred vertical position of the user to take and place the shelf, refer to Figure 2 ,include:

[0070] S201: Generate a vertical plane matrix based on the setting positions and numbers of all light emitters in the light curtain;

[0071] S202: When the vending machine door is opened, the blocked area is located in the vertical plane matrix based on the real-time receiving results of all receivers in the light curtain, and the vertical plane blocked area of ​​the shelf at each moment after the vending machine door is opened is obtained;

[0072] S203: Determine at least one estimated vertical location of the shelf where the user picks up or places the product based on the vertical plane occlusion area of ​​the shelf at all times after the vending machine door is opened.

[0073] In this embodiment, the light emitter installation position and number refer to the specific installation position of each light emitter in the light curtain inside the vending machine and the identification number assigned to each light emitter.

[0074] In this embodiment, generating a vertical plane matrix based on the setting positions and numbers of all light emitters in the light curtain means constructing a matrix data structure that can describe their distribution on the vertical plane according to the specific positions and numbering information of all light emitters in the light curtain.

[0075] In this embodiment, the blocked area is located in the vertical plane matrix based on the real-time receiving results of all receivers in the light curtain, and the vertical plane blocked area of ​​the shelf at each moment after the vending machine door is opened is obtained. This means that the real-time data received by the receivers in the light curtain at each moment is used to determine the area where the light is blocked in the constructed vertical plane matrix, thereby obtaining the blocked area range on the vertical plane of the shelf corresponding to each moment after the vending machine door is opened.

[0076] In this embodiment, the shelf vertical plane shielding area refers to an area on the vertical plane of the shelf of the vending machine that cannot be illuminated by the light because an object (such as a user's hand or body) blocks the light of the light curtain.

[0077] The beneficial effects of the above technology are as follows: in step S201, a vertical plane matrix is ​​generated based on the light source setting position and number, providing an accurate reference framework for the subsequent positioning of the blocked area. Step S202 locates the blocked area in the vertical plane matrix through the real-time reception result of the receiver, and can capture the shelf blockage situation when the vending machine door is opened in real time and accurately. Step S203 determines the vertical position of the user's pick-up and placement shelf based on the blocked area at all times, thereby improving the accuracy and reliability of the position estimation. On the whole, this step-by-step processing method can effectively utilize the light curtain information and accurately infer the vertical position of the user's pick-up and placement shelf, providing an important basis for subsequent product identification work, and helping to improve the accuracy and efficiency of self-service vending machine product identification.

[0078] Embodiment 3:

[0079] Based on Example 2, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition, S203: determining at least one estimated vertical position of the shelf where the user picks up and places the product based on the vertical plane occlusion area of ​​the shelf at all times after the vending machine door is opened, including:

[0080] Generate a dynamic sequence of occlusion areas based on the vertical plane occlusion areas of the shelves at all times after the vending machine door is opened;

[0081] Based on the geometric dynamic features of the dynamic sequence of the occluded area, at least one inferred vertical position of the user picking up and placing the shelf is analyzed.

[0082] In this embodiment, the occlusion region dynamic sequence refers to a series of continuous state records formed by the occlusion region on the vertical plane of the shelf of the vending machine changing over time within a period of time.

[0083] In this embodiment, the geometric dynamic feature refers to the dynamic characteristics and regularities of the occluded area as its geometric attributes such as shape, size, and position change over time.

[0084] The beneficial effects of the above technology are: by generating a dynamic sequence of occluded areas, the changes in the vertical plane occluded area of ​​the shelf after the vending machine door is opened can be fully and dynamically presented. Based on the analysis of the geometric dynamic characteristics of the dynamic sequence of the occluded area, the vertical location of the user's access to the shelf can be more deeply and accurately inferred. This method makes full use of the dynamic information of the occluded area and improves the accuracy and reliability of location inference. It helps to identify products more accurately and improve the efficiency and accuracy of product management in self-service vending machines.

[0085] Embodiment 4:

[0086] On the basis of Example 3, a self-service vending machine commodity recognition method integrating light curtain and AI image recognition analyzes at least one inferred vertical position of the user's shelf based on the geometric dynamic features of the dynamic sequence of the occluded area, including:

[0087] Counting the geometric area of ​​each occlusion area in the occlusion area dynamic sequence, and determining the physical center of each occlusion area in the occlusion area dynamic sequence;

[0088] In the dynamic sequence of the occluded area, a continuous sequence of occluded areas is selected, in which the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold, and the number of continuous frames is not less than the preset number of frames, and the sequence is used as the sequence of the occluded area where the user is estimated to stay;

[0089] The shelf position corresponding to each sequence of occluded areas where the user is estimated to stay on the shelf vertical plane is regarded as a vertical position of the shelf where the user is estimated to pick up and place.

[0090] In this embodiment, the geometric area of ​​the shielding region refers to the area size on a two-dimensional plane occupied by a specific shielding region on a vertical plane of the shelf of the vending machine.

[0091] In this embodiment, the geometric area deviation is an indicator used to measure the degree of difference between the geometric areas of occluded regions at adjacent or different moments.

[0092] In this embodiment, the area deviation threshold is a pre-set critical value used to determine whether the geometric area deviation is acceptable.

[0093] In this embodiment, the physical center jitter amplitude indicates the jitter amplitude of the physical center position of the occlusion area within a certain period of time or between different occlusion areas.

[0094] In this embodiment, the jitter amplitude threshold is an upper limit value of a specified physical center jitter amplitude.

[0095] In this embodiment, the physical center jitter frequency refers to the frequency at which the physical center position of the occlusion area periodically jitters within a certain period of time.

[0096] In this embodiment, the jitter frequency threshold is an upper limit value of the set physical center jitter frequency.

[0097] In this embodiment, the preset number of frames is a number of frames set in advance for determining the continuity of the occluded area.

[0098] In this embodiment, the continuous sequence of occlusion regions refers to an arrangement of a series of continuous occlusion regions having certain association or similarity.

[0099] In this embodiment, inferring the shelf position corresponding to the sequence of occlusion areas where the user stays on the vertical plane of the shelf refers to the specific shelf position corresponding to the sequence of occlusion areas where the user stays, inferred based on analysis, on the vertical plane of the shelf.

[0100] The beneficial effects of the above technology are: by counting the geometric area of ​​the occluded area and determining the physical center, key quantitative data is provided for subsequent analysis. The continuous sequence of occluded areas that meet specific conditions is screened out as the sequence of occluded areas where the user is estimated to stay, which improves the accuracy and reliability of the speculation. The shelf location corresponding to the sequence of occluded areas where the user is estimated to stay is used as the vertical location of the shelf where the user is estimated to take and put, which can more accurately locate the user's possible operation position. This analysis method based on detailed geometric dynamic features effectively eliminates interference factors and improves the accuracy of location speculation. It helps to identify goods more accurately and improve the service quality and management efficiency of self-service vending machines.

[0101] Embodiment 5:

[0102] On the basis of Example 4, a method for identifying goods in a self-service vending machine by integrating light curtain and AI image recognition is used to screen out a continuous sequence of occlusion areas in the dynamic sequence of occlusion areas, in which the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold, and the number of consecutive frames is not less than a preset number of frames, as a sequence of occlusion areas where the user is estimated to stay, including:

[0103] The geometric area deviation between each group of adjacent occlusion areas in the occlusion area dynamic sequence is calculated, and the adjacent occlusion areas whose geometric area deviation does not exceed the area deviation threshold are regarded as the first qualified occlusion area group, and the occlusion areas adjacent to any occlusion area in each first qualified occlusion area group in the occlusion area dynamic sequence are merged with the corresponding first qualified occlusion area group to obtain multiple first merged occlusion area groups, and the average of the geometric area deviations of the two occlusion areas in each first merged occlusion area group is regarded as the geometric area deviation of each first merged occlusion area group, and all first merged occlusion area groups whose geometric area deviation does not exceed the area deviation threshold are regarded as all second qualified occlusion area groups, and each second qualified occlusion area group is continuously merged and compared with the geometric deviation threshold until there is no updated merged occlusion area group whose geometric area deviation does not exceed the area deviation threshold, then all currently obtained qualified occlusion area groups whose continuous frame number is not less than the preset frame number are regarded as the first screened occlusion area sequence;

[0104] Calculating the physical center jitter amplitude and physical center jitter frequency of each first screened occlusion region sequence;

[0105] The first screened occlusion region sequence in which all physical center jitter amplitudes do not exceed the jitter amplitude threshold and the physical center jitter frequency does not exceed the jitter frequency threshold is regarded as the occlusion region sequence in which the user is estimated to stay.

[0106] In this embodiment, the first merged occlusion region group includes three occlusion regions.

[0107] In this embodiment, the method for calculating the geometric area deviation of two occlusion areas may be: first, obtain the geometric areas of the two occlusion areas respectively, then calculate the difference between the two areas, and then divide the difference by the area of ​​one of the occlusion areas (or the average of the two areas). The result obtained is the geometric area deviation of the two occlusion areas.

[0108] In this embodiment, the process of neighbor merging and geometric deviation threshold comparison is the same as the implementation method of the process of obtaining the first merged occlusion area group, calculating the geometric area deviation of the first merged occlusion area group, and comparing the first merged occlusion area group with the area deviation threshold to obtain the second qualified occlusion area group.

[0109] The beneficial effects of the above technology are: by gradually merging and screening the occlusion area groups, the areas where the geometric area deviation meets the requirements can be determined more accurately, thereby improving the accuracy of screening. The physical center jitter amplitude and frequency of each screening area sequence are calculated to evaluate the stability of the occlusion area from more dimensions. The final sequence of occlusion areas where the user is estimated to stay has higher reliability and accuracy. This sophisticated screening and analysis method can effectively eliminate interference and unstable areas and more accurately infer the user's stay location. It helps to improve the accuracy of product identification and the management efficiency of self-service vending machines, providing users with a better service experience.

[0110] Embodiment 6:

[0111] On the basis of Example 5, a self-service vending machine commodity recognition method integrating light curtain and AI image recognition calculates the physical center jitter amplitude and physical center jitter frequency of each first screening occlusion area sequence, including:

[0112] Based on the two-dimensional coordinate value of the physical center of each occlusion area in each first screening occlusion area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter amplitude of each first screening occlusion area sequence is calculated:

[0113]

[0114] Where J is the physical center jitter amplitude of the first filtered occlusion area sequence currently calculated, max() is the maximum value of multiple elements in brackets, and x ij is the horizontal coordinate value of the physical center of the i-th occlusion area in the first screening occlusion area sequence currently calculated in the j-th preset two-dimensional coordinate system, n is the total number of occlusion areas in the first screening occlusion area sequence currently calculated, y ij is the vertical coordinate value of the physical center of the i-th occlusion area in the first screened occlusion area sequence currently calculated in the j-th preset two-dimensional coordinate system;

[0115] Based on the two-dimensional coordinate value of the physical center of each blocking area in each first screening blocking area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter frequency of each first screening blocking area sequence is calculated.

[0116] In this embodiment, the origins of all the preset two-dimensional coordinate systems coincide and the horizontal axes of all the preset two-dimensional coordinate systems are evenly distributed. For example, there are three preset two-dimensional coordinate systems, and the angles between the two horizontal axes of the three preset two-dimensional coordinate systems are 120 degrees.

[0117] The beneficial effects of the above technology are: the above formula can accurately calculate the physical center jitter amplitude of each first screened occlusion area sequence, and quantify the degree of position change of the physical center. The coordinate values ​​of each occlusion area in different preset two-dimensional coordinate systems are taken into account, making the calculation of the jitter amplitude more comprehensive and accurate. At the same time, it provides a basis for calculating the physical center jitter frequency. It helps to more accurately evaluate the stability and reliability of the occlusion area sequence. In this way, the occlusion area sequence where the user is presumed to stay can be screened out more accurately, improving the accuracy of product recognition and the operating efficiency of the self-service vending machine.

[0118] Embodiment 7:

[0119] On the basis of Example 6, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition calculates the physical center jitter frequency of each first screening occlusion area sequence based on the two-dimensional coordinate value of the physical center of each occlusion area in each first screening occlusion area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, including:

[0120] Determine the horizontal coordinate difference and the vertical coordinate difference of the physical centers of all groups of adjacent blocking areas in each first screening blocking area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, and generate a horizontal coordinate difference sequence and a vertical coordinate difference sequence of each first screening blocking area sequence in each preset two-dimensional coordinate system;

[0121] Based on the horizontal coordinate difference sequence and the vertical coordinate difference sequence of each first screened occlusion area sequence in each preset two-dimensional coordinate system, the horizontal autocorrelation function and the vertical autocorrelation function of the physical center of all occlusion areas of each first screened occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system are generated:

[0122]

[0123] In the formula, R xxj (k) is the lateral autocorrelation function of the physical center of all the occluded areas of the first screened occluded area sequence currently calculated in the jth preset two-dimensional coordinate system relative to the period k, Δx ij is the projection displacement of the displacement between the physical center of the i-th occlusion area and the physical center of the i+1-th occlusion area in the first screening occlusion area sequence currently calculated in the abscissa axis direction of the j-th preset two-dimensional coordinate system, Δx (i+k)j is the projection displacement of the displacement between the physical center of the i+kth occlusion area and the physical center of the i+k+1th occlusion area in the first screening occlusion area sequence currently calculated in the abscissa axis direction of the jth preset two-dimensional coordinate system, Δy ijis the projection displacement of the displacement between the physical center of the i-th occlusion area and the physical center of the i+1-th occlusion area in the first screening occlusion area sequence currently calculated in the ordinate axis direction of the j-th preset two-dimensional coordinate system, Δy (i+k)j is the projection displacement of the displacement between the physical center of the i+kth occlusion area and the physical center of the i+k+1th occlusion area in the first screened occlusion area sequence currently calculated in the vertical coordinate axis direction of the jth preset two-dimensional coordinate system;

[0124] Based on the transverse autocorrelation function and the longitudinal autocorrelation function of the physical centers of all occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system, the physical center jitter frequency of each first screening occlusion area sequence is determined.

[0125] In this embodiment, the horizontal coordinate difference sequence refers to the difference in coordinate values ​​of the physical centers of adjacent occlusion areas in each first screened occlusion area sequence on the horizontal axis of the preset two-dimensional coordinate system, and is arranged in time sequence or in the order of occlusion areas to form a sequence.

[0126] In this embodiment, the ordinate difference sequence refers to the difference in coordinate values ​​of the physical centers of adjacent occlusion areas in each first screened occlusion area sequence on the ordinate axis of the preset two-dimensional coordinate system, and is arranged in chronological order or in the order of occlusion areas to form a sequence.

[0127] In this embodiment, the transverse autocorrelation function (longitudinal autocorrelation function) is a function used to describe the similarity or correlation between the physical centers of two groups of partially continuous occlusion areas formed with an interval period k in each first screening occlusion area sequence in the direction of the horizontal axis (or the vertical axis) of a preset two-dimensional coordinate system.

[0128] The beneficial effects of the above technology are: by determining the horizontal coordinate difference and the vertical coordinate difference, a difference sequence is generated, which provides basic data for the subsequent calculation of the autocorrelation function. The horizontal autocorrelation function and the vertical autocorrelation function are generated by complex formulas, which can comprehensively and accurately describe the correlation of the physical center. The physical center jitter frequency is determined based on the autocorrelation function, which improves the accuracy and reliability of the frequency calculation. This sophisticated calculation method helps to more accurately evaluate the dynamic characteristics of the physical center of the occluded area sequence. Thereby more effectively screening out the sequence of occluded areas where users are presumed to stay, improving the accuracy of product recognition and the management effect of self-service vending machines.

[0129] Embodiment 8:

[0130] On the basis of Example 7, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition determines the physical center jitter frequency of each first screening occlusion area sequence based on the horizontal autocorrelation function and the vertical autocorrelation function of the physical center of all occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system, including:

[0131] The reciprocal of the integer period k corresponding to the lateral autocorrelation function when the peak value appears in the lateral autocorrelation function of the physical center of all the occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system is regarded as the lateral jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system;

[0132] The reciprocal of the integer period k corresponding to the longitudinal autocorrelation function when the peak value appears in the longitudinal autocorrelation function of the physical center of all the occluded areas of each first screened occluded area sequence relative to the integer period k in each preset two-dimensional coordinate system is regarded as the longitudinal jitter frequency of the physical center of all the occluded areas of each first screened occluded area sequence in each preset two-dimensional coordinate system;

[0133] Based on the lateral jitter frequency and the longitudinal jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system, the physical center jitter frequency of each first screening occlusion area sequence is determined.

[0134] In this embodiment, the transverse autocorrelation function (or longitudinal autocorrelation function) when a peak appears is the transverse autocorrelation function (or longitudinal autocorrelation function) of a discrete independent variable with an integer period k, and the transverse autocorrelation function (or longitudinal autocorrelation function) formed for the corresponding integer period k when its function value reaches the maximum value.

[0135] The beneficial effects of the above technology are: the inverse of the integer period corresponding to the horizontal autocorrelation function when the peak appears is used as the horizontal jitter frequency, and the inverse corresponding to the peak of the vertical autocorrelation function is used as the vertical jitter frequency. This determination method is accurate and intuitive. The horizontal and vertical jitter frequencies are considered separately, making the evaluation of the jitter frequency more comprehensive and detailed. The physical center jitter frequency is determined based on the horizontal and vertical jitter frequencies, which improves the accuracy and reliability of frequency determination. It helps to more accurately screen out stable occlusion area sequences, thereby more effectively inferring the user's stay position. It improves the accuracy of product recognition and the ability of self-service vending machines to judge user operations, and optimizes user experience and management efficiency.

[0136] Embodiment 9:

[0137] On the basis of Example 8, the self-service vending machine commodity recognition method integrating light curtain and AI image recognition determines the physical center jitter frequency of each first screening occlusion area sequence based on the horizontal jitter frequency and the vertical jitter frequency of the physical center of all occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system, including:

[0138] The square root of the sum of the squares of the horizontal jitter frequency and the vertical jitter frequency of the physical centers of all the occlusion areas of each first screened occlusion area sequence in each preset two-dimensional coordinate system is used as the comprehensive jitter frequency of the physical centers of all the occlusion areas of each first screened occlusion area sequence in each preset two-dimensional coordinate system;

[0139] The average value of the comprehensive jitter frequencies of the physical centers of all the occlusion regions of each first screening occlusion region sequence in all the preset two-dimensional coordinate systems is taken as the physical center jitter frequency of each first screening occlusion region sequence.

[0140] In this embodiment, the comprehensive jitter frequency refers to the value obtained by first performing square root operation on the horizontal jitter frequency and the vertical jitter frequency of the physical center of all occlusion areas of each first filtered occlusion area sequence in each preset two-dimensional coordinate system, and then taking the square root of the result.

[0141] The beneficial effects of the above technology are as follows: by calculating the square root of the sum of the squares of the horizontal and vertical jitter frequencies as the comprehensive jitter frequency, the jitter conditions in the two directions are comprehensively considered, making the evaluation more comprehensive and balanced. Taking the average of the comprehensive jitter frequencies in all preset two-dimensional coordinate systems as the physical center jitter frequency improves the stability and reliability of the jitter frequency determination. This precise calculation method can more accurately reflect the jitter characteristics of the physical center of each first-screened occlusion area sequence. It helps to more strictly and accurately screen out occlusion area sequences that meet the requirements, so as to more accurately infer the vertical position of the user's shelf placement. It improves the accuracy and efficiency of product recognition in self-service vending machines, providing users with better services and shopping experience.

[0142] Embodiment 10:

[0143] Based on Example 1, a self-service vending machine commodity recognition method integrating light curtain and AI image recognition, S4: Based on all the inferred user vertical positions of the shelf and commodity placement monitoring videos each time the vending machine door is opened, AI image recognition is performed on the commodity being placed and picked up this time to obtain the commodity placement result, reference Figure 3 ,include:

[0144] S401: Perform AI image recognition on the surveillance video of product placement and take-out to determine all actual user durations and the type of placement and take-out operation in each actual user duration;

[0145] S402: Based on all actual user durations and the pick-and-place operation type of each actual user duration, identifying all actual user vertical positions for picking up and placing the shelf and the pick-and-place operation type of each actual user vertical position for picking up and placing the shelf in all the inferred user vertical positions for picking up and placing the shelf each time the vending machine door is opened;

[0146] S403: Based on the vertical positions of all actual users picking up and placing goods and the picking and placing operation types of each actual user picking up and placing goods at the vertical positions of the shelf, all actual users picking up and placing goods and the corresponding picking and placing operation types are determined as the picking and placing result of this time.

[0147] In this embodiment, AI image recognition is performed on the monitoring video of commodity picking and placing to determine the duration of all actual users and the type of picking and placing operation during each actual user duration. This means that by applying AI image recognition technology to the monitoring video of commodity picking and placing, the time period (i.e., duration) in which all actual users perform operations is obtained, as well as the type of operation of whether the user is taking or putting commodities during each such time period.

[0148] In this embodiment, the actual user duration period refers to a period of time during which the user actually takes or places a commodity in front of the vending machine.

[0149] In this embodiment, the type of the taking and placing operation during the actual user continuous period is the time during which the actual user continuously performs the operation, specifically, whether it is an operation of taking a commodity or an operation of placing a commodity.

[0150] In this embodiment, based on all actual user durations and the type of pick-and-place operation during each actual user duration, all actual user vertical positions where they pick up and place the shelf and the type of pick-and-place operation during each actual user duration are identified in all inferred vertical positions where they pick up and place the shelf each time the door of the vending machine is opened. This means that based on the actual user duration and the type of pick-and-place operation determined previously, the actual vertical position where the actual user picks up and places the shelf and the type of pick-and-place operation at that position are accurately identified in the previously inferred vertical positions where the user picks up and places the shelf.

[0151] In this embodiment, the actual vertical position of the shelf where the user takes or places the goods is the vertical position area of ​​the shelf corresponding to the actual user taking or placing the goods (ie, the number of layers and the approximate direction of the shelf, such as the left or right).

[0152] In this embodiment, the type of the actual picking and placing operation of the user at the vertical position of the shelf is the type of operation corresponding to whether the specific operation performed is to pick up or place the commodity at the vertical position of the shelf actually picked up by the user.

[0153] The beneficial effects of the above technology are as follows: in step S401, AI image recognition is performed on the monitoring video of commodity picking and placing to determine the actual user duration and the type of picking and placing operation, providing detailed basic information for subsequent analysis. Step S402 identifies the relevant information of the actual user in the inferred vertical location of the shelf based on the actual user duration and the type of picking and placing operation, thereby improving the accuracy and pertinence of the identification. Step S403 determines the actual user's picking and placing of commodities and the corresponding operation type based on the above identification results, and obtains comprehensive and accurate commodity picking and placing results. On the whole, this step-by-step processing method can make full use of the monitoring video and the inferred location information to accurately determine the commodity picking and placing situation, improve the accuracy and reliability of commodity identification in self-service vending machines, and help optimize commodity management and enhance user experience.

[0154] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for identifying goods in a self-service vending machine by integrating light curtain and AI image recognition, characterized in that: include: S1: When the vending machine starts working, the light curtain installed on the upper and inner sides of the door of the vending machine is turned on, wherein the light emitters of the light curtain are arranged in a matrix form, and a receiver is installed on the opposite side of each light emitter. When the light curtain works normally, the light rays transmitted from all groups of light emitters and receivers are parallel or perpendicular to each other; S2: When the vending machine door is opened, the real-time receiving results of all receivers in the light curtain and the plane matrix analysis method are used to determine at least one estimated vertical position of the user to pick up and place the shelf; S3: When the door of the vending machine is opened, the camera installed on the shelf of the vending machine is used to obtain the monitoring video of the product placement; S4: Based on all the inferred vertical positions of the shelves where users pick up and place goods each time the vending machine door is opened and the monitoring videos of the goods being picked up and placed, AI image recognition is performed on the goods being picked up and placed to obtain the results of the goods being picked up and placed.

2. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 1 is characterized in that: S2: When the vending machine door is opened, the real-time receiving results of all receivers in the light curtain and the plane matrix analysis method are used to determine at least one estimated vertical position of the user to pick up and place the shelf, including: S201: Generate a vertical plane matrix based on the setting positions and numbers of all light emitters in the light curtain; S202: When the vending machine door is opened, the blocked area is located in the vertical plane matrix based on the real-time receiving results of all receivers in the light curtain, and the vertical plane blocked area of ​​the shelf at each moment after the vending machine door is opened is obtained; S203: Determine at least one estimated vertical location of the shelf where the user picks up or places the product based on the vertical plane occlusion area of ​​the shelf at all times after the vending machine door is opened.

3. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 2 is characterized in that: S203: Determine at least one estimated vertical location of the shelf where the user picks up or places the product based on the vertical plane occlusion area of ​​the shelf at all times after the vending machine door is opened, including: Generate a dynamic sequence of occlusion areas based on the vertical plane occlusion areas of the shelves at all times after the vending machine door is opened; Based on the geometric dynamic features of the dynamic sequence of the occluded area, at least one inferred vertical position of the user picking up and placing the shelf is analyzed.

4. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 3 is characterized in that: Based on the geometric dynamic features of the dynamic sequence of the occluded area, at least one inferred vertical position of the user picking up and placing the shelf is analyzed, including: Counting the geometric area of ​​each occlusion area in the occlusion area dynamic sequence, and determining the physical center of each occlusion area in the occlusion area dynamic sequence; In the dynamic sequence of the occluded area, a continuous sequence of occluded areas is selected, in which the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold, and the number of continuous frames is not less than the preset number of frames, and the sequence is used as the sequence of the occluded area where the user is estimated to stay; The shelf position corresponding to each sequence of occluded areas where the user is estimated to stay on the shelf vertical plane is regarded as a vertical position of the shelf where the user is estimated to pick up and place.

5. The method for identifying goods in a self-service vending machine by integrating light curtain and AI image recognition according to claim 4, characterized in that: In the dynamic sequence of the occluded area, a continuous sequence of occluded areas is selected, in which the geometric area deviation does not exceed the area deviation threshold, the physical center jitter amplitude does not exceed the jitter amplitude threshold, and the physical center jitter frequency does not exceed the jitter frequency threshold, and the number of consecutive frames is not less than the preset number of frames, and is used as the sequence of the occluded area where the user is estimated to stay, including: The geometric area deviation between each group of adjacent occlusion areas in the occlusion area dynamic sequence is calculated, and the adjacent occlusion areas whose geometric area deviation does not exceed the area deviation threshold are regarded as the first qualified occlusion area group, and the occlusion areas adjacent to any occlusion area in each first qualified occlusion area group in the occlusion area dynamic sequence are merged with the corresponding first qualified occlusion area group to obtain multiple first merged occlusion area groups, and the average of the geometric area deviations of the two occlusion areas in each first merged occlusion area group is regarded as the geometric area deviation of each first merged occlusion area group, and all first merged occlusion area groups whose geometric area deviation does not exceed the area deviation threshold are regarded as all second qualified occlusion area groups, and each second qualified occlusion area group is continuously merged and compared with the geometric deviation threshold until there is no updated merged occlusion area group whose geometric area deviation does not exceed the area deviation threshold, then all currently obtained qualified occlusion area groups whose continuous frame number is not less than the preset frame number are regarded as the first screened occlusion area sequence; Calculating the physical center jitter amplitude and physical center jitter frequency of each first screened occlusion region sequence; The first screened occlusion region sequence in which all physical center jitter amplitudes do not exceed the jitter amplitude threshold and the physical center jitter frequency does not exceed the jitter frequency threshold is regarded as the occlusion region sequence in which the user is estimated to stay.

6. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 5, characterized in that: The physical center jitter amplitude and physical center jitter frequency of each first filtered occlusion area sequence are calculated, including: Based on the two-dimensional coordinate value of the physical center of each occlusion area in each first screening occlusion area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter amplitude of each first screening occlusion area sequence is calculated: Where J is the physical center jitter amplitude of the first filtered occlusion area sequence currently calculated, max() is the maximum value of multiple elements in brackets, and x ij is the horizontal coordinate value of the physical center of the i-th occlusion area in the first screening occlusion area sequence currently calculated in the j-th preset two-dimensional coordinate system, n is the total number of occlusion areas in the first screening occlusion area sequence currently calculated, y ij is the vertical coordinate value of the physical center of the i-th occlusion area in the first screened occlusion area sequence currently calculated in the j-th preset two-dimensional coordinate system; Based on the two-dimensional coordinate value of the physical center of each blocking area in each first screening blocking area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter frequency of each first screening blocking area sequence is calculated.

7. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 6, characterized in that: Based on the two-dimensional coordinate value of the physical center of each occlusion area in each first screening occlusion area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, the physical center jitter frequency of each first screening occlusion area sequence is calculated, including: Determine the horizontal coordinate difference and the vertical coordinate difference of the physical centers of all groups of adjacent blocking areas in each first screening blocking area sequence in each preset two-dimensional coordinate system on the vertical plane of the shelf, and generate a horizontal coordinate difference sequence and a vertical coordinate difference sequence of each first screening blocking area sequence in each preset two-dimensional coordinate system; Based on the horizontal coordinate difference sequence and the vertical coordinate difference sequence of each first screened occlusion area sequence in each preset two-dimensional coordinate system, the horizontal autocorrelation function and the vertical autocorrelation function of the physical center of all occlusion areas of each first screened occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system are generated: In the formula, R xxj (k) is the lateral autocorrelation function of the physical center of all the occluded areas of the first screened occluded area sequence currently calculated in the jth preset two-dimensional coordinate system relative to the period k, Δx ij is the projection displacement of the displacement between the physical center of the i-th occlusion area and the physical center of the i+1-th occlusion area in the first screening occlusion area sequence currently calculated in the abscissa axis direction of the j-th preset two-dimensional coordinate system, Δx (i+k)j is the projection displacement of the displacement between the physical center of the i+kth occlusion area and the physical center of the i+k+1th occlusion area in the first screening occlusion area sequence currently calculated in the abscissa axis direction of the jth preset two-dimensional coordinate system, Δy ij is the projection displacement of the displacement between the physical center of the i-th occlusion area and the physical center of the i+1-th occlusion area in the first screening occlusion area sequence currently calculated in the ordinate axis direction of the j-th preset two-dimensional coordinate system, Δy (i+k)j is the projection displacement of the displacement between the physical center of the i+kth occlusion area and the physical center of the i+k+1th occlusion area in the first screened occlusion area sequence currently calculated in the vertical coordinate axis direction of the jth preset two-dimensional coordinate system; Based on the transverse autocorrelation function and the longitudinal autocorrelation function of the physical centers of all occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system, the physical center jitter frequency of each first screening occlusion area sequence is determined.

8. The method for identifying goods in a self-service vending machine by integrating light curtain and AI image recognition according to claim 7, characterized in that: Based on the transverse autocorrelation function and the longitudinal autocorrelation function of the physical centers of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system relative to the integer period k, the physical center jitter frequency of each first screening occlusion area sequence is determined, including: The reciprocal of the integer period k corresponding to the lateral autocorrelation function when the peak value appears in the lateral autocorrelation function of the physical center of all the occlusion areas of each first screening occlusion area sequence relative to the integer period k in each preset two-dimensional coordinate system is regarded as the lateral jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system; The reciprocal of the integer period k corresponding to the longitudinal autocorrelation function when the peak value appears in the longitudinal autocorrelation function of the physical center of all the occluded areas of each first screened occluded area sequence relative to the integer period k in each preset two-dimensional coordinate system is regarded as the longitudinal jitter frequency of the physical center of all the occluded areas of each first screened occluded area sequence in each preset two-dimensional coordinate system; Based on the lateral jitter frequency and the longitudinal jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system, the physical center jitter frequency of each first screening occlusion area sequence is determined.

9. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 8, characterized in that: Based on the lateral jitter frequency and the longitudinal jitter frequency of the physical center of all the occlusion areas of each first screening occlusion area sequence in each preset two-dimensional coordinate system, determining the physical center jitter frequency of each first screening occlusion area sequence includes: The square root of the sum of the squares of the horizontal jitter frequency and the vertical jitter frequency of the physical centers of all the occlusion areas of each first screened occlusion area sequence in each preset two-dimensional coordinate system is used as the comprehensive jitter frequency of the physical centers of all the occlusion areas of each first screened occlusion area sequence in each preset two-dimensional coordinate system; The average value of the comprehensive jitter frequencies of the physical centers of all the occlusion regions of each first screening occlusion region sequence in all the preset two-dimensional coordinate systems is taken as the physical center jitter frequency of each first screening occlusion region sequence.

10. The self-service vending machine commodity recognition method integrating light curtain and AI image recognition according to claim 1, characterized in that: S4: Based on all the inferred vertical positions of the shelves and the product placement monitoring videos when the vending machine door is opened each time, AI image recognition is performed on the products to be placed and retrieved, and the results of the product placement and retrieval are obtained, including: S401: Perform AI image recognition on the surveillance video of product placement and take-out to determine all actual user durations and the type of placement and take-out operation in each actual user duration; S402: Based on all actual user durations and the pick-and-place operation type of each actual user duration, identifying all actual user vertical positions for picking up and placing the shelf and the pick-and-place operation type of each actual user vertical position for picking up and placing the shelf in all the inferred user vertical positions for picking up and placing the shelf each time the vending machine door is opened; S403: Based on the vertical positions of all actual users picking up and placing goods and the picking and placing operation types of each actual user picking up and placing goods at the vertical positions of the shelf, all actual users picking up and placing goods and the corresponding picking and placing operation types are determined as the picking and placing result of this time.

Images