Vending equipment control method, system and device and selling equipment

Abstract

The invention relates to a selling equipment control method, system and device and selling device.The selling equipment control method comprises the steps that in response to detected preset operation, multiple target picture sets in the selling equipment are collected from multiple collection angles; wherein different target picture sets correspond to different collection angles; and based on a plurality of comparison results obtained by comparing the plurality of target picture sets with the initial picture, determining operation information corresponding to the predetermined operation. Therefore, through the plurality of picture sets acquired from different angles, the increase and decrease conditions of the articles in the selling equipment can be detected more accurately and comprehensively, so that the operation information corresponding to the predetermined operation is automatically determined.

Description

Technical Field

[0001] The present application relates to the field of automatic vending application technology, and in particular to a vending equipment control method, system, device, and vending equipment. Background Art

[0002] For unmanned vending machines, manual review and simple identification by the backend server are often required to determine the types and quantities of goods purchased by users. This way, settlement will be affected by network delays and manual working hours, and accuracy may be affected by manual operations. Summary of the invention

[0003] The present application provides a vending equipment control method, system, device, and vending equipment to solve the technical problem of poor intelligence and accuracy in vending processing.

[0004] In a first aspect, the present application provides a vending equipment control method, the method comprising: in response to detecting a predetermined operation, collecting multiple target image sets within the vending equipment through multiple collection angles; wherein different target image sets correspond to different collection angles; and determining operation information corresponding to the predetermined operation based on multiple comparison results of comparing the multiple target image sets with the initial image respectively.

[0005] In a possible implementation, the operation information corresponding to the predetermined operation is determined based on multiple comparison results of comparing multiple target picture sets with the initial picture, including: determining the initial picture in the vending device before the predetermined operation; comparing multiple target pictures included in the target picture set with the initial picture to obtain comparison results corresponding to the target picture set; and determining the operation information corresponding to the predetermined operation based on multiple comparison results corresponding to the multiple target picture sets.

[0006] In a possible implementation, comparing the multiple target images included in the target image set with the initial image to obtain a comparison result corresponding to the target image set includes: determining the multiple target images included in the target image set; comparing the multiple target images with the initial image respectively to determine the target objects missing from the target images; and in response to the number of target images identical to the target objects missing from the target image set reaching a preset value, determining that the comparison result corresponding to the target image set includes the target object.

[0007] In one possible implementation, the comparing the multiple target images with the initial image respectively to determine the target objects missing from the target images includes: comparing the first quantity and the first type of predetermined features contained in the multiple target images with the second quantity and the second type of predetermined features contained in the initial image respectively; and determining the target objects missing from the target image based on the target quantity and target type of the predetermined features missing from the target image.

[0008] In one possible implementation, determining the operation information corresponding to the predetermined operation based on multiple comparison results corresponding to multiple target image sets includes: in response to the presence of at least one comparison result containing a target object among the multiple comparison results corresponding to the multiple target image sets, determining the operation information corresponding to the predetermined operation based on the target object contained in the at least one comparison result.

[0009] In one possible implementation, after determining the operation information corresponding to the predetermined operation based on multiple comparison results of comparing multiple target image sets with the initial images, the method further includes: determining user information based on the predetermined operation; and generating order information based on the operation information and the user information.

[0010] In one possible implementation, in response to detecting a predetermined operation, collecting multiple target image sets within the vending device through multiple acquisition angles includes: in response to detecting a predetermined operation, determining whether the vending device is in a locked state; if the vending device is in a locked state, collecting multiple target image sets within the vending device through multiple acquisition angles.

[0011] In a possible implementation, the collecting multiple target picture sets in the vending device through multiple collection angles includes: acquiring multiple video clips in the vending device through multiple collection angles within a predetermined period of time; and performing picture frame processing based on the multiple video clips to obtain multiple target picture sets.

[0012] In a second aspect, the present application provides a vending equipment control system, the system comprising: a plurality of shooting modules, different shooting modules corresponding to different acquisition angles; a controller connected to the shooting modules, for acquiring a plurality of target image sets within the vending equipment through a plurality of acquisition angles in response to detecting a predetermined operation; wherein different target image sets correspond to different acquisition angles; and determining operation information corresponding to the predetermined operation based on a plurality of comparison results of comparing the plurality of target image sets with the initial images respectively.

[0013] In a third aspect, the present application provides a vending equipment control device, the device comprising: a collection unit, for collecting multiple target image sets in the vending equipment through multiple collection angles in response to detecting a predetermined operation; wherein different target image sets correspond to different collection angles; a determination unit, for determining operation information corresponding to the predetermined operation based on multiple comparison results of comparing the multiple target image sets with the initial image respectively.

[0014] In a fourth aspect, the present application provides a vending machine, comprising the vending machine control device described in the third aspect.

[0015] In a fifth aspect, the present application provides an electronic device comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to implement any one of the methods described in the first aspect when executing the computer program.

[0016] In a sixth aspect, the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described in the first aspect above.

[0017] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application, in response to detecting a predetermined operation, collects multiple target picture sets in the vending device through multiple collection angles; wherein different target picture sets correspond to different collection angles; based on multiple comparison results of the multiple target picture sets respectively compared with the initial picture, the operation information corresponding to the predetermined operation is determined. In this way, by using multiple different collection angles to collect pictures in the vending device, the situation of the items in the device can be fully reflected from multiple angles, avoiding omissions caused by occlusion of pictures collected from a single angle, so that the changes in the items in the device can be accurately determined by comparison with the initial picture, without the need for manual operation, which is conducive to automatic settlement and has high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0021] Figure 1 A flow chart of a vending machine control method provided in an embodiment of the present application;

[0022] Figure 2 A flow chart of a vending machine control method provided in an embodiment of the present application;

[0023] Figure 3 A flow chart of a vending machine control method provided in an embodiment of the present application;

[0024] Figure 4 A flow chart of a vending machine control method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a vending machine control device provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0028] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0029] In order to solve the technical problems of poor intelligence and accuracy of sales processing in the prior art, the present application provides a sales equipment control method, system, device, and sales equipment, which collects internal pictures of the sales equipment based on multiple different angles and compares them with the initial pictures, so as to automatically and accurately determine the changes of items in the equipment, which is conducive to intelligent settlement.

[0030] Figure 1 A flow chart of a vending machine control method provided in this embodiment is shown as follows: Figure 1 As shown, the method may include:

[0031] S10: In response to detecting a predetermined operation, collecting multiple target picture sets in the vending device through multiple collection angles; wherein different target picture sets correspond to different collection angles;

[0032] S20: Determine operation information corresponding to the predetermined operation based on a plurality of comparison results of comparing the plurality of target image sets with the initial image respectively.

[0033] The vending equipment control method provided in this embodiment can be applied to vending equipment, such as vending machines, vending cabinets, delivery cabinets, etc., such as vending cabinets for storing pre-prepared dishes, meals, or other types of items, etc., or can also be applied to servers, cloud platforms or other terminals that have the ability to control vending equipment.

[0034] In one embodiment, the predetermined operation may refer to a pickup operation, a replenishment operation, a door opening operation, or a door closing operation on the vending device, etc. Capturing a plurality of target image sets in the vending device through a plurality of acquisition angles may refer to capturing a plurality of target image sets in the vending device through a plurality of acquisition angles corresponding to a plurality of shooting modules.

[0035] Here, the shooting module may be a camera, etc., and multiple shooting modules may be respectively arranged at multiple corners of the vending device. For example, each corner corresponds to a collection angle, and different shooting modules are located in different corners, corresponding to different collection angles.

[0036] In one embodiment, a target picture set can be collected by a shooting module, that is, one collection angle corresponds to a target picture set. The target picture set can include multiple target pictures, for example, the target pictures included in the target picture set are slice pictures corresponding to the video clips collected within a predetermined period of time.

[0037] In one embodiment, the initial picture may be a picture representing the initial state of the vending device before the predetermined operation is detected, or may be a picture representing the initial state of the vending device at a predetermined moment.

[0038] For example, the predetermined time may be a time before the current time by a predetermined time period.

[0039] In one embodiment, the initial image may be updated according to a predetermined update period, for example, the initial image may be recaptured every predetermined update period, so that the initial image used as a comparison reference may be updated in a timely manner.

[0040] In one embodiment, collecting multiple target image sets in a vending device through multiple collection angles may include: determining an operation duration corresponding to a predetermined operation; determining the number of collection angles based on the operation duration; and collecting multiple target image sets in the vending device through collection angles based on the number of angles.

[0041] The longer the operation time is, the more angles can be used. For example, when the scheduled operation is a pickup operation, the number of angles can be 4 when the operation time reaches 1 minute; the number of angles can be 3 when the operation time is only 30 seconds, etc. In this way, when the operation time is long and there may be more pickups, pictures can be collected from more angles to avoid omissions.

[0042] In one embodiment, the operation information corresponding to the predetermined operation may include the number and / or type of objects added or reduced in the vending device after the predetermined operation, wherein the objects may be items stored in the vending device.

[0043] For example, when the scheduled operation is a pickup operation, the corresponding operation information may refer to the quantity and / or type of items taken out by the pickup operation, etc. When the scheduled operation is a replenishment operation, the corresponding operation information may refer to the quantity and / or type of items added by the replenishment operation, etc.

[0044] In this way, by collecting pictures from multiple different collection angles inside the vending equipment, the situation of the items in the equipment can be fully reflected from multiple angles, avoiding omissions caused by occlusion of pictures collected from a single angle. Therefore, by comparing with the initial pictures, the changes in the items in the equipment can be accurately determined without manual operation, which is conducive to automatic settlement and high accuracy.

[0045] In some embodiments, Figure 2 As shown, the above step S20 may include:

[0046] S21: Determine the initial image in the vending device before the scheduled operation;

[0047] S22: comparing the multiple target images included in the target image set with the initial image to obtain a comparison result corresponding to the target image set;

[0048] S23: Determine operation information corresponding to the predetermined operation based on multiple comparison results corresponding to the multiple target image sets.

[0049] In an embodiment, before the scheduled operation may refer to a predetermined period of time or a predetermined time before the scheduled operation.

[0050] In one embodiment, determining the initial image in the vending device before the scheduled operation may include: obtaining a video clip in the vending device before the scheduled operation; and determining the initial image in the vending device before the scheduled operation based on the video clip in the vending device before the scheduled operation. The initial image may be a sliced ​​image of the video clip in the vending device before the scheduled operation.

[0051] In one embodiment, the initial image in the vending device before the scheduled operation may refer to the initial image in the vending device updated in the last scheduled update cycle before the scheduled operation.

[0052] In one embodiment, the multiple target pictures included in the target picture set may include slice pictures of the video clip collected by the target picture set corresponding to the collection angle. Here, the slice pictures may be pictures obtained by processing the video clip frame by frame.

[0053] In one embodiment, the number of target pictures included in the target picture set is a predetermined number, such as 10, 20 or 50 pictures.

[0054] In one embodiment, step S22 may include: sequentially comparing a plurality of target pictures included in a plurality of target picture sets with the initial picture to obtain a comparison result corresponding to the target picture set.

[0055] In one embodiment, step S23 may include: in response to the number of comparison results containing the target object reaching a preset threshold, stopping the comparison of the remaining target image sets, and determining the operation information corresponding to the predetermined operation based on multiple comparison results corresponding to the multiple target image sets that have completed the comparison.

[0056] Here, the preset threshold may be 1, 2, or 3, etc., so that when a change in the target object is determined, there is no need to perform subsequent comparison operations, further improving efficiency.

[0057] In one embodiment, responding to the number of comparison results including the target object reaching a preset threshold may include: responding to the number of comparison results including the target object reaching a preset threshold and the target objects included in the comparison results are the same.

[0058] In one embodiment, step S22 may include: determining multiple target pictures included in the target picture set; comparing the multiple target pictures with the initial picture respectively to determine the target objects added or reduced in the target pictures; in response to the number of target pictures with the same target objects added or reduced in the target picture set reaching a preset value, determining that the comparison results corresponding to the target picture set include the target objects.

[0059] In one embodiment, respectively comparing the multiple target images with the initial image to determine the target objects that are increased or decreased in the target images may include: respectively comparing a first quantity and a first type of predetermined features contained in the multiple target images with a second quantity and a second type of predetermined features contained in the initial image; and determining the target objects that are increased or decreased in the target image based on the target quantity and target type of the predetermined features that are increased or decreased in the target image.

[0060] In one embodiment, if the target object added to the target image is determined, after step S20, the method may further include: updating the object information corresponding to the vending device based on the operation information. Here, the object information may refer to the information of all objects contained in the vending device, for example, the object may refer to the goods stored in the vending device.

[0061] In this way, by comparing the initial pictures before the predetermined operation, the content changes of the multiple pictures included in each picture set can be accurately determined, so that the operation information can be accurately obtained through multiple comparison results.

[0062] In some embodiments, the above step S22 may include:

[0063] Determining a plurality of target images included in the target image set;

[0064] Comparing the multiple target images with the initial image respectively to determine target objects missing from the target images;

[0065] In response to the number of target pictures identical to the target object missing from the target picture set reaching a preset value, it is determined that the comparison result corresponding to the target picture set includes the target object.

[0066] In an embodiment, the target object missing in the target image may refer to the target object missing in the target image compared with the initial image.

[0067] In one embodiment, determining the target objects missing from the target image may include: determining the number and / or type of the target objects missing from the target image. Here, the type may refer to category, price, or name, etc.

[0068] In one embodiment, the missing target objects are the same, which may mean that the number and / or type of the missing target objects are the same. For example, if 2 items of product A are missing in target image 1 and 2 items of product A are missing in target image 2, it is determined that the missing target objects in target image 1 and target image 2 are the same.

[0069] In one embodiment, the preset value may be 5, 6, or 8, etc. The preset value may be determined according to the number of target pictures included in the target picture set. For example, the preset value may be equal to a predetermined ratio of the number of target pictures included in the target picture set. Exemplarily, the predetermined ratio may be 80%, etc.

[0070] In one embodiment, the number of target images with the same target object missing from the target image set reaches a preset value, which may refer to a target image set having a preset number of target images with the same target object missing. For example, if the preset value is 8 and the target objects missing from 8 target images in the target image set are all product A, then it can be determined that the comparison result corresponding to the target image set includes product A.

[0071] In this way, the missing objects can be determined by comparing multiple images in a set separately, so that multiple comparisons can be performed for one acquisition angle to avoid omissions, thereby further improving the accuracy of comparison and operation information.

[0072] In some embodiments, the step of comparing the plurality of target images with the initial image respectively to determine the target objects missing from the target images includes:

[0073] Comparing respectively a first number and a first type of the plurality of target images containing the predetermined feature with a second number and a second type of the initial image containing the predetermined feature;

[0074] Based on the number and type of targets with predetermined features missing from the target image, the target objects missing from the target image are determined.

[0075] In one embodiment, the predetermined feature may refer to a label, text, number, barcode, QR code or other identification feature. Each object may include one or more predetermined features.

[0076] In one embodiment, the category may refer to the color, shape or content of the predetermined feature. For example, when the predetermined feature is a label, the category may be color. Different categories may have different corresponding prices.

[0077] In one embodiment, the first quantity and the first type of predetermined features contained in the multiple target images are respectively compared with the second quantity and the second type of predetermined features contained in the initial image, including: the first quantity and the first type of predetermined features contained in the multiple target images are respectively compared with the second quantity and the second type of predetermined features contained in the initial image, to obtain the target quantity and target type of predetermined features missing in the target image.

[0078] In one embodiment, the target quantity and target type of the missing predetermined features may refer to the target quantity and target color of the missing labels. For example, labels of different colors correspond to different prices.

[0079] In an embodiment, determining the target object missing from the target image may include: determining the target object missing from the target image and the price of the target object.

[0080] In one embodiment, the first quantity and the first type of the predetermined features in the multiple target images are compared with the second quantity and the second type of the predetermined features in the initial image, respectively. The comparison can be performed through a predetermined deep learning model.

[0081] In this way, by comparing the predetermined features in the picture, the recognition accuracy is greatly improved in complex environments such as when the goods are placed in a disorderly or overlapping manner and when the camera angle cannot be adjusted.

[0082] In some embodiments, determining the operation information corresponding to the predetermined operation based on the multiple comparison results corresponding to the multiple target picture sets may include:

[0083] In response to at least one comparison result including a target object among multiple comparison results corresponding to multiple target image sets, operation information corresponding to the predetermined operation is determined based on the target object included in the at least one comparison result.

[0084] In one embodiment, each target image set corresponds to a comparison result. When at least one comparison result among multiple comparison results corresponding to multiple target image sets contains a target object, it indicates that an added or missing target object is determined in the images captured at at least one angle.

[0085] In one embodiment, in response to the presence of at least one comparison result including a target object among multiple comparison results corresponding to multiple target image sets, it may include: in response to the presence of at least one comparison result including a target object among multiple comparison results corresponding to multiple target image sets and the target objects included in at least one comparison result are the same.

[0086] In one embodiment, if at least one comparison result includes different target objects, the operation information corresponding to the predetermined operation is determined based on the target object having the largest number of comparison results including the same target object.

[0087] Here, the included target objects are the same, which may refer to the same number and / or type of the included target objects.

[0088] In this way, when there is at least one acquisition angle that determines the comparison result of the target object, the required operation information can be confirmed, thereby eliminating the need for further analysis and improving efficiency.

[0089] In some embodiments, Figure 3 As shown, in the above method, after step S20, the method may further include:

[0090] S30: Determine user information based on the predetermined operation;

[0091] S40: Generate order information based on the operation information and user information.

[0092] In one embodiment, the user information may include at least one of the identity information, account information, and payment information of the user performing the predetermined operation. Determining the user information based on the predetermined operation may refer to determining the user information based on the information input operation associated with the predetermined operation.

[0093] Here, the information input operation may refer to the operation of inputting user information on the vending device before the user performs a predetermined operation such as picking up goods, replenishing goods, opening or closing a door. For example, the information input operation may be at least one operation such as key input, code scanning input, and selection input.

[0094] In one embodiment, generating order information based on the operation information and the user information may include: determining a payment amount based on the operation information; and generating order information based on the payment amount and the user information.

[0095] In one embodiment, step S40 may further include: performing a payment request operation to a payment account corresponding to the user information based on the order information.

[0096] In this way, according to the association between the scheduled operation and the user information, the item and price corresponding to the user information can be quickly determined in combination with the operation information of the scheduled operation, thereby automatically generating order information for processing without manual analysis operation.

[0097] In some embodiments, in the above step S10, in response to detecting a predetermined operation, collecting a plurality of target image sets in the vending device through a plurality of collection angles includes:

[0098] In response to detecting a predetermined operation, determining whether the vending device is in a door-locked state;

[0099] If the vending machine is in a locked state, a plurality of target image sets in the vending machine are collected through a plurality of collection angles.

[0100] In one embodiment, being in the door locked state may refer to that the door of the vending device is in a locked state, or may refer to that the duration between the target time when the door of the vending device changes from the door open state to the door locked state and the current time is shorter than the first time, thereby avoiding the situation where the scheduled operation is not processed for too long after execution.

[0101] In one embodiment, if the vending device is in a locked state, collecting multiple target image sets in the vending device through multiple acquisition angles may include: if the vending device is in a locked state and the vending device is in a locked state for the second time period, collecting multiple target image sets in the vending device through multiple acquisition angles. This can avoid unnecessary acquisition actions caused by repeated door opening operations due to omissions in picking up or replenishing goods by users.

[0102] In this way, by determining that the image collection and analysis process is performed only after the door is locked, it is avoided that the collection and analysis is performed when the user has not completed operations such as picking up or replenishing goods, resulting in inaccurate results, thereby improving the accuracy of automatic settlement.

[0103] In some embodiments, in the above step S10, the collecting of multiple target image sets in the vending device through multiple collecting angles includes:

[0104] Acquire multiple video clips captured from multiple capture angles within a predetermined period of time in the vending device;

[0105] Picture frame processing is performed based on the multiple video clips to obtain multiple target picture sets.

[0106] In one embodiment, capturing multiple video clips in the vending device through multiple capturing angles may refer to capturing multiple video clips in the vending device through capturing angles corresponding to multiple shooting modules.

[0107] In one embodiment, the predetermined period may refer to a predetermined period after the predetermined operation is completed. For example, the predetermined period may refer to a predetermined period after the predetermined operation is detected and the vending device is in a locked state for a second period of time.

[0108] In one embodiment, the lengths of the multiple video clips are the same, for example, all are a predetermined length, and the predetermined length may be 2s, 5s, or 10s.

[0109] In one embodiment, the lengths of the multiple video segments may be different. For example, the length of the video segments captured at one or more predetermined capture angles may be a first length, and the length of the video segments captured at other capture angles may be a second length, and the first length is greater than the second length.

[0110] Exemplarily, the predetermined acquisition angle may be an acquisition angle close to a door of the vending device or located in the middle of the vending device, etc., so that a longer video can be captured at an angle that is conducive to capturing accurate images to obtain more target images.

[0111] In one embodiment, performing picture frame processing based on the multiple video clips may include: slicing the multiple video clips according to picture frames. For example, each video clip corresponds to a target picture set, and the multiple target pictures included in the target picture set are sliced ​​pictures obtained by slicing the video clip.

[0112] In this way, the target image obtained by frame processing of the video clip can more consistently and accurately represent the changes of the items in the vending equipment, avoid the omission of accidental moments caused by taking photos alone, and improve the accuracy of automatic settlement. At the same time, the video is collected within the predetermined period, without the need for continuous collection, reducing resource usage.

[0113] The present application also provides a vending equipment control system, the system comprising:

[0114] Multiple shooting modules, different shooting modules correspond to different acquisition angles;

[0115] A controller connected to the shooting module is used to collect multiple target image sets in the vending device through multiple collection angles in response to detecting a predetermined operation; wherein different target image sets correspond to different collection angles; and based on multiple comparison results of the multiple target image sets being compared with the initial image, determine the operation information corresponding to the predetermined operation.

[0116] Here, the shooting module may be a camera for capturing images or video clips, etc. The controller captures multiple target image sets in the vending device through multiple capture angles, which may include: capturing multiple target image sets in the vending device through capture angles corresponding to the multiple shooting modules.

[0117] The embodiments and specific implementations of the steps executed by the controller can refer to the relevant embodiments and specific implementations in the aforementioned device positioning method, which will not be repeated here.

[0118] As a possible implementation method, a method for locating a vending machine is provided, such as Figure 4 As shown, specific methods may include:

[0119] 1. The solution’s pre-conditions for equipment structure need to be met, that is, to avoid stacking of goods or blocking of the shooting line of sight, multiple cameras need to be distributed in multiple corners of the sales cabinet to ensure that all goods in the cabinet can be fully covered.

[0120] The principle of this solution is to use the video stream data in the vending cabinet from multiple angles of the external camera to integrate the video stream data collected by the camera through the external data receiving module, and perform frame-by-frame image processing on the video stream data to provide it to the system main service; the system main service uses the set program to first save the initial product placement pictures of the vending cabinet from multiple angles as the original comparison pictures for subsequent product sales 1.

[0121] At the same time, the types and appearance of the goods sold are known. In order to improve the recognition accuracy, it has been agreed to use a unified lunch box mold on the outer packaging of pre-prepared meals, and to distinguish the categories of lunch boxes with different colors, for example: a blue label is priced at 15 yuan, a red label is 20 yuan, etc.; in this way, through fixed lunch box packaging and affixing different colored labels on obvious parts of the lunch box appearance, the missing color label can be used as a feature item of the product in the result comparison output of the AI ​​vision chip module and output to the system main service; it also greatly improves the recognition accuracy in complex environments such as vending cabinets where goods are placed in a disorderly and overlapping manner and the camera angle cannot be adjusted.

[0122] When the backend receives a purchase request from a user by scanning the WeChat / Alipay mini-program QR code on the vending machine, the backend will bind and confirm the user's WeChat / Alipay account, and then operate the vending machine to unlock it; until the backend receives the door closing information, it means that the user has completed the purchase of the goods. At this time, the system main service will be notified to settle the goods and the backend service will be notified.

[0123] At this time, after the system main service receives the background instruction, it starts the peripheral data receiving module to start organizing the current video stream and slice the video of the corresponding angle frame by frame. After receiving the sliced ​​picture of the corresponding angle, it uses the picture as a parameter to directly call the data interface provided by the AI ​​vision chip module.

[0124] After the deep learning debugging by the module manufacturer before leaving the factory, the AI ​​vision chip module can compare and distinguish the pixels of the two pictures and the feature items of the goods (such as the color label on the lunch box) at the millisecond level, and provide the missing pixels and feature items of the goods as output parameters to the calling module; at this time, the AI ​​chip compares the saved original picture 1 with the current input picture, and outputs the feature item results of the missing goods to the system main service.

[0125] Because the system's data transmission parameters and calls are all millisecond-level operations, which will not increase time-consuming operations, the program can now slice the video streams of objects placed at different angles at the same time collected by multiple-angle cameras, and then call the AI ​​vision chip module interface for the sliced ​​images at multiple angles. For example, the interface is called 10 times for the image at angle A, and 10 recognition results are received. The results are collected and compared. When a result is the same for more than 8 times, it is determined that this result is the result of the image recognition at angle A. At the same time, the results from multiple angles are summarized. Due to problems such as the placement of cabinets and the shape of goods, there may be some angles where the difference in goods cannot be seen, that is, the result is 0 (no difference). Therefore, by combining the results from multiple angles, if one angle has a result of 1 (difference), the difference in the image can be determined, which can also better improve the accuracy of the product recognition results.

[0126] After the system main service determines the recognition result, it will use its own 4G module to directly inform the backend server of the result through the 4G network, and settle the user's purchase results in real time, avoiding the technical pain points of manual review and settlement delays in the existing industry.

[0127] like Figure 5 As shown, the embodiment of the present application provides a vending equipment control device, which may include:

[0128] The collection unit 100 is used to collect multiple target picture sets in the vending device through multiple collection angles in response to detecting a predetermined operation; wherein different target picture sets correspond to different collection angles;

[0129] The determination unit 200 is used to determine the operation information corresponding to the predetermined operation based on multiple comparison results of comparing multiple target image sets with the initial image respectively.

[0130] In a possible implementation, the determination unit 200 is specifically used to: determine the initial image in the vending device before the predetermined operation; compare the multiple target images included in the target image set with the initial image to obtain the comparison results corresponding to the target image set; and determine the operation information corresponding to the predetermined operation based on the multiple comparison results corresponding to the multiple target image sets.

[0131] In one possible implementation, the determination unit 200 is specifically used to: determine multiple target images included in the target image set; respectively compare the multiple target images with the initial image to determine target objects missing from the target images; in response to the number of target images identical to the target objects missing from the target image set reaching a preset value, determine that the comparison results corresponding to the target image set include the target object.

[0132] In a possible implementation, the determination unit 200 is specifically used to: respectively compare the first number and the first type of predetermined features contained in the multiple target images with the second number and the second type of predetermined features contained in the initial image; and determine the target object missing in the target image based on the target number and the target type of the predetermined features missing in the target image.

[0133] In a possible implementation, the determination unit 200 is specifically used to: in response to the presence of at least one comparison result containing a target object among multiple comparison results corresponding to multiple target image sets, determine operation information corresponding to the predetermined operation based on the target object contained in the at least one comparison result.

[0134] In a possible implementation manner, the determining unit 200 is further configured to: determine user information based on the predetermined operation; and generate order information based on the operation information and the user information.

[0135] In a possible implementation, the acquisition unit 100 is specifically configured to: in response to detecting a predetermined operation, determine whether the vending device is in a locked state; if the vending device is in a locked state, acquire multiple target image sets in the vending device through multiple acquisition angles.

[0136] In a possible implementation, the acquisition unit 100 is specifically configured to: acquire multiple video clips captured from multiple acquisition angles within a predetermined period of time within the vending device; and perform image frame processing based on the multiple video clips to obtain multiple target image sets.

[0137] An embodiment of the present application further provides a vending machine, comprising the vending machine control device described in any one or more of the aforementioned embodiments, or comprising the vending machine control system described in any one or more of the aforementioned embodiments, or for executing the vending machine control method described in any one or more of the aforementioned embodiments.

[0138] like Figure 6As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0139] Memory 113, used for storing computer programs;

[0140] In one embodiment of the present application, the processor 111, when being used to execute the program stored in the memory 113, implements the vending equipment control method provided by any one or more of the aforementioned method embodiments, which at least includes: in response to detecting a predetermined operation, collecting multiple target image sets in the vending equipment through multiple collection angles; wherein different target image sets correspond to different collection angles; and determining the operation information corresponding to the predetermined operation based on multiple comparison results of comparing the multiple target image sets with the initial image respectively.

[0141] In a possible implementation, the operation information corresponding to the predetermined operation is determined based on multiple comparison results of comparing multiple target picture sets with the initial picture, including: determining the initial picture in the vending device before the predetermined operation; comparing multiple target pictures included in the target picture set with the initial picture to obtain comparison results corresponding to the target picture set; and determining the operation information corresponding to the predetermined operation based on multiple comparison results corresponding to the multiple target picture sets.

[0142] In a possible implementation, comparing the multiple target images included in the target image set with the initial image to obtain a comparison result corresponding to the target image set includes: determining the multiple target images included in the target image set; comparing the multiple target images with the initial image respectively to determine the target objects missing from the target images; and in response to the number of target images identical to the target objects missing from the target image set reaching a preset value, determining that the comparison result corresponding to the target image set includes the target object.

[0143] In one possible implementation, the comparing the multiple target images with the initial image respectively to determine the target objects missing from the target images includes: comparing the first quantity and the first type of predetermined features contained in the multiple target images with the second quantity and the second type of predetermined features contained in the initial image respectively; and determining the target objects missing from the target image based on the target quantity and target type of the predetermined features missing from the target image.

[0144] In one possible implementation, determining the operation information corresponding to the predetermined operation based on multiple comparison results corresponding to multiple target image sets includes: in response to the presence of at least one comparison result containing a target object among the multiple comparison results corresponding to the multiple target image sets, determining the operation information corresponding to the predetermined operation based on the target object contained in the at least one comparison result.

[0145] In one possible implementation, after determining the operation information corresponding to the predetermined operation based on multiple comparison results of comparing multiple target image sets with the initial images, the method further includes: determining user information based on the predetermined operation; and generating order information based on the operation information and the user information.

[0146] In one possible implementation, in response to detecting a predetermined operation, collecting multiple target image sets within the vending device through multiple acquisition angles includes: in response to detecting a predetermined operation, determining whether the vending device is in a locked state; if the vending device is in a locked state, collecting multiple target image sets within the vending device through multiple acquisition angles.

[0147] In a possible implementation, the collecting multiple target picture sets in the vending device through multiple collection angles includes: acquiring multiple video clips in the vending device through multiple collection angles within a predetermined period of time; and performing picture frame processing based on the multiple video clips to obtain multiple target picture sets.

[0148] The embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the vending machine control method provided by any one or more of the aforementioned method embodiments are implemented.

[0149] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0151] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0152] In the absence of contradiction, each step in a certain implementation mode or example can be implemented as an independent example, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain implementation mode or example can also be implemented as an independent example, and the order of the steps in a certain implementation mode or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain implementation mode or example can be arbitrarily combined; in addition, the various implementation modes or examples can be arbitrarily combined. For example, some or all steps of different implementation modes or examples can be arbitrarily combined, and a certain implementation mode or example can be arbitrarily combined with the optional methods or optional examples of other implementation modes or examples.

[0153] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A vending equipment control method, characterized in that: The method comprises: In response to detecting a predetermined operation, collecting multiple target picture sets in the vending device through multiple collection angles; wherein different target picture sets correspond to different collection angles; Based on multiple comparison results of the multiple target image sets and the initial image respectively, the operation information corresponding to the predetermined operation is determined.

2. The method according to claim 1, characterized in that The determining operation information corresponding to the predetermined operation based on a plurality of comparison results of comparing the plurality of target image sets with the initial image respectively includes: determining an initial image in the vending device before the predetermined operation; Comparing a plurality of target images included in the target image set with the initial image to obtain a comparison result corresponding to the target image set; Based on multiple comparison results corresponding to multiple target image sets, operation information corresponding to the predetermined operation is determined.

3. The method according to claim 2, characterized in that The comparing the multiple target images included in the target image set with the initial image to obtain a comparison result corresponding to the target image set includes: Determining a plurality of target images included in the target image set; Comparing the multiple target images with the initial image respectively to determine target objects missing from the target images; In response to the number of target pictures identical to the target object missing from the target picture set reaching a preset value, it is determined that the comparison result corresponding to the target picture set includes the target object.

4. The method according to claim 3, characterized in that The step of comparing the multiple target images with the initial image respectively to determine the target objects missing from the target images includes: Comparing respectively a first number and a first type of the plurality of target images containing the predetermined feature with a second number and a second type of the initial image containing the predetermined feature; Based on the number and type of targets with predetermined features missing from the target image, the target object missing from the target image is determined.

5. The method according to claim 2, characterized in that: The determining, based on the multiple comparison results corresponding to the multiple target image sets, the operation information corresponding to the predetermined operation includes: In response to at least one comparison result including a target object among multiple comparison results corresponding to multiple target image sets, operation information corresponding to the predetermined operation is determined based on the target object included in the at least one comparison result.

6. The method according to claim 1, characterized in that After determining the operation information corresponding to the predetermined operation based on the multiple comparison results of the multiple target image sets and the initial image respectively, the method further includes: determining user information based on the predetermined operation; Generate order information based on the operation information and the user information.

7. The method according to claim 1, characterized in that In response to detecting a predetermined operation, collecting a plurality of target picture sets in the vending device through a plurality of collection angles, including: In response to detecting a predetermined operation, determining whether the vending device is in a door-locked state; If the vending machine is in a locked state, a plurality of target image sets in the vending machine are collected through a plurality of collection angles.

8. The method according to claim 1, characterized in that The collecting of multiple target image sets in the vending device through multiple collecting angles includes: Acquire multiple video clips captured from multiple capture angles within a predetermined period of time in the vending device; Picture frame processing is performed based on the multiple video clips to obtain multiple target picture sets.

9. A vending equipment control system, characterized in that: The system comprises: Multiple shooting modules, different shooting modules correspond to different acquisition angles; A controller connected to the shooting module is used to collect multiple target image sets in the vending device through multiple collection angles in response to detecting a predetermined operation; wherein different target image sets correspond to different collection angles; and based on multiple comparison results of the multiple target image sets being compared with the initial image, determine the operation information corresponding to the predetermined operation.

10. A vending equipment control device, characterized in that: The device comprises: A collection unit, configured to collect a plurality of target picture sets in the vending device through a plurality of collection angles in response to detecting a predetermined operation; wherein different target picture sets correspond to different collection angles; The determination unit is used to determine the operation information corresponding to the predetermined operation based on multiple comparison results of comparing multiple target image sets with the initial image respectively.

11. A vending device, characterized in that: Comprising the vending equipment control device of claim 10.

12. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store a computer program; and the processor is used to implement the method according to any one of claims 1 to 8 when executing the computer program.

13. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images