Calibration area determination method, device and equipment for intelligent unmanned goods cabinet

Abstract

The application relates to the technical field of intelligent identification, and discloses a calibration area determination method, device and equipment of an intelligent unmanned goods cabinet, to solve the problem that the method for acquiring a calibration area in related technologies needs to rely on additional hardware devices and has low universality. A first action video is collected through a camera, a video front background recognition process is performed on the first action video after being reversed, a target object in a motion state in a video picture is determined, a first position where a specified mark line in the target object is located is determined based on the target object, finally, the first position where the specified mark line is located is taken as a boundary of the calibration area, a preset pixel width is expanded to the outside of the intelligent unmanned goods cabinet, and the calibration area is obtained. The application embodiment can rely on visual information captured by the camera to acquire the calibration area of the unmanned goods cabinet, and does not rely on additional hardware devices. The production and maintenance cost of the intelligent unmanned goods cabinet can be reduced, and the stability and universality of the calibration area determination method can be improved.

Description

Technical Field

[0001] This application relates to the field of intelligent identification technology, and in particular to a method, apparatus and equipment for determining the calibration area of ​​an intelligent unmanned vending machine. Background Technology

[0002] Currently, smart unmanned vending machines are a branch of the emerging unmanned retail business. Users scan a code to open the door (or use facial recognition), select items from the vending machine, freely take or put items in, and leave after closing the door. Payment is automatically processed in the background. After the user closes the door, relevant information captured during the opening and closing process (such as visual and gravity information) is used to determine the type and quantity of items taken, and this information is used to complete the final payment. In determining whether items are taken out or put in, a defined area is needed as a boundary; moving from one side of this area to the other constitutes an action. Therefore, identifying this defined area in the visual image becomes a fundamental task in determining user shopping behavior.

[0003] In related technologies, methods for obtaining calibration areas typically rely on additional hardware devices, which increase the manufacturing cost of the container during production, processing, and assembly. Furthermore, this reliance on additional hardware reduces the method's versatility, making it impossible to use the same method to obtain calibration areas across multiple types of container units.

[0004] Therefore, it is evident that the methods for obtaining the calibration area in related technologies require additional hardware devices and have low versatility. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and device for determining the calibration area of ​​an intelligent unmanned vending machine, in order to solve the problems that the methods for obtaining the calibration area in related technologies require additional hardware devices and have low versatility.

[0006] In a first aspect, this application provides a method for determining the calibration area of ​​an intelligent unmanned vending machine, the method comprising:

[0007] The first action video is captured by a camera, and the first action video is played in reverse.

[0008] Perform foreground and background recognition processing on the first action video after it has been reversed to identify the target object in motion in the video frame.

[0009] Based on the target object, determine the first position where the specified marker line is located in the target object;

[0010] The first position where the designated marker line is located is taken as the boundary of the calibration area, and the calibration area is extended outward by a preset pixel width to obtain the calibration area. The outer side is the side away from the smart unmanned vending machine from the camera's perspective, and the calibration area is a rectangular area.

[0011] In one possible implementation, before defining the first position where the designated marker line is located as the boundary of the calibration area, the method further includes:

[0012] It is determined that the first position is within a preset range.

[0013] In one possible implementation, the method further includes:

[0014] A second action video is captured by a camera; wherein the second action video is located before the first action video, and the first starting frame of the first action video after being reversed is the same as the state of the target object in the second starting frame of the second action video;

[0015] If the first position is outside the preset range, then the second action video is processed for video foreground and background recognition to determine the target object in motion in the video frame;

[0016] Based on the target object, determine the second position where the specified marker line is located in the target object;

[0017] The second position where the specified marker line is located is taken as the boundary of the calibration area.

[0018] In one possible implementation, before setting the second position where the designated marker line is located as the boundary of the calibration area, the method further includes:

[0019] The second position is determined to be within the preset range.

[0020] In one possible implementation, the method further includes:

[0021] If the second position is outside the preset range, then both the first action video and the second action video are determined to be invalid videos, and both the first action video and the second action video are discarded.

[0022] In one possible implementation, determining the first position of the specified marker line within the target object, based on the target object, includes:

[0023] Determine the position of the designated marker line in the first starting video frame;

[0024] The position of the specified marker line in the first starting point video frame is taken as the first position;

[0025] Determining the second position of the specified marker line within the target object based on the target object includes:

[0026] Determine the position of the designated marker line in the second starting point video frame;

[0027] The position of the specified marker line in the second starting point video frame is taken as the second position.

[0028] In one possible implementation, the method further includes:

[0029] If the target item is detected to have moved from the first side of the marked area to the second side of the marked area, it is determined that the target item will be removed from the smart unmanned vending machine.

[0030] If the target item is detected to have moved from the second side of the calibration area to the first side of the calibration area, it is determined that the target item will be returned to the smart unmanned vending machine.

[0031] Secondly, this application provides a calibration area determination device for an intelligent unmanned vending machine, the device comprising:

[0032] The video capture module is configured to capture a first action video through a camera and then reverse the first action video.

[0033] The dynamic target recognition module is configured to perform foreground and background recognition processing on the first action video after it has been reversed to identify the target object in motion in the video frame.

[0034] The marker line position determination module is configured to determine the first position of a specified marker line in the target object based on the target object;

[0035] The calibration area determination module is configured to take the first position where the specified marker line is located as the boundary of the calibration area, and extend it to the outside of the smart unmanned vending machine by a preset pixel width to obtain the calibration area, wherein the outside is the side away from the smart unmanned vending machine from the camera's perspective, and the calibration area is a rectangular area.

[0036] In one possible implementation, before executing the step of using the first position where the specified marker line is located as the boundary of the calibration region, the calibration region determination module is further configured to:

[0037] It is determined that the first position is within a preset range.

[0038] In one possible implementation, the device further includes:

[0039] The video capture module is configured to capture a second action video through a camera; wherein the second action video is located before the first action video, and the first starting video frame of the first action video after being reversed is in the same state as the target object in the second starting video frame of the second action video;

[0040] The dynamic target recognition module is configured to perform video foreground and background recognition processing on the second action video if the first position is outside the preset range, and determine the target object in motion in the video frame;

[0041] The marker line position determination module is configured to determine a second position of the specified marker line in the target object based on the target object;

[0042] The calibration area determination module is configured to use the second position where the specified marker line is located as the boundary of the calibration area.

[0043] In one possible implementation, before executing the step of using the second position where the specified marker line is located as the boundary of the calibration region, the calibration region determination module is further configured to:

[0044] The second position is determined to be within the preset range.

[0045] In one possible implementation, the calibration region determination module is further configured to:

[0046] If the second position is outside the preset range, then both the first action video and the second action video are determined to be invalid videos, and both the first action video and the second action video are discarded.

[0047] In one possible implementation, the step of determining a first position of a specified marker line within the target object is performed based on the target object, wherein the marker line position determination module is configured to:

[0048] Determine the position of the designated marker line in the first starting video frame;

[0049] The position of the specified marker line in the first starting point video frame is taken as the first position;

[0050] The module for determining the position of the specified marker line within the target object is configured to:

[0051] Determine the position of the designated marker line in the second starting point video frame;

[0052] The position of the specified marker line in the second starting point video frame is taken as the second position.

[0053] In one possible implementation, the device is further configured to:

[0054] If the target item is detected to have moved from the first side of the marked area to the second side of the marked area, it is determined that the target item will be removed from the smart unmanned vending machine.

[0055] If the target item is detected to have moved from the second side of the calibration area to the first side of the calibration area, it is determined that the target item will be returned to the smart unmanned vending machine.

[0056] Thirdly, this application provides an electronic device, comprising:

[0057] Processor and memory;

[0058] The memory is used to store executable instructions of the processor;

[0059] The processor is configured to execute the executable instructions to implement the calibration area determination method for the intelligent unmanned vending machine as described in any one of the first aspects above.

[0060] Fourthly, this application provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the calibration area determination method for an intelligent unmanned vending machine as described in any one of the first aspects above.

[0061] Fifthly, this application provides a computer program product, including a computer program:

[0062] When the computer program is executed by the processor, it implements the calibration area determination method for the intelligent unmanned vending machine as described in any one of the first aspects above.

[0063] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:

[0064] In this embodiment, the calibration area of ​​the unmanned vending machine can be obtained based on visual information captured by a camera, without relying on additional hardware devices. This reduces the production and maintenance costs of intelligent unmanned vending machines, improves the stability and versatility of the calibration area determination method, and can be deployed and used on more types of unmanned intelligent vending machines.

[0065] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 A schematic diagram of the overall process for determining the calibration area of ​​an unmanned vending machine provided in this application embodiment;

[0068] Figure 2 This is a schematic diagram illustrating the effect of specifying a marker line in a target object as provided in an embodiment of this application.

[0069] Figure 3 A schematic diagram illustrating the effect of the first position and range of the designated marker line in the target object provided in this application embodiment;

[0070] Figure 4 A flowchart illustrating the steps of this application to perform a second action video if the first position is outside a preset range, provided as an embodiment of this application.

[0071] Figure 5 A schematic diagram of the first starting video frame provided in an embodiment of this application;

[0072] Figure 6 A schematic diagram of the calibration area provided in the embodiments of this application;

[0073] Figure 7 A schematic diagram of the calibration area determination device for the intelligent unmanned vending machine provided in this application embodiment;

[0074] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0076] Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0077] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0078] Currently, the following methods are used in related technologies to determine the calibration area of ​​intelligent unmanned vending machines:

[0079] 1. Chessboard Marking Method

[0080] This invention discloses a method for extracting feature points of a checkerboard calibration board using a camera and performing joint calibration, belonging to the field of multi-sensor joint calibration. Based on the detected corner coordinates of the checkerboard in the distortion-free image, the top, bottom, left, and right corners are identified as reference points. Relationships between these points are calculated to obtain the two-dimensional coordinates of each feature point on the checkerboard calibration board in the pixel coordinate system. This invention can perform joint calibration using a common checkerboard calibration board. However, this method is highly dependent on the checkerboard device; any misalignment or damage to the device will affect the accuracy of the calibration area determination.

[0081] 2. LED strip calibration method

[0082] A method for calibrating light strips utilizes a camera to extract feature points and employs multi-sensor joint calibration. Based on images captured by the camera, visual algorithms process the images to locate areas where the light strips may exist. Further algorithms confirm the waiting area, thus completing the calibration. Similar to the checkerboard calibration method, the calibration results are highly dependent on the light strip device, and the ambient light level around the container also affects the effectiveness of this method.

[0083] In view of this, this application provides a method, apparatus and equipment for determining the calibration area of ​​an intelligent unmanned vending machine, in order to solve the problems that the methods for obtaining the calibration area in the related art require additional hardware devices and have low versatility.

[0084] The inventive concept of this application can be summarized as follows: A first action video is captured by a camera; the reversed first action video undergoes foreground and background recognition processing to determine a moving target object in the video frame; based on the target object, a first position of a designated marker line within the target object is determined; finally, the first position of the designated marker line is used as the boundary of the calibration area, and a preset pixel width is extended outwards from the smart unmanned vending machine to obtain the calibration area. In summary, the embodiments of this application can obtain the calibration area of ​​the unmanned vending machine based on visual information captured by a camera, without relying on additional hardware devices. This can reduce the production and maintenance costs of smart unmanned vending machines and improve the stability and versatility of the calibration area determination method.

[0085] After introducing the main inventive concepts of the embodiments of this application, the following is a brief description of the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0086] In one possible implementation, this application provides a method for determining the calibration area of ​​an intelligent unmanned vending machine, the overall process of which is shown in the schematic diagram below. Figure 1 As shown, it includes the following:

[0087] In step 101, a first action video is captured by a camera, and the first action video is reversed.

[0088] It should be noted that the first action video is the action video of the smart unmanned vending machine door closing, i.e., the closing action video, and the second action video is the action video of the smart unmanned vending machine door opening, i.e., the opening action video. Since the door frame changes from a closed state to an open state when the door is opened, and changes from an open state to a closed state when the door is closed, both action processes include the position of the door in the closed state when the door is stationary. That is, the embodiment of this application can determine the position of the door axis based on the position of the door in the closed state. However, considering the potential for camera flickering or overexposure during the door opening process, which could lead to video interference and unstable door line detection, this application prioritizes determining the position of the designated marker line in the target object based on the inverted first action video. This effectively reduces abnormal judgments caused by interference introduced into the action video. Furthermore, in this embodiment, the first action video is inverted from the end of the video. The inverted closing action video and the normal opening action video both have the target object (i.e., the cabinet door) in the starting video frame in a closed state, and the cabinet door's state is the same. Therefore, the inverted closing action video and the normal opening action video are essentially the same video, allowing the cabinet door axis position to be identified through the same process as the dynamic visual detection of the opening video. This reduces process design costs, facilitates the identification of the cabinet door axis position, and thus more accurately determines the specific coordinates of the calibrated position.

[0089] In step 102, the first motion video after being reversed is subjected to video foreground and background recognition processing to determine the target object in motion in the video frame.

[0090] In this embodiment of the application, the video foreground and background recognition processing specifically includes: capturing the moving parts in the video frame (i.e., identifying the area where the cabinet door is located) through a foreground and background dynamic detection algorithm, so as to obtain the location of the cabinet door in different states in several consecutive frames during the opening or closing period. Since the cabinet door position detected in each video frame will be different, this embodiment of the application needs to compare and overlap the cabinet door positions detected in these frames to determine the location of the cabinet door in the closed state, that is, to determine the position of the target object in motion in the video frame.

[0091] In step 103, based on the target object, the first position of the specified marker line in the target object is determined.

[0092] In this embodiment, the designated marker line in the target object refers to the cabinet door axis. Based on the position of the cabinet door in the closed state, the first position of the cabinet door axis can be determined. Since the items in the intelligent unmanned vending machine will inevitably pass through the first position of the cabinet door axis during the process of taking out or putting in the vending machine, this application uses the first position of the cabinet door axis as the boundary to form a marked area as a reference for judging whether the items are being taken out or put in, thus determining whether the items are in the process of being taken out or put in.

[0093] In step 104, the first position where the designated marker line is located is taken as the boundary of the calibration area, and the calibration area is extended outward by a preset pixel width to obtain the calibration area.

[0094] For example, if the first action video is the action video of the smart unmanned vending machine door closing, then the target object in motion in the video frame is the door, and the calibration area is the vicinity of the door axis of the smart unmanned vending machine. The designated mark line is the door axis, and the rectangle formed by extending a certain pixel width outward from the door axis is the calibration area required by this application.

[0095] It should be noted that in step 104, the outer side is the side away from the smart unmanned vending machine from the camera's perspective. The calibration area is a rectangular area. The preset pixel width is set manually based on experience. For example, based on the recognition effect of calibration areas with different pixel widths set multiple times, it is determined that the best recognition effect for the movement process of the target item is achieved when the preset pixel width is 0.2 meters. Therefore, the preset pixel width is 0.2 meters.

[0096] In one possible implementation, before designating the first position where the marker line is located as the boundary of the calibration area, this embodiment of the application will also determine that the first position is within a preset range. For example... Figure 2 As shown, the cabinet door axis is represented by the black dotted line in the diagram. The left side represents the interior of the smart unmanned vending machine, and the right side represents the exterior. Due to potential errors in the video foreground / background recognition process, this application sets a preset range to determine the accuracy of the recognition result of the specified marker line (i.e., the cabinet door axis) in the target object. If the first position of the specified marker line in the target object is within the preset range, such as... Figure 3 As shown, in the starting video frame of the first action video, a coordinate system is first established with the lower left corner of the starting video frame as the origin. The preset range is the horizontal coordinate range of 50 to 60. The horizontal coordinate of the first position of the specified marker line in the target object obtained in step 103 is 55, which is within the preset range of 50 to 60. Therefore, the recognition result of the specified marker line (i.e., the cabinet door axis) in the target object is accurate. Based on the first position of the specified marker line as the boundary of the calibration area, the preset pixel width is extended to the outside of the smart unmanned vending machine to obtain the calibration area.

[0097] In another possible implementation, in the reversed first action video (i.e., the reversed door-closing action video), if the first position is outside a preset range, such as Figure 4 As shown, in this embodiment of the application, the following steps will be performed on the second action video:

[0098] In step 401, a second action video is captured using a camera. This second action video precedes the first action video, and the first starting frame of the reversed first action video has the same state as the target object in the second starting frame of the second action video.

[0099] In step 402, if the first position is outside the preset range, then the second action video is subjected to video foreground and background recognition processing to determine the target object in motion in the video frame.

[0100] In step 403, based on the target object, the second position of the specified marker line in the target object is determined.

[0101] In step 404, the second position where the designated marker line is located is used as the boundary of the calibration area.

[0102] In this embodiment, if the position of the door axis determined by the first action video after being reversed is inaccurate, the position of the specified marker line in the target object will be determined based on the position of the specified marker line in the second action video. Therefore, this embodiment can determine the position of the specified marker line in the target object through the door opening or closing video, which can solve the problem of poor calibration area judgment caused by abnormal images (such as shadow images, shaking images, etc.) introduced by the camera due to hardware reasons at the initial opening of the camera.

[0103] It should be noted that this embodiment of the application also uses an invalid motion information filtering module to eliminate any invalid additional movements that may exist in the first and second action videos, such as pedestrians or objects moving nearby, or limbs that may extend into the cabinet body. These movements can interfere with the determination of the cabinet door's movement and position. The invalid motion information filtering module filters out other relatively small motion pixels based on the direction of optical flow movement and the clustering of optical flow information, thereby filtering out some interference. After passing through this module, the movement and position information of the cabinet door can be determined more accurately.

[0104] Furthermore, before using the second position where the designated marker line is located as the boundary of the calibration area, this embodiment of the application will also determine that the second position is within a preset range, ensuring that the identification result of the designated marker line (i.e., the cabinet door axis) in the target object is accurate, and making the position of the finally determined calibration area accurate.

[0105] In one possible implementation, if the second position is outside a preset range, that is, the position of the cabinet door axis determined in the door opening action video is outside the preset range, then the first action video and the second action video are determined to be invalid videos, and the first action video and the second action video are discarded.

[0106] In one possible implementation, step 103, determining the first position of the specified marker line within the target object based on the target object, can be implemented as follows:

[0107] Determine the position of the specified marker line within the first starting video frame, and use this position as the first position. For example... Figure 5 As shown in the figure, the rectangular trapezoidal area is the target object (i.e., the cabinet door of the smart unmanned vending machine). For example, if the position coordinate of the specified marker line (the axis of the cabinet door) is 52, then the position of the specified marker line in the first starting point video frame will be taken as the first position.

[0108] Based on the same principle, in step 403, determining the second position of the specified marker line within the target object, based on the target object, can be implemented as follows:

[0109] Determine the position of the specified marker line in the second starting point video frame, and use the position of the specified marker line in the second starting point video frame as the second position.

[0110] In one possible implementation, after determining the location of the calibration area, if it is detected that a target item has moved from the first side of the calibration area to the second side of the calibration area, such as... Figure 6 In the diagram, the marked area is the rectangular area formed by the dotted line. Moving from the left side of the marked area to the right side of the marked area is equivalent to moving from inside the smart vending machine to outside the smart vending machine, which means that the target item has been moved out of the smart vending machine.

[0111] If the target item is detected to have moved from the second side of the calibration area to the first side of the calibration area, such as Figure 6 If the item moves from the right side of the marked area to the left side of the marked area, which is equivalent to moving from the outside of the smart vending machine to the inside of the smart vending machine, then the item will be placed back into the smart vending machine.

[0112] In summary, the embodiments of this application can obtain the calibration area of ​​the unmanned vending machine based on visual information captured by the camera, without relying on additional hardware devices. This can reduce the production and maintenance costs of intelligent unmanned vending machines and improve the stability and versatility of the calibration area determination method.

[0113] Based on the same inventive concept, embodiments of this application also provide a calibration area determination device for intelligent unmanned vending machines, such as... Figure 7 As shown, the device 700 includes:

[0114] The video capture module 701 is configured to capture a first action video through a camera and to reverse the first action video.

[0115] The dynamic target recognition module 702 is configured to perform video foreground and background recognition processing on the first action video after it has been reversed to determine the target object in motion in the video frame.

[0116] The marker line position determination module 703 is configured to determine the first position of a specified marker line in the target object based on the target object;

[0117] The calibration area determination module 704 is configured to take the first position where the designated marker line is located as the boundary of the calibration area, and extend it to the outside of the smart unmanned vending machine by a preset pixel width to obtain the calibration area, wherein the outside is the side away from the smart unmanned vending machine from the camera's perspective, and the calibration area is a rectangular area.

[0118] In one possible implementation, before executing the step of using the first position where the specified marker line is located as the boundary of the calibration region, the calibration region determination module is further configured to:

[0119] It is determined that the first position is within a preset range.

[0120] In one possible implementation, the device further includes:

[0121] The video capture module is configured to capture a second action video through a camera; wherein the second action video is located before the first action video, and the first starting video frame of the first action video after being reversed has the same state as the target object in the second starting video frame of the second action video;

[0122] The dynamic target recognition module is configured to perform video foreground and background recognition processing on the second action video if the first position is outside the preset range, and determine the target object in motion in the video frame.

[0123] The marker line position determination module is configured to determine a second position of the specified marker line in the target object based on the target object;

[0124] The calibration area determination module is configured to use the second position where the specified marker line is located as the boundary of the calibration area.

[0125] In one possible implementation, before executing the step of using the second position where the specified marker line is located as the boundary of the calibration region, the calibration region determination module is further configured to:

[0126] The second position is determined to be within the preset range.

[0127] In one possible implementation, the calibration region determination module is further configured to:

[0128] If the second position is outside the preset range, then both the first action video and the second action video are determined to be invalid videos, and both the first action video and the second action video are discarded.

[0129] In one possible implementation, the step of determining a first position of a specified marker line within the target object is performed based on the target object, wherein the marker line position determination module is configured to:

[0130] Determine the position of the designated marker line in the first starting video frame;

[0131] The position of the specified marker line in the first starting point video frame is taken as the first position;

[0132] The module for determining the position of the specified marker line within the target object is configured to:

[0133] Determine the position of the designated marker line in the second starting point video frame;

[0134] The position of the specified marker line in the second starting point video frame is taken as the second position.

[0135] In one possible implementation, the device is further configured to:

[0136] If the target item is detected to have moved from the first side of the marked area to the second side of the marked area, it is determined that the target item will be removed from the smart unmanned vending machine.

[0137] If the target item is detected to have moved from the second side of the calibration area to the first side of the calibration area, it is determined that the target item will be returned to the smart unmanned vending machine.

[0138] The following reference Figure 8 To describe an electronic device 130 according to this embodiment of the present application. Figure 8 The electronic device 130 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0139] like Figure 8As shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).

[0140] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0141] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0142] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0143] Electronic device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 130, and / or with any device that enables electronic device 130 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, electronic device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in electronic device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0144] In an exemplary embodiment, this application also provides a computer-readable storage medium including instructions, such as a memory 132 including instructions, which can be executed by a processor 131 of an electronic device 130 to complete the above-described method for determining the calibration area of ​​an intelligent unmanned vending machine. Optionally, the computer-readable storage medium can be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0145] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor 131, implements the calibration area determination method for the intelligent unmanned vending machine provided in this application.

[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0148] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0150] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining a calibration area of an intelligent unmanned cabinet, characterized in that, The method includes: The first action video is captured by a camera, and the first action video is played in reverse. Perform foreground and background recognition processing on the first action video after it has been reversed to identify the target object in motion in the video frame. Based on the target object, determine the first position where the specified marker line is located in the target object; The first position where the designated marker line is located is taken as the boundary of the calibration area, and the calibration area is extended outward by a preset pixel width to obtain the calibration area, wherein the outer side is the side away from the smart unmanned vending machine from the camera's perspective; Before setting the first position where the designated marker line is located as the boundary of the calibration area, the method further includes: It is determined that the first position is within a preset range; The method further includes: A second action video is captured by a camera; wherein the second action video is located before the first action video, and the first starting frame of the first action video after being reversed is the same as the state of the target object in the second starting frame of the second action video; If the first position is outside the preset range, then the second action video is processed for video foreground and background recognition to determine the target object in motion in the video frame; Based on the target object, determine the second position where the specified marker line is located in the target object; The second position where the specified marker line is located is taken as the boundary of the calibration area.

2. The method of claim 1, wherein, Before setting the second position where the designated marker line is located as the boundary of the calibration area, the method further includes: The second position is determined to be within the preset range.

3. The method of claim 2, wherein, The method further includes: If the second position is outside the preset range, then both the first action video and the second action video are determined to be invalid videos, and both the first action video and the second action video are discarded.

4. The method of claim 1, wherein, Determining the first position of a specified marker line within the target object based on the target object includes: Determine the position of the designated marker line in the first starting video frame; The position of the specified marker line in the first starting point video frame is taken as the first position; Determining the second position of the specified marker line within the target object based on the target object includes: Determine the position of the designated marker line in the second starting point video frame; The position of the specified marker line in the second starting point video frame is taken as the second position.

5. The method of claim 1, wherein, The method further includes: If the target item is detected to have moved from the first side of the marked area to the second side of the marked area, it is determined that the target item will be removed from the smart unmanned vending machine. If the target item is detected to have moved from the second side of the calibration area to the first side of the calibration area, it is determined that the target item will be returned to the smart unmanned vending machine.

6. A device for determining a calibration area of an intelligent unmanned cabinet, characterized in that, The device includes: The video capture module is configured to capture a first action video through a camera and then reverse the first action video. The dynamic target recognition module is configured to perform foreground and background recognition processing on the first action video after it has been reversed to identify the target object in motion in the video frame. The marker line position determination module is configured to determine the first position of a specified marker line in the target object based on the target object; The calibration area determination module is configured to take the first position where the specified marker line is located as the boundary of the calibration area, and extend it to the outside of the smart unmanned vending machine by a preset pixel width to obtain the calibration area, wherein the outside is the side away from the smart unmanned vending machine from the camera's perspective, and the calibration area is a rectangular area; Before executing the step of taking the first position where the specified marker line is located as the boundary of the calibration area, the calibration area determination module is further configured to: determine that the first position is within a preset range; The device further includes: a video acquisition module configured to acquire a second action video via a camera; wherein the second action video is located before the first action video, and the first starting video frame of the first action video after being reversed is in the same state as the target object in the second starting video frame of the second action video. The dynamic target recognition module is configured to perform video foreground and background recognition processing on the second action video if the first position is outside the preset range, and determine the target object in motion in the video frame; The marker line position determination module is configured to determine a second position of the specified marker line in the target object based on the target object; The calibration area determination module is configured to use the second position where the specified marker line is located as the boundary of the calibration area.

7. An electronic device, comprising: include: Processor and memory; The memory is used to store the processor-executable instructions; The processor is configured to execute the instructions to implement the calibration area determination method for the intelligent unmanned vending machine as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the calibration area determination method for the intelligent unmanned vending machine as described in any one of claims 1-5.

Images