Picking detection method, device and system based on infrared touch frame

Abstract

The invention discloses a picking detection method, device and system based on an infrared touch frame, and the method comprises the steps: obtaining a picking task queue, and determining a current target storage location according to the picking task queue, the picking task queue comprising to-be-picked storage location information arranged in order; a touch data frame reported by the infrared touch frame in real time is received, and the touch data frame comprises a plurality of touch points generated by hand shielding of an operator; based on a preset time window, the collected touch points are accumulated and stored in a buffer area, the number of the touch points of each storage location is determined according to all the touch points in the buffer area, sorting is conducted according to the number of the touch points, and a storage location candidate list is obtained; the storage location with the largest number of touch points in the storage location candidate list is compared with the current target storage location, and the correctness of the picking operation is judged according to a comparison result; and according to a comparison result, updating the current target storage location of the order picking task queue and controlling the state indication of the corresponding storage location.

Description

Technical Field

[0001] This invention relates to the field of material picking management technology, and in particular to a picking detection method, device and system based on an infrared touch frame. Background Technology

[0002] Against the backdrop of the manufacturing industry's transformation and upgrading towards intelligence and digitalization, the precision and efficiency of production logistics are increasingly becoming crucial links. Material racks, as the core carrier for material storage and management on the production floor, directly impact production cycle time and product quality due to their level of intelligence. Especially in flexible production models with multiple varieties and small batches, material picking operations, due to their high frequency and diverse categories, place higher demands on the accuracy and guidance of the picking process. To improve the intelligence level of the picking process, several technical approaches have emerged in the industry. Among them, solutions based on single-point photoelectric sensors achieve detection by independently deploying sensors in each material compartment; their system complexity and wiring costs are directly proportional to the number of compartments. Visual inspection solutions utilize cameras to capture image information and identify hand movements through image processing algorithms; however, their recognition effect is easily affected by ambient light and occlusion, and the high-precision requirements drive up system computing power and costs. LiDAR scanning solutions detect occlusion by scanning planar point clouds, but their spatial resolution and real-time performance face challenges in high-density warehouse layouts. Overall, the above solutions still have room for improvement in achieving high-precision, low-cost, and high-reliability detection of picking actions in dense warehouse locations. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose a picking and detection method, device, system and computer program product based on an infrared touch frame, which aims to solve the technical problems in the prior art that it is difficult to balance low hardware complexity with high precision hand movements and warehouse location recognition capabilities in scenarios with multiple warehouse locations densely arranged.

[0004] To achieve the above objectives, the present invention provides a picking and detection method based on an infrared touch frame, the method comprising: Obtain the picking task queue, and determine the current target storage location based on the picking task queue, wherein the picking task queue includes storage location information to be picked in sequence; Receive touch data frames reported in real time by the infrared touch frame, wherein the touch data frames include multiple touch points generated by the operator's hand blocking the view; Based on a preset time window, the collected touch points are accumulated and stored in a buffer. The number of touch points for each storage location is determined based on all the touch points in the buffer, and the locations are sorted according to the number of touch points to obtain a candidate list of storage locations. The location with the most touch points in the candidate location list is compared with the current target location, and the correctness of the picking operation is determined based on the comparison result. The current target storage location and the status indication of the corresponding storage location in the picking task queue are updated based on the comparison results.

[0005] Preferably, the touch point includes a touch point number and corresponding touch point coordinates, wherein the touch point coordinates are coordinate values ​​in a two-dimensional coordinate system established by the infrared touch frame.

[0006] Preferably, determining the number of touch points for each storage location based on all touch points in the buffer includes: The coordinates of each touch point in the buffer are matched with the predefined storage location area range, and the total number of touch points falling into each storage location area range is counted as the number of touch points for the corresponding storage location; wherein, each storage location area range is defined based on the set of vertex coordinates in the infrared touch frame coordinate system.

[0007] Preferably, the storage location area is a rectangular area or a polygonal area, wherein, When the storage area is a rectangular area, the system determines whether the touch point coordinates fall within the corresponding rectangular area by judging whether the X coordinate value of the touch point is between the preset minimum X coordinate and maximum X coordinate of the corresponding rectangular area, and whether the Y coordinate value is between the minimum Y coordinate and maximum Y coordinate. When the storage area is a polygonal region, the ray casting method is used to determine whether the coordinates of the touch point fall within the corresponding polygonal region.

[0008] Preferably, determining the correctness of the picking operation based on the comparison results includes: If the warehouse location with the most touch points in the warehouse location candidate list is the current target warehouse location, then it is determined to be a correct picking operation; If the storage location with the most touch points in the candidate storage location list is the storage location where the most recent correct picking was performed, then it is determined to be a duplicate operation. If the warehouse location with the most touch points in the warehouse location candidate list is neither the current target warehouse location nor a completed warehouse location, then it is determined to be an incorrect picking operation. If the storage location with the most touch points in the candidate storage location list is an unidentified area, it is determined to be an invalid touch. The unidentified area consists of touch points that do not fall within the range of any of the storage location areas.

[0009] Preferably, updating the current target storage location of the picking task queue based on the comparison result includes: When a picking operation is determined to be correct, the current target location is marked as a completed location, and the next location to be picked in the picking task queue is updated to the current target location. When an incorrect picking operation, duplicate operation, or invalid touch is detected, the current target storage location remains unchanged.

[0010] Preferably, the status indication of the corresponding storage location includes: Control the indicator light corresponding to the current target storage location to display a first preset color; The indicator light corresponding to the completed storage location will display the second preset color; When an incorrect picking operation is detected, the indicator light corresponding to the location of the incorrect picking operation will display a third preset color and flash as an alarm.

[0011] To achieve the above objectives, the present invention also provides a picking and detection device based on an infrared touch frame, the device comprising: The task acquisition unit is used to acquire the picking task queue and determine the current target storage location based on the picking task queue, wherein the picking task queue includes storage location information to be picked in sequence; The touch point acquisition unit is used to receive touch data frames reported in real time by the infrared touch frame, wherein the touch data frames include multiple touch points generated by the operator's hand blocking the view. The statistics unit is used to accumulate the collected touch points and store them in the buffer based on a preset time window, determine the number of touch points for each storage location based on all the touch points in the buffer, and sort them according to the number of touch points to obtain a candidate list of storage locations. The judgment unit is used to compare the warehouse location with the most touch points in the warehouse location candidate list with the current target warehouse location, and determine the correctness of the picking operation based on the comparison result; The update control unit is used to update the current target storage location of the picking task queue and the status indication of the corresponding storage location based on the comparison result.

[0012] To achieve the above objectives, the present invention also proposes a picking and detection system based on an infrared touch frame, the system comprising: Infrared touch frames are installed around the material rack to form a two-dimensional touch sensing area and detect the operator's hand occlusion in real time to generate touch data frames including multiple touch points; The controller is communicatively connected to the infrared touch frame and is used to receive and process the touch data frames to perform the detection and judgment of picking actions and generate corresponding control commands. LED indicator strips, which communicate with the controller, are installed at each storage location on the material shelf to adjust the status and color of the indicator lights at the corresponding storage location according to the control commands.

[0013] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a computer program that is executed by a processor to implement the steps of a picking and detection method based on an infrared touch frame as described in the above embodiments.

[0014] To achieve the above objectives, the present invention also proposes a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of a picking and detection method based on an infrared touch frame as described in the above embodiments.

[0015] Beneficial effects: The above solution, by introducing an infrared touch frame as a unified coordinate acquisition front end, replaces traditional discrete sensor arrays and other solutions, greatly simplifying hardware deployment and wiring from a system architecture perspective. Simultaneously, by abstracting hand movements into continuous two-dimensional coordinate points, it provides a unified data foundation for subsequent algorithms to achieve precise and flexible judgments, enabling intelligent and automated detection and guidance of picking actions, and achieving accurate perception and judgment of complex picking actions. Furthermore, by defining the data structure of touch points, it provides a unified and standardized input data foundation for all subsequent coordinate processing and region matching algorithms, ensuring the feasibility of the solution and the consistency of processing.

[0016] By matching predefined storage location areas (rectangular or polygonal) with physical coordinates, the detection logic is decoupled from the physical layout. This provides a quantitative basis for subsequent voting decisions, allowing adjustments and additions to the storage location layout to be made without modifying any hardware, only requiring software configuration changes. This significantly improves the system's flexibility and configurability. Introducing ray casting to process polygonal areas provides high-precision point location judgment capabilities for complex or irregular storage location shapes, greatly enhancing the solution's adaptability to different shelving layouts. Whether it's a regular rectangular storage location or an irregular polygonal storage location, accurate judgment can be made, enabling widespread application in various complex actual warehousing environments and improving the solution's versatility and practical value.

[0017] By introducing a statistical (touchpoint number voting) judgment mechanism, the system can not only identify correct picking operations but also effectively distinguish and handle abnormal situations such as duplicate operations, incorrect operations, and invalid touches, greatly improving the system's intelligence and robustness. Furthermore, the status update mechanism based on the judgment results enables automated advancement and closed-loop management of the picking task process. The system can autonomously update the task queue based on the operation results, advancing the process only when the operation is correct; otherwise, it maintains the current state awaiting correction. This ensures the strict orderliness and traceability of the work process, thereby achieving real-time monitoring and automated management of work process compliance and effectively preventing operational errors such as incorrect sequence and duplicate picking.

[0018] By clearly defining the visual instructions triggered by different judgment results, operators can be provided with real-time and clear visual guidance through intuitive and multi-dimensional color and flashing codes. This efficient interaction method can significantly reduce the cognitive load of operators, guide them to complete operations quickly and accurately, thereby effectively improving picking efficiency and accuracy from a human factors perspective and reducing training costs. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a picking and detection method based on an infrared touch frame, as provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of a picking and inspection system provided in an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the overall process of a picking and inspection method provided in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of a picking and detection device based on an infrared touch frame, provided in an embodiment of the present invention.

[0024] The realization of the invention's objective, its functional characteristics, and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The present invention will be described in detail below with reference to the embodiments.

[0027] Reference Figure 1 The diagram shown is a flowchart of a picking and detection method based on an infrared touch frame according to an embodiment of the present invention.

[0028] In this embodiment, the method is implemented based on an infrared touch frame picking and detection system; wherein, the system includes: Infrared touch frames are installed around the material rack to form a two-dimensional touch sensing area and detect the operator's hand occlusion in real time to generate touch data frames including multiple touch points; The controller is communicatively connected to the infrared touch frame and is used to receive and process the touch data frames to perform the detection and judgment of picking actions and generate corresponding control commands. LED indicator strips, which communicate with the controller, are installed at each storage location on the material shelf to adjust the status and color of the indicator lights at the corresponding storage location according to the control commands.

[0029] Reference Figure 2 As shown, the system includes: an infrared touch frame, installed around the perimeter of the material rack to form a two-dimensional touch sensing area covering the entire picking plane of the rack, and to detect the operator's hand obstruction in real time, generating touch data frames containing multiple touch point coordinates; a controller (not shown in the figure) (such as a development platform based on ESP32), which communicates with the infrared touch frame via a USB interface to receive and process the touch data frames, execute the picking action detection and intelligent judgment algorithm of this method, and generate corresponding control commands; and an LED indicator strip (such as a WS2812 series addressable RGB LED strip), which communicates with the controller via a single-wire data protocol. Its LED units are pre-installed physically and logically bound to each storage location on the material rack, and are used to adjust the indicator status (such as on / off, flashing) and color (such as red, green, yellow) of the corresponding storage location in real time according to the control commands, thereby providing intuitive operation guidance and feedback. In specific implementation, the operator faces the material rack equipped with this system, and the operating space in front of them is seamlessly covered by the infrared touch frame. When the hand is placed into this space, the subsequent detection process is triggered.

[0030] Furthermore, the method includes: S11, Obtain the picking task queue, and determine the current target storage location based on the picking task queue, wherein the picking task queue includes storage location information to be picked in sequence.

[0031] S12, receive touch data frames reported in real time by the infrared touch frame, wherein the touch data frames include multiple touch points caused by the operator's hand blocking the view.

[0032] The touch point includes a touch point number and corresponding touch point coordinates, wherein the touch point coordinates are coordinate values ​​in a two-dimensional coordinate system established by the infrared touch frame.

[0033] In this embodiment, the controller receives a specific picking task order from the upper-level Manufacturing Execution System (MES) via a network interface or serial port. This task order is a picking task queue, which contains the identifiers of the storage locations to be picked, arranged in process order, such as ["A01", "B03", "C02"]. During system initialization, the first storage location "A01" in the queue is automatically identified as the current target storage location.

[0034] Reference Figure 3 As shown, the infrared touch frame continuously scans at a high frame rate (e.g., 50Hz). When a hand is detected obstructing the view, a data frame is sent to the controller via the USB interface. Each data frame contains information on all detected touch points at that moment. Each touch point includes a globally unique touch point number (pointId) and its precise coordinates (X, Y) in the two-dimensional Cartesian coordinate system established by the infrared touch frame. The coordinate values ​​are typically represented in a normalized numerical range (e.g., 0-65535). The controller parses these data frames in real time, providing the raw coordinate data stream for subsequent processing.

[0035] S13. Based on a preset time window, accumulate the collected touch points and store them in a buffer. Determine the number of touch points for each storage location based on all the touch points in the buffer, and sort them according to the number of touch points to obtain a candidate list of storage locations.

[0036] Furthermore, in step S13, determining the number of touch points for each storage location based on all touch points in the buffer includes: The coordinates of each touch point in the buffer are matched with the predefined storage location area range, and the total number of touch points falling into each storage location area range is counted as the number of touch points for the corresponding storage location; wherein, each storage location area range is defined based on the set of vertex coordinates in the infrared touch frame coordinate system.

[0037] Furthermore, the storage location area is a rectangular area or a polygonal area, wherein, When the storage area is a rectangular area, the system determines whether the touch point coordinates fall within the corresponding rectangular area by judging whether the X coordinate value of the touch point is between the preset minimum X coordinate and maximum X coordinate of the corresponding rectangular area, and whether the Y coordinate value is between the minimum Y coordinate and maximum Y coordinate. When the storage area is a polygonal region, the ray method is used to determine whether the coordinates of the touch point fall within the corresponding polygonal region. This includes: emitting a ray from the coordinates of the touch point along the horizontal direction, counting the number of intersections between the ray and the boundary line of the polygonal region, and if the number of intersections is odd, then the coordinates of the touch point are determined to fall within the polygonal region.

[0038] In this embodiment, firstly, during system initialization, the algorithm predefines the area range of each storage location within the same infrared touch frame coordinate system based on the actual physical layout of the shelves. Each area range is precisely defined by the set of coordinates of its boundary vertices. For example, a rectangular storage location can be defined by the coordinates of its upper left vertex (minX, minY) and lower right vertex (maxX, maxY); an irregular polygonal storage location is defined by a series of ordered vertex coordinates (x1, y1), (x2, y2)...(xn, yn). The matching process involves traversing the coordinates (Xi, Yi) of all touch points accumulated in the buffer within a preset time window, and for each touch point, determining which predefined storage location area it falls within.

[0039] For rectangular areas, the judgment logic is as follows: if both minX≤Xi≤maxX and minY≤Yi≤maxY are satisfied, then the touch point falls into this rectangular storage location. For polygonal areas, the classic ray method from computational geometry is used for judgment: draw a horizontal ray to the right (positive X-axis direction) from the coordinates of the touch point, and count the number of intersections between the ray and all sides of the polygon; if the total number of intersections is odd, then the touch point is determined to be inside the polygon (i.e., falls into this storage location). After the system completes the classification judgment for all touch points, it counts the total number of touch points belonging to each storage location, and this total number is the number of touch points for that storage location. Finally, the system sorts all storage locations in descending order of their touch point counts, generating an ordered list of candidate storage locations.

[0040] In practical implementation, a minimum effective touch point threshold M is set (preferably M = 43). Within each consecutive preset time window (e.g., 100ms), the system accumulates touch points and checks their quantity. If the accumulated touch point quantity reaches or exceeds the threshold M within a certain time window, it is determined to be a valid operation intention, triggering and entering the subsequent storage location determination process (i.e., performing matching, statistics, and sorting). Conversely, if the system detects that the accumulated touch point quantity fails to reach the threshold M within K consecutive preset time windows (preferably K = 13), it determines that the touch event is invalid or caused by interference, automatically clears the buffer and discards relevant data, waiting for the next valid touch trigger. This mechanism constitutes the core filtering logic at the software level, effectively resisting transient interference and invalid triggers, ensuring that the system only responds to stable and continuous hand insertion actions, thereby significantly improving the accuracy and reliability of detection.

[0041] S14, compare the warehouse location with the most touch points in the warehouse location candidate list with the current target warehouse location, and determine the correctness of the picking operation based on the comparison result.

[0042] Furthermore, in step S14, determining the correctness of the picking operation based on the comparison result includes: If the warehouse location with the most touch points in the warehouse location candidate list is the current target warehouse location, then it is determined to be a correct picking operation; If the storage location with the most touch points in the candidate storage location list is the storage location where the most recent correct picking was performed, then it is determined to be a duplicate operation. If the warehouse location with the most touch points in the warehouse location candidate list is neither the current target warehouse location nor a completed warehouse location, then it is determined to be an incorrect picking operation. If the storage location with the most touch points in the candidate storage location list is an unidentified area, it is determined to be an invalid touch. The unidentified area consists of touch points that do not fall within the range of any of the storage location areas.

[0043] In this embodiment, determining the correctness of the picking operation based on the comparison results is a multi-branch logical decision-making process: if the first-ranked storage location is exactly the "current target storage location" indicated by the system (such as "A01"), then the operation is determined to be a correct picking operation; if the first-ranked storage location is a storage location marked as completed / historical in the task queue (meaning that the material has been correctly picked), then it is determined to be a duplicate operation, and the operator is prompted that there may be a misjudgment; if the first-ranked storage location is neither the current target storage location nor a completed storage location, but any other storage location (such as "B03" or "C02"), then it is determined to be an incorrect picking operation. In addition, there is a special case, that is, if the first-ranked category is "unidentified area", which means that during the statistical process, the largest number of touch points did not fall within any predefined storage location area (possibly because the hand is located in the storage location gap or accidentally touched the edge), then the action is determined to be an invalid touch, and the system issues a warning prompt. This judgment logic closely integrates low-level coordinate matching results with high-level task status (target, completed), enabling intelligent understanding of operational semantics (whether it is correct, whether it is repeated, whether the wrong item was taken).

[0044] S15, update the current target storage location and the status indication of the corresponding storage location in the picking task queue according to the comparison result.

[0045] Furthermore, in step S15, updating the current target storage location of the picking task queue based on the comparison result includes: When a picking operation is determined to be correct, the current target location is marked as a completed location, and the next location to be picked in the picking task queue is updated to the current target location. When an incorrect picking operation, duplicate operation, or invalid touch is detected, the current target storage location remains unchanged.

[0046] Furthermore, in step S15, the control of the status indication of the corresponding storage location includes: Control the indicator light corresponding to the current target storage location to display a first preset color; The indicator light corresponding to the completed storage location will display the second preset color; When an incorrect picking operation is detected, the indicator light corresponding to the location of the incorrect picking operation will display a third preset color and flash as an alarm.

[0047] In this embodiment, the system only initiates the task process when the operation is determined to be a "correct picking operation." Specifically, the current target location (e.g., "A01") is marked as "completed," and the next location to be picked (e.g., "B03") is automatically set as the new current target location from the task queue. If the operation is determined to be an error, a duplicate operation, or an invalid touch, the task queue status remains unchanged, the current target location remains in its original state, and the system waits for the operator to correct the action.

[0048] Controlling the status indicators of corresponding storage locations directly reflects the interaction between the system and the operator. Based on the judgment result and the updated status, the controller immediately drives the LED indicator strip: the LED corresponding to the current target storage location illuminates in the first preset color (e.g., green), guiding the next operation position; the LEDs corresponding to all completed storage locations illuminate in the second preset color (e.g., yellow), providing progress feedback; when an erroneous picking operation occurs, in addition to the target storage location remaining green, the LED corresponding to the mistakenly picked storage location will display a third preset color (e.g., red) and flash or trigger a buzzer, creating a strong audible and visual alarm to immediately stop the error. This real-time, dynamic, multi-color light guidance and alarm mechanism constitutes a clear, closed-loop human-machine interface, greatly reducing the requirements for operator memory and proficiency, enabling new employees to operate quickly and accurately. Simultaneously, through immediate error correction, it significantly reduces the mispicking rate and improves overall picking efficiency and quality.

[0049] Reference Figure 4 The diagram shown is a structural schematic of a picking and detection device based on an infrared touch frame according to an embodiment of the present invention.

[0050] In this embodiment, the device 20 includes: The task acquisition unit 21 is used to acquire the picking task queue and determine the current target storage location based on the picking task queue, wherein the picking task queue includes storage location information to be picked arranged in sequence; The touch point acquisition unit 22 is used to receive touch data frames reported in real time by the infrared touch frame, wherein the touch data frames include multiple touch points generated by the operator's hand blocking the view. The statistics unit 23 is used to accumulate the collected touch points and store them in a buffer based on a preset time window, determine the number of touch points for each storage location based on all the touch points in the buffer, and sort them according to the number of touch points to obtain a candidate list of storage locations. The judgment unit 24 is used to compare the warehouse location with the number of touch points in the warehouse location candidate list with the current target warehouse location, and determine the correctness of the picking operation based on the comparison result; The update control unit 25 is used to update the current target storage location of the picking task queue and the status indication of the corresponding storage location based on the comparison result.

[0051] Each unit module of the device 20 can execute the corresponding steps in the above method embodiment, so the details of each unit module will not be elaborated here. Please refer to the description of the corresponding steps above for details.

[0052] This invention also provides a picking and detection device based on an infrared touch frame. This device includes the picking and detection apparatus based on an infrared touch frame as described above, wherein the picking and detection apparatus based on an infrared touch frame can employ... Figure 4 The structure of the embodiment, correspondingly, can be executed Figure 1 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0053] The device includes: a mobile phone, digital camera, or tablet computer with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, processor, input unit, display unit, and power supply.

[0054] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide access to the memory for the processor and input units.

[0055] The input unit can be used to receive input numerical, character, or image information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in addition to a camera, the input unit of this embodiment may also include a touch-sensitive surface (e.g., a touch screen) and other input devices.

[0056] The display unit can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit may include a display panel, optionally configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar display panel. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor to determine the type of touch event. Subsequently, the processor provides corresponding visual output on the display panel based on the type of touch event.

[0057] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1 The illustrated picking and detection method is based on an infrared touch frame. The computer-readable storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0058] This invention also provides a computer program product, including a computer program / instructions, which are loaded and executed by a processor to implement... Figure 1 This illustrates a picking and detection method based on an infrared touch frame.

[0059] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the device embodiments, equipment embodiments, and storage medium embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions in the method embodiments.

[0060] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A picking and detection method based on an infrared touch frame, characterized in that, The method includes: Obtain the picking task queue, and determine the current target storage location based on the picking task queue, wherein the picking task queue includes storage location information to be picked in sequence; Receive touch data frames reported in real time by the infrared touch frame, wherein the touch data frames include multiple touch points generated by the operator's hand blocking the view; Based on a preset time window, the collected touch points are accumulated and stored in a buffer. The number of touch points for each storage location is determined based on all the touch points in the buffer, and the locations are sorted according to the number of touch points to obtain a candidate list of storage locations. The location with the most touch points in the candidate location list is compared with the current target location, and the correctness of the picking operation is determined based on the comparison result. The current target storage location and the status indication of the corresponding storage location in the picking task queue are updated based on the comparison results.

2. The picking and detection method based on an infrared touch frame according to claim 1, characterized in that, The touch point includes a touch point number and a corresponding touch point coordinate, wherein the touch point coordinate is a coordinate value in a two-dimensional coordinate system established by the infrared touch frame.

3. The picking and detection method based on an infrared touch frame according to claim 1, characterized in that, The step of determining the number of touch points for each storage location based on all touch points in the buffer includes: The coordinates of each touch point in the buffer are matched with the predefined storage location area range, and the total number of touch points falling into each storage location area range is counted as the number of touch points for the corresponding storage location; wherein, each storage location area range is defined based on the set of vertex coordinates in the infrared touch frame coordinate system.

4. The picking and detection method based on an infrared touch frame according to claim 3, characterized in that, The storage area is defined as a rectangular or polygonal region. When the storage area is a rectangular area, the system determines whether the touch point coordinates fall within the corresponding rectangular area by judging whether the X coordinate value of the touch point is between the preset minimum X coordinate and maximum X coordinate of the corresponding rectangular area, and whether the Y coordinate value is between the minimum Y coordinate and maximum Y coordinate. When the storage area is a polygonal region, the ray casting method is used to determine whether the coordinates of the touch point fall within the corresponding polygonal region.

5. The picking and detection method based on an infrared touch frame according to claim 1, characterized in that, The determination of the correctness of the picking operation based on the comparison results includes: If the warehouse location with the most touch points in the warehouse location candidate list is the current target warehouse location, then it is determined to be a correct picking operation; If the storage location with the most touch points in the candidate storage location list is the storage location where the most recent correct picking was performed, then it is determined to be a duplicate operation. If the warehouse location with the most touch points in the warehouse location candidate list is neither the current target warehouse location nor a completed warehouse location, then it is determined to be an incorrect picking operation. If the storage location with the most touch points in the candidate storage location list is an unidentified area, it is determined to be an invalid touch. The unidentified area consists of touch points that do not fall within the range of any of the storage location areas.

6. The picking and detection method based on an infrared touch frame according to claim 5, characterized in that, Updating the current target storage location of the picking task queue based on the comparison result includes: When a picking operation is determined to be correct, the current target location is marked as a completed location, and the next location to be picked in the picking task queue is updated to the current target location. When an incorrect picking operation, duplicate operation, or invalid touch is detected, the current target storage location remains unchanged.

7. The picking and detection method based on an infrared touch frame according to claim 1, characterized in that, The status indication of the corresponding storage location is controlled by: Control the indicator light corresponding to the current target storage location to display a first preset color; The indicator light corresponding to the completed storage location will display the second preset color; When an incorrect picking operation is detected, the indicator light corresponding to the location of the incorrect picking operation will display a third preset color and flash as an alarm.

8. A picking and detection device based on an infrared touch frame, characterized in that, The device includes: The task acquisition unit is used to acquire the picking task queue and determine the current target storage location based on the picking task queue, wherein the picking task queue includes storage location information to be picked in sequence; The touch point acquisition unit is used to receive touch data frames reported in real time by the infrared touch frame, wherein the touch data frames include multiple touch points generated by the operator's hand blocking the view. The statistics unit is used to accumulate the collected touch points and store them in a buffer based on a preset time window, determine the number of touch points for each storage location based on all the touch points in the buffer, and sort them according to the number of touch points to obtain a candidate list of storage locations. The judgment unit is used to compare the warehouse location with the most touch points in the warehouse location candidate list with the current target warehouse location, and determine the correctness of the picking operation based on the comparison result; The update control unit is used to update the current target storage location of the picking task queue and the status indication of the corresponding storage location based on the comparison result.

9. A picking and detection system based on an infrared touch frame, characterized in that, The system includes: Infrared touch frames are installed around the material rack to form a two-dimensional touch sensing area and detect the operator's hand occlusion in real time to generate touch data frames including multiple touch points; The controller is communicatively connected to the infrared touch frame and is used to receive and process the touch data frames to perform the detection and judgment of picking actions and generate corresponding control commands. LED indicator strips, which communicate with the controller, are installed at each storage location on the material shelf to adjust the status and color of the indicator lights at the corresponding storage location according to the control commands.

10. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, implements the steps of a picking and detection method based on an infrared touch frame as described in any one of claims 1 to 7.

Images