Forklift stop floor method, system and medium based on visual recognition fusion intention perception

Abstract

The application provides a forklift parking floor method and system based on visual recognition and fusion intention perception, a visual sensor is installed on a cargo fork lifting device, real-time recognition of features such as a front rack cross beam is performed, combined with continuous monitoring of the operator's control handle signal, it is determined that the identified height is intentionally parked, and then the system automatically takes over the control, accurately lifts the cargo fork to the calculated target height and stops. Automatic completion of accurate height setting reduces the dependence on operator experience and concentration, avoids visual angle error and repeated fine adjustment in manual mode, and improves operation safety and efficiency. Without relying on any preset floor height database, the operator does not need to manually input or select the target floor before performing storage and retrieval, thereby adapting to complex mixed warehouse scenes with different rack specifications and inconsistent floor heights, and realizing flexible and efficient automatic operation.

Description

Technical Field

[0001] This invention relates to the field of industrial vehicles, and more specifically to the control optimization design of forklifts during the lifting process of storing and retrieving goods. Background Technology

[0002] Forklift controlled stopping refers to the operation of precisely stopping the forks carrying goods at a designated height on the target rack during forklift storage or retrieval operations, using the control system. This operation is crucial for ensuring that pallets can be accurately inserted into or detached from the rack beams.

[0003] In existing technologies, two main modes are used to control the stopping position: manual mode and fully automatic control mode. The former involves the operator directly and continuously controlling the fork lifting speed and stopping position via a handle, relying on real-time visual observation and manual fine-tuning. The latter requires manual input of the target shelf and shelf number before operation; the system then retrieves pre-measured and stored shelf height data from a database, automatically driving the forks to that preset height.

[0004] While both manual and fully automatic modes can successfully achieve correct forklift parking for picking and storing goods, they still have certain technical limitations. In manual mode, efficiency heavily relies on the operator's experience and focus, leading to inaccurate positioning, low efficiency, and safety risks due to obstructed view and judgment errors, and also resulting in high labor intensity. In fully automatic mode, on the one hand, operation is cumbersome, requiring manual selection or input of the target storage location before each operation, disrupting the continuity of work. On the other hand, its adaptability is poor. All rack height parameters must be accurately known and fixed in advance. When facing complex mixed storage scenarios with varying rack specifications and heights, or when layout changes occur, the lack of preset data renders it ineffective, resulting in a severe lack of flexibility. Summary of the Invention

[0005] This invention provides a forklift parking method and system based on visual recognition and intent perception. A visual sensor is installed on the forklift lifting device to identify features such as the crossbeams of the preceding rack in real time. Combined with continuous monitoring of the operator's control handle signals, the system determines that the operator intends to park at the identified height. The system then automatically takes over control, precisely raising the forks to the calculated target height and stopping. This automatic and precise height setting reduces reliance on operator experience and focus, avoids perspective errors and repeated fine-tuning inherent in manual mode, and improves operational safety and efficiency. Furthermore, it eliminates the need for any pre-set floor height database and the operator to manually input or select the target floor before storage or retrieval, thus adapting to complex mixed warehousing scenarios with inconsistent rack specifications and floor heights, achieving flexible and efficient automated operation.

[0006] In a first aspect, the present invention provides a forklift parking method based on visual recognition fusion and intent perception, comprising the following steps:

[0007] The vehicle control system detects whether the vision device identifies the shelf beam, which is located between the nth and n+1th layers of the shelf.

[0008] When the rack beam is detected, the vehicle control system records the ground clearance of the forks at that moment, X1.

[0009] The vehicle control system reads the speed deceleration rate P, which is a value reflecting the change in the lifting speed of the forks. The speed deceleration rate P is compared with a preset deceleration rate threshold Q. When the speed deceleration rate P is greater than the deceleration rate threshold Q, the vehicle control system determines that the forks need to be stopped on the (n+1)th layer.

[0010] The fork height is read in real time. When the fork height X2 reaches X1 + hpre, the fork will automatically stop. hpre is the preset height margin.

[0011] In a second aspect, the present invention provides a forklift parking system based on visual recognition fusion and intent perception, the system comprising:

[0012] The beam detection module is configured to detect whether a shelf beam is identified using a vision device, the shelf beam being located between the nth and n+1th layers of the shelf;

[0013] The reference height recording module is configured so that when a rack beam is detected, the vehicle control system records the ground clearance height X1 of the forks at that moment.

[0014] The intent perception module is configured to read the speed deceleration rate P, which is a value reflecting the change in the lifting speed of the forks. The speed deceleration rate P is compared with a preset deceleration rate threshold Q. When the speed deceleration rate P is greater than the deceleration rate threshold Q, the vehicle control system determines that the forks need to be stopped on the (n+1)th layer.

[0015] The fork stop execution module is configured to read the fork's height off the ground in real time. When the fork's height off the ground X2 reaches X1+h pre, the fork will automatically stop. h pre is the preset height margin.

[0016] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described above.

[0017] In a fourth aspect, the present invention provides an electronic device, including a processor and a memory; the processor is connected to the memory.

[0018] The memory is used to store executable program code; the processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method described above.

[0019] In summary, the present invention has the following beneficial effects:

[0020] 1. By dynamically obtaining the target shelf height through real-time visual recognition of the rack beams and combining this with the perception of the driver's deceleration intentions, semi-automatic and precise shelf stopping is achieved. It does not rely on any preset rack height database, nor does it require manual input of target shelf information before operation. It can adaptively handle rack scenarios with different specifications and shelf heights, significantly improving operational flexibility and automation levels in complex warehousing environments.

[0021] 2. The height allowance is the sum of the installation value of the vision device and the storage and retrieval allowance. This ensures that the final floor height not only takes into account the physical relationship of the equipment installation but also reserves safe operating space in advance. This guarantees that a safe and reasonable gap can be maintained between the device and the shelf beams when storing or retrieving goods, effectively preventing the risk of collisions caused by insufficient reserved space.

[0022] 3. Different reservation values ​​are set for picking and storage status. The reservation value for picking is less than the reservation value for storage. When picking, the lower surface of the pallet is in contact with the upper surface of the shelf, while a certain gap is required when placing goods. Therefore, the reservation height for placing goods should be greater than the reservation height for picking, which improves the rationality of operation under different working conditions.

[0023] 4. The installation height of the vision device is set to be less than the upper limit of the difference between the height of the first-level shelf and the lowest point of the forks. This limitation ensures by design that the vision device can identify the shelf beams before the forks are raised to the first level of the shelf, avoiding early recognition blind spots caused by improper installation.

[0024] 5. The maximum height of the cargo is less than the installation height of the vision device. This ensures that the field of view of the vision sensor is not completely obstructed by the cargo below during lifting, guaranteeing the continuous effectiveness of the visual recognition function.

[0025] 6. Upon determining that a stop is necessary, the system is forced to execute a process of first slowing down and then stopping the forks. This effectively avoids the risk of goods shaking, tipping, or even falling due to sudden stops, ensuring the smoothness and safety of the stopping process.

[0026] 7. A critical height for initiating deceleration has been defined. Deceleration and fork stopping are prohibited above this critical point, preventing delayed deceleration and fork stopping due to excessively short operation intervals, thus improving safety.

[0027] 8. The smaller value between the preset speed and the driver's real-time operating speed is taken as the final execution speed, realizing coordinated control and further improving the smoothness of deceleration and fork stopping.

[0028] 9. If the system detects a sudden acceleration in the fork lifting speed exceeding a threshold after determining that the floor has been stopped, the floor-stopping operation will be automatically abandoned. This design can promptly capture scenarios where the driver temporarily changes their operational intentions, avoiding misoperations that conflict with the driver's intentions and ensuring operational safety.

[0029] 10. If the fork lifting speed is detected to be too low, the system automatically switches to manual mode, enabling flexible switching between manual and automatic stop control modes. By proactively exiting automatic control under such boundary conditions, the determinism and reliability of operation are ensured, preventing the system from forcibly intervening under inappropriate operating conditions.

[0030] 11. A real-time safety monitoring mechanism for the target workstation status has been added to inventory operations. When the system detects through vision devices that the target workstation does not meet the inventory conditions, it can automatically trigger an alarm or lock the vehicle. This provides a layer of proactive safety protection, effectively preventing accidents such as collisions, crushing damage, or even vehicle overturning that may result from forced operations due to insufficient storage space, obstacles, or other unsafe conditions. Attached Figure Description

[0031] Figure 1 Flowcharts of several embodiments of the forklift parking method based on visual recognition fusion and intent perception are shown in this specification;

[0032] Figure 2 Schematic diagrams of several embodiments of this specification in a pickup scenario are shown;

[0033] Figure 3 Schematic diagrams of several embodiments of this specification in a cargo delivery scenario are shown;

[0034] Figure 4 Schematic diagrams showing the relative positions of the vision device and the forks in several embodiments of this specification are shown;

[0035] Figure 5 The diagram shows a modular schematic of a forklift parking system based on visual recognition fusion and intent perception, representing several embodiments of this specification.

[0036] Figure 6 A schematic diagram of the structure of an electronic device according to some embodiments of this specification is shown.

[0037] In the diagram: 1. Forklift, 2. Goods, 3. Shelf, 4. Beam, 5. Vision device. Detailed Implementation

[0038] The present invention will be further described in detail below with reference to the accompanying drawings.

[0039] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0040] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0041] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0042] Example 1

[0043] This embodiment provides a practical implementation scheme for a forklift parking method based on visual recognition and intent perception. Regarding hardware implementation and usage environment, it includes... Figure 2-4 As shown: The working environment includes shelf 3, which is a multi-layered structure. The height difference between layers does not need to be equal and can be set arbitrarily. Each layer is separated by beams 4. Goods 2 are placed on each layer.

[0044] A forklift refers to any industrial vehicle equipped with forks. It consists of a body, a lifting frame, a fork lifting mechanism, and forks 1. A forklift also requires a mechatronic intelligent control system and a vision device 5. The fork lifting mechanism drives the lifting frame to rise and fall, and the forks 1 are mounted on the lifting frame. The vision device 5 is also mounted on the lifting frame; therefore, the distance between the vision device 5 and the forks 1 remains constant during lifting. Furthermore, in terms of hardware configuration, forklifts are also equipped with control devices and several auxiliary sensors. The following sections describe each hardware module in detail.

[0045] The core function of vision device 5 is to acquire real-time image information directly in front of the forks, especially to identify the rack beam 4. This device is typically an industrial-grade camera, fixedly mounted on the lifting frame. The vertical distance between the optical center of the camera and the upper surface of the forks 1 is defined as the vision device mounting value h ins, which is a fixed value and will not change during use once installed.

[0046] Furthermore, this value setting can satisfy two key conditions to ensure reliable identification: First, such as Figure 2 and Figure 3 As shown, hins must be less than the difference between the height A1 of the first shelf and the height h0 of the lowest fork point, i.e., hins < A1 - h0. This ensures that the first crossbeam can be detected after the forks start to rise from the lowest point. Second, the maximum height Hs of the loaded goods must be less than hins to ensure that the goods do not obstruct the camera's field of view during the lifting process. The vision device 5 is connected to the vehicle control system via a wiring harness or vehicle network to transmit video streams in real time.

[0047] The vehicle control system is responsible for processing all sensor information, executing core decision-making logic, and outputting control commands. It is typically an industrial controller or embedded industrial computer containing a processor, memory, and various I / O interfaces. Components such as the vision device, control device, forklift motor, and electro-hydraulic proportional valve are all connected to the vehicle control system via electrical wiring. The vehicle control system runs pre-programmed control software to coordinate the operation of the entire system.

[0048] The control device is the direct input device for the driver's intentions, typically an electro-hydraulic proportional control lever. The analog or digital signal generated by the driver moving the lever is interpreted by the vehicle control system as the desired value Va for the fork lifting speed. Simultaneously, the dynamic changes in the lever signal also serve as the raw data source for the vehicle control system to calculate the speed deceleration rate P and perceive the driver's deceleration intentions.

[0049] The fork lifting mechanism is the system's actuator, including a drive motor, hydraulic system, electro-hydraulic proportional valve, and lifting chain. The vehicle control system continuously controls the lifting speed of the forks by adjusting the opening of the electro-hydraulic proportional valve.

[0050] For example, in some embodiments, the auxiliary sensor also includes a weight sensor, which is typically integrated into the fork 1 or mast, to detect whether the fork is carrying goods. Its signal is one of the important bases for the vehicle control system to automatically determine whether the current operation is "picking up" or "storing goods".

[0051] A height sensor provides accurate height feedback and is essential hardware in this case. It is used to collect the real-time height position X of the forks 1, where X is the height of the forks from the ground. Specifically, the height sensor can be a laser rangefinder (as in existing technology) or an encoder connected to the fork lifting mechanism (as in existing technology), determining the real-time height position X by the lifting motion of the fork lifting mechanism.

[0052] Combination Figure 1 The flowchart shows that method 100 runs automatically after the vehicle control system is powered on and enters the operation mode. The specific steps are as follows:

[0053] In box 101, the system performs beam identification and reference height recording.

[0054] The vehicle control system drives the vision device 5 to continuously capture images of the front and uses its built-in image processing algorithm to analyze the video stream in real time to detect whether the shelf beam 4, a characteristic structure, appears in the image. The algorithm can employ existing visual recognition algorithms, such as object detection algorithms based on feature matching or deep learning.

[0055] The crossbeam is located in physical space between the nth and n+1th layers of the rack. When the algorithm successfully identifies the crossbeam, the vehicle control system immediately triggers a synchronous capture action: it reads and freezes the ground clearance of the fork 1 at this moment through the height sensor and records it as the key reference value X1.

[0056] For example, when a forklift approaches a rack with a shelf height of 2.0 meters, the camera detects the beam between the second and third shelves, and the height sensor reports a fork height of 3.8 meters. The system then records X1 = 3.8 meters. Through real-time visual perception, it dynamically establishes the current operational target—the precise spatial reference point of the (n+1)th shelf—eliminating the reliance on preset, static shelf height databases found in traditional fully automatic systems. This allows the system to adaptively handle rack environments of any size or irregular shelf height.

[0057] Furthermore, in some preferred embodiments, the identification reference of the crossbeam 4 is specifically its upper surface; the system determines that the crossbeam reference has been identified only when the horizontal center line of the video image of the vision device 5 coincides with the edge line of the upper surface of the crossbeam 4 in the horizontal direction.

[0058] Under this precise definition, the system's recognition logic not only detects the presence of the crossbeam but also captures a specific, repeatable geometric alignment state. For example, when the outline of the upper edge of the crossbeam 4 in the camera image aligns with the horizontal center line of the image sensor, the vehicle control system triggers a synchronization signal. At this moment, the real-time ground clearance of the fork 1 is captured by the height encoder and recorded as the reference height X1. This means that the recorded X1 value physically corresponds precisely to the height of the upper surface of the crossbeam. This operating method establishes a unified and precise geometric measurement benchmark that does not depend on the crossbeam's texture, color, or lighting conditions, fundamentally eliminating systematic measurement errors caused by inconsistent recognition points, such as aligning with the middle or lower edge of the crossbeam. The resulting X1 value has extremely high accuracy, laying a solid foundation for all subsequent height compensation calculations based on X1.

[0059] In box 102, the system perceives and determines the driver's operational intent. The vehicle control system continuously monitors signals from the control lever and calculates the deceleration rate P of the fork lifting speed in real time. This parameter quantifies the intensity of the driver's deceleration operation.

[0060] One specific way to calculate the deceleration rate P is: P = (maximum speed within the short time window before deceleration - current speed) / maximum speed before deceleration. The system presets an empirical deceleration rate threshold Q, for example, Q=0.5. The vehicle control system compares the real-time calculated P with Q. When the condition P > Q is met, for example, if P=0.6 is measured, the vehicle control system, combined with the context state "the crossbeam has been successfully identified and its height X1 recorded in box 101," comprehensively determines that the driver's intention is to park the forks on the layer above the currently identified crossbeam, i.e., the (n+1)th layer.

[0061] This process achieves extremely natural human-machine interaction: the driver only needs to slow down as they approach the target according to their habits, and the system can intelligently understand their intention to stop at a certain level. Unlike the fully automatic mode, there is no need to perform the tedious operation of manually inputting the target level, thus ensuring the continuity and efficiency of the operation process.

[0062] In box 103, the system performs automatic deceleration and precise fork stop control.

[0063] Once frame 102 makes a stop determination, the system automatically takes over control and completes the subsequent precise positioning.

[0064] First, the system calculates the final target stopping height: X1 + h pre. Here, h pre is a preset height margin, which is finely and critically defined. It includes at least the vision device mounting value h ins, used to compensate for the height difference between the camera's optical center and the fork's bearing surface.

[0065] In some more optimized embodiments, hpre further includes a storage / retrieval reservation value hsa, i.e., hpre = hins + hsa. hsa is distinguished according to the operation type: a retrieval reservation value h1 and a storage reservation value h2, where h1 < h2. Both h1 and h2 are presets.

[0066] The type of task can be identified by analyzing the shelf location status using weight sensors or vision devices. If the weight sensor detects that there is goods on fork 1, it is an inventory task; if there are no goods on fork 1, it is a picking task. If the vision device detects that there are goods on the (n+1)th shelf, it is a picking task; if there are no goods on the (n+1)th shelf, it is an inventory task.

[0067] For example, hins = 0.5 meters; for safe and efficient operation, h1 = 0.05 meters for picking up goods, ensuring fork insertion; h2 = 0.15 meters for storing goods, reserving a safety clearance between the pallet and the goods. If the current state is storage, then the target height = X1 + 0.5 + 0.15 = X1 + 0.65 meters.

[0068] Next, the system controls the forks to smoothly approach the target. To avoid sudden stops, the system decelerates and approaches the target height at a low speed.

[0069] In some preferred embodiments, a deceleration safety determination scheme can be provided here. Specifically, the system sets a starting height for deceleration: hl = X1 + hpre - h3, where h3 is the deceleration buffer value, for example, h3 = 0.1 meters. This is a preset value.

[0070] When X2 >= hl, it indicates that the real-time height of the forks has reached or exceeded the calculated deceleration starting point. The system determines that the remaining height travel is less than the minimum distance h3 required for safe deceleration. In this situation, forcibly initiating automatic deceleration may result in excessively rapid deceleration, a large stopping impact, and a risk of cargo shaking, slipping, or even collision. Therefore, based on the principle of safety first, the system will determine that the deceleration safety distance is insufficient and immediately abandon the current automatic fork-stopping task for the current (n+1)th layer.

[0071] When X2 < hl, it indicates that the forks have not yet passed the deceleration starting point, and there is sufficient height margin for smooth deceleration, with a minimum height margin of h3. Under this condition, the system allows for automatic deceleration and fork stopping. The vehicle control system will control the forks to smoothly decelerate from the current position according to the preset deceleration curve, aiming to reduce the speed to zero exactly when reaching the target height, thereby completing a stable and precise automatic stopping.

[0072] For example, during deceleration, the system can adopt a human-machine co-driving strategy, where the actual execution speed is the smaller of the system's preset smooth curve speed Vp and the driver's real-time handle request speed Va, ensuring that the driver has the highest priority intervention rights.

[0073] In addition, in some preferred embodiments, the system also has an intelligent abandonment mechanism: if the fork speed suddenly accelerates significantly beyond a threshold, such as ≥0.3 m / s, during the automatic stopping process, it indicates that the driver has changed his intention, and the system will automatically abandon stopping and return control.

[0074] Since the system can determine whether the current state is picking up goods or storing goods by detecting whether there are goods 2 on the fork 1 through vision device 5, for example, vision device 5 can detect whether there are goods 2 on the fork 1.

[0075] For inventory status, after the system determines that it is an inventory operation, it will use vision device 5 to perform two key spatial compliance checks on the target storage location on the (n+1)th layer, namely, the safety judgment of the vertical clearance in the vertical direction and the judgment of the horizontal space width in the left and right wind direction.

[0076] In the safety judgment of vertical clearance, the vehicle control system analyzes the image and measures the vertical height clearance between the lower surface of the crossbeam above the (n+1)th shelf layer (i.e., the (n+2)th) layer crossbeam and the upper surface of the cargo 2 currently loaded on the forks. If the calculated real-time clearance value is less than the system's preset safety clearance, the system determines that there is insufficient vertical space, and forcibly storing the cargo risks colliding with the upper crossbeam. Therefore, the system determines that the workstation does not meet the storage conditions.

[0077] In determining the width of the horizontal space, the vehicle control system further uses visual measurement to identify the width of the effective placement space formed by the two side columns or cargo limiting devices in the (n+1)th shelf. If the width of this space is less than the width of the currently loaded cargo 2, the system determines that there is insufficient horizontal space and the cargo cannot be placed, thus also determining that the storage conditions are not met. The width of cargo 2 itself is also identified by the vision device 5.

[0078] When any of the above conditions are triggered, the vehicle control system will immediately stop the automatic stopping process and execute an alarm and / or vehicle locking operation, thereby proactively preventing squeezing or collision accidents caused by insufficient space before physical operations occur.

[0079] When the system determines that a pickup operation is in progress, it will perform an adaptability check on the target pallet through the vision device 5: the vehicle control system identifies and measures the size of the fork holes on the target pallet, such as the width and height of the holes, and compares them with the physical dimensions of the vehicle's forks 1 stored in the system.

[0080] The physical dimensions of fork 1 are known. If the identified socket size does not match the fork size—for example, if the socket width is less than the fork thickness—the system determines that the picking conditions are not met, as forced operation may result in the forks failing to insert, goods tipping over, or equipment damage. The system then triggers an alarm and / or locks the vehicle. This function effectively prevents operational failures and safety accidents caused by picking the wrong pallet size or fork-pallet mismatch.

[0081] Finally, when X2 reaches the target height, the forks automatically stop, completing a precise stop without manual adjustment.

[0082] In summary, Method 100 eliminates the problems of excessive reliance on operator experience, high labor intensity, blind spots, and low positioning accuracy inherent in purely manual modes. Simultaneously, it avoids the enormous upfront costs and system complexity required for high-precision full-area mapping and modeling of the warehouse in fully automated modes. Its core breakthrough lies in the fact that the entire control process requires no prior knowledge or storage of the specific model, specifications, and layer height parameters of the target shelving. The system's height reference relies entirely on the real-time identification and measurement of the shelving beams by the vision device, while control commands originate from the intelligent perception of the driver's natural operational intentions. This gives forklifts equipped with this system a "ready-to-use" universal adaptability. When facing complex mixed warehousing scenarios with varying specifications and layer heights, or dynamically changing warehouse layouts, no pre-configuration is needed; precise and efficient automatic layer stopping operations can be achieved solely through real-time data capture and calculation, significantly improving the intelligence level and operational flexibility of industrial vehicles in discrete and diversified logistics tasks.

[0083] Based on the above detailed description of the shelf access scenario, those skilled in the art will understand that the semi-automatic height control principle of "real-time visual recognition reference plane + operation intention perception" provided by this invention has universal applicability and scalability, and is not limited to the specific application of "recognizing shelf beams". Its core technology lies in dynamically capturing a stable physical reference plane through a vision device and associating this plane with the operator's deceleration intention, thereby triggering automatic positioning.

[0084] Therefore, without departing from the core concept of this invention, by simply replacing the target reference features that the vision system needs to identify, this solution can be seamlessly transferred to various other industrial vehicle operation scenarios that require precise control of lifting height.

[0085] For example, in a loading / unloading scenario at a truck platform. When a forklift needs to precisely place goods on a truck platform, the vision device can be configured to identify the loading / unloading platform plane of the platform as a reference plane. When the system recognizes this plane and senses the driver's intention to slow down, it can automatically control the forks to stop at an appropriate height relative to the platform plane, achieving seamless docking with the platform.

[0086] For example, in cargo stacking scenarios, such as docks and warehouses where goods need to be stacked in multiple layers, the vision device can be configured to identify the upper surface of the already stably stacked goods or the bearing surface of the lower pallet as a reference surface. By recognizing this stacking surface and integrating the operational intent, the system can automatically lift the current goods and accurately place them at the next target height, achieving fast and neat automated stacking.

[0087] Figure 5 A forklift parking system based on visual recognition fusion and intent perception is illustrated. The various embodiments in this specification are described in a progressive manner, with reference allowed for interchangeable parts. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are largely similar to the method embodiments, and therefore described simply; relevant details can be found in the descriptions of the method embodiments. Figure 5 As shown, system 200 includes:

[0088] The beam detection module 201 is configured to detect whether a shelf beam is identified by a vision device, wherein the shelf beam is located between the nth and n+1th layers of the shelf.

[0089] The reference height recording module 202 is configured to record the ground clearance X1 of the forks at the moment when the rack beam is detected by the vehicle control system;

[0090] The intent perception module 203 is configured to read the speed deceleration rate P, which is a value reflecting the change in the lifting speed of the forks, and compare the speed deceleration rate P with a preset deceleration rate threshold Q. When the speed deceleration rate P is greater than the deceleration rate threshold Q, the vehicle control system determines that it is necessary to control the forks to stop on the (n+1)th layer.

[0091] The fork stop execution module 204 is configured to read the fork's height off the ground in real time. When the fork's height off the ground X2 reaches X1 + h pre, the fork will automatically stop. h pre is a preset height margin.

[0092] Figure 6 A block diagram of an electronic device 300 that can implement various embodiments of the present disclosure is shown. For example... Figure 6 As shown, device 300 includes a processor 301, which can perform various appropriate actions and processes based on computer program instructions loaded into random access memory (RAM) 303 according to computer program instructions stored in read-only memory (ROM) 302. RAM 303 may also store various programs and data required for the operation of device 300. The processor 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0093] The various processes and procedures described above, such as method 100, can be executed by processor 301. For example, in some embodiments, method 100 may be implemented as a software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the software program may be loaded and / or installed on device 300 via ROM 302. When the software program is loaded into RAM 303 and executed by processor 301, one or more actions of method 100 described above may be performed.

[0094] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0096] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A forklift parking method based on visual recognition and intent perception, characterized in that, It includes the following steps: The vehicle control system detects whether the vision device identifies the shelf beam, which is located between the nth and n+1th layers of the shelf. When the rack beam reference is identified, the vehicle control system records the ground clearance of the forks at that moment, X1. The vehicle control system reads the speed deceleration rate P, which is a value reflecting the change in the lifting speed of the forks. The speed deceleration rate P is compared with a preset deceleration rate threshold Q. When the speed deceleration rate P is greater than the deceleration rate threshold Q, the vehicle control system determines that the forks need to be stopped on the (n+1)th layer. The fork height is read in real time. When the fork height X2 reaches X1 + h pre, the fork will automatically stop. h pre is the preset height margin.

2. The forklift parking method based on visual recognition fusion and intent perception according to claim 1, characterized in that: The beam reference is specifically the upper surface of the beam. When the beam reference is identified, it specifically means that the horizontal center line of the video image of the vision device coincides with the edge line of the upper surface of the beam in the horizontal direction.

3. The forklift parking method based on visual recognition fusion and intent perception according to claim 1, characterized in that: The preset height margin h pre includes the vision device installation value h ins, which is the height of the vision device relative to the fork.

4. The forklift parking method based on visual recognition fusion and intent perception according to claim 3, characterized in that: The preset height margin hpre is the sum of the vision device installation value h ins and the storage and retrieval reserve value h sa.

5. The forklift parking method based on visual recognition fusion and intent perception according to claim 4, characterized in that: The storage and retrieval reserve value hsa is the retrieval reserve value h1 when in the retrieval state and the storage reserve value h2 when in the storage state, where h1 is less than h2.

6. The forklift parking method based on visual recognition fusion and intent perception according to claim 5, characterized in that: The distinction between the inventory status and the retrieval status is made by visually capturing the location on the shelf using the vision device and / or by identifying whether there are goods on the forks using the vision device.

7. The forklift parking method based on visual recognition fusion and intent perception according to claim 6, characterized in that: If the vehicle is in stock and the system detects that the target workstation does not meet the stocking conditions through the vision device, it will trigger an alarm and / or lock the vehicle.

8. The forklift parking method based on visual recognition fusion and intent perception according to claim 7, characterized in that, The method for determining if the inventory conditions are not met is as follows: when the vision device detects that the height gap between the lower surface of the beam above the (n+1)th shelf and the upper surface of the current goods is less than the preset safety gap, it is determined that the inventory conditions are not met.

9. The forklift parking method based on visual recognition fusion and intent perception according to claim 7, characterized in that, The method for determining if the inventory conditions are not met is as follows: if the vision device detects that the width of the space on the upper surface of the beam is less than the width of the current goods, it is determined that the inventory conditions are not met.

10. The forklift parking method based on visual recognition fusion and intent perception according to claim 6, characterized in that: When the vehicle is in the picking state, if the vision device detects that the size of the target pallet's insertion hole does not match the size of the fork, it will determine that the picking conditions are not met and will perform an alarm operation and / or lock the vehicle.

11. The forklift parking method based on visual recognition fusion and intent perception according to claim 3, characterized in that, The installation value h ins of the vision device needs to meet the condition: h ins < A1-h0, where A1 is the reference height of the first shelf and h0 is the height of the lowest point of the fork from the ground.

12. The forklift parking method based on visual recognition fusion and intent perception according to claim 11, characterized in that, The maximum height Hs of the cargo must satisfy the condition: Hs < h ins.

13. The forklift parking method based on visual recognition fusion and intent perception according to any one of claims 1-12, characterized in that, When the vehicle control system determines that the forks need to be stopped on the (n+1)th layer, it needs to first decelerate and then stop the forks.

14. The forklift parking method based on visual recognition fusion and intent perception according to claim 13, characterized in that, The starting deceleration height hl is X1 + h pre - h3, where h3 is a preset deceleration buffer value. When the vehicle control system determines that the forks need to be stopped on the (n+1)th layer, deceleration and stopping are allowed when the fork's ground clearance is <hl, and deceleration and stopping are not allowed when the fork's ground clearance is >hl.

15. The forklift parking method based on visual recognition fusion and intent perception according to any one of claims 1-12, characterized in that, During the process of the vehicle control system controlling the forks to stop on the (n+1)th layer, the vehicle control system obtains Vp and Va, and takes the minimum value of the two as the actual lifting speed of the forks; where Vp is the preset lifting speed value, which the vehicle control system obtains by reading the preset deceleration curve, and Va is the real-time operation value of the driver, which the vehicle control system obtains through the driver's control device.

16. The forklift parking method based on visual recognition fusion and intent perception according to any one of claims 1-12, characterized in that, When the vehicle control system has determined that the lower level fork stop is required, and it reads that the fork lifting speed V starts to accelerate, and the lifting speed V≥Vmax, then the vehicle control system abandons the lower level fork stop operation; where Vmax is the maximum speed before deceleration.

17. The forklift parking method based on visual recognition fusion and intent perception according to any one of claims 1-12, characterized in that, When the vehicle control system determines that the lower level fork needs to be stopped, and reads that the fork lifting speed V is less than Vmin, the vehicle control system will not automatically control the lifting of the forks and will enter the manual operation mode; where Vmin is the preset minimum fork lifting speed.

18. The forklift parking method based on visual recognition fusion and intent perception according to any one of claims 1-12, characterized in that, The speed deceleration rate P is calculated as follows: P = (maximum speed before deceleration - current speed) / maximum speed before deceleration.

19. The forklift parking method based on visual recognition fusion and intent perception according to any one of claims 1-12, characterized in that, When the vehicle control system determines that it is not necessary to control the forks to stop at the n+1 level, the previously recorded value of the ground clearance X1 is cleared; when the vision device identifies the crossbeam reference between the n+1 and n+2 levels, the value of the fork ground clearance X1 at that moment is recorded.

20. A forklift parking system based on visual recognition and intent perception, characterized in that, The system includes: The beam detection module is configured to detect whether a shelf beam is identified using a vision device, the shelf beam being located between the nth and n+1th layers of the shelf; The reference height recording module is configured so that when a rack beam is detected, the vehicle control system records the ground clearance height X1 of the forks at that moment. The intent perception module is configured to read the speed deceleration rate P, which is a value reflecting the change in the lifting speed of the forks. The speed deceleration rate P is compared with a preset deceleration rate threshold Q. When the speed deceleration rate P is greater than the deceleration rate threshold Q, the vehicle control system determines that the forks need to be stopped on the (n+1)th layer. The fork stop execution module is configured to read the fork's height off the ground in real time. When the fork's height off the ground X2 reaches X1 + hpre, the fork will automatically stop. hpre is a preset height margin.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 19.

22. An electronic device, comprising a processor and a memory; the processor being connected to the memory; The memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to perform the method as described in any one of claims 1 to 19.

Images