Intelligent warehouse logistics state sensing method and system

Abstract

The embodiment of the application provides a kind of intelligent warehousing logistics state sensing method and system, applied to the sensing technical field of computer vision, which obtains the multiple images of warehousing scene, identifies target goods, target handling equipment and functional area boundary, and detects contact event and corresponding contact frame moment based on the relative position change of target goods and target handling equipment;Further construct contact process and contact time interval, and obtain motion characteristic parameters by combining target goods motion process analysis;Detect the crossing behavior and crossing frame moment of target goods crossing functional area boundary, associate the crossing frame moment with contact time interval for judgment;When determining as weak association, modify the contact time interval and re-determine, when determining as deterministic association, generate logistics record, and aggregate to form logistics state information according to functional area, to improve the accuracy of warehousing logistics state sensing.

Description

Technical Field

[0001] This application relates to the field of computer vision perception technology, and in particular to an intelligent warehousing and logistics status perception method and system. Background Technology

[0002] In large-scale intelligent warehousing and logistics centers, there are a wide variety of goods that are frequently moved. Managers need to monitor the status of goods and handling equipment in real time to ensure operational safety, improve operational efficiency, and prevent goods from being collided or lost. Therefore, a method and system that can sense the status of warehousing and logistics is needed to perceive the location and handling status of goods and provide a foundation for intelligent warehouse management.

[0003] In existing technologies, the perception of the status of intelligent warehousing and logistics mainly involves collecting continuous image frames of the warehousing environment through robots or fixed cameras, using target detection and tracking technologies to identify the position and posture of goods, pallets and handling equipment, and judging the handling status of goods based on changes in computer vision information, thereby realizing computer vision-based perception of the status of warehousing and logistics.

[0004] The above-mentioned solutions still have some problems in practical applications. When existing technologies perceive the status of warehouse logistics based on continuous image frames, they usually extract behavioral information independently from two perspectives: changes in spatial relationships between targets and changes in target positions. Based on this, they form judgment results for interactive behavior and regional migration, respectively. However, since the two types of judgment results lack a unified temporal correlation and process constraint mechanism, in situations where there is target occlusion, movement path intersection, or discontinuous image sampling in the warehouse scenario, the correspondence between different behavioral information in the time dimension is easily inconsistent. This makes it difficult to form a stable and consistent behavioral correlation during the logistics status recording process and affects the accurate generation of logistics status records. Summary of the Invention

[0005] This application provides an intelligent warehousing and logistics status perception method and system. By performing time correlation analysis and process constraint processing on the cargo handling process and regional circulation behavior in the warehousing scenario, the accuracy of intelligent warehousing and logistics status perception is improved.

[0006] To achieve the above objectives, this application adopts the following technical solution: This application provides an intelligent warehousing and logistics status perception method, the method comprising: identifying target goods, target handling equipment, and functional area boundaries from multi-frame images of a warehousing scene; detecting contact events and recording corresponding contact frame times based on the relative position changes of the target goods and target handling equipment in the multi-frame images; constructing a contact process and corresponding contact time interval between the target goods and target handling equipment based on the contact events and contact frame times related to the target goods; performing process analysis on the contact process according to the contact frame times to determine the motion characteristic parameters of the target goods; and detecting the relative position changes of the target goods and the functional area boundaries based on the multi-frame images to determine the crossing behavior of the target goods and recording the corresponding contact frame times. The corresponding crossing frame time; perform an association determination between the crossing frame time and the contact time interval of the contact process to obtain a determination result; if the determination result is a first result, then correct the contact time interval of the contact process according to the motion feature parameters, and re-execute the association determination on the corrected contact time interval, the first result being used to characterize a weak association relationship between the crossing frame time and the contact process under the condition of time offset; if the determination result is a second result, then generate a logistics record of the target goods based on the contact process and crossing behavior, the second result being used to characterize a deterministic association relationship between the crossing frame time and the contact process; summarize the logistics record according to the functional area distribution to form the logistics status information of the intelligent warehouse.

[0007] In some possible implementations, the step of detecting contact events and recording the corresponding contact frame times based on the relative position changes of the target cargo and the target handling equipment in the multi-frame images includes: acquiring the spatial position of the target cargo and the spatial position of the target handling equipment in the multi-frame images; for each frame of the multi-frame images, calculating the relative distance value between the target cargo and the target handling equipment based on the spatial positions of the cargo and equipment in the frame image, and recording it to form a relative distance sequence; performing inter-frame difference processing on the relative distance sequence to obtain a relative distance change sequence; and determining that the target cargo and the target handling equipment are in a contact position when the relative distance change sequence shows a continuously decreasing trend in the multi-frame images. Approaching a change state; in the frame image corresponding to the approaching change state, when the relative distance value is less than a preset threshold, a suspected contact event is determined and the corresponding initial contact frame time is recorded; for the suspected contact event, at least one subsequent frame image after the initial contact frame time is selected, and the relative distance value between the target goods and the target handling equipment corresponding to the subsequent frame image is obtained; the difference between the relative distance value corresponding to the subsequent frame image and the relative distance value corresponding to the initial contact frame time is calculated to obtain a distance change maintenance value; when the distance change maintenance value is less than a preset fluctuation threshold, the suspected contact event is determined to be a contact event, and the initial contact frame time of the suspected contact event is marked as the contact frame time of the contact event.

[0008] In some possible implementations, the step of constructing the contact process and corresponding contact time interval between the target cargo and the target handling equipment based on the contact events and contact frame times related to the target cargo includes: for any target cargo, determining the contact events corresponding to the target cargo and the contact frame times corresponding to the contact events, and sorting the contact frame times in chronological order to form a contact frame sequence; using each contact frame time in the contact frame sequence as a reference frame, selecting at least one preceding frame image and at least one subsequent frame image to construct candidate contact segments; for each candidate contact segment, obtaining the relative distance values ​​between the target cargo and the target handling equipment in each frame image within the candidate contact segment, and constructing a relative distance value sequence in chronological order; calculating the relative distance change between adjacent frame images based on the relative distance value sequence, and filtering out the target cargo and target handling equipment based on the relative distance change. The relative distance between the target goods and the target handling equipment is stabilized within a stable interval. The frame images corresponding to the stable interval are mapped to the initial contact process in chronological order. Based on the contact frame time of the frame images in the initial contact process, the initial contact time interval of the initial contact process is determined. If the time interval between two adjacent initial contact time intervals is less than a preset time interval threshold, the corresponding initial contact processes are merged to form a continuous contact process, and the initial contact time intervals are spliced ​​together to form the first contact time interval of the continuous contact process. If the time interval between two adjacent initial contact time intervals is not less than the preset time interval threshold, the initial contact process is marked as a single contact process, and the initial contact time interval is used as the second contact time interval of the single contact process. The continuous contact process and the single contact process together constitute the contact process between the target goods and the target handling equipment, and the first contact time interval and the second contact time interval constitute the contact time interval.

[0009] In some possible implementations, the step of performing process analysis on the contact process according to the contact frame time to determine the motion characteristic parameters of the target cargo includes: for any target cargo, determining the contact process corresponding to the target cargo and the contact time interval corresponding to the contact process, and extracting the cargo spatial position corresponding to each frame image within the contact process; calculating the displacement change between adjacent frame images based on the cargo spatial position, and constructing a displacement change sequence in chronological order; based on the displacement change sequence, and combined with the positional relationship of the contact frame time in the contact time interval, determining the displacement change abrupt point corresponding to the contact frame time, and extracting the displacement change difference value before and after the displacement change abrupt point; dividing the contact process into a pre-contact stage and a post-contact stage according to the contact frame time, and calculating the degree of change dispersion of the pre-contact stage and the post-contact stage respectively; encapsulating the displacement change difference value and the degree of change dispersion of the pre-contact stage and the post-contact stage to obtain the motion characteristic parameters of the target cargo, wherein the motion characteristic parameters are used to characterize the motion state change characteristics of the target cargo during the contact process.

[0010] In some possible implementations, the step of detecting the relative positional change between the target cargo and the functional area boundary based on the multi-frame images, determining the crossing behavior of the target cargo, and recording the corresponding crossing frame time includes: constructing a cargo spatial position trajectory based on the cargo spatial position in the multi-frame images; obtaining boundary structure distribution information of the functional area boundary; determining the interaction crossing point between the target cargo movement trajectory and the functional area boundary based on the cargo spatial position trajectory and the boundary structure distribution information, wherein the interaction crossing point is used to characterize the position where the cargo spatial position trajectory and the boundary structure interact spatially; performing a continuous analysis of the interaction crossing point in chronological order to determine continuous frame images in which the target cargo movement trajectory and the functional area boundary intersect; identifying the path-through process of the target cargo movement trajectory from one side of the functional area boundary to the other side of the functional area boundary in the continuous frame images; determining that a crossing behavior has occurred when the path-through process maintains a unidirectional crossing trend within a continuous preset number of frames, and determining the frame time of the starting interaction crossing point of the path-through process as the crossing frame time.

[0011] In some possible implementations, the step of performing an association determination between the cross-frame moment and the contact time interval of the contact process to obtain a determination result includes: acquiring a frame image corresponding to the cross-frame moment and extracting the spatial position of the target cargo in the frame image; performing segmentation on the relative distance change sequence according to the contact time interval of the contact process to form a spatial distance change sequence corresponding to the contact time interval; matching the cargo spatial position corresponding to the cross-frame moment with the cargo spatial positions corresponding to each contact frame moment within the contact time interval according to the time frame sequence to form a time index correspondence between the cross-frame moment and the contact process; calculating the time interval between the cross-frame moment and each contact frame moment within the contact time interval based on the time index correspondence; determining the association matching degree between the cross-frame moment and the contact process based on the correspondence between the time interval and the spatial distance change sequence; and performing a matching judgment based on the association matching degree and a preset association condition to determine the determination result as a first result or a second result.

[0012] In some possible implementations, the preset association conditions include a first preset association condition and a second preset association condition. The step of matching and judging the association matching degree with the preset association conditions to determine the judgment result as a first result or a second result includes: if the association matching degree meets the first preset association condition, then the judgment result is determined to be a first result; if the association matching degree simultaneously meets the second preset association condition, then the judgment result is determined to be a second result.

[0013] In some possible implementations, the motion feature parameters include displacement change difference values, the degree of dispersion of change in the pre-contact stage, and the degree of dispersion of change in the post-contact stage. If the determination result is a first result, the contact time interval of the contact process is corrected based on the motion feature parameters, and the association determination is re-executed on the corrected contact time interval. This includes: if the determination result is a first result, determining the candidate frame time when the target cargo undergoes a sudden change in motion within the contact time interval based on the displacement change difference values ​​in the motion feature parameters, and using the candidate frame time as a candidate boundary adjustment point; correcting the contact time interval based on the degree of dispersion of change in the pre-contact stage and the degree of dispersion of change in the post-contact stage on both sides of the candidate boundary adjustment point to obtain a corrected contact time interval, wherein the degree of dispersion of change in the pre-contact stage is used to characterize the fluctuation degree of displacement change in the pre-contact stage, and the degree of dispersion of change in the post-contact stage is used to characterize the fluctuation degree of displacement change in the post-contact stage; and re-executing the association determination based on the corrected contact time interval and the crossed frame time.

[0014] In some possible implementations, generating a logistics record for the target cargo based on the contact process and crossing behavior if the determination result is the second result includes: if the determination result is the second result, determining the start and end contact frame times of the target cargo during the contact process based on the cargo spatial position corresponding to each frame image within the contact time interval and the displacement change sequence; determining the regional flow path of the target cargo based on the spatial positional relationship between the crossing frame time and the functional area boundary; associating the start and end contact frame times of the target cargo during the contact process and the regional flow path in chronological order to form a handling process record of the target cargo; storing and encapsulating the handling process record according to the identifier of the target cargo to form a logistics record of the target cargo, wherein the logistics record is used to characterize the regional flow information of the target cargo under the combined action of the contact process and crossing behavior.

[0015] This application provides an intelligent warehousing and logistics status perception system, comprising: a target recognition module for identifying target goods, target handling equipment, and functional area boundaries from multiple frames of images of a warehousing scene; a contact event detection module for detecting contact events and recording corresponding contact frame times based on the relative position changes of the target goods and target handling equipment in the multiple frames of images; a contact process construction module for constructing a contact process and corresponding contact time interval between the target goods and target handling equipment based on the contact events and contact frame times related to the target goods; a motion feature analysis module for performing process analysis on the contact process according to the contact frame times to determine the motion feature parameters of the target goods; and a crossing behavior detection module for detecting the relative position changes of the target goods and the functional area boundaries based on the multiple frames of images to determine the crossing behavior of the target goods. The system includes a crossing behavior module and a crossover behavior module. The first module performs a crossover behavior assessment and records the corresponding crossing frame time. The second module performs a crossover behavior assessment and records the corresponding crossing frame time. The third module performs a crossover behavior assessment and records the corresponding crossing frame time. The fourth module performs a crossover behavior assessment and records the corresponding crossing frame time. The fifth module performs a crossover behavior assessment and records the corresponding crossing frame time. The sixth module performs a crossover behavior assessment and records the corresponding crossing frame time. The seventh module performs a crossover behavior assessment and records the corresponding crossing frame time. The eighth module performs a crossover behavior assessment and records the corresponding crossing frame time. The ninth module performs a crossover behavior assessment and records the corresponding crossing frame time. The eleventh ...

[0016] As can be seen from the above technical solution, this application has the following beneficial effects: 1. This application constructs the contact process and crossing behavior synchronously in consecutive image frames and introduces a time interval-based association determination mechanism to match the correspondence between contact events and area crossing behavior under a unified time constraint framework. Compared with the existing technology that performs contact determination and crossing determination independently, this effectively reduces the problem of fragmentation and mismatch of different visual behaviors in the time dimension, thereby accurately describing the warehousing and logistics operation process and enhancing the structural integrity of the logistics status expression. 2. This application introduces a contact time interval correction mechanism based on motion feature parameters when the association determination result is weak, and performs secondary adjustment on the contact process boundary. The association determination is then re-executed in combination with the corrected time interval, so that the contact process boundary can adapt to the motion change characteristics of the target goods under conditions of occlusion, trajectory intersection, or uneven sampling interval. Compared with the existing technology that only relies on a single determination result, this improves the stability and adaptability of the matching between contact behavior and crossing behavior, thereby enhancing the accuracy of intelligent warehousing and logistics status perception. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the composition of an intelligent warehousing and logistics status perception method according to this application. Figure 2 This is a structural diagram of an intelligent warehousing and logistics status perception system according to this application. Detailed Implementation

[0018] The terms "first," "second," and "third," etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0019] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0020] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the related technologies is given first: Intelligent warehousing refers to a warehousing and logistics system that integrates information sensing technology, automated execution equipment, and data processing mechanisms to enable automatic identification, process awareness, and status management of warehousing operations. Intelligent warehousing can identify and correlate the status of goods, handling equipment, and functional areas, thereby achieving process recording and status awareness of inbound, storage, handling, and outbound processes. Intelligent warehousing is commonly used in e-commerce warehousing centers, manufacturing logistics warehouses, and distribution centers, and is a crucial infrastructure supporting the efficient operation of modern logistics systems.

[0021] Research has revealed that existing technologies for perceiving warehouse logistics status based on continuous image frames typically analyze two dimensions independently: changes in the contact relationship between target goods and target handling equipment, and changes in the position of target goods under functional area boundary constraints. These result in separate contact behavior and crossing behavior determinations. Each of these results relies on its own temporal sampling segments and local spatial judgment conditions, lacking a unified temporal alignment and process constraint mechanism for the entire logistics operation. Furthermore, in actual warehousing scenarios, when targets are occluded causing missing local frame information, multiple target trajectories intersect leading to short-term instability in the correlation, and uneven image sampling intervals causing discretization of event boundary expressions, the temporal correspondence between contact and crossing behaviors is prone to shift. This results in inconsistent start and end boundaries for the same logistics operation across different analysis links. The above analysis shows that existing technologies fail to establish a stable process-level temporal correlation constraint structure between contact and crossing behaviors, making it difficult to form a consistent behavioral correspondence during logistics status generation, thus affecting the coherence and reliability of logistics status expression.

[0022] To address the aforementioned problems, this application provides an intelligent warehousing and logistics status perception method. The method includes: identifying target goods, target handling equipment, and functional area boundaries from multiple frames of images of a warehousing scene; detecting contact events and recording corresponding contact frame times based on the relative position changes of the target goods and target handling equipment in the multiple frames of images; constructing a contact process and corresponding contact time interval between the target goods and target handling equipment based on the contact events and contact frame times related to the target goods; performing process analysis on the contact process according to the contact frame times to determine the motion characteristic parameters of the target goods; and detecting the relative position changes of the target goods and the functional area boundaries based on the multiple frames of images to determine and record the crossing behavior of the target goods. Record the corresponding crossing frame time; perform an association determination between the crossing frame time and the contact time interval of the contact process to obtain a determination result; if the determination result is a first result, then correct the contact time interval of the contact process according to the motion feature parameters, and re-perform the association determination on the corrected contact time interval, the first result being used to characterize a weak association relationship between the crossing frame time and the contact process under the condition of time offset; if the determination result is a second result, then generate a logistics record of the target goods based on the contact process and crossing behavior, the second result being used to characterize a deterministic association relationship between the crossing frame time and the contact process; summarize the logistics record according to the functional area distribution to form the logistics status information of the intelligent warehouse.

[0023] Example 1 like Figure 1 As shown, this application provides an intelligent warehousing and logistics status perception method, the specific steps of which are as follows: S1 identifies target goods, target handling equipment, and functional area boundaries from multiple frames of images of the warehousing scene.

[0024] In the embodiment of step S1, a brief introduction to the following related terms is given: Warehousing scenario: The operating space environment of intelligent warehousing is used to carry out the storage, sorting and handling of goods. The space environment includes different functional areas such as shelving area, aisle area, loading and unloading area and buffer area, and is used to represent the distribution and operating status of goods and target handling equipment in the space.

[0025] Multi-frame images: A collection of multiple image frames acquired sequentially in a warehouse scene by an image acquisition device. These multi-frame images are used to characterize the changes in the warehouse scene over time, providing a continuous visual information basis for identifying target goods, target handling equipment, and their motion relationships.

[0026] Target cargo: refers to cargo objects that are identified and tracked in a warehousing scenario. These cargo objects have corresponding spatial locations and morphological characteristics, and are used to characterize physical units that are handled, stored, or transferred during logistics operations, such as containers.

[0027] Target handling equipment: Equipment objects that perform cargo handling tasks in a warehousing scenario, such as forklifts, handling robots, or conveyor devices. Equipment objects have corresponding spatial positions and motion trajectories, which are used to represent the main body performing handling operations.

[0028] Functional area boundary: refers to the spatial boundary structure that divides different operational functional areas in a warehousing scenario. The spatial boundary structure is used to distinguish the spatial range of different functional areas and serves as a spatial reference for judging whether target goods have crossed areas.

[0029] In one feasible approach, multiple frames of images are acquired by deploying cameras in the warehouse scene. The multiple frames of images are analyzed frame by frame. Based on the target appearance features in the images, candidate targets are identified and the target goods and target handling equipment are distinguished. At the same time, the target goods and target handling equipment are continuously located by combining the position changes in the multiple frames of images. For areas in the warehouse scene where the structural position remains stable, the spatial boundary position is determined by extracting stable structural features from the multiple frames of images, and the spatial boundary position is marked as the functional area boundary.

[0030] S2, based on the relative positional changes of the target cargo and the target handling equipment in the multi-frame images, detect contact events and record the corresponding contact frame times.

[0031] In the embodiment of step S2, a brief introduction to the following related terms is given: Relative position change: In step S2, relative position change refers to the change in the distance relationship between the target cargo and the target handling equipment in the corresponding spatial positions in multiple frames of images over time. It is used to characterize the spatial approach or distance movement relationship between the two.

[0032] Contact event: refers to an event in which the target cargo and the target handling equipment reach a contact determination condition in the vicinity of a certain time frame and enter a stable close or touching state.

[0033] Contact frame time: refers to the time marker of the image frame in a multi-frame image sequence where the corresponding contact event is first determined to meet the contact conditions.

[0034] One possible implementation of step S2 is as follows: S201, Obtain the spatial position of the target cargo in the multi-frame images and the spatial position of the target handling equipment in the multi-frame images; Cargo spatial location: refers to the two-dimensional or three-dimensional position of the target cargo in the image coordinate system or world coordinate system.

[0035] Equipment spatial location: refers to the two-dimensional or three-dimensional position of the target handling equipment in the image coordinate system or world coordinate system.

[0036] S202, for each frame of the multi-frame images, calculate the relative distance between the target cargo and the target handling equipment based on the spatial position of the cargo and the spatial position of the equipment in the frame image, and record them to form a relative distance sequence; Frame image: refers to a single image data acquired continuously in chronological order, used as the basic unit to form a multi-frame image sequence.

[0037] Relative distance value: refers to the geometric distance between the spatial location of the target cargo and the spatial location of the target handling equipment in the same frame image.

[0038] Relative distance sequence: refers to the set of relative distance values ​​corresponding to multiple frames of images arranged in chronological order, used to characterize the distance change process.

[0039] In S202, the method for calculating the relative distance between the target cargo and the target handling equipment is as follows: extract the spatial coordinates of the target cargo and the target handling equipment in each frame of the image, calculate the distance between the two coordinates using the Euclidean distance calculation method, and use the calculation result as the relative distance value of that frame; if two-dimensional image coordinates are used, the distance is calculated based on the pixel coordinate difference; if three-dimensional spatial coordinates are used, the distance is calculated based on the spatial coordinate difference.

[0040] S203, perform inter-frame difference processing on the relative distance sequence to obtain a relative distance change sequence; Inter-frame differential processing: refers to performing difference operations on the relative distance values ​​of adjacent frames to obtain the distance change.

[0041] Relative distance change sequence: refers to a sequence composed of the relative distance difference between adjacent frames, used to characterize the trend of distance change.

[0042] The method for inter-frame difference processing is as follows: the relative distance values ​​of two adjacent frames are selected sequentially according to time, and the difference is calculated by subtracting the relative distance value of the previous frame from the relative distance value of the later frame. The difference results of each pair of adjacent frames are arranged in time order to form a sequence of relative distance changes.

[0043] S204, when the relative distance change sequence shows a continuous decreasing trend in the multi-frame images, it is determined that the target cargo and the target handling equipment are in a state of close change. Continuous decreasing trend: refers to a sequence feature where the relative distance change between multiple adjacent frames is continuously negative and the overall trend is decreasing.

[0044] Approaching change state: refers to the motion state in which the spatial distance between the target cargo and the target handling equipment continues to decrease.

[0045] S205, in the frame image corresponding to the near change state, when the relative distance value is less than a preset threshold, it is determined that there is a suspected contact event and the corresponding initial contact frame time is recorded. Preset threshold: refers to the distance threshold used to determine whether the target goods and the target handling equipment have entered the close contact determination range.

[0046] The preset threshold is set as follows: During the operation of the warehousing system, a sequence of historical operation video frames in which contact events have occurred is selected, and the relative distance values ​​between the target goods and the target handling equipment before and after the contact are extracted from the corresponding frame images. The relative distance value of the most recent frame before the contact occurs is used as the critical contact distance reference value. At the same time, a sequence of historical frame images in which no contact has occurred but the target goods and the target handling equipment are in a normal handling or close operation state is selected, and the corresponding relative distance values ​​are extracted as safe operation distance reference values. The critical contact distance reference value and the safe operation distance reference value are compared, and the boundary distance between the two is taken as the preset threshold. When the relative distance value of the current frame is less than the preset threshold, it indicates that the target goods and the target handling equipment have entered the distance range where contact is possible, thereby triggering the judgment of a suspected contact event.

[0047] Suspected contact event: refers to an event state in which the distance between the target cargo and the target handling equipment meets the contact determination conditions but the stability verification has not yet been completed.

[0048] Initial contact frame time: refers to the time marker corresponding to the first image frame that meets the distance threshold condition.

[0049] S206, for the suspected contact event, select at least one subsequent frame image after the initial contact frame time, and obtain the relative distance value between the target goods and the target handling equipment corresponding to the subsequent frame image; Subsequent frame images: refers to image frames acquired in chronological order after the initial contact frame.

[0050] The method of selecting at least one subsequent frame image after the initial contact frame time can be implemented as follows: selecting several consecutive frames after the initial contact frame time as a set of subsequent frame images (the number of consecutive frames can be dynamically adjusted according to the actual situation, or can be set based on logistics handling experience), or selecting image frames within a fixed time window as a set of subsequent frame images to verify the continuity of the contact state.

[0051] S207, calculate the difference between the relative distance value corresponding to the subsequent frame image and the relative distance value corresponding to the initial contact frame time to obtain the distance change maintenance value; Difference calculation: refers to the subtraction operation between the relative distance values ​​corresponding to two different times.

[0052] The method for calculating the difference is as follows: taking the relative distance value corresponding to the initial contact frame time as the reference value, the relative distance values ​​corresponding to subsequent frames are successively calculated with the reference value, and the absolute value of each difference result is taken and averaged to obtain the distance change preservation value.

[0053] Distance variation retention value: refers to a numerical index used to characterize the degree of stability of relative distance under contact conditions.

[0054] S208, when the distance change value is less than the preset fluctuation threshold, the suspected contact event is determined to be a contact event, and the initial contact frame time of the suspected contact event is marked as the contact frame time of the contact event.

[0055] Preset fluctuation threshold: refers to the allowable fluctuation range threshold used to determine whether the distance is in a stable contact state.

[0056] The preset fluctuation threshold can be determined based on the statistical results of the image acquisition error range, target detection error range, and distance fluctuation between the target goods and the target handling equipment in a static and close-fitting state in the warehousing environment. The upper limit of the statistical result error or the stable interval of the statistical distribution is taken as the setting basis.

[0057] Contact event: refers to an event in which the target cargo and the target handling equipment actually come into contact or are in close proximity, after the distance between the target cargo and the target handling equipment meets the contact determination conditions and stability verification is performed.

[0058] Contact frame time: The time frame identifier corresponding to the contact event, determined after stability verification.

[0059] Step S2 of this application acquires the spatial positions of the target cargo and the target handling equipment in multiple frames of images, constructs a relative distance sequence, and then performs differential processing on the distance changes between adjacent frames to form a distance change trend. Based on this, it combines proximity change state judgment and distance threshold screening, and further introduces a distance judgment boundary determined by comparing historical contact frame images and non-contact operation frame images to perform preliminary identification and stability verification of suspected contact events. Since single-frame spatial distance judgment is easily affected by target occlusion, detection deviation, and instantaneous motion jitter, it may lead to unstable contact judgment results. By constraining the distance change trend in multiple frames, short-term abnormal fluctuations can be filtered out. Combined with threshold triggering and subsequent multi-frame consistency verification, suspected contact states are corrected and confirmed, so that the judgment of contact events simultaneously meets the requirements of continuity and stability, improving the accuracy of determining contact events and contact frame times.

[0060] S3, for the target cargo, construct the contact process and corresponding contact time interval between the target cargo and the target handling equipment based on the contact events and contact frame times related to the target cargo.

[0061] In the embodiment of step S3, a brief introduction to the following related terms is given: Contact process: refers to the interactive behavior process between the target cargo and the target handling equipment in a multi-frame image sequence, from the occurrence of contact to the end of contact, which is a continuous or intermittent time period and is used to characterize the complete temporal evolution process of one or more contact behaviors.

[0062] Contact time interval: refers to the start and end range of the contact process in the time dimension, which is determined by the contact frame time and its corresponding continuous frame range.

[0063] One possible implementation of step S3 is as follows: S301, for any of the target goods, determine the contact event corresponding to the target goods and the contact frame time corresponding to the contact event, and sort the contact frame times in chronological order to form a contact frame sequence; Contact frame sequence: refers to a set of multiple contact frame moments arranged in chronological order, used to reflect the temporal distribution of contact between target goods at different times.

[0064] S302, taking each contact frame moment in the contact frame sequence as a reference frame, at least one preceding frame image is selected forward and at least one subsequent frame image is selected backward to construct a candidate contact segment; Reference frame: refers to the image frame corresponding to the contact frame time as the reference frame for time analysis.

[0065] Preceding frame images: These are frames acquired in chronological order before the reference frame.

[0066] Subsequent frame images: These are frame images acquired in chronological order after the reference frame.

[0067] Candidate contact segment: refers to a time-continuous image segment centered on the reference frame, composed of the preceding frame image and the subsequent frame image.

[0068] The method for constructing candidate contact segments is as follows: taking each contact frame moment as the center, a preset number or preset time range of consecutive frame images are selected as preceding frame images, and a preset number or preset time range of consecutive frame images are selected as following frame images. The preceding frame images, the reference frame, and the following frame images are stitched together in chronological order to form candidate contact segments. The preset number or preset time range is used to reflect the degree of control over the completeness of the coverage of the process before and after the contact occurs (e.g., setting the preset number to 5 frames or setting the preset time range to 2 seconds). It can be adjusted according to the changes in the movement speed of the target goods and the target handling equipment in the warehousing scenario, the image acquisition frame rate, and the duration of the operation to ensure that the candidate contact segments can cover the key behavioral processes before and after the contact occurs.

[0069] S303, for each of the candidate contact segments, obtain the relative distance values ​​between the target goods and the target handling equipment in each frame image within the candidate contact segment, and construct a sequence of relative distance values ​​in chronological order; Relative distance value sequence: refers to the sequence formed by arranging the relative distance values ​​between the target cargo and the target handling equipment in each frame within the candidate contact segment in time, which is used to characterize the distance change process before and after contact.

[0070] S304, calculate the relative distance change between adjacent frame images based on the relative distance value sequence, and filter out the stable range of the relative distance change between the target cargo and the target handling equipment based on the relative distance change; Relative distance change: refers to the difference in relative distance values ​​between adjacent frames, used to characterize the magnitude of distance change.

[0071] Stable range of relative distance change: refers to the continuous frame range in which the relative distance change in the candidate contact segment remains within a preset fluctuation range and the change amplitude is relatively gentle.

[0072] The method for determining the stable interval of the relative distance change between the target cargo and the target handling equipment is as follows: the relative distance change of adjacent frames is compared frame by frame, and continuous frame intervals with an absolute value of change less than a preset fluctuation range are selected; when the number of frames that continuously meet this condition reaches the preset number of continuous frames, the continuous frame interval is determined as the stable interval.

[0073] S305, map the frame images corresponding to the stable interval to the initial contact process in chronological order, and determine the initial contact time interval of the initial contact process based on the contact frame time of the frame images in the initial contact process. Initial contact process: refers to the basic process segment in which the target cargo comes into contact with the target handling equipment, which is composed of frame images corresponding to the stable interval.

[0074] Initial contact time interval: refers to the time range formed by the start frame time and the end frame time corresponding to the initial contact process.

[0075] S306, if the time interval between two adjacent initial contact time intervals is less than a preset time interval threshold, the corresponding initial contact processes are merged to form a continuous contact process, and the initial contact time intervals are spliced ​​together to form the first contact time interval of the continuous contact process. Preset time interval threshold: refers to the time threshold used to distinguish the time interval boundary between a single contact and multiple contacts.

[0076] The preset time interval threshold is set based on the average time interval range of a complete operation of the handling equipment in warehousing operations, and combined with the minimum interval between two consecutive contacts in historical contact events, so that time intervals less than the threshold correspond to the same continuous operation process.

[0077] Merging process: refers to the time splicing and continuous processing of multiple initial contact time intervals that meet the merging conditions.

[0078] The merging process involves the following steps: when there is time overlap or the interval is less than a preset time interval threshold between adjacent initial contact time intervals, the earlier start time and the later end time of the two time intervals are taken to form a continuous time interval.

[0079] Continuous contact process: refers to a complete contact behavior process formed by the combination of multiple consecutive or shortly spaced initial contact processes.

[0080] First contact time interval: refers to the overall time range corresponding to the continuous contact process.

[0081] S307, if the time interval between two adjacent initial contact time intervals is not less than a preset time interval threshold, then the initial contact process is marked as a single contact process, and the initial contact time interval is used as the second contact time interval of the single contact process. Single contact process: refers to contact behaviors that occur independently of each other and with a time interval exceeding a preset time interval threshold.

[0082] Second contact time interval: refers to the independent time range corresponding to a single contact process.

[0083] S308, the continuous contact process and the single contact process together constitute the contact process between the target goods and the target handling equipment, and the first contact time interval and the second contact time interval constitute the contact time interval.

[0084] The contact process is constructed as follows: all the merged continuous contact processes and single contact processes are stored in the same set of contact behaviors of the target cargo, forming the contact process between the target cargo and the target handling equipment.

[0085] The contact time intervals are constructed as follows: the first contact time intervals corresponding to all continuous contact processes and the second contact time intervals corresponding to all single contact processes are summarized and arranged in chronological order to form the contact time intervals corresponding to the target goods.

[0086] Step S3 of this application constructs candidate contact segments centered on the contact frame time and filters stable intervals within the segments based on relative distance changes. This expands a single discrete contact frame into a continuous behavioral segment supported by preceding and following context. The initial contact process is obtained by mapping stable intervals, and multiple initial contact time intervals are further merged or differentiated using time interval thresholds, enabling continuous reconstruction and segmentation of contact behavior in the time dimension. Since relying solely on a single contact frame is susceptible to occlusion, false detection, or instantaneous contact fluctuations, making it difficult to accurately define the contact boundary, the introduction of distance change trend constraints between preceding and following frames allows for supplementary verification of the motion transition process before and after contact, thereby reducing the impact of instantaneous abnormal frames on the judgment results. By filtering out high-fluctuation segments through stable intervals, the contact process boundary is transformed from instantaneous point judgment to continuous interval judgment. Furthermore, multiple contact behaviors are merged or separated using time interval rules, ensuring consistency in the temporal structure of contact behaviors under different work cycles. This improves the completeness of the contact process construction and the accuracy of time interval division, providing a solid data foundation for intelligent warehousing and logistics status perception using computer vision.

[0087] S4. Perform process analysis on the contact process according to the contact frame time to determine the motion characteristic parameters of the target cargo.

[0088] In the embodiment of step S4, a brief introduction to the following related terms is given: Process analysis refers to the process of segmenting and comparing the motion state of the target cargo before and after contact based on the changes in the spatial position of the target cargo in each frame of the image during the contact process. It is used to reflect the impact of contact behavior on the motion state of the target cargo.

[0089] The process analysis method involves extracting the spatial position of the cargo from consecutive frame images during the contact process, calculating the displacement change sequence, and combining the contact frame time to divide the motion process into stages and compare the differences, thereby obtaining the motion state change characteristics. The specific implementation method can be referred to S401-S405.

[0090] Motion characteristic parameters: refers to the set of parameters used to characterize the changes in the motion state of the target cargo during the contact process, including the difference in displacement, the degree of dispersion of changes in the pre-contact stage, and the degree of dispersion of changes in the post-contact stage.

[0091] One possible implementation of step S4: S401, for any of the target goods, determine the contact process corresponding to the target goods and the contact time interval corresponding to the contact process, and extract the spatial position of the goods corresponding to each frame of the image within the contact process; S402, calculate the displacement change between adjacent frame images based on the spatial location of the cargo, and construct a displacement change sequence in chronological order; Displacement change: refers to the change in spatial distance between adjacent frames of the target cargo.

[0092] Displacement change sequence: refers to the set of displacement changes of adjacent frames arranged in chronological order, used to characterize the movement and change process of the target cargo.

[0093] The method for calculating the displacement change between adjacent frames is as follows: extract the spatial coordinates of the target cargo in two adjacent frames, use the coordinate difference calculation method under the same coordinate system to calculate the Euclidean distance change between the two frames, and use this distance value as the displacement change.

[0094] S403, based on the displacement change sequence and the positional relationship of the contact frame time in the contact time interval, determine the displacement change abrupt point corresponding to the contact frame time, and extract the displacement change difference value before and after the displacement change abrupt point. Positional relationship: refers to the temporal positional relationship of the contact frame within the contact time interval, including the temporal correlation between the frame before contact, the contact frame, and the frame after contact.

[0095] Displacement change abrupt change point: refers to the turning frame position in the displacement change sequence where a small change occurs to a large change or a large change occurs to a small change near the contact frame time.

[0096] The method for determining the displacement change abrupt point is as follows: In the displacement change sequence, taking the frame corresponding to the contact frame as the center, the displacement change of at least one adjacent frame before and at least one adjacent frame after the contact frame is selected as the analysis object, and the change amplitude between adjacent displacement changes is calculated sequentially; when the displacement change of a certain frame shows a turning relationship from a small change to a large change or from a large change to a small change compared with the displacement change of the previous frame and the next frame, the frame is determined as the displacement change abrupt point; where large change and small change are distinguished by a preset change amplitude threshold, the preset change amplitude threshold is set according to the displacement change range of the target goods under normal handling state in the warehousing scenario, and is used to distinguish between smooth motion and motion state changes affected by contact.

[0097] Displacement change difference value: refers to the difference between the displacement change in the frame before the abrupt change point and the displacement change in the frame after the abrupt change point, used to characterize the magnitude of change in motion state caused by contact behavior.

[0098] The method for extracting the displacement change difference value is as follows: The displacement change difference value is obtained by calculating the difference between the displacement changes of two adjacent frames before and after the displacement change abrupt point. Specifically, the displacement change value corresponding to the frame before the displacement change abrupt point is selected and the displacement change value corresponding to the frame after the displacement change abrupt point is subtracted, and the absolute value is taken as the displacement change difference value.

[0099] S404, based on the contact frame time, the contact process is divided into a pre-contact stage and a post-contact stage, and the degree of variation dispersion of the pre-contact stage and the post-contact stage is calculated respectively. Pre-contact phase: refers to the frame sequence corresponding to the time period from the start frame of the contact time interval to the moment before the contact frame.

[0100] Post-contact phase: refers to the frame sequence corresponding to the time period from the moment of contact to the end of the contact time interval.

[0101] Degree of variation dispersion: refers to the degree of fluctuation in the amount of displacement change within a corresponding stage, and is used to characterize motion stability.

[0102] The method for calculating the degree of dispersion of changes in the pre-contact and post-contact stages is as follows: extract the displacement changes of each frame in the pre-contact and post-contact stages respectively, calculate the absolute value of the difference between adjacent displacement changes in the stage, and take the average value of all differences as the degree of dispersion of changes in the stage.

[0103] S405, the displacement change difference value and the degree of dispersion of the change between the pre-contact stage and the post-contact stage are encapsulated to obtain the motion characteristic parameters of the target cargo. The motion characteristic parameters are used to characterize the motion state change characteristics of the target cargo during the contact process.

[0104] Encapsulation: refers to combining and storing or outputting multiple calculated numerical parameters according to a preset field structure, so as to make them a unified motion description result.

[0105] The encapsulation method involves: structurally combining the displacement variation difference value, the dispersion of the change before contact, and the dispersion of the change after contact according to the preset parameter field order to form the motion characteristic parameters corresponding to the target cargo, and binding and storing them with the target cargo identifier and the contact time interval.

[0106] Step S4 of this application unfolds the spatial position of the goods during the contact process into a continuous displacement change sequence along the time sequence, transforming the motion state of the target goods from discrete position points into a comparable change process. The displacement change sequence is divided into stages using the contact frame time as the boundary, creating a clear temporal correspondence between the motion segments before and after contact, thus providing a unified reference for difference analysis. Furthermore, by locating the abrupt change points of displacement change, the motion changes introduced by contact are separated from continuous motion, giving the source of change a clear direction. By quantifying the degree of dispersion of changes in the pre-contact and post-contact stages, the difference in motion stability can be expressed numerically. Therefore, the resulting motion characteristic parameters can simultaneously characterize the amplitude and degree of stability changes caused by contact within the same time frame, providing a basis for the identification of contact behavior and state judgment in intelligent warehousing and logistics using computer vision.

[0107] S5, based on the multi-frame images, detect the relative position change between the target cargo and the boundary of the functional area, determine the crossing behavior of the target cargo, and record the corresponding crossing frame time.

[0108] In the embodiment of step S5, a brief introduction to the following related terms is given: Relative position change: In step S5, relative position change refers to the change of the spatial position of the target cargo relative to the boundary of the functional area and the spatial relationship and distance relationship over time. It is used to characterize the evolution of the target cargo as it approaches, contacts and crosses the boundary from one side to the other.

[0109] Crossing behavior: refers to the process by which the spatial location of the target cargo is continuously transferred from one side of the functional area boundary to the other side, forming a stable change in the lateral relationship.

[0110] Crossing frame time: refers to the time marker corresponding to the frame in which the target cargo first establishes a valid crossing association with the functional area boundary during the crossing behavior, and is used as the time positioning reference for the crossing behavior.

[0111] One possible implementation of step S5 is as follows: S501, construct the spatial location trajectory of the target cargo based on its spatial location in multiple frames of images; Cargo spatial location trajectory: refers to the spatial location of the target cargo in each frame of the image connected in chronological order, used to represent the continuous movement path of the target cargo.

[0112] The method for constructing the cargo spatial location trajectory is as follows: extract the cargo spatial location (such as three-dimensional coordinate points) of the target cargo in each frame image; sort the cargo spatial locations according to the frame time order; connect the cargo spatial locations between adjacent frames to form a continuous path, which is the cargo spatial location trajectory.

[0113] S502, Obtain the boundary structure distribution information of the functional area boundary; Boundary structure distribution information refers to the spatial distribution of functional area boundaries in an image, including boundary location, shape, and direction of extension, used to define the spatial division range of functional areas. This boundary structure distribution information serves as a spatial determination benchmark, enabling the analysis of the positional relationship between the target cargo's trajectory and the functional area boundaries. This information can be obtained by identifying functional area boundaries across multiple image frames.

[0114] S503, based on the cargo spatial location trajectory and the boundary structure distribution information, determine the interaction crossing point between the target cargo movement trajectory and the functional area boundary, the interaction crossing point is used to characterize the location where the cargo spatial location trajectory and the boundary structure interact spatially; Interactive crossing point: refers to the location where the spatial trajectory of goods comes into contact with or crosses the boundary of the functional area in space, and is used to identify the intersection of the movement path and the boundary structure.

[0115] The method for determining the interaction crossing point is as follows: the functional area boundary is represented as a closed or open polyline structure formed by connecting the boundary key points, and the spatial position of each frame of cargo in the cargo spatial position trajectory is converted into the corresponding image coordinate point or unified world coordinate point; the shortest Euclidean distance between the cargo spatial position point of each frame and the functional area boundary is calculated; when the shortest Euclidean distance is less than a preset spatial threshold, it is determined that the cargo spatial position point corresponding to the frame has spatial interaction with the functional area boundary, and the frame is taken as the interaction crossing point; or, when the cargo spatial position point crosses from one side of the boundary area to the other side of the boundary between two adjacent frames, the relative lateral relationship between the position point and the boundary of the two frames is combined to determine that a crossing has occurred and the interaction crossing point is determined.

[0116] S504, Perform a continuous analysis on the interactive crossing points in chronological order to determine the continuous frame images of the target cargo movement trajectory intersecting with the functional area boundary; Continuity analysis: This refers to judging the distribution of interaction traversal points in the time dimension to determine whether the interaction behavior continues to exist in adjacent frames.

[0117] The continuity analysis method involves arranging all interactive crossing points in chronological order, calculating the time interval between corresponding frames of adjacent interactive crossing points, determining whether the time interval meets the continuous frame condition, and grouping interactive crossing points that meet the condition into a continuous sequence. The continuous frame condition is set as follows: the frame order difference between corresponding frames of adjacent interactive crossing points is not greater than a preset frame interval threshold, for example, the preset frame interval threshold ranges from 1 to 3 frames. When the target cargo approaches the functional area boundary and crosses it, the interaction between its trajectory and the boundary structure is manifested in short-interval continuous occurrences in time. However, there may be slight detection omissions or boundary extraction fluctuations during image acquisition, causing some frames to fail to form effective interactive crossing points. By limiting the preset frame interval threshold to the range of 1 to 3 frames, a small number of discontinuities are allowed between adjacent interactive crossing points while maintaining temporal continuity. This suppresses the impact of short-term detection instability while avoiding misjudging discrete interactions with large time intervals as continuous crossing processes, ensuring that the continuous sequence has both stability and the ability to constrain actual crossing behavior.

[0118] Continuous frame images: refers to a set of image frames that are temporally adjacent and all have interactive crossing points, used to represent a continuously occurring crossing process.

[0119] S505, in the continuous frame images, identify the path through which the target cargo moves from one side of the functional area boundary to the other side of the support area. Supporting area on one side of the functional area boundary: refers to the spatial area on one side of the functional area boundary, that is, the spatial range formed by the functional area boundary, such as the spatial range on either side of the inbound or outbound area.

[0120] The supporting area on the other side of the functional area boundary refers to the other spatial area opposite to the supporting area on the other side mentioned above.

[0121] Path traversal process: refers to the process by which the spatial location trajectory of goods extends continuously from one side of the functional area boundary to the other side of the support area.

[0122] The method for identifying the path traversal process is as follows: the functional area boundary is divided into two supporting areas, and the spatial location point of each frame of goods is input into a preset area determination rule for side attribute determination. The side attribute determination method is to determine whether the point is within a polygonal area, that is, to determine its supporting area by judging whether the spatial location point of goods is located inside the polygonal area formed by the functional area boundary; the side attribute results of each frame are sorted in chronological order, and when it is detected that the spatial location of goods continuously moves from the first side support area to the second side support area, and there is no situation of returning to the first side support area again during the transfer process, the continuous transfer sequence is determined as the path traversal process.

[0123] S506, when the path traversal process maintains a unidirectional traversal trend within a consecutive preset number of frames, it is determined that a crossing behavior has occurred, and the frame time of the starting interaction crossing point of the path traversal process is determined as the crossing frame time.

[0124] Preset number of consecutive frames: refers to the minimum number of consecutive frames required to determine whether the path traversal process has a stable trend.

[0125] The preset method for the continuous preset frame count is as follows: Based on the inter-frame time interval of multiple images, the typical passage time of the target cargo near the functional area boundary from one support area to another is converted into the corresponding number of frames to obtain the base frame count. Then, a reserved frame count is superimposed on the base frame count to form the continuous preset frame count. The reserved frame count is used to cover short pauses or detection fluctuations near the boundary, so that the continuous preset frame count can completely cover the effective frame range of a crossing process. For example, if the inter-frame time interval of multiple images is 0.1 seconds, and the typical time for the target cargo to pass through the functional area boundary is 2 seconds, then the base frame count is 20 frames. Considering that the cargo may experience short pauses or detection jitters of less than 1 second near the boundary, if about 10 frames are added as reserved frames, the continuous preset frame count can be set to 30 frames to completely cover the crossing behavior process and avoid short-term misjudgments.

[0126] Starting Interactive Crossing Point: The starting interactive crossing point refers to the interactive crossing point at the beginning of the path crossing process, where the spatial location trajectory of the cargo first forms an interactive crossing relationship with the boundary of the functional area. The corresponding frame time is used to mark the start time of the crossing behavior.

[0127] Step S5 of this application constructs a continuous trajectory of the target cargo's spatial location in chronological order and introduces the spatial structure distribution information of the functional area boundary, enabling the target cargo's movement path to form a unified spatial reference relationship with the boundary structure. The spatial relationship between the trajectory and the boundary is mapped frame-by-frame through interactive crossing points. Combined with the temporal continuity analysis of these interactive crossing points, the scattered crossing judgment results are merged in the temporal dimension, thus forming a path-crossing process with continuous constraints. Since single-frame positional relationships are easily affected by detection errors or occlusion, leading to boundary misjudgments, the introduction of trajectory continuity constraints avoids instantaneous misjudgments caused by isolated crossing points. Simultaneously, through the unidirectional trend constraint of the path-crossing process, the determination of crossing behavior does not depend on a single positional change but on the stable transfer of regional attribute relationships in continuous frames. This ensures that the crossing frame time stably corresponds to the time node where the first effective crossing association is formed, thereby improving the reliability of the crossing frame time determination.

[0128] S6, perform an association determination between the cross-frame time and the contact time interval of the contact process to obtain the determination result.

[0129] In the embodiment of step S6, a brief introduction to the following related terms is given: Association determination: This refers to the process of comprehensively analyzing the temporal correspondence and spatial consistency between the crossing frame moment and the contact process based on the time index relationship and spatial distance change sequence, in order to determine whether the crossing behavior and the contact process belong to the same logistics interaction process.

[0130] The method for performing association determination is as follows: a time window spanning the neighboring time of a frame is constructed, and a time consistency analysis is performed on the contact frame times within the time window. At the same time, the consistency statistical results of the spatial distance change trend within the contact time interval are combined to form an association matching degree index, and a graded determination is made based on the index. For specific implementation methods, please refer to S601-S606.

[0131] Judgment Result: The output of the correlation judgment includes a first result and a second result. The first result refers to the judgment result that there is a weak correlation between the crossing frame time and the contact process. It is used to characterize the situation where the crossing behavior of the target goods and the contact process have a certain offset or local inconsistency in the time dimension, but may still belong to the same handling interaction link. This result corresponds to the subsequent triggering of the contact time interval correction process. The second result refers to the judgment result that there is a deterministic correlation between the crossing frame time and the contact process. It is used to characterize the target goods' crossing behavior and the contact process have a consistent correspondence in time, that is, the crossing behavior and the contact process belong to the same stable mapping relationship of the same complete logistics interaction process. This result corresponds to the processing flow of directly generating logistics records.

[0132] One possible implementation of step S6: S601, Obtain the frame image corresponding to the time of the cross-frame, and extract the spatial location of the target cargo in the frame image; S602, based on the contact time interval of the contact process, the relative distance change sequence is segmented to form a spatial distance change sequence corresponding to the contact time interval; Segment extraction: refers to extracting continuous data segments within a corresponding time range from the relative distance change sequence according to the start and end frame times of the contact time interval.

[0133] The method for extracting segments is as follows: based on the start frame number and end frame number corresponding to the contact time interval, sequence data within the corresponding frame range is extracted from the relative distance change sequence to form a continuous subsequence, i.e., the spatial distance change sequence.

[0134] Spatial distance change sequence: refers to the continuous subsequence of relative distance change sequence obtained by segment extraction. It is used to characterize the change characteristics of the distance between the target cargo and the target handling equipment within the local time range during the contact time interval, and serves as a spatial reference sequence for subsequent comparison across the time neighborhood of frames.

[0135] S603, the spatial location of the cargo corresponding to the cross-frame time is matched with the spatial location of the cargo corresponding to each contact frame time in the contact time interval according to the time frame sequence to form a time index correspondence between the cross-frame time and the contact process. Frame sequence number matching: refers to the process of aligning the frame number of the cross-frame moment with the frame numbers of all frames within the contact time interval on the same time axis.

[0136] The frame sequence number matching method is as follows: assign a unique frame sequence number to all frame images, establish a frame sequence according to the timestamp or acquisition order, and map and align the frame sequence number corresponding to the time of crossing frames with the frame sequence number of each frame in the contact time interval to form a one-to-one index relationship between the crossing frames and the contact frames.

[0137] Time index correspondence: refers to the set of correspondences established between cross-frame times and contact times within the contact time interval under a unified frame numbering system.

[0138] S604, Based on the time index correspondence, calculate the time interval between the cross-frame time and each contact frame time within the contact time interval; Time interval: refers to the difference on the time axis between the frame crossing moment and the contact frame moments within the contact time interval.

[0139] The time interval is calculated as follows: obtain the timestamp corresponding to the time of the crossing frame, and obtain the timestamp of each contact frame within the contact time interval one by one; use the time difference calculation method (time of the later frame minus time of the earlier frame) to obtain the time interval sequence, forming the time interval between the crossing frame and the contact frame.

[0140] S605, determine the degree of association and matching between the cross-frame time and the contact process based on the correspondence between the time interval and the spatial distance change sequence; Association matching degree: refers to the comprehensive matching index used to characterize the cross-action and contact process in terms of temporal proximity and spatial change consistency.

[0141] The method for determining the degree of association matching is as follows: A preset number of frames are selected forward and backward from the crossing frame moment to form a crossing time neighborhood interval. The preset number of frames is determined based on the image acquisition frame rate and the average number of frames the target goods pass through near the functional area boundary, for example, by setting the number of frames corresponding to 1 to 3 seconds, so that the time neighborhood can cover the main time range in which the crossing behavior occurs. The number of contact frame moments with a time interval less than a preset time threshold is calculated, and their proportion to the total number of frames in the contact time interval is calculated to obtain the time proportion. The preset time threshold is determined based on the average time difference between the crossing frame moment and the contact occurrence frame in historical contact events. The number of frames with the same direction of distance change in the spatial distance change sequence within the time neighborhood interval is counted, and their proportion is calculated to obtain the spatial consistency proportion. The same direction of distance change refers to the same sign of the distance change between adjacent frames, used to characterize a continuous approach or continuous separation trend. The time proportion and the spatial consistency proportion are combined to obtain the degree of association matching. The combination method can be product calculation or weighted calculation. If it is weighted calculation, the weight is used to adjust the contribution ratio of the time proportion and the spatial consistency proportion to the degree of association matching, such as setting both to 0.5.

[0142] S606, Based on the degree of association matching and the preset association conditions, a matching judgment is made to determine the judgment result as either the first result or the second result.

[0143] Matching judgment: A graded judgment process based on the comparison between the degree of association matching and the preset threshold range, used to determine the association strength between the cross-frame time and the contact process.

[0144] Preset association conditions: This is a set of conditions used to classify and determine the degree of association matching. The preset association conditions include a first preset association condition and a second preset association condition. The first preset association condition refers to an association matching degree greater than or equal to a first preset threshold and less than a second preset threshold. It is used to characterize a weak association state where there is insufficient temporal and spatial consistency between the cross-frame time and the contact process, but there is a partial correspondence. The second preset association condition refers to an association matching degree greater than or equal to a second preset threshold. It is used to characterize a strong association state where there is a stable and consistent correspondence between the cross-frame time and the contact process in terms of temporal and spatial comprehensive features.

[0145] It should be noted that both the first and second preset thresholds are statistically determined based on the correlation matching degree corresponding to the labeled cross-behavior and contact process samples in historical warehousing operations. After sorting the correlation matching degree of all samples according to the order of magnitude, the value corresponding to the low quantile position is selected as the first preset threshold, and the value corresponding to the high quantile position is selected as the second preset threshold, with the first preset threshold being less than the second preset threshold. Because the correlation matching degree in historical samples usually naturally forms two distribution intervals: low-value clusters and high-value clusters, dividing them by quantiles allows the first preset threshold to correspond to the boundary level of weak correlation and transitional state, and the second preset threshold to correspond to the starting level of stable strong correlation state. Thus, even with detection errors and fluctuations in operation rhythm, the distinction between weak correlation, transitional correlation, and strong correlation can still be maintained, and the judgment stability can be ensured.

[0146] The matching and judgment methods are as follows: If the degree of association matching meets the first preset association condition, then the determination result is determined to be the first result. The first result is used to characterize the weak association relationship under the condition that there is a time offset between the cross-frame time and the contact process. If the degree of association matching simultaneously meets the second preset association condition, then the determination result is determined to be the second result, which is used to characterize the deterministic association between the cross-frame time and the contact process.

[0147] Step S6 of this application introduces the crossing frame moment into a unified temporal neighborhood analysis framework and performs collaborative alignment processing with the time index relationship and spatial distance change sequence within the contact time interval. This enables the crossing behavior and the contact process to complete corresponding mapping under the same time reference. At the same time, through the joint constraint of time interval constraint and spatial change consistency statistics, the correlation between the crossing frame moment and the contact process can be quantitatively characterized from two dimensions: temporal proximity and spatial evolution consistency. This avoids the problem of false or missed correlation caused by relying solely on a single time difference or a single spatial feature. Furthermore, it maintains the ability to stably classify the correlation judgment results under the conditions of detection error and operational rhythm fluctuations. This provides an interpretable basis for determining whether the crossing behavior and the contact process belong to the same logistics interaction link.

[0148] S7. If the determination result is the first result, then the contact time interval of the contact process is corrected according to the motion characteristic parameters, and the association determination is re-executed for the corrected contact time interval.

[0149] In the embodiment of step S7, a brief introduction to the following related terms is given: The correction in step S7 refers to adjusting the start or end boundary of the contact time interval based on motion characteristic parameters, so that the adjusted time interval is more consistent with the time range of the actual contact of the target goods, and is used to eliminate the time offset caused by target detection delay or short-term false detection.

[0150] The correction method is as follows: by identifying the location point where the motion state changes significantly during the contact process, the frame corresponding to the location point is used as the boundary candidate point, and combined with the difference in motion stability on both sides of the boundary, the original contact time interval is expanded or contracted forward or backward. For specific implementation methods, see S701-S703.

[0151] One possible implementation of step S7: S701, if the determination result is the first result, then based on the displacement change difference value in the motion characteristic parameters, determine the candidate frame moment when the target cargo undergoes a sudden change in motion within the contact time interval, and use the candidate frame moment as the candidate boundary adjustment point; The method for determining candidate frame times is as follows: during the current contact process, the motion characteristic parameters of the target cargo corresponding to the contact process are determined. Based on the displacement change difference value in the motion characteristic parameters, the frame time corresponding to the frame image of the target cargo with the sudden displacement change is identified as the candidate frame time of the first result. If there are multiple frame times corresponding to displacement change difference values, the earliest frame time is selected as the candidate frame time.

[0152] Candidate boundary adjustment point: refers to the frame time corresponding to the location where the displacement change difference value changes abruptly, and is used as a reference benchmark point for adjusting the contact interval boundary.

[0153] S702, based on the degree of dispersion of the changes in the pre-contact stage and the degree of dispersion of the changes in the post-contact stage on both sides of the candidate boundary adjustment point, the contact time interval is boundary corrected to obtain the corrected contact time interval. Boundary correction: refers to the process of repositioning the start or end frame of the contact time interval based on the difference in motion stability near the frame where the candidate boundary adjustment point is located.

[0154] One implementation method for boundary correction is as follows: Step 1: When the dispersion of changes in the pre-contact stage is greater than that in the post-contact stage and the difference between the two is greater than a preset difference threshold, the following search is performed in the pre-contact stage: starting from the frame with a dispersion higher than the preset stability threshold, scan frame by frame until the first frame with a dispersion lower than or equal to the preset stability threshold is found, and the increase in dispersion of each intermediate frame relative to the previous frame does not exceed the preset increase. The moment of the first frame that meets the conditions is determined as the stable starting frame, and the stable starting frame is used as the starting boundary of the corrected contact time interval.

[0155] Step 2: When the dispersion of changes in the post-contact stage is greater than that in the pre-contact stage and the difference between the two is greater than a preset difference threshold, the following search is performed in the post-contact stage: starting from the frame with a dispersion higher than the preset stability threshold, scan frame by frame until the first frame with a dispersion lower than or equal to the preset stability threshold is found, and during this scanning process, the increase in dispersion of each intermediate frame relative to the previous frame does not exceed the preset increase. The moment of the first frame that meets the conditions is determined as the stable end frame, and the stable end frame is used as the end boundary of the corrected contact time interval.

[0156] Step 3: When the difference between the dispersion of changes in the pre-contact stage and the dispersion of changes in the post-contact stage is less than or equal to a preset difference threshold, then within the contact time interval, a stable frame interval with a dispersion of at least n consecutive frames (e.g., the preset number of consecutive frames n=5) that is lower than the preset stability threshold is searched, and the starting frame time of the stable frame interval is taken as the starting boundary of the corrected contact time interval, and the ending frame time is taken as the ending boundary of the corrected contact time interval.

[0157] The preset difference threshold, preset stability threshold, and preset recovery range are all statistically determined based on contact process samples already marked in historical warehousing operations. The preset difference threshold is taken as the upper bound of the difference in dispersion before and after contact in historical samples; the preset stability threshold is taken as the high quantile value (e.g., 90%) of the dispersion distribution during the stable contact phase; the preset recovery range is defined as 50% of the difference between the dispersion of the previous frame and the stability threshold, that is, the increase in dispersion of the current frame compared to the previous frame is allowed to be no more than 0.5 × (AB), where A is the dispersion of the previous frame and B is the preset stability threshold. The purpose of using this recovery range constraint is to allow a small recovery in dispersion caused by slight detection noise during the scanning process, while avoiding misjudging large fluctuations as a stable transition.

[0158] Corrected contact time range: refers to the new contact time range after boundary adjustment, used to more accurately characterize the time period during which the target goods actually come into contact.

[0159] The dispersion of the change in the pre-contact stage is used to characterize the degree of fluctuation in the displacement change in the pre-contact stage, and the dispersion of the change in the post-contact stage is used to characterize the degree of fluctuation in the displacement change in the post-contact stage. S703, based on the corrected contact time interval and the crossed frame time, re-execute the association determination.

[0160] It should be noted that the specific method for performing the association determination can be found in the embodiment section of step S6.

[0161] Step S7 of this application introduces a candidate boundary adjustment point determination mechanism based on motion feature parameters when the determination result is the first result. This transforms the contact time interval correction from a single time threshold determination into a structured adjustment process with the position of motion mutation as the anchor point, thus providing a clear temporal positioning basis for boundary correction. Furthermore, by combining the difference in the degree of dispersion of changes between the pre-contact and post-contact stages, the stability of both sides of the candidate boundary adjustment point is compared and analyzed. Short-term fluctuations are suppressed by preset stability thresholds and preset recovery amplitudes, enabling the boundary adjustment process to distinguish between changes in the actual motion state and local fluctuations caused by detection noise. In cases of large differences, boundary extrapolation or inward contraction is achieved by determining a stable entry point on one side. In cases of small differences, the overall reconstruction of the contact interval is achieved by constraining the continuity of the stable frame interval. This allows the corrected contact time interval to converge to a low-fluctuation and continuously stable frame set range, reducing time offsets caused by detection delays, occlusion, and instantaneous false detections, improving the temporal correspondence consistency between the contact process and the crossing behavior, and enhancing the accuracy of time sequence information in subsequent logistics records.

[0162] S8. If the determination result is the second result, then the logistics record of the target goods is generated based on the contact process and crossing behavior.

[0163] In the embodiment of step S8, a brief introduction to the following related terms is given: Logistics records of target goods: refers to a complete set of temporal information that characterizes the target goods during a single handling interaction, consisting of contact process and crossing behavior. It includes contact time information, spatial location change information, and regional flow information, and is used to reflect the movement path and corresponding temporal sequence relationship of the target goods between warehousing functional areas.

[0164] One possible implementation of step S8: S801, if the determination result is the second result, then based on the cargo spatial position corresponding to each frame image within the contact time interval, and combined with the displacement change sequence, determine the start contact frame time and end contact frame time of the target cargo during the contact process. Initial contact frame time: refers to the first frame time corresponding to when the target cargo changes from a non-contact motion state to a contact stable change state within the contact time interval, and is used to characterize the time starting point of the contact process.

[0165] End of contact frame time: refers to the last frame time when the target cargo transitions from a stable contact state to a non-contact motion state within the contact time interval, and is used to characterize the end point of the contact process.

[0166] S802, determine the regional flow path of the target goods based on the spatial positional relationship between the cross-frame time and the boundary of the functional area; Spatial positional relationship: refers to the relative positional relationship between the target cargo spatial position and the functional area boundary in the same coordinate system at the time of the frame crossing, including the positional changes before and after crossing the boundary, and the changes in distance from the boundary.

[0167] Regional flow path: refers to the spatial migration sequence of target goods from one functional area to another during the crossing process, used to characterize the regional-level movement trajectory of goods in the storage space.

[0168] The method for determining the regional circulation path of the target cargo is as follows: based on the spatial relationship between the cargo's location and the functional area boundary at the time of the crossing frame, the change in the regional affiliation of the target cargo before and after the time of the crossing frame is determined; the functional area where the cargo was located before the crossing and the functional area where it was located after the crossing are sequentially connected in chronological order to form a one-way path structure from the starting functional area to the target functional area, which serves as the regional circulation path.

[0169] S803, the starting contact frame time and ending contact frame time of the target goods in the contact process and the regional flow path are associated in chronological order to form a handling process record of the target goods; Handling process record: refers to the structured record formed by aligning the contact process time information with the cross-behavioral spatial path information on a unified time axis. It is used to characterize the complete process information of the contact time period of the target goods in a single handling operation and the relationship between the target goods and the functional area when entering and leaving.

[0170] S804, the transport process record is stored and packaged according to the identification of the target goods to form the logistics record of the target goods.

[0171] Storage encapsulation: refers to the process of binding the handling process records according to the unique identifier of the target goods and organizing and storing them according to a certain data structure, so that the handling process records of the same target goods can be accumulated and managed in chronological order.

[0172] The storage and encapsulation method involves mapping and binding the handling process records one-to-one with the target cargo identifier, and writing them into the record set corresponding to the target cargo in chronological order to form a traceable historical handling sequence structure.

[0173] Logistics records are used to characterize the regional flow information of target goods under the combined effects of contact process and crossing behavior.

[0174] Step S8 of this application determines the start and end contact frame times within the contact time interval when the determination result is the second result. This limits the contact process to a stable change interval with clear time boundaries, thereby reducing the interference of transition frames or abnormal frames on the representation of the contact process. At the same time, it constructs a regional flow path by combining the spatial positional relationship between the crossing frame time and the functional area boundary. This allows the spatial migration process of the target goods from one functional area to another to be expressed in a temporal order and to form a correspondence with the contact time interval. Furthermore, it unifies and associates the temporal information of the contact process with the spatial changes of the crossing behavior, so that the originally separate temporal and spatial dimension information is integrated under the same handling process framework. This allows the generated logistics record to simultaneously reflect the contact time range and regional flow path changes of the target goods.

[0175] S9, the logistics records are summarized according to the distribution of the functional areas to form the logistics status information of the intelligent warehouse.

[0176] Summarizing: This refers to the process of classifying and organizing the logistics records corresponding to the target goods according to the functional area as the classification dimension, and merging and statistically analyzing multiple logistics records in the same functional area according to the preset time order or spatial area order. It is used to transform scattered single handling process records into a structured regional logistics information set.

[0177] The logistics status information of intelligent warehousing refers to the overall status expression of the warehouse based on the summary of logistics records of multiple target goods. It is used to characterize the distribution of goods inflow, outflow and residence between functional areas, and to reflect the intensity and flow relationship of logistics activities in different functional areas within the warehouse space within a unit of time or statistical period.

[0178] The logistics status information of smart warehousing can be stored or output in the form of functional area flow matrix, functional area inflow and outflow sequence, or functional area time series statistical chart. For example, the regional flow matrix constructed with functional areas as row and column indices is used to represent the correspondence of goods flow between different functional areas, the regional inflow and outflow sequence sorted by time is used to represent the logistics change process of each functional area in the time dimension, and the time segmented statistical chart is used to characterize the logistics change trend of each functional area in different statistical periods.

[0179] This application embodiment uses computer vision to perform target detection and region segmentation on multi-frame images of a warehouse scene, obtaining target goods, target handling equipment, and functional area boundaries. The position of the target goods is continuously expressed on the timeline of the frame sequence, allowing the target's motion state to be described in a unified temporal coordinate system. Contact events are extracted through temporal analysis of the relative positional changes of the target goods and target handling equipment. Crossing behavior is identified by combining the positional changes of the target goods and functional area boundaries, enabling the contact process and the area crossing process to form corresponding spatial relationship change features in the same visual sequence, thus providing a unified temporal and spatial basis for subsequent correlation analysis. Furthermore, by analyzing the contact time zone... The system correlates and determines the time of contact with the frame crossing, and introduces a time interval correction mechanism based on motion feature parameters when there is an offset. It uses the motion mutation point and stable change interval to relocate the contact boundary, so that the time deviation caused by occlusion, detection jitter or inter-frame error is constrained to the stable motion interval, thereby improving the temporal consistency between contact events and crossing behavior. The corrected contact process and crossing behavior are summarized according to the functional area distribution to form a regional flow expression of the target goods, so that the warehousing logistics status information can reflect the migration path of goods between different functional areas and their corresponding contact process in a structured way, improving the traceability and expression of logistics behavior in complex warehousing scenarios.

[0180] Example 2 like Figure 2 As shown, based on Embodiment 1, this application provides an intelligent warehousing and logistics status perception system, the system comprising: The target recognition module is used to identify target goods, target handling equipment, and functional area boundaries from multiple frames of images of a warehouse scene; The contact event detection module is used to detect contact events and record the corresponding contact frame times based on the relative position changes of the target cargo and the target handling equipment in the multi-frame images. The contact process construction module is used to construct, for the target cargo, the contact process between the target cargo and the target handling equipment and the corresponding contact time interval based on the contact events and contact frame times related to the target cargo. The motion feature analysis module is used to perform process analysis on the contact process according to the contact frame time to determine the motion feature parameters of the target cargo. The crossing behavior detection module is used to detect the relative position change between the target cargo and the boundary of the functional area based on the multi-frame images, determine the crossing behavior of the target cargo, and record the corresponding crossing frame time. The association determination module is used to perform an association determination between the cross-frame time and the contact time interval of the contact process, and obtain a determination result; The correction control module is used to correct the contact time interval of the contact process according to the motion feature parameters when the determination result is the first result, and to re-execute the association determination on the corrected contact time interval. The first result is used to characterize the weak association relationship between the cross-frame time and the contact process under the condition of time offset. The logistics record generation module is used to generate a logistics record for the target goods based on the contact process and crossing behavior when the determination result is the second result. The second result is used to characterize that there is a deterministic correlation between the crossing frame time and the contact process.

[0181] The information aggregation module is used to aggregate the logistics records according to the distribution of the functional areas to form the logistics status information of the smart warehouse.

[0182] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this application. Various changes and modifications can be made to this application without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of this application as claimed. The scope of protection of this application is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent warehousing and logistics status perception, characterized in that, The method includes: Identify target goods, target handling equipment, and functional area boundaries from multi-frame images of a warehouse scene; Based on the relative positional changes of the target cargo and the target handling equipment in the multi-frame images, contact events are detected and the corresponding contact frame times are recorded. For the target cargo, the contact process and corresponding contact time interval between the target cargo and the target handling equipment are constructed based on the contact events and contact frame times related to the target cargo. The contact process is analyzed according to the contact frame time to determine the motion characteristic parameters of the target cargo. Based on the multi-frame images, the relative positional change between the target cargo and the boundary of the functional area is detected, the crossing behavior of the target cargo is determined, and the corresponding crossing frame time is recorded; The correlation determination is performed between the cross-frame time and the contact time interval of the contact process to obtain the determination result; If the determination result is the first result, the contact time interval of the contact process is corrected according to the motion feature parameters, and the association determination is re-executed on the corrected contact time interval. The first result is used to characterize the weak association relationship between the cross-frame time and the contact process under the condition of time offset. If the determination result is the second result, then the logistics record of the target goods is generated based on the contact process and crossing behavior. The second result is used to characterize the deterministic correlation between the crossing frame time and the contact process. The logistics records are summarized according to the distribution of the functional areas to form the logistics status information of the intelligent warehouse.

2. The method according to claim 1, characterized in that, The step of detecting contact events and recording the corresponding contact frame times based on the relative positional changes of the target cargo and the target handling equipment in the multi-frame images includes: Obtain the spatial position of the target cargo in multiple frames of images and the spatial position of the target handling equipment in multiple frames of images; For each of the multiple frames of images, the relative distance between the target cargo and the target handling equipment is calculated based on the spatial position of the cargo and the spatial position of the equipment in the frame image, and recorded to form a relative distance sequence; The relative distance sequence is subjected to inter-frame difference processing to obtain a relative distance change sequence; When the relative distance change sequence shows a continuous decreasing trend in the multi-frame images, it is determined that the target cargo and the target handling equipment are in a state of close change. In the frame image corresponding to the near change state, when the relative distance value is less than a preset threshold, a suspected contact event is determined and the corresponding initial contact frame time is recorded; For the suspected contact event, at least one subsequent frame image after the initial contact frame time is selected, and the relative distance value between the target cargo and the target handling equipment corresponding to the subsequent frame image is obtained; The difference between the relative distance value corresponding to the subsequent frame image and the relative distance value corresponding to the initial contact frame time is calculated to obtain the distance change retention value; When the distance change remains less than a preset fluctuation threshold, the suspected contact event is determined to be a contact event, and the initial contact frame time of the suspected contact event is marked as the contact frame time of the contact event.

3. The method according to claim 2, characterized in that, The step of constructing the contact process and corresponding contact time interval between the target cargo and the target handling equipment based on the contact events and contact frame times related to the target cargo includes: For any of the target goods, determine the contact event corresponding to the target goods and the contact frame time corresponding to the contact event, and sort the contact frame times in chronological order to form a contact frame sequence; Using each contact frame moment in the contact frame sequence as a reference frame, at least one preceding frame image is selected forward and at least one subsequent frame image is selected backward to construct candidate contact segments. For each candidate contact segment, the relative distance value between the target cargo and the target handling equipment in each frame image within the candidate contact segment is obtained, and a sequence of relative distance values ​​is constructed in chronological order. The relative distance change between adjacent frame images is calculated based on the relative distance value sequence, and a stable range of relative distance change between the target cargo and the target handling equipment is selected based on the relative distance change. The frame images corresponding to the stable interval are mapped to the initial contact process in chronological order, and the initial contact time interval of the initial contact process is determined based on the contact frame time of the frame images in the initial contact process. If the time interval between two adjacent initial contact time intervals is less than a preset time interval threshold, the corresponding initial contact processes are merged to form a continuous contact process, and the initial contact time intervals are spliced ​​together to form the first contact time interval of the continuous contact process. If the time interval between two adjacent initial contact time intervals is not less than a preset time interval threshold, then the initial contact process is marked as a single contact process, and the initial contact time interval is used as the second contact time interval of the single contact process. The continuous contact process and the single contact process together constitute the contact process between the target cargo and the target handling equipment, and the first contact time interval and the second contact time interval constitute the contact time interval.

4. The method according to claim 3, characterized in that, The step of performing process analysis on the contact process according to the contact frame time to determine the motion characteristic parameters of the target cargo includes: For any of the target goods, determine the contact process corresponding to the target goods and the contact time interval corresponding to the contact process, and extract the spatial position of the goods corresponding to each frame of the image within the contact process; The displacement change between adjacent frame images is calculated based on the spatial location of the cargo, and a displacement change sequence is constructed in chronological order. Based on the displacement change sequence, and combined with the positional relationship of the contact frame time in the contact time interval, the displacement change abrupt point corresponding to the contact frame time is determined, and the displacement change difference value before and after the displacement change abrupt point is extracted. Based on the contact frame time, the contact process is divided into a pre-contact stage and a post-contact stage, and the degree of variation dispersion of the pre-contact stage and the post-contact stage is calculated respectively. The displacement variation difference value and the degree of dispersion of the changes in the pre-contact stage and the post-contact stage are encapsulated to obtain the motion characteristic parameters of the target cargo. The motion characteristic parameters are used to characterize the motion state change characteristics of the target cargo during the contact process.

5. The method according to claim 4, characterized in that, The step of detecting the relative positional change between the target cargo and the functional area boundary based on the multi-frame images, determining the crossing behavior of the target cargo, and recording the corresponding crossing frame time includes: Construct a cargo spatial location trajectory based on the cargo spatial location in multiple frames of images; Obtain the boundary structure distribution information of the functional area boundary; Based on the cargo spatial location trajectory and the boundary structure distribution information, the interaction crossing point between the target cargo movement trajectory and the functional area boundary is determined. The interaction crossing point is used to characterize the location where the cargo spatial location trajectory and the boundary structure interact spatially. The interaction crossing points are analyzed in chronological order to determine the continuous frame images of the target cargo movement trajectory intersecting with the functional area boundary; In the continuous frame images, the path of the target cargo movement trajectory from one side of the functional area boundary to the other side of the support area is identified. When the path traversal process maintains a unidirectional traversal trend within a consecutive preset number of frames, it is determined that a crossing behavior has occurred, and the frame time of the starting interaction crossing point of the path traversal process is determined as the crossing frame time.

6. The method according to claim 5, characterized in that, The step of associating the cross-frame time with the contact time interval of the contact process to obtain a determination result includes: Obtain the frame image corresponding to the time spanning the frame, and extract the spatial location of the target cargo in the frame image; Based on the contact time interval of the contact process, the relative distance change sequence is segmented to form a spatial distance change sequence corresponding to the contact time interval; The spatial location of the cargo corresponding to the crossing frame time is matched with the spatial location of the cargo corresponding to each contact frame time within the contact time interval according to the time frame sequence, so as to form a time index correspondence between the crossing frame time and the contact process. Based on the time index correspondence, calculate the time interval between the cross-frame time and each contact frame time within the contact time interval; The degree of correlation and matching between the time interval and the spatial distance change sequence is determined based on the correspondence between the time interval and the contact process. The matching judgment is made based on the degree of association and the preset association conditions, and the judgment result is determined to be either the first result or the second result.

7. The method according to claim 6, characterized in that, The preset association conditions include a first preset association condition and a second preset association condition. The step of matching the degree of association with the preset association conditions to determine whether the result is a first result or a second result includes: If the degree of association matching meets the first preset association condition, then the determination result is determined to be the first result; If the degree of association matching simultaneously meets the second preset association condition, then the determination result is determined to be the second result.

8. The method according to claim 7, characterized in that, The motion characteristic parameters include displacement variation difference values, the degree of variation dispersion in the pre-contact stage, and the degree of variation dispersion in the post-contact stage. If the determination result is the first result, the contact time interval of the contact process is corrected based on the motion characteristic parameters, and the association determination is re-executed on the corrected contact time interval, including: If the determination result is the first result, then based on the displacement change difference value in the motion characteristic parameters, the candidate frame time when the target cargo undergoes a sudden change in motion within the contact time interval is determined, and the candidate frame time is used as the candidate boundary adjustment point; Based on the degree of dispersion of changes in the pre-contact stage and the degree of dispersion of changes in the post-contact stage on both sides of the candidate boundary adjustment point, the contact time interval is boundary-corrected to obtain a corrected contact time interval. The degree of dispersion of changes in the pre-contact stage is used to characterize the degree of fluctuation of displacement changes in the pre-contact stage, and the degree of dispersion of changes in the post-contact stage is used to characterize the degree of fluctuation of displacement changes in the post-contact stage. Based on the corrected contact time interval and the crossed frame time, the association determination is re-executed.

9. The method according to claim 7, characterized in that, If the determination result is the second result, then a logistics record for the target goods is generated based on the contact process and crossing behavior, including: If the determination result is the second result, then based on the cargo spatial position corresponding to each frame image within the contact time interval, and combined with the displacement change sequence, the start contact frame time and end contact frame time of the target cargo during the contact process are determined. Based on the spatial relationship between the crossing frame time and the functional area boundary, the regional flow path of the target goods is determined; The starting and ending contact frames of the target cargo during the contact process and the flow path of the area are associated in chronological order to form a record of the handling process of the target cargo. The transport process records are stored and packaged according to the identification of the target goods to form the logistics record of the target goods. The logistics record is used to characterize the regional flow information of the target goods under the combined action of contact process and crossing behavior.

10. An intelligent warehousing and logistics status sensing system, characterized in that, The system includes: The target recognition module is used to identify target goods, target handling equipment, and functional area boundaries from multiple frames of images of a warehouse scene; The contact event detection module is used to detect contact events and record the corresponding contact frame times based on the relative position changes of the target cargo and the target handling equipment in the multi-frame images. The contact process construction module is used to construct, for the target cargo, the contact process between the target cargo and the target handling equipment and the corresponding contact time interval based on the contact events and contact frame times related to the target cargo. The motion feature analysis module is used to perform process analysis on the contact process according to the contact frame time to determine the motion feature parameters of the target cargo. The crossing behavior detection module is used to detect the relative position change between the target cargo and the boundary of the functional area based on the multi-frame images, determine the crossing behavior of the target cargo, and record the corresponding crossing frame time. The association determination module is used to perform an association determination between the cross-frame time and the contact time interval of the contact process, and obtain a determination result; The correction control module is used to correct the contact time interval of the contact process according to the motion feature parameters when the determination result is the first result, and to re-execute the association determination on the corrected contact time interval. The first result is used to characterize the weak association relationship between the cross-frame time and the contact process under the condition of time offset. The logistics record generation module is used to generate a logistics record for the target goods based on the contact process and crossing behavior when the determination result is the second result. The second result is used to characterize the deterministic correlation between the crossing frame time and the contact process. The information aggregation module is used to aggregate the logistics records according to the distribution of the functional areas to form the logistics status information of the smart warehouse.

Images