A personnel trajectory positioning and tracking method and system based on RFID and AI video recognition

Abstract

The present application relates to the technical field of personnel positioning tracking, and provides a personnel trajectory positioning tracking method and system based on RFID and AI video recognition, which comprises the following steps: acquiring video data in an industrial environment; judging whether an event of a personnel target being blocked by an automated device occurs based on the video data; if the event of the personnel target being blocked by the automated device occurs, acquiring predetermined motion information of the automated device; calculating a predicted reappearance space-time window of the personnel target based on the predetermined motion information and a state of the personnel target before being blocked; processing the video data within the predicted reappearance space-time window, searching for and re-recognizing the personnel target, so as to realize personnel trajectory positioning tracking. The present application has the advantages that it can effectively deal with visual blocking of the personnel target by the automated device, and realize rapid and accurate re-recognition and trajectory tracking after the blocking, thereby improving the continuity and accuracy of personnel positioning tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of personnel positioning and tracking, and in particular to a personnel trajectory positioning and tracking method and system based on RFID and AI video recognition. BACKGROUND

[0002] In modern industrial production and logistics warehouse environments, in order to improve management efficiency and operation safety, personnel positioning and tracking systems based on the combination of radio frequency identification (RFID) and artificial intelligence (AI) video recognition technology are usually deployed. Such systems obtain the identity and approximate location of personnel by having them wear RFID tags, and simultaneously use cameras spread throughout the area for visual capture and analysis in order to achieve accurate recording of personnel activity trajectories. However, in actual applications, especially in complex scenarios where humans and automated equipment are highly mixed, the reliability and accuracy of existing technologies face serious challenges. When multiple signal sources are intertwined, visual occlusion is frequent, and environmental characteristics are variable, the system often has difficulty maintaining continuous and error-free tracking of a specific individual. In addition to industrial and warehouse environments, personnel positioning and tracking systems based on the integration of RFID and AI video recognition are gradually being introduced into activities, exhibitions, and other densely populated public scenarios. In such cases, the system uses RFID tags to identify personnel and count their entry and exit from a region, and uses AI visual recognition technology to monitor crowd flow and detect abnormal behavior, thereby providing auxiliary support in order maintenance, emergency evacuation, and personnel scheduling. These application scenarios, although not primarily focused on industrial automation, still place higher demands on the accuracy and anti-interference capabilities of the system, especially in conditions of frequent occlusion, dense crowds, and highly dynamic environments. Existing technologies still face significant challenges in continuous tracking and identity restoration.

[0003] In existing technologies, when a personnel target is occluded by automated equipment, the system often has difficulty accurately predicting when and where it will reappear, resulting in a broken tracking chain. Even after the target reappears, due to the long occlusion time or complex environment, the re-identification algorithm based on body shape, gait, and other characteristics has greatly reduced reliability, and may even cause tracking drift, incorrectly shifting the tracking focus to other objects, thereby failing to achieve continuous and accurate trajectory positioning and tracking of a specific individual. In addition, in certain special work areas such as cold storage, environmental factors may cause the imaging quality of the camera to decline, further exacerbating the difficulty of personnel target re-identification. Therefore, how to effectively deal with the visual occlusion of personnel targets by automated equipment and achieve rapid and accurate re-identification and trajectory tracking after occlusion is a key problem that current technologies urgently need to solve.

[0004] In view of the above problems, existing technologies need to be improved. SUMMARY

[0005] The purpose of the present application is to provide a personnel trajectory positioning tracking method and system based on RFID and AI video recognition, which can effectively deal with the visual obstruction of automatic equipment to personnel targets, and realize fast and accurate re-identification and trajectory tracking after obstruction, improving the continuity and accuracy of personnel positioning tracking.

[0006] The present application provides a personnel trajectory positioning tracking method based on RFID and AI video recognition, comprising:

[0007] Obtaining video data in an industrial environment;

[0008] Based on the video data, it is judged whether an event of personnel target being obstructed by automatic equipment occurs;

[0009] If the event of personnel target being obstructed by automatic equipment occurs, the predetermined motion information of the automatic equipment is obtained;

[0010] Based on the predetermined motion information and the state of the personnel target before being obstructed, the predicted reappearance space-time window of the personnel target is calculated;

[0011] In the predicted reappearance space-time window, the video data is processed, the personnel target is searched and re-identified, so as to realize personnel trajectory positioning tracking.

[0012] Through the above scheme, the visual obstruction of automatic equipment to personnel targets can be effectively dealt with, and fast and accurate re-identification and trajectory tracking can be realized after obstruction, improving the continuity and accuracy of personnel positioning tracking.

[0013] Optionally, the present application also proposes that when the actual motion of the automatic equipment deviates from the predetermined motion information, and the personnel target moves during the obstruction, the step of calculating the predicted reappearance space-time window of the personnel target based on the predetermined motion information and the state of the personnel target before being obstructed, comprises:

[0014] Collecting response data of each radio frequency reader covering the work area in the industrial environment;

[0015] Analyzing the response data to determine whether the automatic equipment has an unexpected stillness corresponding to the deviation between the actual motion and the predetermined motion information, and determining the moving path of the personnel target during the obstruction;

[0016] If the unexpected stillness occurs, the time component of the predicted reappearance space-time window is corrected based on the duration of the unexpected stillness;

[0017] Based on the moving path of the personnel target, the spatial component of the predicted reappearance space-time window is corrected.

[0018] Through the above scheme, the correction accuracy of the predicted spatiotemporal window is further improved when the automated equipment moves abnormally or the personnel target moves during the shielding, and the robustness of the tracking is ensured.

[0019] Optionally, the application further proposes that the step of judging whether the event of the personnel target being shielded by the automated equipment occurs based on the video data comprises:

[0020] In the video data, the intersection-over-union of the personnel target bounding box in the continuous image frames is analyzed;

[0021] When the intersection-over-union of the personnel target bounding box in the continuous image frames is lower than a preset threshold, and the coincidence degree of the last visible position of the personnel target and the bounding box of the automated equipment exceeds a preset shielding coincidence ratio, it is judged that the event of the personnel target being shielded by the automated equipment occurs.

[0022] Through the above scheme, a specific method for judging the shielding event of the personnel target is provided, and the accuracy and automation degree of the shielding event identification are improved.

[0023] Optionally, the application further proposes that when the number of moving targets in the shielding area formed by the automated equipment is greater than one, the response data is analyzed to judge whether the automated equipment has an unexpected stillness corresponding to a deviation between the actual motion and the predetermined motion information, and the moving path of the personnel target during the shielding is determined, comprising:

[0024] Before the personnel target is shielded by the automated equipment, the reference response characteristics of the radio frequency tag corresponding to the personnel target are acquired;

[0025] The response data is analyzed to judge whether the automated equipment has an unexpected stillness corresponding to a deviation between the actual motion and the predetermined motion information, and the moving path characteristics are parsed from the response data;

[0026] All moving path characteristics are compared with the reference response characteristics respectively, and the target moving path characteristics matched with the reference response characteristics are identified;

[0027] The target moving path characteristics are determined as the moving path of the personnel target during the shielding.

[0028] Through the above scheme, the problem of how to accurately identify the moving path of a specific personnel target under the condition of multi-target shielding is solved, and the tracking accuracy in complex scenes is improved.

[0029] Optionally, the application further proposes that the step of comparing all moving path characteristics with the reference response characteristics respectively to identify the target moving path characteristics matched with the reference response characteristics comprises:

[0030] The real-time response characteristics of the moving path characteristics are compared with the reference response characteristics.

[0031] In the comparison process, it is judged whether the state of the real-time response feature changes from the state of matching the reference response feature to the state of not matching, to obtain a matching state judgment result;

[0032] If the matching state judgment result is yes, the response feature change process changing to the state of not matching is analyzed, and it is judged whether the response feature change process is caused by the personnel target performing a specific work action;

[0033] If it is determined that the personnel target performs a specific work action, the changed real-time response feature is set as an updated reference response feature, and comparison is performed between the updated reference response feature and the movement path feature;

[0034] All movement path features are compared with the reference response feature respectively, or all movement path features are compared with the updated reference response feature respectively, and the movement path feature recognized as matching is determined as a target movement path feature.

[0035] Through the above scheme, the case that the personnel target may perform a specific work action to cause the response feature to change during the shielding period is considered, and the accuracy of matching recognition is improved by dynamically updating the reference response feature.

[0036] Optionally, the application further provides that the step of analyzing the response feature change process changing to the state of not matching and judging whether the response feature change process is caused by the personnel target performing a specific work action includes:

[0037] Selecting, in the shielding area formed by the automated equipment, real-time response features other than the response feature changing to the state of not matching, and recording the real-time response features as response features of other movement paths;

[0038] Based on the response features of the other movement paths, a background change feature representing global disturbance is determined;

[0039] The background change feature is separated from the response feature change process as a local change feature;

[0040] Based on the local change feature, it is determined whether the response feature change process is caused by the personnel target performing a specific work action.

[0041] Through the above scheme, a method for distinguishing between a specific work action of a personnel and a global environmental disturbance is provided, so that the judgment of the response feature change is more accurate and misjudgment is avoided.

[0042] Optionally, the application further provides that the step of determining, based on the response features of the other movement paths, a background change feature representing global disturbance includes:

[0043] determine a contribution degree index of each of the other moving paths based on respective historical response features of the other moving paths;

[0044] weight changes in the response features of the other moving paths according to the contribution degree index to generate the background change feature.

[0045] Through the above scheme, the global background change feature is more accurately extracted by weighting the response features of different moving paths, and the robustness of background disturbance recognition is improved.

[0046] Optionally, the present application also proposes that the step of determining a contribution degree index of each of the other moving paths based on respective historical response features of the other moving paths comprises:

[0047] identify a change point in the time series of the respective historical response features of each of the other moving paths;

[0048] calculate a dispersion statistical value based on time series data after the change point;

[0049] If no change point is identified, calculate a dispersion statistical value based on preset latest time series data in the historical response features;

[0050] determine the contribution degree index based on the dispersion statistical value.

[0051] Through the above scheme, a specific method for determining the contribution degree index is provided, making the generation of the background change feature more refined and adaptive.

[0052] Optionally, the present application also proposes that if an event of a personnel target being blocked by an automated device occurs, the step of obtaining predetermined motion information of the automated device comprises:

[0053] According to the video frame corresponding to the event of the personnel target being blocked by the automated device, the identity number of the automated device is identified, and a query request containing the camera number, timestamp, blocking position and device number is sent to the dispatch control system of the automated device;

[0054] After the dispatch control system receives the query request, the task database is queried and the predetermined motion information of the automated device is returned.

[0055] Through the above scheme, the specific process of obtaining the predetermined motion information of the automated device is clear, ensuring the timeliness and accuracy of the data required for predicting the spatio-temporal window.

[0056] Optionally, the present application also proposes a personnel trajectory positioning and tracking system based on RFID and AI video recognition, for performing personnel trajectory positioning and tracking based on RFID and AI video recognition, comprising:

[0057] The video data acquisition module is configured to acquire video data in an industrial environment.

[0058] The occlusion event judgment module is configured to judge whether an event of a personnel target being occluded by an automated device occurs based on the video data.

[0059] The motion information acquisition module is configured to acquire predetermined motion information of the automated device if the event of the personnel target being occluded by the automated device occurs.

[0060] The reappearing window calculation module is configured to calculate a predicted reappearing spatiotemporal window of the personnel target based on the predetermined motion information and a state of the personnel target before being occluded.

[0061] The personnel positioning and tracking module is configured to process the video data, search for and re-identify the personnel target in the predicted reappearing spatiotemporal window, so as to realize personnel trajectory positioning and tracking.

[0062] The above scheme provides a system for implementing the above method, so that the method can be actually deployed and applied, and has good operability.

[0063] As can be seen from the above, the personnel trajectory positioning and tracking method and system based on RFID and AI video recognition provided by the present application introduce the concept of a predicted reappearing spatiotemporal window, and combine the predetermined motion information of the automated device, so as to accurately predict the time and spatial range of the reappearance of the personnel target after being occluded, thereby realizing rapid re-identification and continuous tracking, and having the advantages of being able to effectively cope with the visual occlusion of the personnel target by the automated device, and realizing rapid and accurate re-identification and trajectory tracking after occlusion, and improving the continuity and accuracy of personnel positioning and tracking. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The method flowchart of the personnel trajectory positioning and tracking method based on RFID and AI video recognition in one embodiment of the present application is shown in the figure.

[0065] Figure 2 The method flowchart of the personnel trajectory positioning and tracking method based on RFID and AI video recognition in one embodiment of the present application is shown in the figure.

[0066] Figure 3 The method flowchart of the personnel trajectory positioning and tracking method based on RFID and AI video recognition in one embodiment of the present application is shown in the figure.

[0067] Figure 4 The method flowchart of the personnel trajectory positioning and tracking method based on RFID and AI video recognition in one embodiment of the present application is shown in the figure.

[0068] Figure 5Figure 4 is a flow chart of a method for a personnel trajectory positioning and tracking method based on RFID and AI video recognition according to another embodiment of the present application;

[0069] Figure 6 Figure 5 is a flow chart of a method for a personnel trajectory positioning and tracking method based on RFID and AI video recognition according to another embodiment of the present application;

[0070] Figure 7 Figure 6 is a flow chart of a method for a personnel trajectory positioning and tracking method based on RFID and AI video recognition according to another embodiment of the present application;

[0071] Figure 8 Figure 7 is a flow chart of a method for a personnel trajectory positioning and tracking method based on RFID and AI video recognition according to another embodiment of the present application;

[0072] Figure 9 Figure 8 is a flow chart of a method for a personnel trajectory positioning and tracking method based on RFID and AI video recognition according to another embodiment of the present application;

[0073] Figure 10 Figure 9 is a system block diagram of a personnel trajectory positioning and tracking system based on RFID and AI video recognition according to another embodiment of the present application;

[0074] BRIEF DESCRIPTION OF DRAWINGS

[0075] 1. A personnel trajectory positioning and tracking system based on RFID and AI video recognition; 11. A video data acquisition module; 12. An occlusion event judgment module; 13. A motion information acquisition module; 14. A reappearing window calculation module; 15. A personnel positioning and tracking module. DETAILED DESCRIPTION

[0076] The technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0077] It should be noted that similar reference numerals and letters refer to like items throughout the accompanying drawings, and once an item is defined in one drawing, that definition should be applied to all like items through the drawings, unless the context clearly dictates otherwise. Also, in the description of the application, the terms "first", "second", and so on are used merely as identifiers, and are not intended to signify or imply relative importance.

[0078] Traditional existing personnel trajectory positioning and tracking systems based on RFID and AI video recognition have been widely applied in industrial production, logistics and warehousing, and other high-automation operation scenarios, and have also begun to expand to personnel-intensive public places such as assemblies, for realizing identity recognition, area management and trajectory tracking of large-scale crowds. However, in actual deployment, the tracking stability and recognition accuracy of the system in complex dynamic environments still have significant deficiencies, especially under conditions of frequent occlusion of targets, mixed use of labels or serious environmental interference.

[0079] Taking the industrial production and logistics and warehousing environment as an example, when the target personnel coexist and frequently interact with automated equipment or goods with RFID tags, the tracking system is prone to recognition interruption. Specifically, at the work entrance, if the target personnel and goods enter the identification area at the same time, they may trigger RFID identification together, and the system is difficult to establish a high-confidence initial identity and video entity correspondence relationship, resulting in uncertain initial binding. In the subsequent work process, the target personnel are often completely occluded by large equipment (such as stackers, automated guided vehicles, etc.), causing the video tracking chain to break, making it difficult to continuously obtain their motion trajectories. In addition, when the RFID tools or labels carried by the target personnel are temporarily handed over to others for use, the location signal source will be misjudged as the continuation of the original target, further interfering with the overall recognition accuracy. In some special areas, such as personnel wearing uniform, insufficient environmental light or limited camera shooting angle, the quality of visual information decreases significantly, and the system may completely lose the effective recognition ability of individuals, and cannot complete uninterrupted and accurate trajectory tracking.

[0080] For example, assume that in a large automated logistics center, a personnel management system combining RFID and video monitoring is deployed. When an employee enters the warehouse operation area, the RFID tag worn by the employee interacts with the RFID reading device at the entrance. However, if at the same time an unmanned forklift carrying a large cargo passes by, and the cargo also has an RFID tag attached to it, the reading device may collect the RFID signals of both the employee and the cargo at the same time. At this time, in the entrance camera picture, the employee's figure may be blocked by the tall cargo, causing the system to be unable to accurately bind the employee's identity information to the visual entity in the video for initial binding. During the subsequent movement of the employee, his figure may be completely blocked by the automated guided vehicle (AGV) and the tall shelf column that are driving alternately, making it difficult for the re-identification algorithm based on visual features to maintain continuous tracking. If the employee hands over a handheld data scanning terminal with an independent RFID tag to another employee during the work process, the system may mistakenly associate the movement track of the tool to the original employee, causing the generated track to deviate from the true situation. Further, when the employee enters a special operation area, such as a cold storage, all personnel wear uniform cold-weather clothing, the visual features are highly similar, and the cold storage entrance camera lens may condense thin fog due to temperature difference, causing the imaging quality to decrease, at which time the video system will have difficulty in identifying specific individuals, causing the tracking chain to be completely broken, and the complete activity path of the employee cannot be obtained.

[0081] To this end, the present application proposes a personnel trajectory positioning and tracking method based on RFID and AI video recognition, which combines Figure 1 as shown, comprising:

[0082] S1, acquiring video data in an industrial environment;

[0083] S2, based on the video data, determining whether an event of a personnel target being blocked by an automated device occurs;

[0084] S3, if the event of the personnel target being blocked by the automated device occurs, acquiring predetermined motion information of the automated device;

[0085] S4, based on the predetermined motion information and the state of the personnel target before being blocked, calculating a predicted reappearance space-time window of the personnel target;

[0086] S5, processing the video data within the predicted reappearance space-time window, searching for and re-identifying the personnel target, to realize personnel trajectory positioning and tracking.

[0087] The judgment of whether the event of the personnel target being blocked by the automated device occurs refers to real-time analysis of the video stream to identify the case that the personnel target is visually blocked by the automated device running in the industrial environment, which can be realized by using image processing, target detection or motion analysis technology, such as by analyzing the change of the target bounding box, the decline of the target confidence or the coincidence degree of the target and the device boundary, which is mainly to discover the visual interruption of the personnel target in the video tracking in time, so as to trigger the subsequent trajectory recovery mechanism. In addition, the acquisition of the predetermined motion information of the automated device refers to the acquisition of the data such as the moving trajectory, speed, direction or work plan of the automated device planned to be executed in a specific time period, which can be realized by reading from the device control system, the scheduling system or the task management database, such as by querying the preset path planning or work instruction of the device, which is mainly to provide kinematic basis for predicting the possible appearing position of the personnel target after being blocked. Further, the calculation of the predicted reappearance space-time window of the personnel target refers to predicting the time range and space area of the visual reappearance of the personnel target based on the known motion law of the automated device and the last visible state of the personnel target before being blocked, which can be realized by using motion model, probability prediction or geometric calculation method, such as by combining the device moving path with the last known speed direction for deduction, which is mainly to narrow the range of subsequent video search, improve the efficiency and accuracy of re-identification. Finally, the search and re-identification of the personnel target refers to analyzing the video data in the predicted space-time range to rediscover and confirm the personnel target that was blocked before, which can be realized by using target detection, feature matching or multi-modal fusion technology, such as using a deep learning model for target detection and combining RFID signal for identity confirmation, which is mainly to restore the interrupted tracking chain and ensure the continuity and integrity of the personnel trajectory.

[0088] The scheme of the present application takes video data in an industrial environment as the basis to provide original information for subsequent analysis and processing. When the system determines that an event of a person target being occluded by an automated device occurs based on the video data, this triggers the core logic of the scheme. Once the occlusion event is identified, the system immediately obtains the predetermined motion information of the automated device that caused the occlusion. These predetermined motion information, combined with the last visible state of the person target before being occluded, are used to calculate a predicted reappearance spatio-temporal window of the person target. This spatio-temporal window defines the time period and spatial region in which the person target is likely to reappear visually. Subsequently, within this predicted reappearance spatio-temporal window, the system performs targeted processing of the video data, concentrating resources to search for and re-identify the person target. In this way, even if the person target temporarily disappears in the video, the system can use the known motion rules of the automated device and the last state of the person to make predictions, so as to quickly capture and confirm the identity of the target when it reappears, realizing continuous positioning and tracking of the person's trajectory, effectively overcoming the limitations of traditional video tracking in occlusion scenarios.

[0089] In some preferred embodiments, the present application is implemented as follows: First, through multiple high-definition network cameras deployed in an industrial plant, real-time video stream data is continuously obtained, which is transmitted to a central processing server for analysis. When the AI visual analysis module on the server processes the video data, it detects the person target and automated devices such as automated guided vehicles or robotic arms in the picture in real time. If the bounding box of a person target overlaps with that of an automated device, and the visibility or confidence of the person target significantly decreases, the system determines that an event of the person target being occluded by the automated device occurs. Once such an occlusion event is identified, the system immediately sends a query request to the central dispatch control system of the automated device to obtain the predetermined motion information of the automated device, such as the running path, speed, and stopping points in the future period of time. At the same time, the system records the last known position, moving direction, and speed of the person target before being occluded. Based on these predetermined motion information and the state of the person target before being occluded, the system uses a motion prediction model, such as a prediction algorithm based on device path and person inertial motion, to calculate a predicted reappearance spatio-temporal window of the person target, which contains a time range and a spatial region. Subsequently, within this predicted spatio-temporal window, the system activates more refined video analysis algorithms, such as re-identification models based on feature matching or deep learning, to perform intensive search on the video data. Once the person target is successfully identified within the predicted window, the system associates it with the previous tracking chain, thereby realizing continuous positioning and tracking of the person's trajectory.

[0090] Optionally, in combination with Figure 2When the actual movement of the automation equipment deviates from the predetermined movement information and the personnel target moves during the occlusion, S4 calculates the predicted reappearance space-time window of the personnel target based on the predetermined movement information and the state of the personnel target before being occluded, including:

[0091] S41, collect response data of each radio frequency reader covering the operation area in the industrial environment;

[0092] S42, analyze the response data to determine whether the automation equipment has an unexpected stillness corresponding to the deviation between the actual movement and the predetermined movement information, and determine the moving path of the personnel target during the occlusion;

[0093] S43, if the unexpected stillness occurs, correct the time component of the predicted reappearance space-time window based on the duration of the unexpected stillness;

[0094] S44, correct the space component of the predicted reappearance space-time window based on the moving path of the personnel target.

[0095] The response data of the radio frequency reader refers to the electronic record containing the tag identification code, signal strength, time stamp and other information generated by the radio frequency reader after receiving the radio frequency tag signal, which can be collected in real time by using an ultra-high frequency (UHF) RFID system or an active RFID system, and the purpose is to obtain the real-time position and state information of the personnel target and the automated equipment in the blocked area; the unintended static refers to the fact that the automated equipment fails to continuously move according to the preset trajectory or speed during the predetermined movement due to reasons such as failure, command error or external interference, and instead stops at a certain position, which can be determined by comparing the real-time position data of the automated equipment with the predetermined movement information, and when the real-time position does not change for a long time or the change is much smaller than the expected value, and the purpose is to identify the movement abnormality of the automated equipment and provide a basis for subsequent time correction; the moving path refers to the actual trajectory of the personnel target in space during the blocking period of the automated equipment, which can be determined by continuously analyzing the response data of the radio frequency reader in combination with positioning technologies such as signal strength, time difference of arrival (TDOA) or angle of arrival (AOA), and the purpose is to obtain the real position change of the personnel target during the blocking period and provide a basis for subsequent spatial correction; the time component of the correction and prediction of the spatiotemporal window refers to adjusting the time range of the reappearance of the personnel target according to the duration of the unintended static of the automated equipment, which can be achieved by adding the duration of the unintended static to the original predicted reappearance time range, or recalculating the reappearance time point according to the time point and duration of the static occurrence, and the purpose is to ensure that the predicted reappearance time is consistent with the actual situation; the spatial component of the correction and prediction of the spatiotemporal window refers to adjusting the spatial range of the reappearance of the personnel target according to the actual moving path of the personnel target during the blocking period, which can be achieved by superimposing the moving path of the personnel target on the original predicted reappearance spatial range, or re-determining the reappearance position according to the end point of the moving path, and the purpose is to ensure that the predicted reappearance position is consistent with the actual situation.

[0096] In some preferred embodiments, when the actual motion of the automated equipment deviates from the predetermined motion information and the personnel target moves during the occlusion, the predicted reappearance spatiotemporal window of the personnel target can be implemented as follows: First, in an industrial environment such as a large automated warehouse, multiple ultra-high frequency (UHF) radio frequency identification (RFID) readers can be deployed at key locations in the work area, such as passageways, shelf area entrances, etc. When the RFID tag worn by the personnel or the RFID tag on the automated equipment enters the reading range of the reader, the reader will collect and upload the response data in real time, which can include the unique identification code of the tag, the received signal strength indication (RSSI) value, and the accurate timestamp. These data are transmitted to a central data processing server for storage and preliminary processing. Then, the data processing server continuously analyzes the collected response data. For example, for the automated equipment, the server can compare its current location information (obtained by analyzing the response data of its RFID tag) with the pre-set motion plan (such as the predetermined path and speed obtained from the scheduling control system). If it is found that the RSSI value of the automated equipment at a certain location remains stable for a long time without expected changes, or its location update frequency is much lower than expected, it can be judged that the equipment has an unexpected stop. At the same time, for the occluded personnel target, the server tracks the RSSI value changes of its RFID tag and the signal switching between different readers, and estimates and records the continuous location points of the personnel target during the occlusion in real time, so as to determine its moving path. Further, if the system judges that the automated equipment has an unexpected stop, for example, an AGV stops at a certain intersection for 5 minutes, the system will immediately obtain the duration of 5 minutes. The original predicted reappearance time window of the personnel target, for example, the reappearance is expected to be between 10 seconds and 20 seconds after the occlusion, will now be corrected based on the 5-minute unexpected stop time. Specifically, the start and end times of the original predicted time window can be delayed by 5 minutes, so that the new predicted reappearance time window becomes between 5 minutes 10 seconds and 5 minutes 20 seconds, to reflect the impact of the automated equipment stop on the reappearance time of the personnel. Finally, the system will use the moving path of the personnel target during the occlusion to correct the spatial component of the predicted reappearance spatiotemporal window. For example, if the personnel target was located at point A before being occluded, the original predicted reappearance spatial window was a region near point A. But during the occlusion, it is found through the RFID data that the personnel target actually moved to point B. Then, the system will adjust the center point or range of the predicted reappearance spatial window to the vicinity of point B instead of point A, so as to more accurately indicate the possible reappearance position of the personnel target. Through these corrections, the system can more accurately predict the reappearance of the personnel target and provide more accurate guidance for subsequent video re-identification.

[0097] Optionally, in combination withFigure 3 As shown, S2 determines whether the event of the person target being occluded by the automated device occurs based on the video data, and the step includes:

[0098] S21, in the video data, analyzing the intersection-over-union of the person target bounding box in consecutive image frames;

[0099] S22, when the intersection-over-union of the person target bounding box in consecutive image frames is lower than a preset threshold, and the overlap degree of the last visible position of the person target and the bounding box of the automated device exceeds a preset occlusion overlap ratio, it is determined that the event of the person target being occluded by the automated device occurs.

[0100] The intersection-over-union of the person target bounding box in consecutive image frames refers to the quantitative evaluation of the degree of overlap of the bounding boxes generated by the person target detected in two or more consecutive image frames in the video stream. Specifically, it refers to the ratio of the intersection area of two bounding boxes to their union area. The purpose is to reflect the position and shape change of the person target in a short period of time. When the ratio is low, it may indicate that the person target has moved quickly, deformed, or been occluded. The preset threshold refers to a pre-set value used to determine whether the intersection-over-union meets a certain condition. It can be a critical value obtained by statistical analysis, machine learning training or expert experience setting on a large amount of actual scene data. The purpose is to distinguish between normal motion of the person target and critical state that may occur occlusion or dramatic change. The overlap degree of the last visible position of the person target and the bounding box of the automated device refers to the spatial overlap between the bounding box corresponding to the last position of the person target clearly identified in the video before it disappears or is completely occluded from the video, and the bounding box corresponding to the current position of the automated device. It can be quantified by calculating the ratio of the intersection area of the two bounding boxes to the area of the person target bounding box. The purpose is to confirm whether the disappearance or change of the person target is indeed related to the presence of the automated device, thereby excluding other non-occlusion factors that cause the visual target to disappear. The preset occlusion overlap ratio refers to a pre-set ratio used to determine whether the overlap between the person target and the automated device meets the occlusion condition. It can be a percentage or ratio value determined by simulating or testing different occlusion scenarios, and combining the sensitivity requirements of the system to the occlusion event. The purpose is to ensure that only when there is a sufficient degree of visual occlusion between the person target and the automated device, the occlusion event is determined, avoiding false positives.

[0101] In some preferred embodiments, the determination of whether an event of a personnel target being occluded by an automated device occurs can be implemented as follows. First, the system performs frame-by-frame analysis on the video data in the industrial environment using a deep learning model (e.g., a real-time object detection model based on the YOLOv8 or FasterR-CNN architecture) to identify and obtain the bounding box information of the personnel target and the automated device in each frame. For consecutive video frames, such as the current frame and the previous frame, the system extracts the bounding box coordinates of the personnel target (e.g., [x1, y1, x2, y2]). Then, the intersection over union (IoU) of the bounding boxes of the personnel target in the two consecutive frames is calculated. The formula for calculating the intersection over union is: IoU = (A∩B) / (A∪B), where A and B represent the areas of the two bounding boxes, respectively.

[0102] Specifically, when the calculated intersection over union is lower than a preset threshold (e.g., 0.3, which can be determined by training and verification on a large amount of normal motion and occlusion scene data), the system further checks the spatial relationship between the personnel target and the automated device. At this time, the system obtains the last clearly detected bounding box position of the personnel target before the intersection over union is lower than the threshold, and obtains the bounding box position of the automated device in the current frame. Then, the overlap of the last visible position bounding box of the personnel target and the bounding box of the automated device is calculated. The overlap can be defined as the ratio of the intersection area of the two bounding boxes to the area of the personnel target bounding box. For example, if the personnel target bounding box is P and the automated device bounding box is M, the overlap can be calculated as (P∩M) / P. When this overlap exceeds a preset occlusion overlap ratio (e.g., 0.7, indicating that most of the area of the personnel target is occluded by the automated device), the system determines that an event of the personnel target being occluded by the automated device occurs. For example, if a worker suddenly disappears in the video, and his position before disappearing has a high overlap with the bounding box of an AGV that is moving, and the intersection over union of his bounding box before and after disappearing decreases significantly, the system can accurately identify that it is an occlusion event. This method can effectively avoid false positives caused by non-occlusion situations such as personnel moving out of the frame, sudden changes in light causing target detection failure, etc.

[0103] Optionally, in combination with Figure 4 As shown in FIG. 4B, when the number of moving targets in the occlusion area formed by the automated device is greater than one, the step S42 of analyzing the response data to determine whether the automated device has deviated from the expected stationary state corresponding to the deviation between the actual motion of the automated device and the predetermined motion information, and determining the moving path of the personnel target during the occlusion period includes:

[0104] S421, before the personnel target is occluded by the automated device, obtaining the baseline response characteristics of the radio frequency tag corresponding to the personnel target;

[0105] S422, analyze the response data to determine whether the automated equipment has an actual movement corresponding to the deviation of the predetermined movement information and an unexpected stillness, and parse the movement path characteristics from the response data;

[0106] S423, compare all movement path characteristics with the reference response characteristics respectively, and identify the target movement path characteristics matched with the reference response characteristics;

[0107] S424, determine the target movement path characteristics as the movement path of the personnel target during the shielding period.

[0108] Wherein, the reference response characteristics of the radio frequency tag refer to the unique signal pattern or fingerprint generated by the radio frequency tag worn by the personnel target in a specific radio frequency reader network when the personnel target is not shielded by the automated equipment, which can specifically include but is not limited to the combination of signal strength indication (RSSI), signal phase, signal time of arrival (TOA), signal angle of arrival (AOA), and the trend of these parameters changing with time or position, and its purpose is to establish a reliable reference for subsequent identification and comparison; the movement path characteristics refer to the radio frequency signal characteristic sequence of all possible moving targets (including personnel targets and other moving targets) analyzed from the radio frequency reader response data in the shielding area of the automated equipment, which can be a comprehensive description of the changes of signal strength, frequency, phase, etc. of multiple radio frequency tags within a period of time, reflecting the moving track of the target in space, and its purpose is to extract all potential moving track information from complex response data to provide data basis for subsequent screening; the target movement path characteristics refer to the specific movement path radio frequency signal characteristic sequence matched with the personnel target identified from all movement path characteristics after comparison with the reference response characteristics, and its purpose is to accurately identify the real moving track of the personnel target from multiple moving targets.

[0109] In some preferred embodiments, specifically, when a worker wearing a radio frequency tag enters a narrow passage frequently traversed by an automated forklift, the system continuously collects the signal strength, signal phase, and signal arrival time data of the worker's radio frequency tag before the worker is blocked by the forklift, and stores them as the worker's baseline response characteristics. When the forklift blocks the worker, and there are other mobile devices or goods with radio frequency tags in the passage at the same time, the radio frequency reader receives mixed response data from multiple radio frequency tags. At this time, the system analyzes these response data in real time, first determines whether the forklift is in an unexpected stationary state, and at the same time, through signal processing algorithms, parses and clusters these mixed response data into multiple independent moving path characteristics, each of which represents a potential moving target. Subsequently, the system compares all the parsed moving path characteristics with the worker's baseline response characteristics one by one. The comparison process can use signal pattern matching, correlation analysis, or machine learning classification methods. For example, if the signal strength change pattern and phase change trend of a moving path characteristic are highly consistent with the worker's baseline response characteristics, the moving path characteristic will be identified as a target moving path characteristic matching the worker. Finally, this identified target moving path characteristic is determined as the actual moving path of the worker during the period of being blocked by the forklift, thereby providing an accurate basis for subsequent trajectory correction.

[0110] Optionally, in combination with Figure 5 As shown, S423 compares all moving path characteristics with baseline response characteristics to identify a target moving path characteristic matching the baseline response characteristics, including:

[0111] S4231, compares the real-time response characteristics of the moving path characteristic with the baseline response characteristics;

[0112] S4232, in the comparison process, determines whether the state of the real-time response characteristics changes from matching the baseline response characteristics to not matching, to obtain a matching state determination result;

[0113] S4233, if the matching state determination result is yes, analyzes the response characteristic change process that changes to the not matching state, and determines whether the response characteristic change process is caused by a personnel target performing a specific work action;

[0114] S4234, if it is determined that the personnel target performs a specific work action, sets the real-time response characteristics after the change as updated baseline response characteristics, and compares the updated baseline response characteristics with the moving path characteristic;

[0115] S4235, compare all the moving path features with the reference response features respectively, or compare all the moving path features with the updated reference response features respectively, and determine the moving path features identified as matching as target moving path features.

[0116] wherein the real-time response feature of the moving path feature refers to the signal feature of the radio frequency identification tag collected by the radio frequency reader at a specific time point or time period during the occlusion of the personnel target by the automated equipment, which can be specifically a received signal strength indication (RSSI), phase, frequency offset, signal-to-noise ratio or combination thereof of the radio frequency signal, and aims to reflect the instant radio frequency signal performance of the personnel target in the occlusion area; the reference response feature refers to the signal feature of the radio frequency identification tag collected and recorded by the radio frequency reader in a stable state before the occlusion of the personnel target by the automated equipment, which can be specifically the average RSSI value, phase stability or specific frequency response mode of the radio frequency tag in a non-interference or low-interference environment before the personnel target enters the occlusion area, and aims to provide an initial reference standard for comparison and identification; the matching state judgment result refers to the judgment about the similarity or difference degree between the real-time response feature and the reference response feature obtained by comparing the two, which can be specifically whether the Euclidean distance, cosine similarity or correlation coefficient between the real-time response feature and the reference response feature is within an acceptable range based on a pre-set similarity threshold, and aims to determine whether the real-time response feature still conforms to the expected reference mode; the response feature change process refers to the trajectory or mode of the parameters of the real-time response feature changing over time during the transition from the matching state to the non-matching state with the reference response feature, which can be specifically the continuous rising, falling or fluctuation trend of the real-time RSSI value, phase or frequency in a short period of time, and aims to provide a data basis for analyzing the cause of the feature change; the specific work action refers to the work-related behavior of the personnel target in the occlusion area of the automated equipment that can cause significant changes in the response feature of the radio frequency identification tag, which can be specifically picking up or putting down metal tools, wearing or removing protective equipment, changing body posture (such as squatting or standing), or operating equipment with electromagnetic interference, and aims to distinguish between response feature changes caused by personnel behavior and environmental interference; the updated reference response feature refers to the changed real-time response feature as a new reference standard after judging that the personnel target performs a specific work action to cause the response feature to change, which can be specifically the signal feature of the radio frequency tag in a new stable state after the personnel target performs a specific work action, such as a new average RSSI value or phase mode after picking up a metal tool, and aims to adapt to the behavior change of the personnel target and maintain the accuracy of subsequent comparison.

[0117] In some preferred embodiments, the present solution is implemented as follows. Assume that in an automated warehouse, an employee E17 wears a radio frequency identification tag, and before entering an area shielded by a large automated stacker, the system has acquired the baseline response characteristics of his radio frequency tag, for example, his average received signal strength indicator value is -60 dBm, and the phase is stable at 180 degrees. When E17 enters the shielded area, the system continues to collect the real-time response characteristics of his radio frequency tag. At a certain time, the system detects that the real-time response characteristics of E17 have changed, for example, the received signal strength indicator value suddenly drops to -75 dBm, and the phase fluctuates greatly, which no longer matches the baseline response characteristics. The system determines that the state of the real-time response characteristics has changed from matching to mismatching. At this time, the system analyzes the response characteristic change process. For example, by analyzing the rate, duration, and change pattern of the received signal strength indicator value and the phase change, the system can identify that this is a typical signal attenuation and interference pattern caused by the person picking up a metal tool (for example, a heavy wrench). If the system determines that this change is caused by E17 performing a specific job action (picking up a metal tool), it will set the current transitioned real-time response characteristics (for example, the new average received signal strength indicator value -75 dBm and the fluctuating phase pattern) as the updated baseline response characteristics. Subsequently, the system will continue to compare with the subsequent movement path characteristics of E17 based on this updated baseline response characteristics. At the same time, for other moving targets in the shielded area that do not perform specific job actions, the system will still use their original baseline response characteristics for comparison. Finally, the system identifies all movement path characteristics that match the original baseline response characteristics or the updated baseline response characteristics, and determines them as the movement path of E17 during the shielding period. In this way, even if E17 picks up or puts down a tool during the shielding period, the system can continue to accurately track his position.

[0118] Optionally, in combination with Figure 6 As shown in S4233, the step of analyzing the response characteristic change process that changes to the mismatching state to determine whether the response characteristic change process is caused by the personnel target performing a specific job action includes:

[0119] A1, selecting real-time response characteristics in the shielding area formed by the automated equipment, except for the response characteristics that change to the mismatching state, and recording them as response characteristics of other movement paths;

[0120] A2, determining a background change characteristic representing global disturbance based on the response characteristics of other movement paths;

[0121] A3, separating the background change characteristic from the response characteristic change process as a local change characteristic;

[0122] A4, based on the local variation feature, determining whether the response feature variation process is caused by the specific operation action of the personnel target.

[0123] Wherein, the response feature variation process refers to the continuity change in value or pattern of the real-time response feature of the personnel target's RFID tag during the time when the real-time response feature changes from the state of matching the reference response feature to the state of not matching, which can be recorded and characterized by time series data, difference sequence or trend analysis, and the purpose is to capture the dynamic process leading to the occurrence of the state of not matching; the response feature of other moving paths refers to the real-time response data generated by the RFID tags of all moving objects (such as other personnel, other automated equipment or goods with RFID tags) in the shielding area formed by the automated equipment, except for the personnel target whose response feature variation process is being analyzed, which can be recorded by using indicators such as radio frequency signal strength, phase, frequency offset or signal-to-noise ratio, and the purpose is to provide a reference benchmark reflecting the overall change of the current environment; the background variation feature representing global disturbance refers to the change of environmental factors caused by non-specific personnel target operation actions, which has a universal impact on all or most of the response features of RFID tags, such as the movement of large equipment, the fluctuation of electromagnetic environment or the overall change of temperature and humidity, which can be extracted from the response features of other moving paths by using statistical average, trend line fitting or principal component analysis, and the purpose is to quantify and identify these universal environmental disturbances; the local variation feature refers to the unique variation part caused by the personnel target's own behavior (especially specific operation actions) after removing or separating the global disturbance (i.e. background variation feature) from the response feature variation process of the personnel target's RFID tag, which can be obtained by using difference calculation, residual analysis or specific filter processing, and the purpose is to highlight and isolate the signal change directly related to the personnel target's operation action; the specific operation action refers to the standardized operation that the personnel target may perform during being shielded, which will have a significant and identifiable impact on the response feature of the RFID tag worn or carried by the personnel target, such as carrying heavy objects, operating handheld devices, changing body posture or interacting with specific objects in the environment, which can be identified and judged by using pre-trained models, expert rule bases or feature pattern libraries established based on historical data, and the purpose is to distinguish the response feature difference caused by the personnel target's normal movement and operation action.

[0124] In some preferred embodiments, when the system detects that the real-time response feature of the personnel target's radio frequency tag has transitioned from a matching state to a non-matching state, in order to accurately determine whether this change is caused by the personnel target performing a specific job action, the following steps can be implemented: First, the system continuously monitors all active radio frequency tag signals in the shadow area formed by the automated equipment. When the target personnel's tag signal becomes non-matching, the system selects and records the real-time response features of all other radio frequency tags in these active signals, such as their received signal strength indication (RSSI) values, phase information, or frequency offsets, and uses these data as the response features of other moving paths. These data can be organized into a time series reflecting the dynamic changes of non-target tags in the area. Then, based on the response features of these other moving paths, the system can determine the background change feature representing global disturbance. For example, the response features of these other moving paths can be aggregated, such as calculating their moving average, median, or performing principal component analysis, to extract the environmental noise or interference patterns commonly present in the area. For example, if a large forklift in the area starts, its electromagnetic interference may cause the RSSI values of all tags to generally decrease, and this general downward trend is the background change feature. Subsequently, the system separates the background change feature determined earlier from the response feature change process of the target personnel's radio frequency tag, thereby obtaining the local change feature. This can be achieved by differencing or adaptive filtering the real-time response change of the target personnel's tag with the background change feature, to eliminate the influence of environmental factors on the target tag signal, so that the remaining signal change can reflect the personnel's own behavior. Finally, the system determines whether the response feature change process is caused by the personnel target performing a specific job action based on this separated local change feature. For example, a classification model such as a support vector machine or a neural network can be pre-trained, with the time series of the local change feature as input and the judgment result of whether it is a specific job action as output. Alternatively, a rule base can be established, for example, if the local change feature presents a specific amplitude fluctuation or a specific pattern (such as periodic attenuation of the signal when carrying heavy objects) within a short time, it is determined to be a specific job action. In this way, the system can accurately distinguish whether the change in the response feature is caused by environmental changes or the personnel's own job action.

[0125] Optionally, in combination with Figure 7 As shown, the step of A2 of determining the background change feature representing global disturbance based on the response features of other moving paths includes:

[0126] A21, based on the historical response features of each of the other moving paths, determines a contribution degree index of each of the other moving paths;

[0127] A22, according to the contribution index, the response characteristics of other moving paths are weighted to generate background change characteristics.

[0128] The historical response characteristics of each of the other moving paths refer to a pattern or trend formed by continuously recording and analyzing radio frequency signal response data of non-target moving entities (such as other automated equipment, goods with RFID tags, etc.) in a period of time, which can specifically include a series of time-varying parameter sets such as signal strength, signal arrival time, reading frequency, signal phase, etc. at different time points and different positions of the moving paths, and the purpose is to provide a basis for evaluating the potential impact of these non-target moving entities on the overall radio frequency environment.

[0129] The contribution index refers to a value quantitatively evaluating the degree of influence of each non-target moving entity on the overall change of the radio frequency environment according to the historical response characteristics of each of the other moving paths, which can be calculated by analyzing the stability, activity, interaction frequency with the radio frequency reader, or the residence time in a specific area of the historical response characteristics, and the purpose is to distinguish the weight of different non-target moving entities on the background signal change, so as to give more reasonable consideration in subsequent processing.

[0130] The background change characteristics refer to comprehensive characteristics that can represent global or universal disturbances of the radio frequency environment generated by weighting the response characteristic changes of other moving paths in the shielding area except for the personnel target, which can be obtained by multiplying the response characteristic changes of each of the other moving paths by the corresponding contribution index and then accumulating or averaging, and the purpose is to accurately reflect the universal interference in the environment, so as to effectively separate the local change caused by the specific work action of the personnel target from the response characteristics of the personnel target.

[0131] Optionally, as shown in Figure 8 The step of A21 of determining the contribution index of each of the other moving paths based on the historical response characteristics of each of the other moving paths includes:

[0132] A211, in the time sequence of the historical response characteristics of each of the other moving paths, a change point is identified;

[0133] A212, based on the time sequence data after the change point, a dispersion statistical value is calculated;

[0134] A213, if the change point is not identified, the dispersion statistical value is calculated based on the preset latest time sequence data in the historical response characteristics;

[0135] A214, the contribution index is determined based on the dispersion statistical value.

[0136] The change point refers to a point at which the data characteristics at a certain time point in the time series data change obviously, which can be achieved by using a statistical method or a machine learning method. The dispersion statistical value refers to a statistical quantity for measuring the fluctuation degree or dispersion degree of a group of data, which can be achieved by using a statistical index such as variance, standard deviation, mean absolute deviation, interquartile range, etc. The preset latest time series data refers to a latest data set selected from the historical response characteristics in order to reflect the current data characteristics in the case where no change point is identified, which can be determined by using a fixed time window, a fixed number of data points, or a method based on a moving average, etc. The contribution index refers to a numerical value for quantifying the influence degree of each other mobile path on the global disturbance or background change, which can be achieved by using the reciprocal of the dispersion statistical value, the normalized processing result of the dispersion statistical value, or a weight value obtained by mapping based on the dispersion statistical value through a preset function.

[0137] In some preferred embodiments, the specific process of determining the contribution index of each other mobile path can be as follows. First, for each other mobile path, such as a stationary shelf or a continuously moving AGV, the system will continuously collect its radio frequency response characteristics and form a time series. In order to identify whether there is an obvious change in the time series, a cumulative sum control chart (CUSUM) algorithm can be used. The algorithm calculates the cumulative deviation of the response characteristic value from the target mean value, and when the cumulative deviation exceeds a preset threshold, a change point is identified. For example, if the response characteristics of a certain shelf have been stable around a certain RSSI value in the past period of time, but suddenly deviate continuously due to environmental interference or the deployment of a new device nearby, the CUSUM algorithm can timely detect this change and mark the change point.

[0138] Secondly, once the change point is identified, the system will calculate the dispersion statistical value based on the time series data after the change point. For example, the standard deviation of these data can be calculated. The standard deviation can intuitively reflect the degree of data fluctuation, and the larger the standard deviation, the more unstable the response characteristics of the mobile path. If no change point is identified, the system will calculate the standard deviation based on the preset latest time series data in the historical response characteristics, such as the latest response characteristic data of a preset time length.

[0139] Finally, based on the calculated dispersion statistics, a contribution index can be determined. For example, the standard deviation can be normalized, and then its inverse taken as the contribution index. Specifically, if the standard deviation of the response characteristics of a certain moving path is σ, then its contribution index can be calculated as 1 / (1+σ). In this way, the smaller the standard deviation, the higher the contribution index, indicating that the response characteristics of this moving path are more reliable in contributing to the background changes; on the contrary, the larger the standard deviation, the lower the contribution index. In this way, the system can dynamically evaluate the stability of each non-target moving path, and adjust its weight in the background change feature generation accordingly, so as to more accurately separate out the response feature changes caused by the specific work actions of the personnel target.

[0140] Optionally, in combination with Figure 9 As shown, if the event of the personnel target being blocked by the automated device occurs, the step of obtaining the predetermined motion information of the automated device includes:

[0141] S31, according to the video frame corresponding to the event of the personnel target being blocked by the automated device, the identity number of the automated device is identified, and a query request containing the camera number, timestamp, blocking position and device number is sent to the dispatch control system of the automated device;

[0142] S32, after the dispatch control system receives the query request, the task database is queried and the predetermined motion information of the automated device is returned.

[0143] Wherein, the identity number of the automated device refers to the code for uniquely identifying the automated device, which can include serial number, asset number or internal identification code, etc., and its purpose is to accurately identify the automated device that has the blocking event; the dispatch control system refers to the central control platform responsible for managing and coordinating the operation of the automated device, which can be a centralized software system or a distributed control network, and its purpose is to receive query requests and provide operation data of the device; the task database refers to a collection of data of task plans, running states and historical trajectories of the automated device, which can be a relational database, a non-relational database or a distributed file system, and its purpose is to provide the dispatch control system with the required data source for query; the predetermined motion information refers to the motion trajectory, speed, stopping point, task target, etc. data planned to be executed by the automated device within a certain time period, which can include path coordinate sequence, timestamp, speed curve or task instruction set, and its purpose is to describe the expected behavior of the automated device during the blocking period.

[0144] In some preferred embodiments, when the system detects that the silhouette of an employee is completely blocked by a moving AGV, it triggers the process of obtaining the scheduled motion information of the AGV. Specifically, the system immediately analyzes the video frames that led to the blocking event, uses a pre-trained image recognition model to identify the unique identity number of the AGV, such as "AGV-007", from its appearance features (e.g., vehicle color, specific logo, or laser radar shape on the roof). At the same time, the system records the corresponding camera number (e.g., "Cam-A3"), the precise timestamp (e.g., "2023-10-26 14:35:12.345"), and the blocking position information (e.g., the center point of the blocking area in pixel coordinates or relative scene coordinates) where the personnel target's last visible position coincides with the AGV's bounding box. Subsequently, the system encapsulates this information into a query request and sends it to the central dispatch control system of the AGV through a network interface (e.g., TCP / IP protocol). This dispatch control system can be deployed on independent server clusters and maintains a distributed task database containing all AGV task plans and real-time states. After receiving the query request, the dispatch control system quickly retrieves the task records related to the AGV within a specific time period in the task database based on the AGV identity number, timestamp, and blocking position provided in the request. For example, it may query that "AGV-007" is scheduled to perform a transportation task from "A area shelf 12" to "B area loading and unloading port" during "14:35:00 to 14:36:00", and it is accompanied by detailed path point sequences, preset speed curves, and estimated arrival times. The dispatch control system packages these detailed scheduled motion information (e.g., a series of path coordinate points with timestamps and corresponding speed values) and returns it to the personnel trajectory positioning and tracking system through the network. In this way, the system can obtain the precise motion trajectory of the AGV during the blocking period, thereby providing accurate data basis for subsequent personnel trajectory prediction and tracking.

[0145] A personnel trajectory positioning and tracking system based on RFID and AI video recognition for performing personnel trajectory positioning and tracking based on RFID and AI video recognition, the personnel trajectory positioning and tracking system 1 comprising:

[0146] a video data acquisition module 11 for acquiring video data in an industrial environment;

[0147] a blocking event judgment module 12 for judging whether an event of a personnel target being blocked by an automated device occurs based on the video data;

[0148] a motion information acquisition module 13 for acquiring scheduled motion information of the automated device if the event of the personnel target being blocked by the automated device occurs;

[0149] A re-occurrence window calculation module 14 is configured to calculate a predicted re-occurrence spatio-temporal window of the personnel target based on the predetermined motion information and the state of the personnel target before being occluded;

[0150] A personnel positioning and tracking module 15 is configured to search and re-identify the personnel target in the predicted re-occurrence spatio-temporal window by processing the video data, so as to realize personnel trajectory positioning and tracking.

[0151] The video data acquisition module refers to a hardware or software unit responsible for collecting visual information from the industrial environment, which can be implemented by using a high-definition network camera, an industrial video acquisition card or a video stream server, and the purpose thereof is to provide raw data input for subsequent video analysis;

[0152] The occlusion event judgment module refers to a logic processing unit for analyzing the video data to identify the occlusion of the personnel target by the automated device, which can be implemented by using a software module based on image processing and computer vision algorithms, for example, by analyzing the intersection-over-union and overlap of the target bounding box, and the purpose thereof is to timely discover the key event that may interrupt the personnel tracking;

[0153] The motion information acquisition module refers to an interface or processing unit for acquiring the predetermined motion trajectory and state data of the related automated device when the occlusion event occurs, which can be implemented by using a software interface or protocol converter for data communication with the dispatching control system of the automated device, and the purpose thereof is to provide key motion parameters for predicting the re-occurrence position of the personnel;

[0154] The re-occurrence window calculation module refers to a calculation unit for predicting the spatio-temporal range in which the personnel target may re-occur based on the known motion information and the state of the personnel before being occluded, which can be implemented by using a software module integrating a prediction algorithm and a state estimation model, and the purpose thereof is to narrow the range of subsequent personnel search and improve the efficiency of re-identification;

[0155] The personnel positioning and tracking module refers to a comprehensive processing unit for searching and re-identifying the personnel target by deeply processing the video data in the predicted spatio-temporal range, which can be implemented by using a software system combining a deep learning model and a multi-target tracking algorithm, and the purpose thereof is to quickly and accurately restore the continuous tracking of the personnel after the personnel is occluded.

[0156] In some preferred embodiments, the application is implemented as follows: the video data acquisition module can be composed of multiple high-resolution IP cameras deployed in the industrial site, which are connected to a central video server through Ethernet, which is responsible for receiving, storing and preprocessing video streams, such as frame synchronization and preliminary encoding. The occlusion event judgment module can be a software application running on a high-performance computing unit (e.g., an industrial PC equipped with a GPU), which integrates a deep learning-based target detection model (such as YOLO or FasterR-CNN) for real-time identification of personnel targets and automated equipment in the video, and determines the occurrence of occlusion events by calculating the intersection-over-union and overlap of the personnel target bounding box and the automated equipment bounding box. The motion information acquisition module can be a software interface that communicates with the factory's manufacturing execution system (MES) or automated equipment scheduling system through ModbusTCP or OPCUA protocols, which sends a query request to the scheduling system when the occlusion event is triggered by the judgment module to obtain the current task plan, preset path and speed information of the specific automated equipment (such as AGV, robotic arm). The recurrence window calculation module can be an algorithm service running on the same high-performance computing unit, which receives device motion data from the motion information acquisition module and personnel target last visible position and speed information from the occlusion event judgment module, uses state estimation algorithms such as Kalman filtering or particle filtering, combined with the device motion model, to predict the spatial area and time point where the personnel target may appear in the future period of time, forming a three-dimensional spatio-temporal prediction window. The personnel positioning and tracking module can be a software component that integrates multi-target tracking algorithms (such as DeepSORT or ByteTrack) and re-identification models (such as feature matching based on ReID network), which processes the specific area of the video stream within the spatio-temporal range provided by the recurrence window calculation module, searches for visual entities similar to the features of the occluded personnel target, and performs identity re-confirmation, thereby restoring and continuing the trajectory tracking of the personnel. Through the configuration of these specific components, the system can achieve continuous and accurate trajectory tracking of personnel in complex industrial environments, even in the face of frequent occlusions.

[0157] By the technical solution, the personnel trajectory positioning and tracking system based on RFID and AI video recognition is provided, the system modularizes and organically integrates video data acquisition, shielding event judgment, motion information acquisition, reproduction window calculation and personnel positioning and tracking and other function modules, and effectively solves the problem that the traditional method is difficult to coordinate the work of each function module in actual application. The systematic design makes the data flow and processing flow more smooth, and each module can efficiently transmit information and respond, thereby improving the efficiency and accuracy of personnel trajectory positioning and tracking, especially in the complex scene that personnel targets are frequently shielded by automated equipment in industrial environment, the system can continuously track the personnel, and the reliability of tracking is improved.

[0158] The above merely describes the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for locating and tracking personnel trajectories based on RFID and AI video recognition, characterized in that, include: Acquire video data in industrial environments; Based on video data, determine whether there are events where people are obscured by automated equipment; If an event occurs in which a person is obscured by automated equipment, then the predetermined motion information of the automated equipment is obtained; Based on the predetermined motion information and the state of the person target before being occluded, the predicted spatiotemporal window for the person target is calculated. Within the predicted spatiotemporal window, the video data is processed to search for and re-identify personnel targets in order to achieve personnel trajectory positioning and tracking. When the actual movement of the automated equipment deviates from the predetermined movement information, and the person target moves during the occlusion period, the step of calculating the predicted spatiotemporal window of the person target based on the predetermined movement information and the state of the person target before being occluded includes: Collect response data from various RFID readers covering the work area in an industrial environment; Analyze the response data to determine whether the automated equipment has experienced an unexpected stationary state corresponding to a deviation between the actual movement and the predetermined movement information, and determine the movement path of the personnel target during the occlusion period; If the unexpected stillness occurs, the time component of the predicted recurrence time-space window is corrected based on the duration of the unexpected stillness. Based on the movement path of personnel targets, the spatial components of the predicted spatiotemporal window are corrected. When the number of moving targets within the shielding area formed by the automated equipment is greater than one, the step of analyzing the response data, determining whether the automated equipment has experienced an unexpected stationary state corresponding to a deviation between the actual movement and the predetermined movement information, and determining the movement path of the personnel target during the shielding period includes: Before a person is obscured by automated equipment, acquire the baseline response characteristics of the RFID tag corresponding to the person. Analyze the response data to determine whether the automated equipment has experienced an unexpected stationary state corresponding to a deviation between the actual movement and the predetermined movement information, and parse the movement path characteristics from the response data; All of the movement path features are compared with the baseline response features to identify the target movement path features that match the baseline response features; The target movement path features are determined as the movement path of the personnel target during the occlusion period; The step of comparing all the movement path features with the baseline response features to identify the target movement path features that match the baseline response features includes: The real-time response features of the mobile path features are compared with the baseline response features; During the comparison process, it is determined whether the state of the real-time response feature changes from a state that matches the benchmark response feature to a state that does not match, and a matching state judgment result is obtained. If the matching status judgment result is yes, then analyze the response characteristic change process of the transition to the non-matching status, and determine whether the response characteristic change process is caused by the personnel target performing a specific operation action; If it is determined that the cause is the execution of a specific operation by a personnel target, the transformed real-time response feature is set as the updated baseline response feature, and a comparison is made between the updated baseline response feature and the movement path feature. All of the aforementioned movement path features are compared with the baseline response features, or all of the aforementioned movement path features are compared with the updated baseline response features. The movement path features that are identified as matching are determined as the target movement path features.

2. The method for personnel trajectory positioning and tracking based on RFID and AI video recognition according to claim 1, characterized in that, The step of determining whether an event has occurred where a person target is obscured by automated equipment based on video data includes: In video data, the intersection-union ratio (IUU) of the bounding boxes of people in consecutive image frames is used for analysis; When the intersection-over-union ratio of the bounding boxes of a person target in consecutive image frames is lower than a preset threshold, and the overlap between the last visible position of the person target and the bounding box of the automated device exceeds a preset occlusion overlap ratio, it is determined that an event has occurred in which the person target is occluded by the automated device.

3. The method for personnel trajectory positioning and tracking based on RFID and AI video recognition according to claim 1, characterized in that, The analysis of the response characteristic change process in the transition to a mismatched state, and the step of determining whether the response characteristic change process is caused by the execution of a specific task action by a person, includes: Within the shielded area formed by the automated equipment, the real-time response characteristics, excluding those that transition to a mismatched state, are selected and recorded as the response characteristics of other movement paths. Based on the response characteristics of other movement paths, the background change characteristics characterizing global disturbances are determined; The background change features are separated from the response feature change process and used as local change features; Based on the aforementioned local change characteristics, it is determined whether the change in response characteristics is caused by the execution of specific operational actions by the personnel target.

4. The method for personnel trajectory positioning and tracking based on RFID and AI video recognition according to claim 3, characterized in that, The step of determining the background change characteristics characterizing the global disturbance based on the response characteristics of other movement paths includes: Based on the historical response characteristics of each of the other mobile paths, the contribution index of each other mobile path is determined. Based on the contribution index, the changes in response characteristics of other movement paths are weighted to generate background change characteristics.

5. A method for locating and tracking personnel based on RFID and AI video recognition according to claim 4, characterized in that, The step of determining the contribution index of each other mobile path based on their respective historical response characteristics includes: Identify change points in the time series of historical response characteristics for each of the other movement paths; Calculate the dispersion statistics based on the time series data after the change point; If the change point is not identified, the dispersion statistic is calculated based on the latest time series data preset in the historical response features. The contribution index is determined based on the aforementioned dispersion statistics.

6. The method for locating and tracking personnel based on RFID and AI video recognition according to claim 1, characterized in that, The step of obtaining predetermined motion information of the automated equipment if an event occurs in which a person target is obscured by the automated equipment includes: Based on the video frame corresponding to the event where a person is obscured by automated equipment, the identification number of the automated equipment is identified, and a query request containing the camera number, timestamp, obstruction location and equipment number is sent to the scheduling and control system of the automated equipment. After receiving the query request, the scheduling and control system queries the task database and returns the scheduled motion information of the automated equipment.

7. A personnel trajectory positioning and tracking system based on RFID and AI video recognition, used to perform personnel trajectory positioning and tracking based on RFID and AI video recognition, characterized in that, include: The video data acquisition module is used to acquire video data in industrial environments. The occlusion event detection module is used to determine, based on video data, whether an event has occurred in which a person or target is occluded by automated equipment. The motion information acquisition module is used to acquire the predetermined motion information of the automated equipment if an event occurs in which a person target is obscured by the automated equipment. The reproduction window calculation module is used to calculate the predicted reproduction spatiotemporal window of the person target based on the predetermined motion information and the state of the person target before being occluded. The personnel location and tracking module is used to process the video data within the predicted reproduction time window, search for and re-identify personnel targets, so as to realize personnel trajectory location and tracking. When the actual movement of the automated equipment deviates from the predetermined movement information, and the person target moves during the occlusion period, the step of calculating the predicted spatiotemporal window for the person target based on the predetermined movement information and the state of the person target before being occluded includes: Collect response data from various RFID readers covering the work area in an industrial environment; Analyze the response data to determine whether the automated equipment has experienced an unexpected stationary state corresponding to a deviation between the actual movement and the predetermined movement information, and determine the movement path of the personnel target during the occlusion period; If the unexpected stillness occurs, the time component of the predicted recurrence time-space window is corrected based on the duration of the unexpected stillness. Based on the movement path of personnel targets, the spatial components of the predicted spatiotemporal window are corrected. When the number of moving targets within the shielding area formed by the automated equipment is greater than one, the analysis of the response data to determine whether the automated equipment has experienced an unexpected stationary state corresponding to a deviation between the actual movement and the predetermined movement information, and the determination of the movement path of the personnel target during the shielding period, includes: Before a person is obscured by automated equipment, acquire the baseline response characteristics of the RFID tag corresponding to the person. Analyze the response data to determine whether the automated equipment has experienced an unexpected stationary state corresponding to a deviation between the actual movement and the predetermined movement information, and parse the movement path characteristics from the response data; All of the movement path features are compared with the baseline response features to identify the target movement path features that match the baseline response features; The target movement path features are determined as the movement path of the personnel target during the occlusion period; The step of comparing all the movement path features with the baseline response features to identify the target movement path features that match the baseline response features includes: The real-time response features of the mobile path features are compared with the baseline response features; During the comparison process, it is determined whether the state of the real-time response feature changes from a state that matches the benchmark response feature to a state that does not match, and a matching state judgment result is obtained. If the matching status judgment result is yes, then analyze the response characteristic change process of the transition to the non-matching status, and determine whether the response characteristic change process is caused by the personnel target performing a specific operation action; If it is determined that the cause is the execution of a specific operation by a personnel target, the transformed real-time response feature is set as the updated baseline response feature, and a comparison is made between the updated baseline response feature and the movement path feature. All of the aforementioned movement path features are compared with the baseline response features, or all of the aforementioned movement path features are compared with the updated baseline response features. The movement path features that are identified as matching are determined as the target movement path features.

Images