Transform-based multi-target motion trail prediction device and real-time processing system thereof

Abstract

The invention discloses a Transform-based multi-target motion trail prediction device and a real-time processing system thereof, and relates to the technical field of trail prediction, and the device comprises a light transmission region construction module which is used for generating a vertical extension length based on an original gray level image, and generating a light transmission region image based on the vertical extension length; the geometric shear offset calculation module is used for generating geometric shear offset based on the light-passing area image; the physical centroid calculation module is used for calculating a local observation centroid and calculating a physical centroid coordinate based on the local observation centroid and the geometric shear offset; and the pure track generation module is used for determining a correlation result based on the physical centroid coordinates and generating a pure physical track based on the correlation result. The operation efficiency and safety in narrow roadway warehousing operation are improved.

Description

Technical Field

[0001] This invention relates to the field of trajectory prediction technology, and in particular to a multi-target motion trajectory prediction device based on Transformer and its real-time processing system. Background Technology

[0002] With the continuous development of intelligent warehousing and logistics automation technologies, mobile robots are increasingly being widely used in high-density warehousing operations, especially in narrow aisle storage areas. Mobile robots need to complete tasks such as handling, inspection, and assisted picking within limited space. In these scenarios, to improve storage capacity per unit area, warehouse racks are typically arranged back-to-back to form continuous aisle rows, with multiple rows of vertical columns on either side, creating a regular and dense spatial structure. Simultaneously, in actual operations, mobile robots often need to collaborate with human pickers within the same aisle. Robots typically use onboard vision sensors to perceive the working environment in real time, acquiring information about the picker's position changes and predicting future movement trends based on historical location data, thereby enabling path planning and safe obstacle avoidance. In the aforementioned warehousing environment, the dense arrangement of rack columns, the narrowness of the aisle space, and the continuous movement of the robot itself present complex spatial occlusion relationships and perspective changes in the visual perception process, placing high demands on the stable acquisition of target positions.

[0003] In existing technologies, when predicting the movement trajectory of pickers in aisles based on visual images, it is usually necessary to calculate the position of the target in the visible pixel area of ​​the image and use this position directly as the target's motion input data. When pickers are in an aisle environment formed by back-to-back shelf pillars, their body outline is often periodically obscured by pillars at different distances in front and behind. As the robot moves and the viewing angle changes, the visible area undergoes asymmetrical changes in the horizontal direction, causing the target position calculated based on visible pixels to produce obvious lateral jumps in a short period of time.

[0004] Existing methods often mistake such positional changes for actual lateral movement of the target, thus introducing a large amount of non-realistic motion information into the trajectory prediction process. This leads to unstable prediction results and may even cause the robot to frequently take unnecessary avoidance or braking actions, making it difficult to meet the actual requirements for the stability and reliability of motion trajectory prediction in narrow aisle storage scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies that fail to meet the actual requirements for stability and reliability of motion trajectory prediction in narrow aisle warehousing scenarios, and to propose a Transformer-based multi-target motion trajectory prediction device and its real-time processing system.

[0006] To address the problems existing in the prior art, the present invention adopts the following technical solution: A Transformer-based multi-target motion trajectory prediction device includes: The light-transmitting region construction module is used to generate a vertical extension length based on the original grayscale image, and to generate a light-transmitting region image based on the vertical extension length. The geometric shearing bias calculation module is used to generate geometric shearing bias amounts based on the image of the light-transmitting region; The physical centroid calculation module is used to calculate the local observed centroid and to calculate the physical centroid coordinates based on the local observed centroid and the geometric shear offset. The pure trajectory generation module is used to determine the correlation results based on the physical centroid coordinates and generate a pure physical trajectory based on the correlation results. The motion trend prediction module is used to predict the future motion trend of the target picker based on the pure physical trajectory.

[0007] Preferably, generating the vertical extension length based on the original grayscale image includes: Acquire the raw grayscale image captured by the robot's vision sensor; Edge extraction is performed on the original grayscale image to obtain a set of vertical edges; The reference height is obtained by locating the vertical midline of the original grayscale image using coordinates. A threshold is preset for the reference height to obtain the width threshold; Get the pixel width of each vertical edge in the vertical edge set at the reference height; The vertical extension length is obtained by statistically analyzing the vertically connected pixel heights of each vertical edge in the vertical edge set.

[0008] Preferably, generating an image of the illuminated area based on the vertical extension length includes: Pixels with a width greater than the width threshold and a vertical extension length that extends through the original grayscale image are identified as near-end pillar pixels. Generate a near-pillar mask based on near-pillar pixels; Pixels with a width less than or equal to the width threshold and whose vertical extension length does not penetrate the original grayscale image are identified as far-end pillar pixels. Generate a far-end pillar mask based on the far-end pillar pixels; The near-end pillar mask and the far-end pillar mask are subjected to NOT and AND union logic operations to obtain the light-transmitting area image.

[0009] Preferably, generating a geometric clipping bias based on the image of the illuminated area includes: Connectivity analysis of the light-transmitting area image yields the slit region; Obtain the pixel geometric center corresponding to the slit region; The near-end pillar is obtained by matching the edge attributes of the horizontal coordinate range of the slit region based on the near-end pillar mask. The midpoint coordinates of the horizontal spacing between the near-end columns are calculated to obtain the theoretical gap center. The difference between the pixel geometric center and the theoretical gap center is calculated to obtain the geometric shear offset of the slit region.

[0010] Preferably, calculating the local observation centroid includes: Moving target detection is performed on the original grayscale image to obtain the original pixel region; The intersection operation is performed between the set of pixel coordinates corresponding to the original pixel region and the range of pixel coordinates corresponding to each slit region to obtain the overlapping pixel set. Count the number of pixels in the overlapping pixel set; The slit region corresponding to the largest number of pixels is determined to be occupied by the target in the slit. Determine the local observation centroid of the pixel region corresponding to the target picker within the target-occupied slit.

[0011] Preferably, calculating the physical centroid coordinates based on the locally observed centroid and the geometric shear offset includes: Get the pixel width of the slit occupied by the target; Perform perspective projection calculations on the pixel region corresponding to the target picker to obtain the average pixel width; Multiply the calculated ratio of the pixel width occupied by the target in the slit to the average pixel width by the geometric shearing offset to obtain the preliminary proportional displacement. The initial proportional displacement is multiplied by the preset gain coefficient to obtain the final offset compensation amount; The physical centroid coordinates are obtained by subtracting the final offset compensation from the local observed centroid.

[0012] Preferably, determining the correlation results based on the physical centroid coordinates includes: Calculate the displacement vector between the coordinates of two physical centroids at adjacent time points; The horizontal vector perpendicular to the direction of the shelf uprights is defined as the noise-sensitive vector. Calculate the projection component of the displacement vector in the direction of the noise-sensitive vector; Calculate the rate of change of geometric shear bias over time; The correlation between the changing trend of the projection components and the rate of change of the geometric shear bias over time is determined, and the correlation results are obtained.

[0013] Preferably, generating a pure physical trajectory based on the correlation results includes: The projected components and the preset column width threshold are numerically compared to obtain the comparison results; Based on the correlation and comparison results, the components of the displacement vector on the noise-sensitive vector are set to zero to obtain the pure physical displacement vector. The pure physical trajectory is obtained by performing vector superposition on the pure physical displacement vectors corresponding to each time step.

[0014] To address the aforementioned issues, embodiments of the present invention also provide a real-time processing system for multi-target motion trajectory prediction based on Transformer, which includes a multi-target motion trajectory prediction device based on Transformer.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention analyzes the vertical extension length of the original grayscale image acquired by the robot to distinguish between near-end and far-end columns, and constructs an image of the illuminated area based on this. This allows for accurate depiction of the effective gap space between shelf columns in complex back-to-back aisle environments. By clearly defining the column occlusion area and the illuminated area, the vision system no longer relies solely on the instantaneously visible pixel area of ​​the target, but introduces a constraint description of the environmental structure. This effectively reduces target position fluctuations caused by alternating column occlusion and changes in viewing angle, providing a more stable and reliable spatial basis for subsequent target position calculations.

[0016] 2. This invention calculates the geometric shear offset of the slit region based on the image of the light-transmitting area, and combines it with the local observed centroid of the target within the slit to compensate and correct the target position, obtaining the physical centroid coordinates corresponding to the real spatial position. It can quantitatively correct the asymmetric changes in the visible area caused by the obstruction of the pillar, avoid misjudging the visual obstruction changes as the target's own lateral movement, fundamentally suppress the unreal jumps in the horizontal direction of the target position, and make the obtained target position sequence more consistent with the actual movement law.

[0017] 3. Based on obtaining the physical centroid coordinates, this invention analyzes the changes in displacement along the noise-sensitive direction at adjacent time points and combines this with the time-varying relationship of the geometric shear offset to identify and suppress non-real displacements, generating a pure physical trajectory. This pure physical trajectory is then input into a prediction model to predict future motion trends. By effectively filtering noise displacements before trajectory prediction, abnormal displacements caused by environmental occlusion are avoided from being introduced into the prediction process, resulting in smoother and more stable prediction results. This significantly reduces frequent braking or avoidance behaviors caused by the robot misjudging target motion, thereby improving operational efficiency and safety in narrow aisle warehousing operations. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a functional block diagram of a Transformer-based multi-target motion trajectory prediction device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Example: This example provides multi-target motion trajectory prediction based on Transformer. See [link to example]. Figure 1 Specifically, including: The light-transmitting region construction module is used to generate a vertical extension length based on the original grayscale image, and to generate a light-transmitting region image based on the vertical extension length. In an embodiment of the present invention, generating a vertical extension length based on the original grayscale image includes: Acquire the raw grayscale image captured by the robot's vision sensor; Specifically, during the robot's movement within the warehouse operation area, its onboard vision sensor is activated, continuously receiving optical signals from the surrounding environment. The received optical signals are converted into analog electrical signals, which are then converted into digital image data containing red, green, and blue components via an analog-to-digital converter. Subsequently, according to the luminance component extraction rules, the red, green, and blue components corresponding to each pixel in the digital image data are converted into a single luminance value, and each pixel retains only this luminance value. The image obtained after the above processing, containing the luminance information of each pixel, is the original grayscale image captured by the robot's vision sensor.

[0021] Edge extraction is performed on the original grayscale image to obtain a set of vertical edges; Specifically, the vertical Sobel operator is selected as the edge detection operator. The vertical Sobel operator is convolved with the original grayscale image pixel by pixel to calculate the vertical gradient value corresponding to each pixel position in the original grayscale image. Then, the distribution of vertical gradient values ​​of all pixels in the original grayscale image is statistically analyzed. The gradient values ​​in the first 30% to 40% of the distribution are selected as the gradient threshold. The vertical gradient value of each pixel position is compared with the gradient threshold. Pixels with gradient values ​​greater than the gradient threshold are retained. Then, these retained pixels are grouped and aggregated according to the characteristics of spatial continuous distribution. Each group of consecutive pixels obtained after aggregation constitutes a vertical edge set.

[0022] The reference height is obtained by locating the vertical midline of the original grayscale image using coordinates. Specifically, the total number of pixels in the original grayscale image in the horizontal direction is counted, and half of this total number is calculated. The column pixel position corresponding to this value is used as the vertical centerline of the original grayscale image. Then, the pixel range in the vertical direction of the original grayscale image is determined. From the starting pixel row to the ending pixel row of this range, all pixels on the vertical centerline are traversed, and the brightness value corresponding to each traversed pixel is counted. Pixels whose brightness values ​​are within the middle 50% interval of all counted brightness values ​​are selected, and the average position of these pixels in the vertical direction is determined. The pixel row coordinates corresponding to this average position are the reference height corresponding to the vertical centerline of the original grayscale image.

[0023] Specifically, the raw grayscale image acquired by the robot's vision sensor is image data directly generated after the robot's onboard vision sensing device performs optical imaging on the warehouse environment. This data uses pixel values ​​of different brightness gradients to represent the optical reflection characteristics of objects such as shelves and pickers in the environment. It has not undergone any subsequent processing and can objectively reflect the brightness distribution of each object in the environment. The vertical edge set is the set of pixels obtained after performing edge detection on the raw grayscale image. These pixels correspond to areas in the image where the brightness and darkness of objects change drastically in the vertical direction. They correspond to the vertical contour boundaries of objects in the actual warehouse environment and can be used to distinguish the spatial arrangement of different objects. The reference height of the vertical centerline of the raw grayscale image is the actual spatial height corresponding to the pixel position of the straight line running vertically through the center of the raw grayscale image. This height can be used as a benchmark scale for subsequent positioning and measurement of the relative positions of objects in the warehouse environment, and is used to establish the correspondence between image pixels and actual spatial dimensions.

[0024] A threshold is preset for the reference height to obtain the width threshold; Specifically, the actual horizontal width of the near-end shelf upright in the storage operation area is measured at the reference height. Combined with the shooting distance and field of view parameters of the robot vision sensor, the number of pixels corresponding to the actual horizontal width is calculated. 1.2 times the number of pixels is taken as the upper limit threshold, and 0.8 times the number of pixels is taken as the lower limit threshold. The upper limit threshold and the lower limit threshold together form the width threshold.

[0025] Get the pixel width of each vertical edge in the vertical edge set at the reference height; Specifically, locate the pixel row corresponding to the reference height, traverse each vertical edge in the vertical edge set, determine the minimum and maximum column coordinates of the consecutive pixels corresponding to the vertical edge within the pixel row, subtract the minimum column coordinate from the maximum column coordinate value, and add 1 to get the pixel width of the vertical edge at the reference height.

[0026] The vertical extension length is obtained by statistically analyzing the vertically connected pixel heights of each vertical edge in the vertical edge set.

[0027] Specifically, for each vertical edge in the set of vertical edges, select any pixel within that edge as the starting point, and traverse upwards along the vertical direction to find adjacent pixels until a pixel that does not belong to that vertical edge is encountered, recording the number of pixels traversed upwards; then, from the starting point, traverse downwards along the vertical direction to find adjacent pixels until a pixel that does not belong to that vertical edge is encountered, recording the number of pixels traversed downwards; add the number of pixels traversed upwards and the number of pixels traversed downwards, and add 1 to obtain the result, which is the vertical extension length of that vertical edge.

[0028] Specifically, the width threshold is a pre-set pixel width standard based on the actual spatial scale corresponding to the aforementioned reference height. This standard matches the actual width characteristics of different types of objects in the warehouse environment at this height position and can be used as a basis for distinguishing the vertical edges of different objects. The pixel width of each vertical edge in the vertical edge set at the reference height refers to the number of pixels occupied by each vertical edge in the horizontal direction when it is at the image position corresponding to the reference height. This number corresponds to the horizontal spatial width of the object at this height position in the actual environment and can reflect the actual size of the object at this position. The vertical extension length obtained by statistically analyzing the vertically connected pixel height of each vertical edge in the vertical edge set refers to the total number of pixels that are continuously connected along the vertical direction of the image for each vertical edge. This total corresponds to the extension range of the object in the vertical direction in the actual environment and can reflect the longitudinal spatial distribution of the object.

[0029] In an embodiment of the present invention, generating a light-transmitting region image based on the vertical extension length includes: Pixels with a width greater than the width threshold and a vertical extension length that extends through the original grayscale image are identified as near-end pillar pixels. Specifically, the process first retrieves the pixel width and vertical extension length of each vertical edge in the acquired set of vertical edges, along with a pre-defined width threshold and the total number of rows of pixels in the original grayscale image in the vertical direction. For each vertical edge, its pixel width is compared with the width threshold, filtering out edges whose pixel width is greater than the threshold. Then, for these filtered vertical edges, the vertical extension length is checked to see if it equals the total number of rows of pixels in the original grayscale image. If the vertical extension length of a certain vertical edge equals the total number of rows, it is determined that the vertical extension length of that vertical edge extends through the original grayscale image. The vertical edges obtained after these two filtering processes are the qualified vertical edges, and all pixels corresponding to these vertical edges are marked as near-end pillar pixels.

[0030] Generate a near-pillar mask based on near-pillar pixels; Specifically, a blank binary image with the exact same size as the original grayscale image is initialized first, and all pixels in this image are initially assigned the first identifier. Then, each pixel in the original grayscale image is traversed, and it is determined whether the pixel belongs to a labeled near-pillar pixel. If it does, the pixel identifier at the corresponding position in the blank binary image is modified to the second identifier; otherwise, the pixel identifier at that position is kept as the first identifier. After traversing and modifying the identifiers of all pixels, the resulting binary image is the near-pillar mask generated based on the near-pillar pixels.

[0031] Specifically, a vertical edge whose pixel width is greater than the width threshold and whose vertical extension length runs through the original grayscale image refers to the vertical edge portion in the set of vertical edges where the pixel width at the reference height exceeds the preset width threshold range, and the continuous pixel extension range along the vertical direction covers the entire vertical pixel area of ​​the original grayscale image. This portion corresponds to the area of ​​the shelf uprights that are close to the robot's vision sensor in the warehouse environment. Near-end upright pixels refer to the pixels corresponding to the vertical edges selected by the above judgment conditions. These pixels correspond to the pixel mapping of the physical contour of the shelf uprights close to the robot in the actual warehouse scene, which can accurately reflect the spatial position and morphological characteristics of the near-end shelf uprights. The near-end upright mask generated based on the near-end upright pixels refers to the binary image formed by marking the positions of the near-end upright pixels in the original grayscale image as a specific identifier and the positions of the remaining pixels as another identifier. This mask can clearly distinguish the area corresponding to the near-end shelf uprights in the original grayscale image from other environmental areas, providing a clear regional division basis for subsequent identification of far-end uprights or target objects.

[0032] Pixels with a width less than or equal to the width threshold and whose vertical extension length does not penetrate the original grayscale image are identified as far-end pillar pixels. Specifically, the process first retrieves the pixel width and vertical extension length of each vertical edge in the vertical edge set, along with the determined width threshold and the total number of pixel rows in the original grayscale image in the vertical direction. For each vertical edge, its pixel width is compared with the width threshold, filtering out vertical edges whose pixel width is less than or equal to the width threshold. Then, for these filtered vertical edges, the vertical extension length is checked to see if it is equal to the total number of vertical pixel rows in the original grayscale image. If the vertical extension length of a certain vertical edge is not equal to the total number of rows, it is determined that the vertical extension length of that vertical edge does not penetrate the original grayscale image. The vertical edges obtained after the above two filtering processes are the vertical edges that meet the conditions, and all pixels corresponding to these vertical edges are marked as far-end pillar pixels.

[0033] Generate a far-end pillar mask based on the far-end pillar pixels; Specifically, a blank binary image with the exact same size as the original grayscale image is initialized first, and all pixels in this image are initially assigned the first identifier. Then, each pixel in the original grayscale image is traversed, and it is determined whether the pixel belongs to a labeled far-end pillar pixel. If it does, the pixel identifier at the corresponding position in the blank binary image is modified to the second identifier; otherwise, the pixel identifier at that position is kept as the first identifier. After traversing and modifying the identifiers of all pixels, the resulting binary image is the far-end pillar mask generated based on the far-end pillar pixels.

[0034] The near-end pillar mask and the far-end pillar mask are subjected to NOT and AND union logic operations to obtain the light-transmitting area image.

[0035] Specifically, first, a NOT operation is performed on the near-end pillar mask, modifying the pixel positions marked with the second identifier to the first identifier, and simultaneously modifying the pixel positions marked with the first identifier to the second identifier. The same NOT operation is performed on the far-end pillar mask. Then, a union logic operation is performed on the two NOT-processed masks, iterating through the corresponding pixel identifiers in both masks. If the pixel identifier at the corresponding position in either mask is the second identifier, then the corresponding position in the new blank image is marked with the second identifier; otherwise, it is marked with the first identifier. After completing the traversal and marking of all positions, the resulting image is the image of the illuminated area.

[0036] Specifically, a vertical edge whose pixel width is less than or equal to a width threshold and whose vertical extension length does not penetrate the original grayscale image refers to a vertical edge set whose pixel width at a reference height is within or below a pre-set width threshold, and whose continuous pixel extension along the vertical direction does not cover the entire vertical pixel area of ​​the original grayscale image. This part corresponds to the area of ​​the shelf uprights that are far from the robot's vision sensor in the warehouse environment. The far-end upright pixels refer to the pixels corresponding to the vertical edges selected by the above criteria. These pixels correspond to the pixel mapping of the physical contour of the shelf uprights that are far from the robot in the actual warehouse scenario, and can reflect the spatial position and morphological characteristics of the far-end shelf uprights. The far-end upright mask generated based on the far-end upright pixels refers to the original... In a grayscale image, the position of the pixel belonging to the far-end column is marked with a specific identifier, and the positions of the remaining pixels are marked with another identifier, forming a binary image. This mask can clearly distinguish the area corresponding to the far-end shelf column in the original grayscale image from other environmental areas. The NOT and union logic operation performed on the near-end column mask and the far-end column mask refers to first converting the marked column areas in the near-end column mask and the far-end column mask into non-column areas, and then converting the non-column areas into column areas, respectively. Then, the parts marked as non-column areas in the two masks after NOT processing are merged. The light-transmitting area image obtained by this operation refers to the corresponding area in the image that can reflect the unobstructed space between the near-end and far-end shelf columns in the actual warehousing scene, where light can pass normally. This image can clearly define the effective spatial range of the gap between the two layers of columns.

[0037] The geometric shearing bias calculation module is used to generate geometric shearing bias amounts based on the image of the light-transmitting region; In an embodiment of the present invention, generating a geometric clipping bias based on an image of the illuminated region includes: Connectivity analysis of the light-transmitting area image yields the slit region; Specifically, first, an image of the illuminated area is acquired. Pixels in this image are labeled with a first identifier and a second identifier, with the second identifier corresponding to the illuminated area. Starting from the top-left pixel of the illuminated area image, each pixel is traversed row by row and column by column. If a pixel is labeled with the second identifier and is not marked, then starting from that pixel, the process expands in four adjacent directions: up, down, left, and right, continuously collecting adjacent pixels labeled with the second identifier until expansion is impossible. This group of consecutive pixels is then marked as an independent group. The above operation is repeated until all pixels have been traversed. Each independent group is a slit region.

[0038] Obtain the pixel geometric center corresponding to the slit region; Specifically, for each slit region, all pixels within that region are traversed, and the horizontal column coordinates and vertical row coordinates of each pixel are accumulated. The total number of pixels in that region is counted, and the sum of the accumulated horizontal column coordinates is divided by the total number of pixels to obtain the average horizontal column coordinates of that region. The sum of the accumulated vertical row coordinates is divided by the total number of pixels to obtain the average vertical row coordinates of that region. The image pixel position determined by the average horizontal column coordinates and the average vertical row coordinates is the geometric center of the pixel corresponding to that slit region.

[0039] The near-end pillar is obtained by matching the edge attributes of the horizontal coordinate range of the slit region based on the near-end pillar mask. Specifically, first, determine the starting and ending column coordinates of each slit region in the horizontal direction to obtain the horizontal coordinate range of that slit region. Then, retrieve the near-end column mask; the region marked with the second identifier in this mask corresponds to the near-end column pixels. Calculate the starting and ending column coordinates of each second-identified region in the near-end column mask to obtain the horizontal coordinate range of each near-end column. Compare the horizontal coordinate range of the slit region with the horizontal coordinate range of each near-end column to find the horizontal coordinate range of the near-end column directly adjacent to the horizontal coordinate range of the slit region. The actual shelf column mapped by the near-end column mask region corresponding to this range is the matched near-end column.

[0040] Specifically, the slit region refers to a group of pixels in the image that is spatially continuous and identified by the second identifier. This group corresponds to the mapping area in the image of the narrow gap space formed between the near and far shelf uprights in an actual warehouse scenario, which is the effective gap portion through which light can pass normally. The pixel geometric center corresponding to the slit region refers to the image pixel position corresponding to the average of the horizontal column coordinates and vertical row coordinates of all pixels contained in each slit region, and the average of these two average values. The center position of the slit space is mapped to the point in the image; the near-end pillars obtained by edge attribute matching of the horizontal coordinate range of the slit area according to the near-end pillar mask refer to the horizontal coordinate range corresponding to the pixel marked as the second identifier in the near-end pillar mask. The start and end coordinates of the slit area in the horizontal direction are compared with the horizontal coordinate range of each area in the near-end pillar mask to find the near-end pillar mask area adjacent to the horizontal coordinate range of the slit area. The actual shelf pillar in the warehouse scene mapped by the pixel corresponding to this area is the near-end shelf pillar adjacent to the slit.

[0041] The midpoint coordinates of the horizontal spacing between the near-end columns are calculated to obtain the theoretical gap center. Specifically, first, select two adjacent proximal columns obtained from the matching process. Determine the horizontal end column coordinates and start column coordinates of each proximal column. Take the horizontal end column coordinates of the first proximal column as its adjacent side boundary coordinates, and take the horizontal start column coordinates of the second proximal column as its adjacent side boundary coordinates. Subtract the values ​​of these two adjacent side boundary coordinates; the result is the horizontal distance between the two proximal columns. Then, calculate the average value of these two adjacent side boundary coordinates. The horizontal column coordinate position corresponding to this average value is the theoretical center of the gap between the two proximal columns.

[0042] The difference between the pixel geometric center and the theoretical gap center is calculated to obtain the geometric shear offset of the slit region.

[0043] Specifically, first retrieve the horizontal column coordinates of the pixel geometric center corresponding to the slit region, then retrieve the horizontal column coordinates of the theoretical gap center corresponding to the slit region. Subtract the horizontal column coordinates of the theoretical gap center from the horizontal column coordinates of the pixel geometric center. The resulting value is the geometric shear offset of the slit region. The sign of this value reflects the horizontal offset direction of the slit region relative to the theoretical gap center, and the magnitude of the value reflects the degree of offset.

[0044] Specifically, the theoretical gap center refers to the horizontal spacing calculated by selecting two adjacent proximal columns, statistically determining the starting and ending coordinates of each proximal column in the horizontal direction, and calculating the distance between the starting and ending coordinates of the adjacent sides of the two proximal columns. The midpoint value of the coordinate range corresponding to this horizontal spacing is then taken. The image position corresponding to this midpoint value corresponds to the center spatial position of the gap between proximal columns in the ideal arrangement in the actual warehousing scenario. The geometric shear offset of the slit region, obtained by calculating the difference between the pixel geometric center and the theoretical gap center, refers to the result obtained by subtracting the horizontal coordinate value of the theoretical gap center from the horizontal coordinate value of the pixel geometric center corresponding to the slit region. This result corresponds to the spatial offset of the slit region relative to the ideal center of the proximal column gap in the actual scenario, and can quantify the deviation of the slit position caused by dynamic factors.

[0045] The physical centroid calculation module is used to calculate the local observed centroid and to calculate the physical centroid coordinates based on the local observed centroid and the geometric shear offset. In an embodiment of the present invention, calculating the local observed centroid includes: Moving target detection is performed on the original grayscale image to obtain the original pixel region; Specifically, at least two frames of raw grayscale images are continuously acquired. Two adjacent frames are selected, and the pixels at corresponding positions in these two frames are traversed row by row and column by column, calculating the difference in brightness value for each pixel. The distribution of brightness value differences across all pixels in these two frames is statistically analyzed, and the differences falling within the top 5% are selected as the difference threshold. The brightness value difference for each pixel position is compared with this threshold; if the difference is greater than the threshold, the pixel is marked as a suspected moving pixel. All suspected moving pixels are traversed, and spatially adjacent suspected moving pixels are grouped into a continuous pixel group, which constitutes the original pixel region.

[0046] The intersection operation is performed between the set of pixel coordinates corresponding to the original pixel region and the range of pixel coordinates corresponding to each slit region to obtain the overlapping pixel set. Specifically, the coordinate information of all pixels contained in the original pixel region is retrieved to form an original pixel coordinate set. Simultaneously, the horizontal start column coordinates, horizontal end column coordinates, vertical start row coordinates, and vertical end row coordinates of each slit region in the image are retrieved to determine the pixel coordinate range corresponding to each slit region. Pixel coordinates are extracted one by one from the original pixel coordinate set, and it is checked whether the coordinates fall within the coordinate range of a certain slit region. If they do, the pixel coordinates are added to the temporary set of the corresponding slit region. The temporary set corresponding to each slit region is the overlapping pixel set corresponding to that region.

[0047] Specifically, the original pixel region refers to the continuous pixel group determined by identifying the pixel parts in the original grayscale image whose brightness changes over time. This group corresponds to the pixel mapping area of ​​the target picker in the actual warehouse scenario, which can reflect the spatial position and outline range of the picker. The overlapping pixel set obtained by intersecting the pixel coordinate set corresponding to the original pixel region and the pixel coordinate range corresponding to each slit region refers to the set of pixels that belong to both the original pixel region and a certain slit region. This set corresponds to the pixel mapping of the body part of the target picker in the actual scenario, which is located between the near and far columns.

[0048] Count the number of pixels in the overlapping pixel set; Specifically, for each overlapping pixel set corresponding to a slit region, starting from the coordinates of the first pixel in the set, all pixel coordinates within the set are traversed one by one, with a count performed for each coordinate traversed, until all coordinates in the set have been traversed. The count result for each overlapping pixel set is recorded, and this result represents the number of pixels in that overlapping pixel set.

[0049] The slit region corresponding to the largest number of pixels is determined to be occupied by the target in the slit. Specifically, the number of pixels in the overlapping pixel sets corresponding to all slit regions is collected, these numbers are compared, and the overlapping pixel set corresponding to the largest number is found. The slit region corresponding to this overlapping pixel set is determined, and this slit region is identified as the target occupying the slit.

[0050] Determine the local observation centroid of the pixel region corresponding to the target picker within the target-occupied slit.

[0051] Specifically, the overlapping pixel set corresponding to the target occupying the slit is retrieved. This set represents the pixel portion of the target picker's pixel region within the slit occupied by the target. All pixel coordinates within this set are traversed, and the horizontal column coordinates and vertical row coordinates of all pixels are accumulated, resulting in a total number of pixels in the set. The sum of the horizontal column coordinates is divided by the total number of pixels to obtain the average horizontal coordinate; the sum of the vertical row coordinates is divided by the total number of pixels to obtain the average vertical coordinate. The image pixel position determined by the combined average horizontal and vertical coordinates is the local observation centroid of the pixel region corresponding to the target picker within the slit occupied by the target.

[0052] Specifically, counting the number of pixels in the overlapping pixel set refers to the value obtained by counting the number of pixels contained in each overlapping pixel set. This value reflects the size of the area occupied by the target picker in the corresponding slit area. Determining the slit area corresponding to the largest number of pixels as the target occupied slit means selecting the slit area corresponding to the overlapping pixel set with the largest count value. This area corresponds to the gap space between the near and far columns where the target picker is mainly located in the actual warehouse scenario. Determining the local observation centroid of the pixel area corresponding to the target picker in the target occupied slit means averaging the horizontal and vertical coordinates of the pixel area belonging to the target picker in the target occupied slit. This position corresponds to the center position of the target picker in the gap as observed from the robot's vision perspective in the actual scenario.

[0053] In embodiments of the present invention, calculating the physical centroid coordinates based on the locally observed centroid and the geometric shear offset includes: Get the pixel width of the slit occupied by the target; Specifically, retrieve the starting and ending column coordinates of the slit occupied by the target in the horizontal direction of the image. Subtract the value of the starting column coordinate from the value of the ending column coordinate, and then add one to the result. The calculated value is the pixel width of the slit occupied by the target, which corresponds to the pixel coverage area of ​​the slit occupied by the target in the horizontal direction of the image.

[0054] Perform perspective projection calculations on the pixel region corresponding to the target picker to obtain the average pixel width; Specifically, first determine the focal length parameters of the vision sensor and the current shooting distance. Then, traverse each pixel row in the vertical direction of the pixel area corresponding to the target picker, determine the horizontal start and end column coordinates of the target pixel area within each pixel row, and calculate the pixel width corresponding to each pixel row. Combining the focal length and shooting distance, convert the pixel width corresponding to each pixel row into an equivalent pixel width under the same depth plane. Count the equivalent pixel widths of all pixel rows, and divide the sum of these equivalent pixel widths by the total number of pixel rows to obtain the average pixel width.

[0055] Multiply the calculated ratio of the pixel width occupied by the target in the slit to the average pixel width by the geometric shearing offset to obtain the preliminary proportional displacement. Specifically, the pixel width of the slit occupied by the target is divided by the average pixel width to obtain the ratio. The geometric shear offset corresponding to the target occupying the slit is then retrieved, and the ratio is multiplied by the geometric shear offset. The resulting value is the preliminary proportional displacement, which reflects the scaling effect of the ratio of the slit width to the target width on the observation offset.

[0056] Specifically, the pixel width of the slit occupied by the target refers to the difference between the starting and ending column coordinates of the slit in the horizontal direction of the image, plus one. This result corresponds to the pixel mapping of the horizontal space width between the near and far columns where the target picker is located in the actual warehouse scenario. The average pixel width calculated by perspective projection of the pixel area corresponding to the target picker refers to the average width of the pixel area of ​​the target picker at different vertical positions converted into the average width under the same depth plane by combining the perspective imaging parameters of the vision sensor. This average value corresponds to the average pixel mapping of the actual horizontal width of the target picker's body in the actual scene. The preliminary proportional displacement obtained by multiplying the ratio of the pixel width of the slit occupied by the target to the average pixel width by the geometric shearing offset refers to the value obtained by scaling the geometric shearing offset after combining the proportional relationship between the slit width and the target width. This value quantifies the influence of the width ratio of the slit to the target on the observation offset.

[0057] The initial proportional displacement is multiplied by the preset gain coefficient to obtain the final offset compensation amount; Specifically, the gain coefficient is determined based on the actual spacing between near-end columns in the warehouse scenario and the pixel resolution of the visual sensor. The product of the initial proportional displacement value and the gain coefficient value yields the final offset compensation amount, which matches the intensity of the influence of dynamic factors on the observed offset in the actual scenario.

[0058] The physical centroid coordinates are obtained by subtracting the final offset compensation from the local observed centroid.

[0059] Specifically, the horizontal column coordinates of the local observation centroid corresponding to the target picker are retrieved, and the final offset compensation value is subtracted from the horizontal column coordinates. The result is the horizontal column coordinates of the physical centroid. Combined with the vertical row coordinates of the local observation centroid, the physical centroid coordinates are formed. These coordinates correspond to the mapping position of the target picker's actual spatial center in the image.

[0060] Specifically, the final offset compensation amount refers to the degree of offset that needs to be corrected for the local observation centroid after matching the influence intensity of the dynamic moiré effect in the actual scene through the gain coefficient; the physical centroid coordinates obtained by subtracting the final offset compensation amount from the local observation centroid refer to the pixel mapping of the actual spatial center position of the target picker in the image after eliminating the observation offset caused by the dynamic moiré effect.

[0061] The pure trajectory generation module is used to determine the correlation results based on the physical centroid coordinates and generate a pure physical trajectory based on the correlation results. In embodiments of the present invention, determining the correlation result based on the physical centroid coordinates includes: Calculate the displacement vector between the coordinates of two physical centroids at adjacent time points; Specifically, two consecutive adjacent moments are selected and denoted as time t1 and time t2. The physical centroid coordinates corresponding to time t1 are retrieved as coordinate A, which includes a horizontal column coordinate component A1 and a vertical row coordinate component A2. The physical centroid coordinates corresponding to time t2 are retrieved as coordinate B, which includes a horizontal column coordinate component B1 and a vertical row coordinate component B2. The horizontal displacement component is obtained by subtracting A1 from B1, and the vertical displacement component is obtained by subtracting A2 from B2. The vector formed by the horizontal and vertical displacement components is the displacement vector between the two physical centroid coordinates at adjacent moments.

[0062] The horizontal vector perpendicular to the direction of the shelf uprights is defined as the noise-sensitive vector. Specifically, observe the actual arrangement direction of the shelf uprights in the storage operation area to determine that the shelf uprights are distributed vertically. Take the horizontal direction perpendicular to this vertical direction as the target direction, select a unit vector along this target direction, and define this unit vector as the noise sensitive vector. The direction of this vector corresponds to the direction in which the observed noise caused by the dynamic moiré effect is mainly distributed.

[0063] Specifically, the displacement vector refers to the vector containing horizontal and vertical changes obtained by selecting the physical centroid coordinates corresponding to two consecutive moments, subtracting the corresponding components of the previous moment from the horizontal and vertical components of the physical centroid coordinates at the later moment, and this vector corresponds to the spatial displacement state of the target picker at adjacent moments in the actual warehousing scenario; the horizontal vector perpendicular to the direction of the shelf uprights is defined as the noise sensitive vector, which means that the shelf uprights are arranged in a vertical direction, and the vector along the horizontal direction and perpendicular to the vertical direction is taken as the noise sensitive vector. This vector corresponds to the direction in which the observed noise caused by the dynamic Moiré effect is mainly distributed.

[0064] Calculate the projection component of the displacement vector in the direction of the noise-sensitive vector; Specifically, the horizontal and vertical components of the displacement vector and the noise-sensitive vector are retrieved. The horizontal components of the displacement vector and the noise-sensitive vector are multiplied together, and then the vertical components of the displacement vector and the noise-sensitive vector are multiplied together. The two products are added together to obtain the dot product. The magnitude of the noise-sensitive vector is calculated, and the dot product is divided by this magnitude. The resulting value is the projection component of the displacement vector along the direction of the noise-sensitive vector.

[0065] Calculate the rate of change of geometric shear bias over time; Specifically, the geometric shear offset of the target occupying the slit at time t1 is denoted as offset C1, and the geometric shear offset of the target occupying the slit at time t2 is denoted as offset C2. The time interval between time t2 and time t1 is calculated, and the value of C2 minus C1 is divided by the time interval. The result is the rate of change of the geometric shear offset with time.

[0066] The correlation between the changing trend of the projection components and the rate of change of the geometric shear bias over time is determined, and the correlation results are obtained.

[0067] Specifically, at least five consecutive time intervals are selected, and the changes in the projected components and geometric shear bias over time are calculated for each group. These changes in the projected components are treated as one set of data, and the rate of change of the geometric shear bias over time is treated as another set. The correlation coefficient between the two sets of data is calculated. Based on the actual impact data of the dynamic Mohr effect in the warehousing scenario, a correlation threshold is determined. The calculated correlation coefficient is compared with this threshold. If the correlation coefficient is greater than the threshold, the two are considered strongly correlated; otherwise, they are considered weakly correlated. This determination result is the correlation result.

[0068] Specifically, calculating the projection component of the displacement vector in the noise-sensitive vector direction refers to decomposing the displacement vector into components in the noise-sensitive vector direction through vector dot product operations. These values ​​reflect the degree of displacement change in the noise-sensitive direction. Calculating the rate of change of the geometric shear bias over time involves selecting the geometric shear bias at two consecutive moments, subtracting the bias at the previous moment from the bias at the later moment, and dividing the result by the time interval between the two moments. The resulting value corresponds to the rate of change of the geometric shear bias over time. The correlation result obtained by judging the correlation between the trend of the projection component and the rate of change of the geometric shear bias over time refers to determining the degree of correlation between the variation law of the projection component and the rate of change of the geometric shear bias through statistical analysis. This result can be used to determine whether the projection component is caused by noise due to the dynamic Moiré effect.

[0069] In an embodiment of the present invention, generating a pure physical trajectory based on the correlation results includes: The projected components and the preset column width threshold are numerically compared to obtain the comparison results; Specifically, first, the pixel width of the near-end shelf uprights in the warehouse scene is measured in the image. The pixel width values ​​of multiple near-end uprights are counted and their average value is calculated. This average value is determined as the upright width threshold. Then, the projection component of the displacement vector in the direction of the noise-sensitive vector is retrieved. The value of this projection component is compared with the value of the upright width threshold. If the projection component value is less than or equal to the upright width threshold, the comparison result is recorded as within the threshold range; if the projection component value is greater than the upright width threshold, the comparison result is recorded as outside the threshold range.

[0070] Specifically, first determine the reference height of the original grayscale image. Select at least five evenly distributed near-end shelf uprights within the storage operation area. For each near-end shelf upright, determine the horizontal start column coordinates and horizontal end column coordinates of the vertical edge corresponding to the reference height in the pixel row. Subtract the horizontal start column coordinates from the horizontal end column coordinates and add one to obtain the pixel width of each near-end shelf upright at the reference height. Count the pixel width values ​​of all selected near-end shelf uprights. Sum these values ​​and divide them by the total number of selected near-end shelf uprights. The average value obtained is the preset upright width threshold.

[0071] Based on the correlation and comparison results, the components of the displacement vector on the noise-sensitive vector are set to zero to obtain the pure physical displacement vector. Specifically, the previously obtained correlation results are retrieved and compared with the above comparison results. If the correlation results show a strong correlation between the projection component and the rate of change of the geometric shear bias over time, and the comparison results are within the threshold range, then the component of the displacement vector in the direction of the noise-sensitive vector is selected, and the value of this component is set to zero. If the above conditions are not met, the original value of this component is retained. The components of the displacement vector in other directions are kept unchanged. The vector obtained after the above processing is the pure physical displacement vector.

[0072] The pure physical trajectory is obtained by performing vector superposition on the pure physical displacement vectors corresponding to each time step.

[0073] Specifically, the physical centroid coordinates at the initial moment are retrieved as the starting point coordinates of the trajectory. Then, the pure physical displacement vectors at each subsequent moment are selected sequentially. The position coordinates of the previous moment are added to the pure physical displacement vector at the current moment to obtain the physical centroid coordinates at the current moment. Connecting the physical centroid coordinates of all moments in chronological order forms a continuous path, which is the pure physical trajectory.

[0074] Specifically, the comparison result obtained by numerically comparing the projected component with the preset column width threshold refers to the result obtained by comparing the projection component of the displacement vector in the direction of the noise-sensitive vector with the pixel width value of the shelf column in the actual warehousing scenario in the image, which corresponds to the preset column width threshold. This result can reflect whether the projection component is within the noise fluctuation range caused by the dynamic moiré effect. The pure physical displacement vector obtained by setting the component of the displacement vector in the direction of the noise-sensitive vector to zero according to the correlation result comparison result refers to the vector obtained by setting the component of the displacement vector in the direction of the noise-sensitive vector to zero and retaining the components in other directions if the correlation result shows that the projection component is strongly correlated with the rate of change of the geometric shear offset and the comparison result shows that the projection component does not exceed the column width threshold. This vector corresponds to the actual spatial displacement state of the target picker. The pure physical trajectory obtained by vector superposition of the pure physical displacement vectors corresponding to each time moment refers to the path formed by a series of continuous coordinate points obtained by successively accumulating the pure physical displacement vectors of consecutive time moments, starting from the physical centroid coordinates of the initial time moment. This path corresponds to the actual movement trajectory of the target picker in the actual warehousing scenario.

[0075] The motion trend prediction module is used to predict the future motion trend of the target picker based on the pure physical trajectory.

[0076] Specifically, the time window length is first determined based on the acquisition frequency of the pure physical trajectory. A continuous sequence of historical trajectory coordinate points is selected as the input sequence. The horizontal and vertical coordinates of each trajectory point, as well as the time interval velocity change between adjacent trajectory points, are extracted as basic features. Simultaneously, the location distribution information of the shelf uprights in the warehouse scenario is incorporated as environmental constraint features. These features are combined in chronological order to form a multi-dimensional temporal feature matrix. After standardization, this feature matrix is ​​mapped into a high-dimensional feature vector through an embedding layer. Then, a position encoding based on sine and cosine functions is added to preserve temporal information, resulting in the input features for the Transformer model. The input features are then fed into the Transformer encoder, which consists of a multi-head self-attention layer and a feedforward neural network layer. The self-attention mechanism captures different time intervals in the historical trajectory. The dependencies and spatial relationships between time steps are used to generate a global trajectory feature representation. A Transformer decoder is constructed, using the global trajectory features output by the encoder as contextual information. A cross-attention mechanism is used to guide the decoder to focus on key information related to future motion in the global features. An autoregressive approach is used to gradually generate trajectory feature vectors for multiple future time steps. Based on the speed constraints of pickers and the spatial constraints of shelf gaps in a warehouse scenario, a loss function is designed and gradient descent optimization algorithm is used to iteratively adjust the model parameters until the model converges. Finally, the future trajectory feature vector output by the decoder is mapped to a specific coordinate value sequence through the output layer. This sequence is the future motion trajectory of the target picker, and its directional change trend, speed change law, and spatial position evolution represent the future motion trend of the target picker.

[0077] This invention also provides a real-time processing system for multi-target motion trajectory prediction based on Transformer, which includes a multi-target motion trajectory prediction device based on Transformer.

[0078] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-target motion trajectory prediction device based on Transformer, characterized in that, Includes the following steps: The light-transmitting region construction module is used to generate a vertical extension length based on the original grayscale image, and to generate a light-transmitting region image based on the vertical extension length. The geometric shearing bias calculation module is used to generate geometric shearing bias amounts based on the image of the light-transmitting region; The physical centroid calculation module is used to calculate the local observed centroid and to calculate the physical centroid coordinates based on the local observed centroid and the geometric shear offset. The pure trajectory generation module is used to determine the correlation results based on the physical centroid coordinates and generate a pure physical trajectory based on the correlation results. The motion trend prediction module is used to predict the future motion trend of the target picker based on the pure physical trajectory.

2. The multi-target motion trajectory prediction device based on Transformer according to claim 1, characterized in that, Generate the vertical extension length based on the original grayscale image, including: Acquire the raw grayscale image captured by the robot's vision sensor; Edge extraction is performed on the original grayscale image to obtain a set of vertical edges; The reference height is obtained by locating the vertical midline of the original grayscale image using coordinates. A threshold is preset for the reference height to obtain the width threshold; Get the pixel width of each vertical edge in the vertical edge set at the reference height; The vertical extension length is obtained by statistically analyzing the vertically connected pixel heights of each vertical edge in the vertical edge set.

3. The Transformer-based multi-target motion trajectory prediction device according to claim 2, characterized in that, The image of the illuminated region is generated based on the vertical extension length, including: Pixels with a width greater than the width threshold and a vertical extension length that extends through the original grayscale image are identified as near-end pillar pixels. Generate a near-pillar mask based on near-pillar pixels; Pixels with a width less than or equal to the width threshold and whose vertical extension length does not penetrate the original grayscale image are identified as far-end pillar pixels. Generate a far-end pillar mask based on the far-end pillar pixels; The near-end pillar mask and the far-end pillar mask are subjected to NOT and AND union logic operations to obtain the light-transmitting area image.

4. The Transformer-based multi-target motion trajectory prediction device according to claim 3, characterized in that, Generate geometric clipping bias based on the image of the illuminated region, including: Connectivity analysis of the light-transmitting area image yields the slit region; Obtain the pixel geometric center corresponding to the slit region; The near-end pillar is obtained by matching the edge attributes of the horizontal coordinate range of the slit region based on the near-end pillar mask. The midpoint coordinates of the horizontal spacing between the near-end columns are calculated to obtain the theoretical gap center. The difference between the pixel geometric center and the theoretical gap center is calculated to obtain the geometric shear offset of the slit region.

5. The Transformer-based multi-target motion trajectory prediction device according to claim 4, characterized in that, Calculating the local observed centroid includes: Moving target detection is performed on the original grayscale image to obtain the original pixel region; The intersection operation is performed between the set of pixel coordinates corresponding to the original pixel region and the range of pixel coordinates corresponding to each slit region to obtain the overlapping pixel set. Count the number of pixels in the overlapping pixel set; The slit region corresponding to the largest number of pixels is determined to be occupied by the target in the slit. Determine the local observation centroid of the pixel region corresponding to the target picker within the target-occupied slit.

6. The Transformer-based multi-target motion trajectory prediction device according to claim 5, characterized in that, The physical centroid coordinates are calculated based on the locally observed centroid and the geometric shear offset, including: Get the pixel width of the slit occupied by the target; Perform perspective projection calculations on the pixel region corresponding to the target picker to obtain the average pixel width; Multiply the calculated ratio of the pixel width occupied by the target in the slit to the average pixel width by the geometric shearing offset to obtain the preliminary proportional displacement. The initial proportional displacement is multiplied by the preset gain coefficient to obtain the final offset compensation amount; The physical centroid coordinates are obtained by subtracting the final offset compensation from the local observed centroid.

7. The multi-target motion trajectory prediction device based on Transformer according to claim 1, characterized in that, The correlation results are determined based on the physical centroid coordinates, including: Calculate the displacement vector between the coordinates of two physical centroids at adjacent time points; The horizontal vector perpendicular to the direction of the shelf uprights is defined as the noise-sensitive vector. Calculate the projection component of the displacement vector in the direction of the noise-sensitive vector; Calculate the rate of change of geometric shear bias over time; The correlation between the changing trend of the projection components and the rate of change of the geometric shear bias over time is determined, and the correlation results are obtained.

8. The Transformer-based multi-target motion trajectory prediction device according to claim 7, characterized in that, Pure physical trajectories are generated based on the correlation results, including: The projected components and the preset column width threshold are numerically compared to obtain the comparison results; Based on the correlation and comparison results, the components of the displacement vector on the noise-sensitive vector are set to zero to obtain the pure physical displacement vector. The pure physical trajectory is obtained by performing vector superposition on the pure physical displacement vectors corresponding to each time step.

9. A real-time processing system for multi-target motion trajectory prediction based on Transformer, characterized in that, The system includes the Transformer-based multi-target motion trajectory prediction device as described in any one of claims 1-8.

Images