An AGV visual positioning system based on edge computing

Abstract

The application discloses an AGV visual positioning system based on edge calculation and relates to the technical field of visual positioning.The AGV acquires visual change information through an event triggering mode in a driving process based on a section constraint set, collects corresponding image information, inertial information and wheel speed information when a triggering condition in the section constraint set is met, forms observation data, a pose solution module, the AGV performs pose solution by using the observation data and geometric coding landmark information in the section constraint set, obtains a current pose, synchronously generates a positioning availability level and corresponding cooperative request data, an edge checking module, after the edge node receives the cooperative request data, the current pose is checked for consistency according to the section constraint set and a pose correction amount is calculated; the application collects image, inertial and wheel speed data only when the triggering condition is met, avoids the calculation burden caused by continuous image collection and high-frequency feature extraction, and effectively reduces the generation of redundant observation data.

Description

Technical Field

[0001] This invention relates to the field of visual positioning technology, and in particular to an AGV visual positioning system based on edge computing. Background Technology

[0002] With the development of intelligent manufacturing, Automated Guided Vehicles (AGVs), as key equipment for automated material handling and scheduling, have been widely used in warehousing and logistics, industrial manufacturing, and port transportation. Positioning technology is the core foundation for AGVs to achieve autonomous navigation, path planning, and motion control. Existing AGV positioning methods mainly include absolute positioning based on magnetic strips, QR codes, or reflectors, and relative positioning based on the fusion of LiDAR, visual sensors, and inertial sensors. Visual perception-based positioning methods, due to their rich information content and strong environmental adaptability, have become an important development direction for current AGV positioning technology. Meanwhile, the rise of edge computing technology has enabled some computing tasks to be moved from the cloud to edge nodes closer to the equipment, providing a new technical path to reduce communication latency and improve system response speed.

[0003] However, existing localization methods still have some areas for improvement. First, current visual localization methods rely on continuous image acquisition and high-frequency feature extraction, resulting in high computational resource consumption and power consumption, and easily introducing a large amount of redundant information, reducing the quality of effective observations. Second, they lack effective utilization of the computational capabilities of edge nodes, making it difficult to verify and correct localization results in a timely manner, leading to easy accumulation of errors, and failing to fully integrate prior environmental information for consistency verification. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an AGV visual positioning system based on edge computing to solve the problem of easily introducing a large amount of redundant information and reducing the quality of effective observation.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides an AGV visual positioning system based on edge computing, comprising:

[0008] In the segment modeling module, edge nodes generate a segment positioning list according to the AGV operating environment and send it to the AGV. After receiving the segment positioning list, the AGV forms a segment constraint set corresponding to the segment positioning list.

[0009] The event perception module allows the AGV to acquire visual change information through event triggering during its journey, based on a segment constraint set. When the triggering conditions in the segment constraint set are met, it collects corresponding image information, inertial information, and wheel speed information to form observation data.

[0010] The pose calculation module uses observation data and geometrically encoded landmark information from the segment constraint set to perform pose calculation, obtain the current pose, and simultaneously generate the positioning availability level and corresponding collaborative request data.

[0011] The edge verification module, after receiving the collaborative request data, performs consistency verification on the current pose according to the segment constraint set and calculates the pose correction amount, and sends the pose correction amount to the AGV;

[0012] The constraint control module updates the current pose of the AGV after receiving the pose correction amount, and outputs vehicle control commands based on the updated pose under the constraint of the positioning availability level.

[0013] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the segment positioning list includes geometrically coded landmark information within each segment, spatial pose information corresponding to the geometrically coded landmark information, visible range information corresponding to the geometrically coded landmark information, and positioning constraint information and motion constraint information corresponding to each segment.

[0014] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the formation of a segment constraint set corresponding to the segment positioning list refers to the AGV extracting information content corresponding to the current operating segment from the segment positioning list and structuring the extracted information content.

[0015] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, wherein: the acquisition of visual change information through event triggering specifically includes:

[0016] Under the triggering condition constraints recorded in the segment constraint set, the AGV continuously receives the brightness change event stream output by the event trigger source, and performs event polarity statistics, event space clustering, event change density calculation and event direction consistency calculation on the brightness change event stream within the preset event time window to obtain visual change information;

[0017] The visual change information is compared with the triggering conditions in the segment constraint set item by item to generate a triggering determination result, which includes a trigger passed state and a trigger failed state.

[0018] When the AGV trigger determination result is a trigger pass state, the trigger time is recorded and the event time window boundary is locked. At the same time, the trigger hysteresis rule is started. The trigger determination result is frozen within the holding period limited by the trigger holding duration parameter, and new trigger determination requests are blocked within the suppression period limited by the trigger suppression duration parameter, and the system enters a stable trigger state.

[0019] When the AGV trigger determination result is a trigger failure state, maintain the non-trigger state, update the event time window and continue to receive the brightness change event stream, and return to the trigger determination process to execute subsequent trigger determinations.

[0020] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the formation of observation data specifically includes:

[0021] When the AGV is in a stable triggering state, image information is collected according to the triggering time corresponding to the stable triggering state, and inertial information and wheel speed information corresponding to the triggering time are collected synchronously.

[0022] Time alignment is performed on image information, inertial information, and wheel speed information to obtain time-aligned data corresponding to the trigger time. This data is then encapsulated with visual change information corresponding to the trigger time to form observation data.

[0023] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the step of obtaining the current pose and simultaneously generating the positioning availability level and corresponding collaborative request data specifically includes:

[0024] The AGV performs pose calculation based on the observation data and the geometrically encoded landmark information in the segment constraint set to obtain the current pose corresponding to the observation data;

[0025] Based on the geometric reprojection error, the number of landmark matches, and the observation consistency results obtained during the pose calculation process, the availability of the current pose is evaluated, and a positioning availability level is generated.

[0026] The current pose, positioning availability level, and corresponding observation data are encapsulated to form collaborative request data.

[0027] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the step of performing pose calculation processing refers to the AGV establishing a geometric correspondence between the geometrically encoded landmark information in the observation data and the corresponding spatial pose information recorded in the segment constraint set, and performing spatial geometric calculation by minimizing the reprojection error of the geometrically encoded landmark information on the image plane under the constraint of the geometric correspondence.

[0028] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the calculation of the pose correction amount specifically includes:

[0029] Based on the spatial pose information and segment constraint information corresponding to the geometrically encoded landmark information recorded in the segment constraint set, a consistency check is performed on the current pose in the collaborative request data;

[0030] When the consistency check fails, the edge node re-executes the pose calculation process based on the observation data and the segment constraint set to calculate the pose correction amount;

[0031] When the consistency check passes, the edge node calculates the pose correction based on the current pose and the segment constraint set.

[0032] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the AGV updating its current pose after receiving the pose correction amount means that the AGV performs algebraic addition operations on the position coordinate correction component in the pose correction amount and the position coordinate component in the current pose according to the component correspondence method in the environmental coordinate system, and performs algebraic addition operations on the attitude angle correction component in the pose correction amount and the attitude angle component in the current pose to obtain the updated current pose.

[0033] As a preferred embodiment of the edge computing-based AGV visual positioning system of the present invention, the output vehicle control command specifically includes:

[0034] The AGV generates vehicle motion control parameters based on the updated current pose, and combines the positioning availability level to constrain the vehicle motion control parameters, resulting in constrained vehicle motion control parameters.

[0035] The constrained vehicle motion control parameters are used as vehicle control commands and output.

[0036] The beneficial effects of this invention are as follows: By introducing a segment constraint set and adopting an event-triggered approach to acquire visual change information, image, inertial, and wheel speed data are only collected when the triggering conditions are met. This avoids the computational burden caused by continuous image acquisition and high-frequency feature extraction, effectively reducing the generation of redundant observation data, thereby reducing computational resource consumption and power consumption, and improving the quality of effective observation information. Simultaneously, by sending the pose calculation results to the edge nodes in the form of collaborative request data, and utilizing the environmental prior information in the segment constraint set to perform consistency verification on the current pose, and calculating the pose correction amount when necessary, timely verification and correction of the positioning results are achieved. This fully leverages the computational capabilities of the edge nodes, suppresses the continuous accumulation of positioning errors, and improves the overall stability and reliability of positioning. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1This is a flowchart of an AGV vision positioning system based on edge computing.

[0039] Figure 2 A schematic diagram of the segment location list and segment constraint set generated.

[0040] Figure 3 This is a flowchart for event trigger determination and observation data formation.

[0041] Figure 4 A flowchart for generating vehicle control commands. Detailed Implementation

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0043] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0044] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0045] Reference Figures 1-4 This is one embodiment of the present invention, which provides an AGV visual positioning system based on edge computing, including the following steps:

[0046] In the segment modeling module, edge nodes generate a segment positioning list according to the AGV operating environment and send it to the AGV. After receiving the segment positioning list, the AGV forms a segment constraint set corresponding to the segment positioning list.

[0047] The edge nodes generate a segment location list according to the AGV operating environment and send it to the AGV, specifically:

[0048] The edge node reads the boundary information of the work area of ​​the AGV's operating environment. The boundary information of the work area is represented by a sequence of polygonal boundary vertices in a unified plane coordinate system. The sequence of boundary vertices is arranged in spatial order. The edge node determines the position of the channel centerline of the work area based on the sequence of polygonal boundary vertices, and divides the work area at the intersection of the channel centerlines to obtain multiple operating segments. Each operating segment corresponds to an independent spatial range, which is represented by a sequence of polygonal boundary vertices.

[0049] Edge nodes read landmark deployment registration information within the spatial range of each operating segment. The landmark deployment registration information records the unique number of the geometrically coded landmark information and the corresponding installation plane coordinates. The unique number of the geometrically coded landmark information is associated with the installation plane coordinates to form the spatial pose information corresponding to the geometrically coded landmark information. The spatial pose information includes the plane coordinates and the installation orientation angle.

[0050] The edge node reads the installation height and installation orientation angle corresponding to the geometrically coded landmark information from the landmark deployment registration information, and reads the AGV camera installation height and camera field of view from the AGV parameter configuration file. The edge node uses the AGV camera installation position as the viewpoint, determines the horizontal and vertical angles of the spatial view cone using the camera field of view, and uses the height difference between the camera installation height and the landmark installation height as the projection reference height. The edge node performs geometric projection of the spatial view cone on the ground plane corresponding to the operating section, calculates the intersection area of ​​the spatial view cone and the ground plane, and performs orientation rotation correction on the projected area according to the installation orientation angle of the geometrically coded landmark information, thereby obtaining the visible range information of the geometrically coded landmark information within the operating section. The visible range information is represented by a sequence of polygon boundary vertices.

[0051] Edge nodes perform spatial intersection operations on the polygon boundary vertex sequences of the operating segment's spatial range and visible range information. The non-empty result of the spatial intersection operation is used as the visible range condition, and a set of geometrically encoded landmark information numbers that meet the visible range condition is selected to form positioning constraint information.

[0052] Edge nodes read path shape registration information, which records the upper limit of speed and the upper limit of turning angular velocity corresponding to the running section, and uses the upper limit of speed and the upper limit of turning angular velocity as motion constraint information.

[0053] Edge nodes combine and encapsulate geometrically encoded landmark information, spatial pose information, visible range information, positioning constraint information, and motion constraint information according to the operating segment to form a segment positioning list.

[0054] After receiving the segment positioning list, the AGV forms a segment constraint set corresponding to the segment positioning list, specifically:

[0055] After receiving the segment positioning list, the AGV extracts the geometrically encoded landmark information, spatial pose information, visible range information, positioning constraint information, and motion constraint information corresponding to the current operating segment. The AGV then structures and organizes the extracted information to form a segment constraint set corresponding to the segment positioning list. This segment constraint set is used to limit the constraints in subsequent visual change information acquisition, observation data formation, and pose calculation processes.

[0056] The event perception module allows the AGV to acquire visual change information during its journey by triggering events based on a set of segment constraints. When the triggering conditions in the set of segment constraints are met, the module collects corresponding image information, inertial information, and wheel speed information to form observation data.

[0057] The AGV acquires visual change information during its journey based on a set of segment constraints through event triggering. Specifically:

[0058] During operation, the AGV continuously receives brightness change event streams from the event trigger source. The brightness change event stream is output by the event trigger source when the pixel brightness change exceeds a set brightness threshold. Each event in the brightness change event stream contains at least pixel position coordinates, brightness change polarity, and event timestamp. By only outputting brightness change events and not the complete image, the impact of irrelevant static scenes on subsequent processing is reduced.

[0059] It should also be noted that the brightness threshold is set according to the minimum brightness change resolution of the event trigger source. The brightness threshold is set to a value that is not less than the minimum brightness change resolution of the event trigger source in order to avoid invalid triggering caused by sensor noise or slight light fluctuations.

[0060] Under the triggering condition constraints recorded in the segment constraint set, the AGV divides the brightness change event stream into consecutive event time windows according to the event timestamp. The event time window is limited by the start timestamp and the end timestamp. The event time window is updated at fixed time intervals (e.g., 5 milliseconds) during the driving process, so that the brightness change event can form analyzable data segments in time.

[0061] The AGV reads the brightness change events contained in the brightness change event stream within each event time window, and classifies the brightness change events according to the brightness change polarity field recorded in the events. Brightness change events with positive polarity are added to the positive polarity event set, and those with negative polarity are added to the negative polarity event set. The number of events in the positive polarity event set and the number of events in the negative polarity event set are counted separately to obtain the total number of positive and negative polarity events. Based on the number of positive and negative polarity events, a polarity ratio value is calculated. The polarity ratio value is defined as the proportion of the larger of the positive and negative polarity event counts to the sum of the two counts, expressed as:

[0062] ;

[0063] in, Indicates the polarity ratio value. Indicates the number of positive polarity events. This indicates the number of negative polarity events.

[0064] The number of positive polarity events, the number of negative polarity events, and the polarity ratio are used as the event polarity statistics results. The polarity ratio in the event polarity statistics results is compared with a preset ratio threshold. When the polarity ratio is greater than the preset ratio threshold, it is determined that the brightness change event shows a consistent brightness change direction within the event time window. When the polarity ratio is not greater than the preset ratio threshold, it is determined that the brightness change event does not have directional consistency, thereby suppressing scattered brightness change events caused by random noise or illumination jitter.

[0065] It should also be noted that the preset ratio threshold is set according to the polarity distribution characteristics of the event trigger source under the condition of no effective scene change. The preset ratio threshold is set to 0.6 to ensure that at least 60% of the brightness change events within the event time window have the same polarity, so that the brightness change caused by the real spatial structure can be distinguished from random noise events with a near-equilibrium polarity distribution.

[0066] After completing the event polarity statistics, the AGV reads the pixel position coordinates corresponding to each brightness change event within the event time window. The pixel position coordinates are represented in two-dimensional coordinate form, and event spatial clustering is performed on the brightness change events based on these coordinates. For any two brightness change events within the event time window, the AGV calculates the pixel spatial distance between them. The pixel spatial distance is calculated using Euclidean distance, expressed as:

[0067] ;

[0068] in, Indicates the first The brightness change event and the first Pixel spatial distance between brightness change events Indicates the first The pixel x-coordinate of each brightness change event. Indicates the first The pixel x-coordinate of each brightness change event. Indicates the first The pixel ordinate of each brightness change event. Indicates the first The pixel ordinate of each brightness change event.

[0069] The AGV compares the pixel spatial distance with a preset pixel distance threshold. When the pixel spatial distance between two brightness change events is less than the preset pixel distance threshold, the two brightness change events are divided into the same event cluster. This allows brightness change events that are continuously distributed in the pixel spatial location to be merged into the same event cluster, thus forming an event spatial clustering result. This distinguishes the brightness change region generated by the same spatial structure from spatially dispersed noise events.

[0070] In the specific implementation process, the event space clustering adopts a density connectivity clustering method based on a preset pixel distance threshold. By judging the neighborhood connectivity of brightness change events in the pixel space, brightness change events that meet the preset pixel distance threshold condition are gradually merged into the same event cluster, without relying on the preset number of event clusters, thereby adapting to the dynamic changes in the number of brightness change events within the event time window.

[0071] It should also be noted that the preset pixel distance threshold is set based on the pixel resolution of the event triggering source and the minimum projection size of the geometrically coded landmark information on the image plane. In this embodiment, the preset pixel distance threshold is set to 3 pixels. By limiting the preset pixel distance threshold to no more than half of the minimum projection width of the geometrically coded landmark information, brightness change events caused by the edge of the same physical structure are grouped into the same event cluster in the pixel space, and spatially randomly distributed noise events are suppressed.

[0072] After completing event space clustering, the AGV statistically analyzes the brightness change polarity distribution corresponding to brightness change events within each event cluster, obtaining the number of positive and negative polarity events within the event cluster. The AGV calculates the directional consistency value corresponding to the event cluster based on the proportion of positive and negative polarity events within the cluster. The directional consistency value is defined as the ratio between the number of events corresponding to the brightness change polarity with the largest proportion within the event cluster and the total number of brightness change events within the cluster. When the proportion of a certain brightness change polarity within an event cluster exceeds a preset directional consistency threshold, it is determined that the brightness changes within the event cluster maintain a consistent direction. The preset directional consistency threshold is determined based on the statistical characteristics of the brightness change polarity distribution in the event space clustering results. The AGV calculates the average and fluctuation of the brightness change polarity proportion within the event cluster and uses the difference between the average and the fluctuation as the preset directional consistency threshold.

[0073] The AGV summarizes the directional consistency judgment results of each event cluster. When at least one event cluster within the event time window meets the directional consistency judgment condition, it determines that the directional consistency of the event corresponding to the event time window is established, and outputs the directional consistency judgment result corresponding to the event time window as the event directional consistency result. This allows the brightness change to be identified as a continuous change of the same physical structure within the event time window, rather than being caused by random noise or instantaneous illumination disturbance, thus providing a basis for directional consistency judgment for triggering the determination.

[0074] After obtaining the event direction consistency result, the AGV counts the total number of brightness change events within the event time window and calculates the event change density based on the event time window length. The event change density is the ratio of the number of brightness change events within the event time window to the duration of the event time window. The event change density is used to represent the degree of temporal concentration of brightness changes within the event time window, allowing the intensity of brightness changes to participate in subsequent trigger condition determination in a definite numerical form.

[0075] The AGV combines event polarity statistics, event spatial clustering results, event change density, and event direction consistency results to form visual change information. It then compares this visual change information with each trigger condition in the segment constraint set: when the event change density is greater than or equal to the event change density threshold, a pass result is generated; when the event change density is less than the threshold, a fail result is generated. Based on the event spatial clustering results, the event spatial cluster size (the number of brightness change events contained in the largest event cluster within the event time window) is calculated, and this size is compared with an event spatial cluster size threshold. When the event spatial cluster size is greater than or equal to the threshold, a pass result is generated; when the event spatial cluster size is less than the threshold, a fail result is generated. Simultaneously, it determines whether the event direction consistency result is valid.

[0076] To further explain, the event change density threshold is set based on the rate of background brightness change events generated by the event trigger source under conditions of no effective scene change. It is used to limit the minimum temporal concentration of brightness change events within the event time window. The event change density threshold is set above the background noise event change density but below the upper limit of event output of the event trigger source under high-frequency change conditions, thus ensuring that the triggering condition is met only when there is continuous brightness change within the event time window. The event spatial clustering size threshold is set based on the minimum projection size of the geometrically coded landmark information on the image plane. It is used to limit the number of brightness change events or the pixel range covered by the event clusters in the event spatial clustering results. The event spatial clustering size threshold is not less than the minimum number of brightness change events required to constitute the minimum projection area of ​​the geometrically coded landmark information, to ensure that the event clusters can correspond to physical objects with a clear spatial structure.

[0077] When the visual change information satisfies all the triggering conditions in the segment constraint set, the AGV generates a trigger pass status and records the corresponding trigger time timestamp at the same time. The trigger time is used for time alignment of subsequent multi-source information collection. When the visual change information does not satisfy any triggering condition, the AGV generates a trigger fail status.

[0078] When the AGV determines that the trigger has passed, it locks the event time window boundary corresponding to the trigger moment and uses the locked event time window boundary as the trigger reference moment, immediately initiating the trigger hysteresis rule. The trigger hysteresis rule consists of a trigger hold duration parameter and a trigger suppression duration parameter. The trigger hold duration parameter is set based on the continuous stabilization time of the brightness change event within the event time window, used to limit the minimum duration for which the trigger determination result needs to remain valid. The hold period corresponding to the trigger hold duration parameter is determined by extending the trigger hold duration parameter backward from the trigger moment. The trigger suppression duration parameter is set based on the event output frequency of the event trigger source under continuous brightness change conditions, used to limit the duration of consecutive trigger determinations. The minimum time interval between the trigger and hold periods is determined by extending the trigger and hold period by the trigger and hold period duration parameters. During the hold period, the AGV freezes the current trigger judgment result, ensuring that the trigger judgment result does not change with the instantaneous fluctuations of the brightness change event. During the hold period, the AGV blocks new trigger judgment requests, so that the brightness change event arriving during the hold period is only used to update the event time window and does not participate in the new trigger judgment. Under the joint constraints of the trigger hold period parameter and the trigger and hold period parameter, the AGV enters a stable trigger state, so that the subsequent acquisition of image information, inertial information, and wheel speed information all use the same trigger time as the time reference.

[0079] When the trigger determination result is a trigger failure state, the AGV maintains the non-trigger state, updates the start and end timestamps of the event time window, continues to receive brightness change event streams, and returns to the trigger determination process to perform event polarity statistics, event space clustering, event change density calculation, and event direction consistency calculation on the new event time window until the trigger determination result changes to a trigger success state.

[0080] Preferably, compared to existing methods, this invention uses a brightness change event flow analysis mechanism based on event triggering sources to jointly constrain visual changes in multiple dimensions of time, space, and direction. It triggers multi-source information acquisition only when the conditions of stability, continuity, and clear spatial structure characteristics are met, thereby reducing the acquisition of invalid data. At the same time, by combining event time windows and trigger hysteresis rules, the trigger determination has noise resistance and stability, thereby reducing the computational load and improving real-time response capability while ensuring positioning reliability.

[0081] When the triggering conditions in the segment constraint set are met, the corresponding image information, inertial information, and wheel speed information are collected to form observation data, specifically:

[0082] In a stable trigger state, based on the trigger time timestamp recorded when the trigger pass state is generated, the vehicle-mounted image acquisition center is controlled to acquire a frame of image information at the time point corresponding to the trigger time. The image information includes at least complete pixel matrix data and an image timestamp corresponding to the image acquisition time. At the same time, the AGV reads inertial information covering the trigger time from the inertial measurement center. The inertial information includes angular velocity data and linear acceleration data. It also reads wheel speed information covering the trigger time from the wheel speed acquisition center. The wheel speed information includes the rotational speed data of each drive wheel corresponding to the trigger time. Both the inertial information and the wheel speed information carry their respective sampling timestamps.

[0083] Based on image timestamps, inertial information timestamps, and wheel speed information timestamps, time alignment processing is performed on image information, inertial information, and wheel speed information. The time alignment processing uses the trigger time timestamp as a unified reference time, and performs time interpolation or time truncation on the inertial information and wheel speed information to obtain inertial information data and wheel speed information data corresponding to the trigger time timestamp, so that image information, inertial information, and wheel speed information correspond to the same motion state under the same time reference.

[0084] The time-aligned image information, inertial information, and wheel speed information are uniformly encapsulated with the visual change information corresponding to the trigger time. The encapsulation content includes the trigger time timestamp, time-aligned image information, time-aligned inertial information, time-aligned wheel speed information, and visual change information indicating the trigger cause, forming observation data that corresponds one-to-one with the trigger time.

[0085] The pose calculation module uses observation data and geometrically encoded landmark information from the segment constraint set to perform pose calculation, obtain the current pose, and simultaneously generate the positioning availability level and corresponding collaborative request data.

[0086] The AGV uses observation data and geometrically encoded landmark information from the segment constraint set to perform pose calculation, obtain the current pose, and simultaneously generate the positioning availability level and corresponding collaborative request data. Specifically:

[0087] The AGV reads image information from the observation data and detects image features corresponding to geometrically coded landmark information in the image information. The image features include the corner coordinates or contour feature coordinates of the geometrically coded landmark information in the image coordinate system. At the same time, the AGV reads the geometrically coded landmark information corresponding to the current operating segment and the spatial pose information corresponding to the geometrically coded landmark information from the segment constraint set. The spatial pose information is represented in a unified environmental coordinate system, which represents the fixed spatial position of the geometrically coded landmark information in the AGV's operating environment.

[0088] The AGV establishes a geometric correspondence between image feature coordinates and spatial pose information. This geometric correspondence links the geometrically encoded landmark information in the image coordinate system with the spatial pose information in the environmental coordinate system. Based on this geometric correspondence, the AGV constructs spatial geometric constraint equations by combining inertial and wheel speed information from the observation data. Inertial information provides attitude change constraints, wheel speed information provides displacement change constraints, and the spatial geometric constraint equations describe the position and attitude relationship of the AGV in the environmental coordinate system.

[0089] The AGV performs spatial geometric solution on the spatial geometric constraint equations. By minimizing the reprojection error of the geometrically encoded landmark information on the image plane, it obtains the current pose corresponding to the observation data. The spatial geometric solution process is represented as follows:

[0090] ;

[0091] in, This represents the current pose calculation result obtained through spatial geometry calculation. This indicates the number of geometrically coded landmark matches involved in the spatial geometric solution. Indicates the first The actual observed feature coordinates of each geometrically encoded landmark in the image plane. Indicates the first Spatial pose information of a geometrically coded landmark in the environmental coordinate system. Expressed in posture Conditions, No. The theoretical feature coordinates are obtained by projecting geometrically encoded landmark information from the environmental coordinate system onto the image plane.

[0092] During the spatial geometry solution process, the AGV synchronously counts the number of geometrically coded landmarks matched in the solution and records the geometric reprojection error value corresponding to each geometrically coded landmark.

[0093] The AGV performs a usability assessment of the current pose based on the spatial geometry calculation results. The usability assessment uses geometric reprojection error, the number of geometrically coded landmark matches, and the observation consistency result as evaluation indicators. Among them, the geometric reprojection error is determined by the deviation between the actual observed feature coordinates and the theoretical projected feature coordinates of each geometrically coded landmark information on the image plane during the spatial geometry calculation process, and is used to reflect the degree of geometric fit between the current pose and the image observation. The number of geometrically coded landmark matches is determined by the number of geometrically coded landmark information that participated in the spatial geometry calculation and successfully established a geometric correspondence, and is used to reflect the number of geometric constraints on which the current pose calculation depends. The observation consistency result is determined by the consistency of the geometric reprojection error distribution corresponding to each geometrically coded landmark during the spatial geometry calculation process.

[0094] The AGV judges the geometric reprojection error of each geometrically coded landmark participating in the spatial geometric solution one by one. When all geometric reprojection errors are within the preset error range, the observation consistency result is determined to be valid; when at least one geometrically coded landmark has a geometric reprojection error that exceeds the preset error range, the observation consistency result is determined to be invalid.

[0095] To further explain, the preset error range is determined based on the physical size of the geometrically encoded landmark information and its spatial projection relationship to the image plane, which is recorded in the segment constraint set. Preferably, the preset error range is limited to a preset ratio range of the minimum projection feature size of the geometrically encoded landmark information on the image plane, so that the geometric reprojection error does not exceed the distinguishable feature range of the corresponding geometrically encoded landmark information on the image plane, thereby ensuring that the geometric reprojection error can truly reflect the pose calculation accuracy without introducing misjudgments caused by changes in image resolution or feature scale.

[0096] When the geometric reprojection error is within the preset error range, the number of geometrically encoded landmark matches meets the minimum matching requirement, and the observation consistency result is valid, the positioning availability level is determined: the AGV will rate the current pose as high availability; when any two conditions in the evaluation index are met, the AGV will rate the current pose as medium availability; if only one evaluation index is met, the AGV will rate the current pose as low availability, thereby generating the positioning availability level corresponding to the current pose.

[0097] The AGV encapsulates the current pose, positioning availability level, and observation data corresponding to the current pose in a unified manner. The encapsulation content includes the current pose's position and attitude information, positioning availability level identifier, and observation data index information, forming collaborative request data.

[0098] The edge verification module performs consistency verification on the current pose based on the segment constraint set and calculates the pose correction amount after the edge node receives the collaborative request data. The pose correction amount is then sent to the AGV.

[0099] Edge nodes read the geometrically coded landmark information corresponding to the current operating segment, the spatial pose information corresponding to the geometrically coded landmark information, and the segment constraint information from the segment constraint set. Based on the spatial pose information and segment constraint information, the edge nodes perform consistency verification on the current pose in the collaborative request data. The consistency verification includes at least the following judgments: determining whether the current pose is within the operating segment space defined by the segment constraint information; determining whether the current pose and the spatial pose information corresponding to the geometrically coded landmark information satisfy a preset spatial consistency relationship; and determining whether the current pose meets the motion constraint conditions defined by the segment constraint information.

[0100] When the consistency check fails, the edge node determines that its current pose is inconsistent with the segment constraint set. In this state, the edge node reads observation data from the collaboration request data and, combined with the geometrically encoded landmark information and corresponding spatial pose information recorded in the segment constraint set, re-executes the pose calculation process. This re-executed pose calculation establishes a geometric correspondence between the geometrically encoded landmark information in the observation data and the corresponding spatial pose information recorded in the segment constraint set, and optimizes the reprojection error to obtain a new pose calculation result. The edge node compares this new pose calculation result with the current pose in the collaboration request data and calculates the pose correction based on the difference in position coordinates and attitude angles.

[0101] When the consistency check result is passed, the edge node determines that the current pose satisfies the reliable interval constraints and is consistent with the segment constraint set. In the consistency check-passed state, the edge node does not re-execute the complete pose calculation process; instead, it calculates the pose correction based on the spatial constraints of the current pose and the segment constraint set. The segment constraint set pre-records the position coordinate constraint range and attitude angle constraint range for the current operating segment. The position coordinate constraint range is limited by the minimum and maximum values ​​of the operating segment's spatial boundary in the environmental coordinate system, and the attitude angle constraint range is limited by the allowed heading angle range of the operating segment. The pose correction is obtained by restricting the position coordinate components and attitude angle components of the current pose to their respective constraint ranges to obtain the projection result. The component difference vector between the projection result and the current pose is used as the pose correction, which is used to correct the cumulative offset of the current pose without changing the segment consistency.

[0102] After the edge node completes the pose correction calculation, it sends the pose correction to the AGV for the AGV to perform update processing on the current pose. The pose correction includes position coordinate correction components and attitude angle correction components.

[0103] Preferably, the present invention uses a consistency verification and pose correction mechanism based on segment constraint sets at the edge node side, enabling the current pose calculated by the AGV to undergo secondary verification and constraint correction under structured spatial constraints, thus avoiding the cumulative error propagation problem caused by relying solely on the results of a single visual calculation; when the consistency verification fails, a recalculation mechanism is used to correct significant deviations, and when the consistency verification passes, a lightweight correction is achieved through constraint mapping, thereby reducing computational overhead while ensuring positioning reliability and improving the stability, continuity, and overall operating efficiency of the positioning results.

[0104] The constraint control module updates the current pose of the AGV after receiving the pose correction amount, and outputs vehicle control commands based on the updated pose under the constraint of the positioning availability level.

[0105] The AGV receives the pose correction amount sent by the edge node and performs a fusion calculation on the current pose and the pose correction amount in the environmental coordinate system. The fusion calculation is performed according to the component correspondence method: the position coordinate correction component in the pose correction amount is algebraically added to the position coordinate component in the current pose to obtain the updated position coordinates; the attitude angle correction component in the pose correction amount is algebraically added to the attitude angle component in the current pose to obtain the updated attitude angle; the updated position coordinates and the updated attitude angle together constitute the updated current pose.

[0106] The AGV reads the preset travel path information corresponding to the current operating segment from the segment constraint set. The preset travel path information is represented in the environmental coordinate system and consists of a set of path reference points arranged in the travel sequence. Each path reference point includes at least reference position coordinates and a reference heading angle, used to describe the expected travel trajectory of the AGV within the operating segment. Based on the updated current pose, the AGV determines the path reference point in the preset travel path that is spatially closest to the updated current pose and uses this closest path reference point as the current path tracking reference. The AGV calculates the pose deviation between the updated current pose and the path reference point. The pose deviation includes position deviation and heading deviation. The position deviation is determined by the difference between the updated current position coordinates and the position coordinates of the path reference point, and the heading deviation is determined by the difference between the updated current attitude angle and the reference heading angle corresponding to the path reference point.

[0107] Vehicle motion control parameters are calculated based on position and heading deviations. These parameters include target speed, steering angle, and acceleration. The target speed is determined by the forward deviation of the current position along the preset travel path and the path curvature, ensuring the AGV moves stably along the path. The steering angle is calculated by converting the position deviation into a corresponding angle correction and algebraically combining it with the heading deviation. This ensures the steering angle reflects both the lateral offset of the current position relative to the preset travel path and the angle difference between the current heading and the path reference heading, guiding the AGV to adjust its steering towards the preset travel path. The acceleration is calculated based on the difference between the target speed and the current speed, used to control the vehicle's speed change process.

[0108] The AGV reads the positioning availability level and applies constraint processing to the vehicle motion control parameters according to the control constraint rules corresponding to that level. These control constraint rules are determined by a pre-stored positioning availability level control rule table, which records the defined correspondence between the positioning availability level and the allowable range of vehicle motion control parameters. The table sets upper limits for target driving speed, steering angle, and acceleration parameters for different positioning availability levels, ensuring that each level corresponds to a defined set of vehicle motion control parameter constraint ranges. This establishes a one-to-one numerical constraint relationship between the positioning availability level and the vehicle motion control behavior.

[0109] According to the positioning availability level control rule table, the AGV reads the upper limits of the target driving speed parameter, steering angle parameter, and acceleration parameter corresponding to the current positioning availability level, and performs constraint processing on the vehicle motion control parameters. The constraint processing is accomplished as follows: when the target driving speed parameter exceeds the corresponding upper limit value, the target driving speed parameter is truncated to the upper limit value; when the steering angle parameter exceeds the corresponding upper limit value, the steering angle parameter is truncated to the upper limit value; when the acceleration parameter exceeds the corresponding upper limit value, the acceleration parameter is truncated to the upper limit value.

[0110] The AGV uses the constrained target driving speed parameters, steering angle parameters, and acceleration parameters as vehicle control commands, and outputs vehicle control commands to drive the vehicle to perform motion control within the control range defined by the positioning availability level.

[0111] In summary, this invention introduces a segment constraint set and employs an event-triggered approach to acquire visual change information. Image, inertial, and wheel speed data are only collected when trigger conditions are met, avoiding the computational burden of continuous image acquisition and high-frequency feature extraction. This effectively reduces the generation of redundant observation data, thereby lowering computational resource consumption and power consumption, and improving the quality of effective observation information. Simultaneously, by sending the pose calculation results to edge nodes in the form of collaborative request data, and utilizing the environmental prior information in the segment constraint set to perform consistency verification on the current pose, and calculating pose corrections when necessary, timely verification and correction of the positioning results are achieved. This fully leverages the computational capabilities of edge nodes, suppresses the continuous accumulation of positioning errors, and improves the overall stability and reliability of positioning.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An AGV visual positioning system based on edge computing, characterized in that: include, In the segment modeling module, edge nodes generate a segment positioning list according to the AGV operating environment and send it to the AGV. After receiving the segment positioning list, the AGV forms a segment constraint set corresponding to the segment positioning list. The segment positioning list includes geometrically coded landmark information within each segment, spatial pose information corresponding to the geometrically coded landmark information, visible range information corresponding to the geometrically coded landmark information, and positioning constraint information and motion constraint information corresponding to each segment. The formation of the segment constraint set corresponding to the segment positioning list refers to the AGV extracting information content corresponding to the current operating segment from the segment positioning list and structuring the extracted information content. The event perception module allows the AGV to acquire visual change information through event triggering during its journey, based on a segment constraint set. When the triggering conditions in the segment constraint set are met, it collects corresponding image information, inertial information, and wheel speed information to form observation data. The pose calculation module uses observation data and geometrically encoded landmark information from the segment constraint set to perform pose calculation, obtain the current pose, and simultaneously generate the positioning availability level and corresponding collaborative request data. The edge verification module, after receiving the collaborative request data, performs consistency verification on the current pose according to the segment constraint set and calculates the pose correction amount, and sends the pose correction amount to the AGV; The constraint control module updates the current pose of the AGV after receiving the pose correction amount, and outputs vehicle control commands based on the updated pose under the constraint of the positioning availability level.

2. The AGV visual positioning system based on edge computing as described in claim 1, characterized in that: The method of obtaining visual change information through event triggering is as follows: Under the triggering condition constraints recorded in the segment constraint set, the AGV continuously receives the brightness change event stream output by the event trigger source, and performs event polarity statistics, event space clustering, event change density calculation and event direction consistency calculation on the brightness change event stream within the preset event time window to obtain visual change information; The visual change information is compared with the triggering conditions in the segment constraint set item by item to generate a triggering determination result, which includes a trigger passed state and a trigger failed state. When the AGV trigger determination result is a trigger pass state, the trigger time is recorded and the event time window boundary is locked. At the same time, the trigger hysteresis rule is started. The trigger determination result is frozen within the holding period limited by the trigger holding duration parameter, and new trigger determination requests are blocked within the suppression period limited by the trigger suppression duration parameter, and the system enters a stable trigger state. When the AGV trigger determination result is a trigger failure state, maintain the non-trigger state, update the event time window and continue to receive the brightness change event stream, and return to the trigger determination process to execute subsequent trigger determinations.

3. The AGV visual positioning system based on edge computing as described in claim 1, characterized in that: The formation of observation data specifically includes: When the AGV is in a stable triggering state, image information is collected according to the triggering time corresponding to the stable triggering state, and inertial information and wheel speed information corresponding to the triggering time are collected synchronously. Time alignment is performed on image information, inertial information, and wheel speed information to obtain time-aligned data corresponding to the trigger time. This data is then encapsulated with visual change information corresponding to the trigger time to form observation data.

4. The AGV visual positioning system based on edge computing as described in claim 1, characterized in that: The process of obtaining the current pose and simultaneously generating the positioning availability level and corresponding collaborative request data specifically involves: The AGV performs pose calculation based on the observation data and the geometrically encoded landmark information in the segment constraint set to obtain the current pose corresponding to the observation data; Based on the geometric reprojection error, the number of landmark matches, and the observation consistency results obtained during the pose calculation process, the availability of the current pose is evaluated, and a positioning availability level is generated. The current pose, positioning availability level, and corresponding observation data are encapsulated to form collaborative request data.

5. The AGV visual positioning system based on edge computing as described in claim 4, characterized in that: The aforementioned pose calculation process refers to the AGV establishing a geometric correspondence between the geometrically encoded landmark information in the observation data and the corresponding spatial pose information recorded in the segment constraint set, and performing spatial geometric calculation by minimizing the reprojection error of the geometrically encoded landmark information on the image plane under the constraints of the geometric correspondence.

6. The AGV visual positioning system based on edge computing as described in claim 1, characterized in that: The calculation of the pose correction amount is specifically as follows: Based on the spatial pose information and segment constraint information corresponding to the geometrically encoded landmark information recorded in the segment constraint set, a consistency check is performed on the current pose in the collaborative request data; When the consistency check fails, the edge node re-executes the pose calculation process based on the observation data and the segment constraint set to calculate the pose correction amount; When the consistency check passes, the edge node calculates the pose correction based on the current pose and the segment constraint set.

7. The AGV visual positioning system based on edge computing as described in claim 1, characterized in that: The AGV updating its current pose after receiving the pose correction means that, in the environmental coordinate system, the AGV performs algebraic addition on the position coordinate correction component in the pose correction and the position coordinate component in the current pose, and performs algebraic addition on the attitude angle correction component in the pose correction and the attitude angle component in the current pose, in a component correspondence manner, to obtain the updated current pose.

8. The AGV visual positioning system based on edge computing as described in claim 1, characterized in that: The output vehicle control commands are specifically as follows: The AGV generates vehicle motion control parameters based on the updated current pose, and combines the positioning availability level to constrain the vehicle motion control parameters, resulting in constrained vehicle motion control parameters. The constrained vehicle motion control parameters are used as vehicle control commands and output.

Images