Machine vision assisted automated guided vehicle obstacle avoidance system and method

Abstract

The present application relates to the technical field of image recognition, and particularly relates to a machine vision assisted automatic guided vehicle obstacle avoidance system and method. The method comprises the following steps: acquiring a field of view image, extracting a motion area by frame by frame gray difference and calculating a centroid trajectory; analyzing a centroid vector to determine a driving direction and a meeting direction and classifying a risk; constructing a shielding or buffer zone according to the opposite direction / same direction; and adjusting the steering and speed according to the risk to achieve obstacle avoidance. The present application performs time sequence and speed classification analysis on the frame by frame centroid trajectory based on the vehicle-mounted vision, generates a semantic shielding and buffer zone by combining smoothness / mutation detection, and dynamically adjusts the obstacle avoidance action, realizes high robustness, real-time classification avoidance and low-cost engineering deployment for various meeting scenarios, thereby significantly improving the driving safety and traffic efficiency and reducing the probability of collision and sudden stop.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a machine vision assisted automatic guided vehicle obstacle avoidance system and method. BACKGROUND

[0002] Automatic guided vehicles are widely used in warehouse, production line and logistics distribution scenarios, and the core task is to safely and stably complete the cargo carrying and path following in a complex dynamic environment. Existing obstacle avoidance techniques usually adopt multi-sensor fusion (such as laser radar, millimeter wave, ultrasonic wave and vision), geometry-based path planning (such as potential field method, dynamic window method) or visual-based optical flow / background subtraction and deep learning detection methods to detect obstacles and generate avoidance instructions.

[0003] However, in actual application, under the condition of relying only on vehicle-mounted vision, only the distance or the presence of obstacles is output, it is difficult to develop fine strategies for different meeting types, and in the same direction following scenario, only distance is used to judge proximity, ignoring trajectory overlap and motion vector features, which may lead to poor timing of intervention and affect the accuracy of obstacle avoidance. SUMMARY

[0004] Therefore, it is necessary to provide a machine vision assisted automatic guided vehicle obstacle avoidance system and method to solve at least one of the above technical problems.

[0005] To achieve the above purpose, a machine vision assisted automatic guided vehicle obstacle avoidance method, the method comprises the following steps:

[0006] Step S1: acquiring a field of view image set of an automatic guided vehicle, wherein the field of view image set comprises a plurality of field of view images;

[0007] Step S2: extracting the gray value and time series difference of each pixel point in the field of view image set to obtain the gray change rate of the pixel point; and confirming the motion area image of the vehicle itself according to the spatial distribution of the gray change rate;

[0008] Step S3: confirming the driving direction of the vehicle itself by the change of the center of mass vector of the pixels in the motion area image, and judging the meeting direction of the vehicle in the field of view image and the vehicle itself and confirming the meeting risk level according to the driving direction;

[0009] Step S4: when the meeting direction is opposite meeting, generating a meeting avoidance shield area according to the angle between the center of mass vector and the center line in the motion area image and the area expansion rate; when the meeting direction is same meeting, generating a following avoidance buffer area according to the center of mass vector modulus and the area overlap ratio in the motion area image;

[0010] Step S5: adjusting the steering and speed of the trolley itself based on the collision risk level to realize obstacle avoidance for the collision avoidance shielding area or the following avoidance buffer area.

[0011] In the present specification, a machine vision assisted automatic guided trolley obstacle avoidance system is provided for performing the machine vision assisted automatic guided trolley obstacle avoidance method described above, which comprises:

[0012] An image acquisition module is configured to acquire a field of view image set of the automatic guided trolley, wherein the field of view image set comprises a plurality of field of view images.

[0013] A region division module is configured to extract the time series difference of the gray value of each pixel point in the field of view image set, acquire the gray change rate of the pixel point, and confirm the motion region image of the trolley itself according to the spatial distribution of the gray change rate.

[0014] A direction confirmation module is configured to confirm the driving direction of the trolley itself by the centroid vector change of the pixels in the motion region image, and judge the collision direction of the trolley and the trolley itself in the field of view image according to the driving direction and confirm the collision risk level.

[0015] A collision avoidance module is configured to generate a collision avoidance shielding area according to the angle between the centroid vector and the center line in the motion region image and the region expansion rate when the collision direction is opposite, and generate a following avoidance buffer area according to the modulus value of the centroid vector in the motion region image and the region overlap ratio when the collision direction is the same.

[0016] An obstacle avoidance control module is configured to adjust the steering and speed of the trolley itself based on the collision risk level to realize obstacle avoidance for the collision avoidance shielding area or the following avoidance buffer area.

[0017] The present application has the following beneficial effects:

[0018] I. By frame-by-frame gray difference, motion region extraction and centroid position trajectory analysis of the field of view image, the method can accurately identify the motion region of the trolley and the other trolley and restore the centroid change vector. Combined with the time smoothing strategy (if the directions of a plurality of centroid change vectors are consistent, it is judged as a stable direction, and if it is suddenly changed, it is judged as a U-turn / turning direction), the robustness and noise resistance of the driving direction determination are significantly improved, and the misjudgment and jitter are reduced.

[0019] II. Based on the centroid position change speed, a clear speed threshold and sampling strategy (sampling interval 200-500 ms; reasonable speed range 0.05-1.5 m / s) are given, and the risk is quantified into three levels (low / medium / high) (0.05-0.4 m / s, 0.4-0.9 m / s, >0.9 m / s and continuous condition or short-term burst condition are judged as high risk), which can realize real-time risk assessment and early warning, and facilitate the control layer to take different obstacle avoidance strategies according to the level.

[0020] III. According to the meeting direction, opposite "meeting avoidance shielding area" (according to the angle between centroid vector and center line and area expansion rate) and same "following avoidance buffer area" (according to the modulus value of centroid vector and area overlap ratio) are constructed, so that the avoidance area has directionality and spatial quantization basis, so that semantic clear and executable buffer / shielding area can be generated under different meeting types.

[0021] IV. The current driving state (low speed / normal / high speed) of the vehicle and the area overlap ratio (low / medium / high approach) of the two vehicles are combined to map the specific buffer area range (for example, low speed and low approach: 1.0-2.0 m; normal and medium approach: 2.0-3.5 m; high speed or high approach: 3.5-5.0 m), which realizes the dynamic adjustment of the buffer area, ensures the safety margin, and maintains the traffic efficiency as much as possible under low risk.

[0022] V. This method only relies on vehicle-mounted vision, real-time image processing and existing driving / steering execution mechanism, and can realize obstacle avoidance action by directly adjusting speed and direction under the instruction of vehicle-mounted controller. The comprehensive effect is: reducing the probability of collision / stop, improving the driving stability and traffic efficiency of the vehicle, facilitating the integration with the existing automatic guided vehicle platform and engineering deployment. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 It is a step flowchart of a machine vision assisted automatic guided vehicle obstacle avoidance method;

[0024] Figure 2 It is Figure 1 It is a detailed implementation step flowchart of step S3;

[0025] Figure 3 It is a scene diagram of a machine vision assisted automatic guided vehicle obstacle avoidance method of the application;

[0026] The implementation of the application, functional characteristics and advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0027] The technical method of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0028] In addition, the drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. Identical reference numerals in the drawings represent identical or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily have to correspond to physically or logically independent entities. The functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0029] It should be understood that although the terms "first", "second" and the like can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, a first element can be referred to as a second element, and similarly a second element can be referred to as a first element. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0030] To achieve the above-mentioned purpose, please refer to Figures 1 to 3 A machine vision assisted automatic guided vehicle obstacle avoidance method, the method comprising the following steps:

[0031] Step S1: acquiring a set of field of view images of the automatic guided vehicle, wherein the set of field of view images comprises a plurality of field of view images;

[0032] In an embodiment, the automatic guided vehicle can collect the working scene image in front of the vehicle in real time through a visual collection device (such as an industrial camera, a multi-view vision sensor or a depth camera, etc.) installed at the front end of the vehicle body. The visual collection device can collect the scene in front of the vehicle according to a set sampling period (such as 100 milliseconds) to form a set of field of view images containing a plurality of field of view images at different time points.

[0033] In some embodiments, to improve the integrity of image acquisition, the image resolution of the visual acquisition device can be configured to be 1280x720 pixels or 1920x1080 pixels, and parameters such as exposure time, white balance, and frame rate are set to adapt to different lighting conditions in the working environment. For example, the sampling frame rate can be set to 10 to 20 frames per second to obtain a continuous sequence of field of view images, ensuring that continuous and clear image data can be acquired even at low speed and turning state of the trolley.

[0034] For example, in the case of excessive lighting or dark areas in the working scene, a dynamic exposure compensation function can be enabled in the visual acquisition device to prevent overexposure or underexposure of the image, thereby ensuring the clarity and effective information amount of the acquired field of view image. The system can determine the quality of each frame of field of view image through an image clarity evaluation index, and if the image quality is below a preset threshold, reacquisition or enhancement processing can be triggered.

[0035] Step S2: Extracting the gray value and time sequence difference of each pixel point in the field of view image set to obtain the gray change rate of the pixel point; and confirming the motion area image of the trolley itself according to the spatial distribution of the gray change rate;

[0036] In an embodiment, each field of view image in the field of view image set obtained in step S1 is preprocessed, and the preprocessing process can include image graying, noise suppression, and edge smoothing operations. Graying processing can convert a color image into a gray image for subsequent calculation of the gray value of each pixel point.

[0037] In some embodiments, the gray change rate of a pixel point can be calculated based on the gray difference between two consecutive frames of gray images. For example, for a first time gray image and a second time gray image, the gray difference between the corresponding pixel points is calculated and divided by the time interval to obtain the gray change rate. The greater the gray change rate, the more obvious the brightness change of the pixel point in that time period.

[0038] For example, a sliding time window can be used to perform difference processing on consecutive multiple frames of gray images to obtain the cumulative gray change rate of the pixel point, so as to reduce the influence of burst noise on the result. When the cumulative gray change rate of a certain pixel point is higher than a preset change threshold, it can be determined that the pixel point is in the motion area.

[0039] In another embodiment, the spatial distribution of the gray change rate of all pixel points in the image can be counted, and the motion area image of the trolley itself can be determined based on the region in the gray change rate set. For example, when the gray change rates of the pixel points in a certain connected region are all in the high value interval and the area exceeds a preset area threshold, it can be determined that the region is the motion area image of the trolley.

[0040] Step S3: The driving direction of the vehicle itself is determined by the centroid vector change of the pixels in the motion region image, and the meeting direction of the vehicle in the field of view image and the vehicle itself is determined according to the driving direction, and the meeting risk level is determined.

[0041] In an embodiment, the two-dimensional coordinates of all the pixels in the motion region image identified in step S2 are counted, and the centroid position of the pixel region is calculated. Then, the centroid position change vector is calculated in a plurality of continuous sampling time points, and the driving direction of the automatic guided vehicle is determined by the direction and amplitude change of the centroid vector.

[0042] The centroid coordinates at the current time and the centroid coordinates at the last time are connected to form a centroid movement vector. When the centroid movement vector tends to the right side of the image, it is determined that the vehicle is in a right driving state. When the centroid movement vector tends to the left side of the image, it is determined that the vehicle is in a left driving state. When the centroid movement vector changes along the longitudinal direction of the image, it is determined to be a straight driving state.

[0043] For example, on the basis of determining the driving direction of the vehicle itself, the motion direction of other vehicles in the field of view image can be identified and compared with the driving direction of the vehicle itself to determine whether there is a meeting situation. When the motion direction of the other vehicles is opposite to the driving direction of the vehicle itself and the predicted trajectories of the motion regions of the two vehicles overlap, it is determined to be a meeting scene.

[0044] In other embodiments, the meeting risk level can be determined according to the spatial distance and relative speed in the meeting scene. For example, when the predicted collision distance between the two vehicles is less than 1.5 meters and the relative speed is greater than 0.6 meters per second, it is determined to be a high-risk meeting; when the predicted collision distance is between 1.5 meters and 3 meters and the relative speed is between 0.3 meters per second and 0.6 meters per second, it is determined to be a medium-risk meeting; and when the predicted collision distance is greater than 3 meters and the relative speed is less than 0.3 meters per second, it is determined to be a low-risk meeting.

[0045] Step S4: When the meeting direction is opposite, the meeting avoidance shielding area is generated according to the angle between the centroid vector and the center line in the motion region image and the region expansion rate; when the meeting direction is the same, the following avoidance buffer area is generated according to the centroid vector modulus in the motion region image and the region overlap ratio.

[0046] In an embodiment, the two-dimensional coordinates of all the pixels in the motion region image identified in step S2 are counted, and the centroid position of the pixel region is calculated. Then, the centroid position change vector is calculated in a plurality of continuous sampling time points, and the driving direction of the automatic guided vehicle is determined by the direction and amplitude change of the centroid vector. Figure 3When the system determines that the meeting direction is opposite, the angle between the centroid vector of the current motion region image and the center line of the channel can be used as the meeting risk angle, and the rate of change of the motion region area over time (i.e., the region expansion rate) can be used for joint evaluation. When the angle is greater than a set angle threshold and the region expansion rate is in an upward trend, it indicates that the opposite vehicle is approaching quickly, and at this time the system can construct a meeting avoidance shield zone in front of the motion region to limit the forklift to continue driving forward to avoid collision risk.

[0047] For example, when the angle between the centroid vector and the center line is greater than 20 degrees and the region expansion rate is in a growth trend for three consecutive sampling periods, the system defines a rectangular shield zone in the possible intersection area of the two vehicles, and stops the forklift at the boundary of the shield zone to wait for the opposite vehicle to pass first.

[0048] In other embodiments, when it is determined that the meeting direction is the same, a buffer zone can be generated based on the centroid vector module value of the motion region image and the overlap ratio between the motion regions. When the centroid vector module value is small and the overlap ratio of the motion region exceeds a preset threshold, it indicates that the motion trend of the front vehicle and the forklift is consistent and the distance is close, and to avoid rear-end collision, a following avoidance buffer zone can be constructed between the two vehicles.

[0049] For example, when the motion region centroid vector module value is less than 30 pixels and the motion region overlap ratio is greater than 40%, the system generates a following avoidance buffer zone between the motion regions of the two vehicles, and limits the forklift to follow the front vehicle at low speed in the buffer zone until the overlap ratio drops below the set release threshold.

[0050] Step S5: Adjust the steering and speed of the forklift itself based on the meeting risk level to achieve obstacle avoidance.

[0051] In an embodiment, when the system determines that the meeting direction is opposite and has generated a meeting avoidance shield zone, the system adjusts the steering angle and driving speed of the forklift according to the meeting risk level. For example, when the meeting risk level is high, the system controls the forklift to slow down before reaching the meeting avoidance shield zone, and when the forklift is less than 0.5 meters from the boundary of the shield zone, the driving speed is reduced to the minimum safe speed (e.g., 0.2 meters per second), and the original direction is maintained to stop advancing to wait for the opposite vehicle to pass first. When the meeting risk level is medium, the system can control the forklift to slowly steer and offset before entering the shield zone, so that the vehicle center line gradually approaches the original channel edge, and at the same time the speed is reduced to a low speed range (e.g., 0.3 to 0.5 meters per second), thereby leaving a meeting passage space for the opposite vehicle to pass without completely stopping.

[0052] In some embodiments, when the system determines that the oncoming vehicles are in the same direction and has generated a following avoidance buffer zone, the system adjusts the following speed and longitudinal acceleration of the trolley according to the risk level of the oncoming vehicles. For example, when the risk level of the oncoming vehicles is high, the system controls the trolley to run at a speed of 0.2-0.3 m / s lower than the speed of the front vehicle within the buffer zone, and limits the longitudinal acceleration within 0.1 m / s2to maintain a safe following distance; when the risk level of the oncoming vehicles is low, the system allows the trolley to maintain a speed close to that of the front vehicle within the buffer zone, and maintains a stable spacing through slight acceleration adjustment.

[0053] As an example of the present application, reference is made to Figure 2 In this example, the step S3 comprises:

[0054] Step S31: confirming the timestamp of the motion region image, and marking the motion region image of the timestamp as the center motion region image;

[0055] Step S32: confirming the trolley motion change region image by comparing the pixel gray scale changes of the center motion region image and the motion region images of adjacent timestamps;

[0056] Step S33: obtaining the centroid position of the motion region of each frame based on the coordinate distribution of the pixels in the trolley motion change region image of the frame;

[0057] Step S34: comparing the centroid positions between adjacent frames frame by frame to obtain the change trajectory of the centroid position; determining the actual driving direction of the trolley according to the change trajectory of the centroid position;

[0058] Step S35: determining the oncoming direction of the trolley and the trolley itself in the field of view image through the driving direction.

[0059] In an embodiment, each motion region image identified in step S2 can be added with a timestamp in chronological order, and the motion region image corresponding to the current sampling time is taken as the center motion region image for subsequent direction identification processing. The center motion region image and the motion region images of the previous and next timestamps are extracted, and the gray scale changes are calculated pixel by pixel. When the gray scale change is higher than the gray scale change threshold, the corresponding position is determined as the trolley motion change region, thereby forming the trolley motion change region image.

[0060] The coordinates of all the pixels in the motion change region image are counted, and the two-dimensional geometric centroid position of the region is calculated to represent the center position of the motion region in the field of view image. The centroid positions of consecutive multiple frames are connected in chronological order to form the change trajectory of the centroid. If the change direction of the centroid tends to the left side of the image, it is determined as left driving; if it tends to the right side of the image, it is determined as right driving; if it tends to the longitudinal direction of the image, it is determined as straight driving.

[0061] In some embodiments, the system can compare the driving direction of the trolley with the moving direction of the other vehicle, and determine opposite meeting if the two directions are opposite, and determine same direction meeting if the two directions are consistent and the distance is close. The relative direction of the two vehicles is determined to provide a basis for subsequent meeting risk level determination.

[0062] Preferably, the trajectory of the change of the position of the center of mass further comprises:

[0063] The change vector of the center of mass is determined by the trajectory of the change of the position of the center of mass;

[0064] The change vector of the center of mass is time-smoothed, and when the directions of a plurality of change vectors of the center of mass are consistent, the consistent direction is taken as the stable driving direction; and when the directions of the continuous change vectors of the center of mass suddenly change, it is determined that the trolley is currently in a turning or changing direction state, and the driving direction is updated accordingly.

[0065] In an embodiment, the change vector of the center of mass can be calculated based on the coordinate difference of the positions of the center of mass of adjacent two frames, the direction of the change vector of the center of mass represents the direction of the movement of the center of mass, and the length of the vector represents the amplitude of the movement of the center of mass.

[0066] The system can count the direction distribution of the change vector of the center of mass within a preset time window (for example, 5 continuous frames), and when the directions of the vectors are in the same direction interval, the direction is recorded as the stable driving direction. When the directions of the change vectors of the center of mass of two or more continuous frames deviate from the original direction interval and there is a sudden direction mutation (for example, the deflection angle is greater than 45°), the system can determine that the trolley is in a turning or changing direction state, and the driving direction is updated according to the changed direction of the change vector of the center of mass, thereby improving the timeliness and accuracy of the determination of the driving direction.

[0067] It should be noted that in actual running scenarios, the trolley can have temporary stop, slight jitter or short-time deviation, which can cause small perturbations of the change vector of the center of mass, thereby misjudging the driving direction. Based on this, the amplitudes of the change vectors of the center of mass of a plurality of continuous frames (for example, three to five continuous frames) can be threshold determined, and when the amplitude is lower than a preset micro-motion threshold (for example, lower than 2 pixels), the system ignores the small change and does not update the driving direction. If there is a significant direction mutation after ignoring the micro-motion, it is determined that the trolley is truly turning or changing direction, and the driving direction is updated.

[0068] In addition, when the trolley is driving in a corner, a slope or a narrow passage, the change vector of the center of mass can be affected by image distortion or change of viewing angle. In a preferred embodiment, the historical driving direction of the trolley and the direction of the center line of the passage can be combined for auxiliary judgment to prevent false meeting direction judgment caused by single-frame center of mass anomaly, thereby improving the robustness of the whole meeting risk determination and obstacle avoidance decision.

[0069] Preferably, the step S35 comprises the following steps:

[0070] Step S351: match the driving direction of the current tow vehicle with the driving direction of the other tow vehicles detected in the field of view image, and determine the meeting scene when the included angle between the two directions meets the opposite relationship, and determine the meeting risk level according to the centroid change speed of the tow vehicle;

[0071] Step S352: if the included angle meets the same direction or crossing relationship, determine the non-meeting scene, and maintain the driving direction of the current tow vehicle.

[0072] In an embodiment, firstly, the driving direction of each other tow vehicle detected in the field of view image is estimated (based on the centroid change trajectory and the centroid change vector thereof). Then, the driving direction of the current tow vehicle is compared with the driving direction of the detected other tow vehicles two by two. If the included angle of the directions is close to 180 degrees (for example, the included angle is greater than 150 degrees and less than 210 degrees), the system determines that the tow vehicles are driving in opposite directions, i.e., a meeting scene.

[0073] After determining the meeting scene, the system evaluates the meeting risk level in combination with the centroid change speed of the tow vehicle (i.e., the moving speed of the centroid between consecutive frames). For example, if the centroid speed is greater than a preset high-speed threshold and the approaching distance between the two vehicles is less than a safe distance (for example, less than 1.5 meters), it is determined as a high-risk level; if the speed is lower and the approaching distance is larger, it is determined as a medium or low-risk level. The risk level is used for subsequent generation of meeting avoidance shielding area or following avoidance buffer area and speed adjustment strategy.

[0074] When the included angle of the directions is less than 30 degrees or greater than 330 degrees, the system determines that the tow vehicles are driving in the same direction; when the included angle of the directions is between 30 degrees and 150 degrees or between 210 degrees and 330 degrees, it is determined as a crossing relationship. For these non-meeting scenes, the tow vehicle does not need to generate a special avoidance area, can maintain the current lane driving direction, and continue to monitor the centroid change trajectory of the subsequent frames to update the driving direction judgment.

[0075] Preferably, determining the meeting risk level according to the centroid change speed of the tow vehicle comprises:

[0076] The centroid position data during the driving of the tow vehicle is acquired in real time, and the centroid position change speed is calculated; wherein the reasonable value range of the centroid change speed is 0.05 meters per second to 1.5 meters per second, and the sampling time interval is set to 200 milliseconds to 500 milliseconds;

[0077] When the centroid change speed is between 0.05 meters per second and 0.4 meters per second, the meeting risk level is determined as low risk; when the centroid change speed is between 0.4 meters per second and 0.9 meters per second, the meeting risk level is determined as medium risk;

[0078] When the centroid change speed is greater than 0.9 meters per second and the duration exceeds 1.5 seconds, or the centroid change speed is detected twice in succession within 0.5 seconds and exceeds 1.2 meters per second, the system determines that the risk level of the meeting vehicle is high risk.

[0079] In an embodiment, the centroid position data of the trolley is obtained in successive frames at a preset sampling time interval, which is preferably set to 200 milliseconds to 500 milliseconds, so as to ensure that the calculated speed can reflect the short-time motion characteristics of the vehicle and avoid data noise caused by too fast sampling. The centroid change speed is obtained by dividing the difference between the centroid position coordinates of adjacent frames by the time interval, and the reasonable value range is set to 0.05 meters per second to 1.5 meters per second, so as to cover the full range of the trolley running at low speed, accelerating or moving quickly for a short time.

[0080] When the centroid change speed is between 0.05 meters per second and 0.4 meters per second, the system determines that the risk level of the meeting vehicle is low risk. This range is suitable for the trolley running at low speed and smoothly, and the relative speed between the vehicles is small, so the collision risk is low. When the centroid change speed is between 0.4 meters per second and 0.9 meters per second, the system determines that the risk level of the meeting vehicle is medium risk. This range corresponds to the state of the trolley running at medium speed or approaching the oncoming vehicle, prompting the system to prepare for avoidance or adjustment strategy in advance. When the centroid change speed is greater than 0.9 meters per second and the duration exceeds 1.5 seconds, or the centroid change speed is detected twice in succession within 0.5 seconds and exceeds 1.2 meters per second, the system determines that the risk level of the meeting vehicle is high risk. This determination is used to quickly identify the scenario of the trolley approaching the oncoming vehicle at high speed, so as to ensure that the system can generate a meeting avoidance shielding area or a following avoidance buffer area in time, and adjust the speed and steering of the trolley to prevent collision.

[0081] Preferably, when the meeting direction is the same direction in step S4, the following avoidance buffer area is generated according to the centroid vector modulus value in the motion region image and the region overlap ratio, which includes:

[0082] When the meeting direction is the same direction, the motion region contours of the trolley and the oncoming vehicle are extracted in the motion region image, and the first centroid position and the second centroid position are calculated by calculating the centroid positions.

[0083] The motion trajectories of the centroids of the two vehicles are confirmed based on the first centroid position and the second centroid position, and the modulus values of the respective centroid vectors are calculated according to the continuous displacement changes of the centroids.

[0084] Region overlap detection is performed on the motion regions of the two vehicles in the current frame, the proportion of the overlapping area of the two regions to the motion region of the trolley is calculated, and the region overlap ratio is obtained.

[0085] The region range size of the buffer area is set according to the modulus value of the centroid vector and the region overlap ratio, and the following avoidance buffer area is obtained.

[0086] In an embodiment, the contours of the host vehicle and the approaching vehicle in the motion area image can be identified by image segmentation or pixel classification techniques, and the contour pixel coordinates can be counted. The contour pixel coordinates can be averaged to obtain the centroid positions of the host vehicle and the approaching vehicle, which are denoted as the first centroid position and the second centroid position, respectively.

[0087] The centroid vector can be calculated by the changes of the centroid positions in consecutive frames. The direction of the centroid vector represents the motion direction of the vehicle, and the modulus represents the motion speed of the vehicle. The larger the modulus is, the faster the vehicle moves, which can be used for the subsequent dynamic adjustment of the buffer size.

[0088] The intersection area of the motion area of the host vehicle and the motion area of the approaching vehicle can be calculated by pixel-level comparison, and then divided by the total area of the motion area of the host vehicle to obtain the overlap ratio. The overlap ratio can reflect the proximity of the two vehicles in space. The larger the overlap ratio is, the closer the distance between the vehicles is, and a larger buffer is needed. By mapping the centroid vector modulus and the area overlap ratio to the buffer length and width, for example, the buffer length is elongated as the centroid modulus increases, and the buffer width is widened as the area overlap ratio increases. The preferred buffer length ranges from 1 meter to 5 meters, and the width ranges from 0.5 meters to 2 meters, to ensure that the buffer can cover the potential collision risk and not affect the passage of other vehicles.

[0089] Preferably, step S5 comprises the following steps:

[0090] setting the current driving state of the host vehicle according to the centroid vector modulus;

[0091] setting the proximity of the host vehicle and the approaching vehicle according to the area overlap ratio;

[0092] selecting the corresponding buffer range based on the current driving state and the proximity;

[0093] drawing a buffer area corresponding to the buffer range in the front of the field of view image based on the driving direction of the host vehicle, to form a following avoidance buffer.

[0094] In an embodiment, the vehicle driving speed and acceleration information can be reflected by calculating the modulus of the centroid position change vector of the host vehicle in consecutive frames. A smaller centroid vector modulus indicates that the host vehicle is in a low-speed or slow-driving state, and a larger centroid vector modulus indicates that the host vehicle is in a high-speed driving state. The system divides the modulus into several driving state levels, such as low speed, normal, medium speed, and high speed, for subsequent buffer size selection.

[0095] The area overlap ratio is obtained by calculating the proportion of the intersection area of the vehicle motion area and the other vehicle motion area in the total area of the vehicle motion area. The larger the area overlap ratio, the higher the degree of spatial proximity between the two vehicles. The system can divide the area overlap ratio into low, medium and high levels, which are used for dynamic adjustment of the buffer zone range.

[0096] Different combinations of buffer zone size tables can be preset, for example: when the travel state is low speed and the proximity is low, the buffer zone length is 1 meter and the width is 0.5 meters; when the travel state is medium speed and the proximity is high, the buffer zone length is 3 meters and the width is 1.5 meters; when the travel state is high speed and the proximity is high, the buffer zone length is 5 meters and the width is 2 meters. Through the combination of travel state and proximity, the system dynamically selects the most suitable buffer zone range to balance safety and traffic efficiency.

[0097] The selected buffer zone length is extended along the direction of travel of the trolley, and the width is expanded along the direction perpendicular to the direction of travel to generate a buffer area. If a vehicle or obstacle is detected in the buffer area, the system can trigger speed adjustment or avoidance action.

[0098] Preferably, the corresponding buffer zone range is selected based on the combination of the level of the current travel state and the level of the proximity, which includes:

[0099] The current travel state of the trolley is determined based on the centroid change speed, wherein 0.05 meters per second to 0.4 meters per second is a low speed state, 0.4 meters per second to 0.9 meters per second is a normal state, and greater than 0.9 meters per second is a high speed state;

[0100] The proximity is determined according to the area overlap ratio of the trolley and the other vehicle in the channel, wherein the overlap ratio less than 20% is low proximity, 20% to 50% is medium proximity, and higher than 50% is high proximity;

[0101] The corresponding buffer zone range is selected in combination with the current travel state and the proximity, wherein the buffer zone range includes 1.0 meters to 2.0 meters for low speed and low proximity, 2.0 meters to 3.5 meters for normal and medium proximity, and 3.5 meters to 5.0 meters for high speed or high proximity.

[0102] In an embodiment, the centroid position data of the vehicle in consecutive frames is obtained, and the centroid continuous displacement change speed is calculated: when the centroid change speed is between 0.05 meters per second and 0.4 meters per second, the trolley is determined to be in a low speed state; when the centroid change speed is between 0.4 meters per second and 0.9 meters per second, it is determined to be in a normal state; when the centroid change speed is greater than 0.9 meters per second, it is determined to be in a high speed state. This division can cover the low speed cruise, medium speed operation and high speed emergency movement scenarios of the trolley, ensuring that the buffer zone can adapt to the safety distance requirements at different speeds.

[0103] The area overlap ratio is obtained by calculating the proportion of the intersection area of the vehicle motion area and the other vehicle motion area to the total area of the vehicle motion area: when the overlap ratio is less than 20%, it is determined as low proximity; when the overlap ratio is between 20% and 50%, it is determined as medium proximity; and when the overlap ratio is higher than 50%, it is determined as high proximity. The proximity can intuitively reflect the space tightness between the forklift and the other vehicle, and provide a basis for adjusting the length and width of the buffer zone.

[0104] The buffer zone range is preset according to the combination of the travel state and the proximity: low speed state and low proximity: the buffer zone length is set to 1.0-2.0 meters; normal state and medium proximity: the buffer zone length is set to 2.0-3.5 meters; high speed state or high proximity: the buffer zone length is set to 3.5-5.0 meters. The buffer zone width can be adjusted according to the width of the vehicle itself and the safety distance, and the preferred range is 0.5-2.0 meters, so as to ensure that the buffer zone covers the potential collision risk area, while not affecting the passage of other forklifts.

[0105] Taking the driving direction of the forklift as the reference, the selected buffer zone is extended in the forward direction and appropriately expanded in the lateral direction to form a complete buffer area. In order to improve the stability of the buffer zone, the buffer zone size can be smoothed for continuous multiple frames of results, so as to avoid frequent changes of the buffer zone caused by short-term centroid speed fluctuations or area overlap errors, and ensure the safety and reliability of the following avoidance.

[0106] Preferably, the step S5 of adjusting the steering and speed of the forklift to realize obstacle avoidance is specifically: adjusting the speed and direction of the forklift under the instruction of the vehicle-mounted controller through the driving motor and the steering mechanism, to realize the obstacle avoidance action based on the risk level.

[0107] In an embodiment, the risk level of the meeting or the size information of the following avoidance buffer zone obtained according to the foregoing steps is used to judge the obstacle avoidance priority and the emergency degree. For example: when the risk level is high or the buffer zone is close to the upper limit, an emergency deceleration or sharp turning action is triggered; when the risk level is medium, a smooth deceleration or slight direction adjustment is triggered; and when the risk level is low, only the speed is fine-tuned or the original driving direction is maintained.

[0108] The target speed and acceleration are calculated according to the obstacle avoidance action, and instructions are sent to the driving motor control module to realize the acceleration and deceleration of the forklift. In a preferred embodiment, the acceleration adjustment range is 0.2-0.8 m / s2, so as to ensure the smooth obstacle avoidance of the forklift, and avoid damage to the equipment or goods caused by excessive emergency operation. The steering angle and its change rate are calculated according to the obstacle avoidance requirement, and the steering mechanism is controlled to realize the direction adjustment. In a preferred embodiment, the steering angle adjustment range is ±15 degrees, and the steering angle change rate is controlled within 10 degrees per second, so as to ensure the smoothness of the turning action and avoid the risk of excessive deviation or rollover.

[0109] The system simultaneously adjusts the speed and direction to make the trolley travel along the preset trajectory of the buffer area, avoiding collision with incoming trolleys or obstacles. The trolley state and buffer area changes are monitored in real time in consecutive frames, and the speed and direction are dynamically corrected to ensure continuous, smooth and safe and reliable obstacle avoidance.

[0110] It should be noted that in some embodiments, the following special cases should also be considered to ensure the safety and reliability of the trolley in complex or abnormal scenarios: when the trolley is in a low-speed stable state but detects a stationary obstacle in the front buffer area, the system can preferentially perform the action of reducing the speed to zero and slightly turning to prevent collision, while waiting for the obstacle to be cleared or re-planning the path; when the trolley encounters a situation where a trolley moving in the same direction approaches but the buffer area does not reach the maximum value, it can achieve smooth avoidance by gradually reducing speed and fine-tuning the steering, avoiding the impact of sudden stopping or sudden turning on the goods and the trolley itself; when the trolley is in a narrow passage or complex intersection area, the system can combine inertial sensor and ultrasonic / laser ranging sensor data to jointly close-loop adjust the speed and steering, ensuring stable passage even in the case of centroid position mutation or sensor data fluctuation; when communication delay or command anomaly is detected, the trolley can temporarily enter a safe parking mode, maintaining the current driving state or slowly reducing speed, waiting for communication to be restored before performing obstacle avoidance action; the system can predict and smooth the obstacle avoidance trajectory for consecutive multiple frames to prevent frequent buffer area adjustments due to short-range ranging or centroid fluctuation, ensuring smooth and reliable obstacle avoidance action.

[0111] Especially important is that step S2 further comprises the following steps:

[0112] Step S21: Traverse each pixel point in the field of view image set, calculate the gray value difference between consecutive frames, and obtain the gray change sequence of each pixel point;

[0113] Step S22: Differentiate the gray change sequence by time difference, calculate the gray change rate of each pixel point, and record its size and change direction; perform spatial statistical analysis on the gray change rate of the field of view image to generate a gray change rate distribution map;

[0114] Step S23: According to the spatial distribution characteristics of the gray change rate in the gray change rate distribution map, identify the pixel block with significant and consistent gray change in the continuous area, and determine it as a possible trolley self-motion area;

[0115] Step S24: Perform morphological processing on the identified candidate motion area to remove noise points and fill holes to obtain a complete and continuous trolley motion area image.

[0116] In an embodiment, each pixel in the set of field-of-view images is traversed, and a difference in gray scale values between consecutive frames is calculated to obtain a sequence of gray scale changes for each pixel. The difference in gray scale values can be obtained by subtracting the gray scale of the previous frame from the current frame, and the result can be positive or negative, indicating an increase or decrease in brightness of the pixel. The sequence of gray scale changes is used to describe the brightness fluctuation of the pixel over time, providing basic data for identifying the moving region.

[0117] The sequence of gray scale changes is differentiated with respect to time to obtain a rate of change of gray scale per unit time for each pixel, and the direction of change is recorded, where the rate of change can represent the change amplitude using an absolute value, and the direction can represent a trend of brightness increase or decrease. Spatial statistical analysis is performed on the rate of change of gray scale of the entire field-of-view image, such as calculating the average rate of change of gray scale in a local neighborhood, to generate a distribution map of the rate of change of gray scale. The distribution map can reflect the region of pixel brightness change caused by the movement of the trolley, facilitating subsequent identification of the moving region.

[0118] According to the spatial distribution characteristics of the rate of change of gray scale in the distribution map of the rate of change of gray scale, a pixel block with a significant and consistent change in gray scale in a continuous region is identified, and is determined as a candidate trolley self-moving region. A threshold of the rate of change of gray scale can be set to filter small amplitude changes and reduce the interference of ambient light or noise on the identification result. The determination of the continuous region can be based on a connectivity analysis of the pixel neighborhood to ensure that the identified region is a continuous block rather than discrete points.

[0119] Morphological processing is performed on the identified candidate moving region, including erosion, dilation, opening operation, and closing operation, to remove noise points and fill holes, obtaining a complete and continuous trolley moving region image. The morphological processing can eliminate small area noise points, while filling the holes caused by light or obstruction to ensure the continuity and smoothness of the moving region image. The final moving region image can be used as input for subsequent centroid calculation and travel direction determination to ensure accuracy and reliability.

[0120] In particular, step S23 further comprises the following steps:

[0121] Step S231: region division is performed on the distribution map of the rate of change of gray scale, dividing the image into a plurality of small candidate blocks; the mean and variance of the rate of change of gray scale of the pixels in each candidate block are calculated to determine whether the gray scale change is significant and consistent;

[0122] Step S232: connectivity detection is performed on the candidate blocks that meet the conditions of significance and consistency, merging the blocks that are spatially adjacent and similar in features to form a preliminary continuous region; morphological processing is performed on the preliminary continuous region, including noise removal and small hole filling, to obtain a processed continuous region;

[0123] Step S233: boundary extraction is performed on the processed continuous region, the spatial position and shape characteristics of each region are recorded, it is determined that the region is a vehicle self-motion region, and the determined vehicle motion region is output.

[0124] In an embodiment, the gray scale change rate distribution map can be regionally divided, and the image is divided into a plurality of small candidate blocks, for example, each block has a size of 8x8 to 16x16 pixels; the mean value and variance of the pixel gray scale change rate in each candidate block are counted as the gray scale change characteristic index of the block; it is determined whether the gray scale change is significant and consistent, for example, the candidate block with a mean value exceeding a set threshold and a variance lower than a set threshold can be considered as significant and consistent; this operation can filter out the gray scale fluctuations caused by ambient light, shadow or local noise, and only retain the pixel changes caused by the vehicle self-motion.

[0125] The candidate blocks meeting the significance and consistency conditions are subjected to connectivity detection: the blocks that are spatially adjacent and have similar gray scale change characteristics are merged to form a preliminary continuous region; the preliminary continuous region is subjected to morphological processing, including erosion denoising, inflation hole filling, opening and closing operation, and the like, to remove small area noise and fill small holes, so as to obtain a processed continuous region; the connectivity detection and morphological processing ensure that the recognized motion region is continuous and complete, facilitating subsequent centroid calculation.

[0126] The boundary of the processed continuous region is extracted, and the spatial position and shape characteristics of each region are recorded; for example, the boundary coordinates or the minimum circumscribed rectangle and the like can be used to describe the geometric shape of the region; the boundary information and the spatial position are marked and output, and it is determined that the region is a vehicle self-motion region, which is used in subsequent steps such as centroid calculation and travel direction determination.

[0127] In the present specification, a machine vision assisted automatic guided vehicle obstacle avoidance system is provided for performing the machine vision assisted automatic guided vehicle obstacle avoidance method described above, and the machine vision assisted automatic guided vehicle obstacle avoidance system comprises:

[0128] An image acquisition module is configured to acquire a set of field of view images of the automatic guided vehicle, wherein the set of field of view images comprises a plurality of field of view images.

[0129] A region division module is configured to extract the difference between the gray scale value and the time sequence of each pixel point in the set of field of view images, to obtain the gray scale change rate of the pixel point; and to confirm the motion region image of the vehicle itself according to the spatial distribution of the gray scale change rate.

[0130] A direction confirmation module is configured to confirm the travel direction of the vehicle itself by the centroid vector change of the pixels in the motion region image, and to determine the meeting direction of the vehicle in the field of view image and the collision risk level according to the travel direction.

[0131] The meeting avoidance module is configured to generate a meeting avoidance shielding area according to an angle between a center of mass vector and a center line and a region expansion rate in a moving area image when a meeting direction is opposite, and to generate a following avoidance buffer area according to a center of mass vector module value and a region overlap ratio in the moving area image when the meeting direction is same.

[0132] The obstacle avoidance control module is configured to adjust a steering and a speed of the carrier vehicle to realize obstacle avoidance based on a meeting risk level for the meeting avoidance shielding area or the following avoidance buffer area.

[0133] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the application being defined by the appended claims and not by the above description, and all changes falling within the meaning and range of equivalence of the elements of the patent file are therefore intended to be embraced by the application.

[0134] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Various modifications to the embodiments will be readily apparent to persons having ordinary skill in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A machine vision-assisted obstacle avoidance method for automated guided transport vehicles, characterized in that, Includes the following steps: Step S1: Obtain the field-of-view image set of the automated guided vehicle, wherein the field-of-view image set contains several field-of-view images; Step S2: Extract the grayscale value and time series difference of each pixel in the field of view image set to obtain the grayscale change rate of the pixel; confirm the motion area image of the transport vehicle itself based on the spatial distribution of the grayscale change rate; Step S3: Determine the driving direction of the transport vehicle itself by the change of the centroid vector of the pixels in the motion area image, and determine the meeting direction of the transport vehicle and the transport vehicle itself in the field of view image based on the driving direction and confirm the meeting risk level. Step S4: When the oncoming traffic is in the direction of the meeting, a meeting avoidance shielding zone is generated based on the angle between the centroid vector and the center line in the motion region image and the region expansion rate; when the oncoming traffic is in the same direction, a following avoidance buffer zone is generated based on the centroid vector magnitude in the motion region image and the region overlap ratio. Step S5: Based on the risk level of passing vehicles, adjust the steering and speed of the transport vehicle to achieve obstacle avoidance in the passing avoidance shield zone or the following avoidance buffer zone.

2. The obstacle avoidance method for automated guided transport vehicles assisted by machine vision according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Confirm the timestamp of the motion region image and mark the motion region image with that timestamp as the center motion region image; Step S32: By comparing the pixel grayscale changes of the central motion region image with the motion region images of adjacent time stamps, the image of the area where the transport vehicle's motion changes is confirmed; Step S33: Based on the coordinate distribution of pixels in the image of the moving area of ​​the transport vehicle in each frame, obtain the centroid position of the moving area in that frame; Step S34: Compare the centroid positions between adjacent frames frame by frame to obtain the trajectory of the centroid position change; determine the actual driving direction of the transport vehicle based on the trajectory of the centroid position change. Step S35: Determine the direction of travel between the transport vehicle and itself in the field of view image based on the direction of travel.

3. The obstacle avoidance method for automated guided transport vehicles assisted by machine vision according to claim 2, characterized in that, Obtaining the trajectory of the change in the centroid position also includes: The centroid change vector is determined by the trajectory of the change in the centroid's position. The centroid change vector is smoothed over time. When multiple centroid change vectors are in the same direction, the corresponding consistent direction is taken as the stable driving direction. If the direction of consecutive centroid change vectors changes abruptly, it is determined that the transport vehicle is currently in a U-turn or change of direction state, and the driving direction is updated accordingly.

4. The obstacle avoidance method for automated guided transport vehicles assisted by machine vision according to claim 2, characterized in that, Step S35 includes the following steps: Step S351: Match the current travel direction of the transport vehicle with the travel directions of other transport vehicles detected in the field of view image. When the angle between the two directions satisfies the opposite direction relationship, it is determined to be a meeting scenario, and the meeting risk level is determined according to the change speed of the transport vehicle's center of gravity. Step S352: If the included angle satisfies the same direction or cross relationship, it is determined to be a non-meeting scenario, and the current driving direction of the transport vehicle is maintained.

5. The obstacle avoidance method for an automated guided transport vehicle assisted by machine vision according to claim 4, characterized in that, Determining the risk level of a collision based on the rate of change of the center of gravity of the transport vehicle includes: The system acquires the center of gravity position data of the transport vehicle in real time during its movement and calculates the rate of change of the center of gravity position; the reasonable range of the rate of change of the center of gravity is 0.05 meters per second to 1.5 meters per second, and the sampling time interval is set to 200 milliseconds to 500 milliseconds. When the velocity of the center of gravity changes between 0.05 meters per second and 0.4 meters per second, the risk level of passing other vehicles is determined to be low; when the velocity of the center of gravity changes between 0.4 meters per second and 0.9 meters per second, the risk level of passing other vehicles is determined to be medium. When the velocity of the center of gravity changes by more than 0.9 meters per second and lasts for more than 1.5 seconds, or when the velocity of the center of gravity changes by more than 1.2 meters per second is detected twice within 0.5 seconds, the risk level of passing vehicles is determined to be high.

6. The obstacle avoidance method for an automated guided transport vehicle assisted by machine vision according to claim 1, characterized in that, In step S4, when the oncoming traffic is coming from the same direction, a following avoidance buffer is generated based on the centroid vector magnitude and region overlap ratio in the motion region image, including: When the oncoming vehicles are traveling in the same direction, the contours of the moving regions of the vehicle and the oncoming vehicle are extracted from the moving region image, and the centroid positions are calculated to obtain the first centroid position and the second centroid position. The motion trajectories of the two vehicle centers of mass are determined based on the first and second center of mass positions, and the magnitude of their respective center of mass vectors is calculated based on the continuous displacement changes of the center of mass. In the current frame, perform region overlap detection on the motion regions of the two vehicles, calculate the proportion of the overlapping area of ​​the two regions to the motion region of the vehicle itself, and obtain the region overlap ratio. The size of the buffer zone is determined by setting the magnitude of the centroid vector and the region overlap ratio, thus obtaining the following avoidance buffer zone.

7. The obstacle avoidance method for an automated guided transport vehicle assisted by machine vision according to claim 6, characterized in that, Step S5 includes the following steps: The current travel state of this vehicle is set according to the magnitude of the centroid vector; The proximity of this vehicle and the approaching vehicle is set according to the area overlap ratio; The corresponding buffer range is selected based on the current travel status and the degree of proximity. Based on the direction of travel of the transport vehicle, a buffer zone corresponding to the buffer zone range is drawn in front of the field of view image to form a following and avoidance buffer zone.

8. The obstacle avoidance method for an automated guided transport vehicle assisted by machine vision according to claim 7, characterized in that, The corresponding buffer range is selected based on the combination of the current travel status level and the proximity level, including: The current travel status of the transport vehicle is determined based on the change speed of its center of gravity. The speed range is 0.05 meters per second to 0.4 meters per second, which is considered a low-speed state; 0.4 meters per second to 0.9 meters per second, which is considered a normal state; and greater than 0.9 meters per second, which is considered a high-speed state. The degree of proximity is determined by the overlap ratio between the areas where the transport vehicle and the incoming vehicle are located in the passageway. An overlap ratio of less than 20% is considered low proximity, 20% to 50% is considered medium proximity, and more than 50% is considered high proximity. Based on the combination of the current travel status and the degree of approach, select the corresponding buffer range. The buffer range is set as follows: low speed and low approach: 1.0 meter to 2.0 meter; normal and medium approach: 2.0 meter to 3.5 meter; high speed or high approach: 3.5 meter to 5.0 meter.

9. The obstacle avoidance method for an automated guided transport vehicle assisted by machine vision according to claim 1, characterized in that, In step S5, adjusting the steering and speed of the transport vehicle to achieve obstacle avoidance specifically involves adjusting the speed and direction of the transport vehicle under the command of the on-board controller via the drive motor and steering mechanism to achieve obstacle avoidance action based on risk level.

10. A machine vision-assisted obstacle avoidance system for an automated guided transport vehicle, characterized in that, For performing the machine vision-assisted obstacle avoidance method for automated guided vehicles as described in claim 1, the machine vision-assisted obstacle avoidance system for automated guided vehicles includes: The image acquisition module is used to acquire a set of field-of-view images of the automated guided vehicle, wherein the set of field-of-view images contains several field-of-view images; The region segmentation module is used to extract the gray value and time series difference of each pixel in the field image set to obtain the gray value change rate of the pixel; and to identify the motion area image of the transport vehicle itself based on the spatial distribution of the gray value change rate. The direction confirmation module is used to confirm the driving direction of the transport vehicle itself by the change of the centroid vector of the pixels in the motion area image, and to determine the meeting direction of the transport vehicle and the transport vehicle itself in the field of view image based on the driving direction and to confirm the meeting risk level. The oncoming traffic avoidance module is used to generate an oncoming traffic avoidance shielding zone based on the angle between the centroid vector and the center line in the motion region image and the region expansion rate when the oncoming traffic is from the opposite direction; and to generate a following avoidance buffer zone based on the magnitude of the centroid vector in the motion region image and the region overlap ratio when the oncoming traffic is from the same direction. The obstacle avoidance control module is used to adjust the steering and speed of the transport vehicle based on the obstacle avoidance shielding zone or the following obstacle avoidance buffer zone to achieve obstacle avoidance.

Images