Industrial vehicle pedestrian anti-collision detection and early warning system based on 5G RedCap module

Abstract

The invention provides an industrial vehicle pedestrian anti-collision detection and early warning system based on a 5G RedCap module. The industrial vehicle pedestrian anti-collision detection and early warning system comprises a camera which carries out real-time video shooting on an operation environment of an industrial vehicle to form an environment video stream; the distance sensor measures the real-time distance and the real-time relative movement speed between the tracked target object and the industrial vehicle; on the edge computing terminal, a target recognition model recognizes a target object from the environment video stream, a target tracking model tracks the recognized target object, and a collision risk recognition model predicts whether a collision risk occurs between the target object and the industrial vehicle or not according to the real-time distance and the real-time relative movement speed between the target object and the industrial vehicle; when the collision risk is predicted, a risk early warning strategy is generated to perform braking or deceleration control on the industrial vehicle; and the 5G RedCap module sends the environment video stream, the real-time distance between the target object and the industrial vehicle, the real-time relative movement speed, the collision risk prediction result and the risk early warning strategy to a background server.

Description

Technical Field

[0001] The present application relates to the fields of Industrial Internet of Things (AIoT) and intelligent traffic management, and in particular to an industrial vehicle pedestrian collision avoidance detection and warning system based on a 5GRedCap module. Background Art

[0002] With the rapid development of industrial automation, the internal environment of modern factories is becoming more and more complex, and the phenomenon of special vehicles and pedestrians coexisting densely frequently occurs. How to effectively ensure the safety of vehicles and pedestrians has become an important challenge. Existing anti-collision systems mostly rely on traditional sensors and processors, which often have delays in real-time response and data processing, resulting in poor anti-collision effects. Summary of the invention

[0003] The purpose of this application is to propose an industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module to solve or alleviate the technical problems existing in the prior art.

[0004] An industrial vehicle pedestrian collision avoidance detection and warning system based on a 5G RedCap module, comprising:

[0005] A camera, used for capturing a real-time video of the working environment of the industrial vehicle to form an environmental video stream;

[0006] A distance sensor, used to measure the real-time distance and real-time relative moving speed between the tracked target and the industrial vehicle;

[0007] an edge computing terminal, on which a target recognition model, a target tracking model, and an early warning model are deployed, wherein the target recognition model is used to recognize a target object from the environmental video stream, the target tracking model is used to track the recognized target object, and the collision risk recognition model is used to predict whether there is a risk of collision between the target object and the industrial vehicle according to the real-time distance and real-time relative moving speed between the target object and the industrial vehicle, and when the collision risk is predicted, generate a risk early warning strategy to brake or decelerate the industrial vehicle;

[0008] The 5G RedCap module is used to send the environmental video stream, the real-time distance between the target object and the industrial vehicle, the real-time relative moving speed, the prediction result of the collision risk, and the risk warning strategy to the background server.

[0009] Optionally, the number of the cameras is at least 4, so that at least one camera is installed in each of the four directions of the front, back, left, and right of the industrial vehicle to perform panoramic video capture of the working environment of the industrial vehicle.

[0010] Optionally, the edge computing terminal is also used to generate an alarm signal when the collision risk is predicted.

[0011] Optionally, the industrial vehicle pedestrian avoidance detection and warning system based on the 5G RedCap module further includes: at least one of a warning light and a buzzer, the warning light being used to generate a warning light signal under the drive of the alarm signal, and the buzzer being used to generate a warning sound signal under the drive of the alarm signal.

[0012] Optionally, the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module also includes: a remote monitoring center connected to the background server to receive the monitoring data sent by the background server and perform real-time analysis.

[0013] Optionally, the target recognition model is used to perform the following steps to recognize a target object from the environment video stream:

[0014] Decomposing the environment video stream into single video frames to form an original video frame sequence;

[0015] Preprocessing the original video frame sequence to filter out key frames and form a key video frame sequence;

[0016] Performing illumination normalization processing on the key video frame sequence and then extracting multi-scale features to obtain a multi-scale feature map;

[0017] Fusing the multi-scale feature maps to obtain a feature representation of the target object;

[0018] The target feature representation is classified to identify the target object from the environment video stream.

[0019] Optionally, performing illumination normalization processing on the key video frame sequence includes:

[0020] Performing adaptive histogram equalization on each key frame in the key video frame sequence to obtain an adaptive equalized histogram;

[0021] Performing illumination compensation on the adaptive equalization histogram to obtain an illumination-compensated image frame;

[0022] Nonlinear mapping is performed on the illumination-compensated image frame to obtain an illumination-normalized image frame.

[0023] Optionally, based on Adaptive histogram equalization is performed on each key frame in the key video frame sequence to obtain an adaptive equalized histogram, where I adapt(x, y) represents the pixel value at the position (x, y) after adaptive histogram equalization, L represents the possible brightness level of the image, N represents the total number of pixels in the local area, g(k) represents the number of pixels at the kth brightness level in the local area, that is, the local histogram, and the local area is defined by a window in the image, which surrounds the current pixel (x, y) and includes the pixels around (x, y);

[0024] Based on I comp (x, y) = I adapt (x, y)-log(I illum (x, y)) performs illumination compensation on the adaptive equalization histogram to obtain an image frame after illumination compensation, wherein I comp (x, y) represents the pixel value after illumination compensation at the position (x, y), I illum (x, y) represents the estimated illumination component at the position (x, y), which can be estimated by analyzing the high brightness area in the image;

[0025] Based on I nl (x, y) = α·tanh(β·I comp (x, y)) performs nonlinear mapping on the image frame after illumination compensation to obtain a mapped image frame, wherein I nl (x, y) represents the pixel value at the position (x, y) after nonlinear mapping.

[0026] Tanh represents the hyperbolic tangent function, which is used to map the compensated pixel value into a preset range to enhance the contrast. α and β are used to adjust the intensity of the nonlinear mapping. α controls the range of the mapped value, and β controls the sensitivity of the mapping.

[0027] based on The mapping image frame is contrast enhanced to obtain a normalized illumination image frame, I lce (x, y) represents the pixel value after local contrast enhancement at the position (x, y), min(I nl) and max(I nl ) represent the minimum and maximum values ​​of the image after nonlinear mapping.

[0028] Optionally, the target tracking model is used to perform the following steps to track the identified target object:

[0029] Extracting a color histogram, edge features, and texture features from the environment video stream to locate a target object in the environment video stream;

[0030] The target object and the target object are predicted based on Calculating a cost function between the target object and the target object, and They represent the predicted position and speed of the target object i obtained by predicting the state of the target object; p x，j and v x，j is the predicted position and speed of the target object j obtained by predicting the state of the target object; σ p and σ v is the standard deviation of the position and velocity measurement noise, determined by describing the noise characteristics of the camera; w p and w v is a weight factor used to balance the importance of position and speed in cost calculation;

[0031] Based on the minimization of the cost function, the identified target object is tracked.

[0032] Optionally, the collision risk identification model is used to predict whether there is a risk of collision between the target object and the industrial vehicle according to the real-time distance and the real-time relative moving speed between the target object and the industrial vehicle, and generate a risk warning strategy when the collision risk is predicted, so as to brake or decelerate the industrial vehicle based on Predict the remaining time before the collision occurs, d(t) is the distance between the target object and the vehicle at the current moment, v rel (t) is the relative speed; by setting a threshold TTC threshold To determine whether there is a collision risk, if TTC(t)<TTC threshold , it is considered that there is a collision risk, and based on Generate the corresponding warning strategy, Δv(t) is the deceleration that needs to be applied, k p and k d are the proportional and differential control coefficients used to adjust the braking intensity.

[0033] The solution of the embodiment of the present application effectively solves the safety and collision avoidance problem when special vehicles and pedestrians densely coexist in an industrial environment, as follows:

[0034] (1) Real-time video monitoring: Use cameras to capture real-time video of the working environment of industrial vehicles to form an environmental video stream. This method can intuitively reflect the dynamic changes of the vehicle's surrounding environment and provide basic data for subsequent target recognition and tracking.

[0035] (2) High-precision distance and speed measurement: Distance sensors are used to measure the distance and relative moving speed between the tracked target (such as pedestrians, other vehicles, etc.) and the industrial vehicle in real time. This data is crucial for accurately predicting collision risks because it directly reflects the spatial relationship and dynamic changes between the target and the vehicle.

[0036] (3) Edge computing and intelligent analysis: Target recognition models, target tracking models, and collision risk recognition models are deployed on edge computing terminals. These models use environmental video streams and distance sensor data for real-time processing to extract targets from videos, track their motion trajectories, and predict collision risks based on real-time distance and speed. The application of edge computing reduces data transmission delays and improves the system's real-time response capabilities.

[0037] (4) Risk warning and automatic control: When the collision risk identification model predicts a potential collision risk, a risk warning strategy is immediately generated. These strategies may include controlling the industrial vehicle to brake or slow down, thereby effectively avoiding the occurrence of collision accidents. This automated control mechanism can quickly respond to potential threats and improve safety.

[0038] (5) Efficient communication of 5G RedCap module: The 5G RedCap module is used to quickly and reliably send key data (including environmental video streams, real-time distance and speed data, collision risk prediction results, risk warning strategies, etc.) to the background server. As a lightweight version of the 5G network, 5G RedCap is designed for IoT scenarios. It can reduce power consumption and costs while ensuring data transmission speed, and is very suitable for the application of industrial vehicle collision avoidance detection and warning systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Hereinafter, some specific embodiments of the present application will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0040] Figure 1 This is a structural schematic diagram of an industrial vehicle pedestrian avoidance detection and warning system based on a 5G RedCap module according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0042] Figure 1 This is a schematic diagram of the structure of an industrial vehicle pedestrian collision avoidance detection and warning system based on a 5G RedCap module according to an embodiment of the present application. Figure 1 As shown, it includes:

[0043] A camera, used for capturing a real-time video of the working environment of the industrial vehicle to form an environmental video stream;

[0044] A distance sensor, used to measure the real-time distance and real-time relative moving speed between the tracked target and the industrial vehicle;

[0045] an edge computing terminal, on which a target recognition model, a target tracking model, and an early warning model are deployed, wherein the target recognition model is used to recognize a target object from the environmental video stream, the target tracking model is used to track the recognized target object, and the collision risk recognition model is used to predict whether there is a risk of collision between the target object and the industrial vehicle according to the real-time distance and real-time relative moving speed between the target object and the industrial vehicle, and when the collision risk is predicted, generate a risk early warning strategy to brake or decelerate the industrial vehicle;

[0046] The 5G RedCap module is used to send the environmental video stream, the real-time distance between the target object and the industrial vehicle, the real-time relative moving speed, the prediction result of the collision risk, and the risk warning strategy to the background server.

[0047] Optionally, the number of the cameras is at least 4, so that at least one camera is installed in each of the four directions of the front, back, left, and right of the industrial vehicle to perform panoramic video capture of the working environment of the industrial vehicle.

[0048] To this end, at least one camera is installed in the front, back, left, and right directions of the industrial vehicle to achieve panoramic video shooting of the working environment. This technical choice brings many significant benefits, which are specifically analyzed as follows in combination with the technical solution:

[0049] (1) All-round visual coverage: Through the layout of cameras in four directions, the system can obtain 360-degree panoramic video around the industrial vehicle. This all-round visual coverage greatly reduces blind spots, allowing potential obstacles or pedestrians in any direction to be captured in a timely manner, providing comprehensive data support for subsequent target recognition and tracking.

[0050] (2) Improve the accuracy of target recognition and tracking: Since the camera can capture images from multiple directions at the same time, the target recognition model can be analyzed based on richer data, thereby improving the accuracy and robustness of target recognition. Similarly, the target tracking model can more accurately track the movement trajectory of the target object and reduce tracking loss caused by perspective switching or occlusion.

[0051] (3) Enhanced collision risk prediction capabilities: Panoramic video capture enables the system to more comprehensively assess the spatial relationship and dynamic changes between the target and the industrial vehicle. Combined with real-time distance and relative movement speed data, the collision risk identification model can more accurately predict potential collision risks and take early warning and control measures.

[0052] (4) Improve emergency response speed: In an emergency, panoramic video surveillance can quickly locate the source of danger and trigger the corresponding emergency response mechanism. For example, when a pedestrian is detected suddenly in front of the vehicle, the system can immediately initiate braking or deceleration control, thereby effectively avoiding a collision accident.

[0053] (5) Optimizing driving assistance decisions: Panoramic video streams can also serve as an important input to the driving assistance system, providing drivers with more intuitive information about the surrounding environment. By integrating advanced image processing technology and artificial intelligence algorithms, the system can further analyze video content and provide drivers with auxiliary decision-making support such as road condition warnings and obstacle avoidance.

[0054] (6) Improve system reliability and redundancy: The simultaneous operation of multiple cameras also improves system reliability and redundancy. Even if a camera fails or is blocked, other cameras can still continue to work, ensuring the normal operation of the entire system.

[0055] Optionally, the edge computing terminal is also used to generate an alarm signal when the collision risk is predicted.

[0056] Optionally, the industrial vehicle pedestrian avoidance detection and warning system based on the 5G RedCap module further includes: at least one of a warning light and a buzzer, the warning light being used to generate a warning light signal under the drive of the alarm signal, and the buzzer being used to generate a warning sound signal under the drive of the alarm signal.

[0057] In the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module, the edge computing terminal is introduced to generate an alarm signal when a collision risk is predicted, and a warning light and / or buzzer are further equipped as an alarm device. This technical choice brings many significant benefits, which are specifically analyzed in combination with the technical solution as follows:

[0058] (1) Instant warning: When the edge computing terminal predicts a collision risk, it immediately generates an alarm signal and drives the warning light and / or buzzer. This instant warning can quickly attract the attention of the driver and other people around, alerting them to potential dangerous situations so that they can take necessary avoidance or protective measures.

[0059] (2) Enhanced safety: Through the dual warning of vision (warning light) and hearing (buzzer), the system can more effectively convey information about collision risks. This multi-dimensional warning method is more eye-catching and effective than a single signal, and can significantly improve the safety of the industrial vehicle operating environment.

[0060] (3) Improving driver reaction speed: After receiving the alarm signal, the driver can react quickly, adjust the vehicle's driving state or take other evasive measures. This quick reaction speed is crucial to avoiding collision accidents.

[0061] (4) Reduce human misjudgment: The edge computing terminal predicts collision risks based on real-time data and intelligent algorithms, which is more accurate and reliable than human judgment. By automatically triggering alarm signals, the system can reduce safety accidents caused by human misjudgment or negligence.

[0062] (5) Improving the intelligence level of the system: The introduction of alarm signals and alarm devices is an important manifestation of the improvement of the intelligence level of the system. These functions enable the system to autonomously monitor, analyze and respond to potential dangers, providing more comprehensive and intelligent protection for the safe driving of industrial vehicles.

[0063] (6) Enhanced system scalability: As part of the system, alarm signals and alarm devices can be integrated and linked with other safety equipment or systems. For example, alarm signals can trigger the emergency braking system or share information with other safety monitoring platforms, thereby further improving the safety management level of the entire industrial vehicle operating environment.

[0064] Optionally, the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module also includes: a remote monitoring center connected to the background server to receive the monitoring data sent by the background server and perform real-time analysis.

[0065] In the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module, a remote monitoring center is introduced to connect with the backend server to receive monitoring data and conduct real-time analysis. This technical choice brings many significant benefits:

[0066] (1) Real-time remote monitoring capability: The remote monitoring center can receive real-time monitoring data from the backend server, including environmental video streams, real-time distance and speed data, collision risk prediction results, etc. This real-time data transmission and processing capability enables monitoring personnel to understand the working environment and safety status of industrial vehicles at any time.

[0067] (2) Enhanced data analysis and decision support: The remote monitoring center is equipped with professional data processing and analysis software, which can conduct in-depth analysis and mining of the received monitoring data. Through big data analysis, the monitoring center can discover potential safety hazards and trends, and provide strong data support for decision-making.

[0068] (3) Improve monitoring efficiency and coverage: Compared with traditional on-site monitoring methods, the remote monitoring center can realize centralized monitoring and management of multiple industrial vehicles and work areas. This centralized monitoring mode not only improves monitoring efficiency, but also greatly expands the coverage of monitoring.

[0069] (4) Convenient for remote maintenance and management: The remote monitoring center can also communicate and maintain remotely with other parts of the system (such as edge computing terminals, 5G RedCap modules, etc.). When the system fails or needs to be upgraded, the monitoring center can remotely perform fault diagnosis and repair operations, improving the reliability and maintainability of the system.

[0070] Optionally, the target recognition model is used to perform the following steps to recognize a target object from the environment video stream:

[0071] Decomposing the environment video stream into single video frames to form an original video frame sequence;

[0072] Preprocessing the original video frame sequence to filter out key frames and form a key video frame sequence;

[0073] Performing illumination normalization processing on the key video frame sequence and then extracting multi-scale features to obtain a multi-scale feature map;

[0074] Fusing the multi-scale feature maps to obtain a feature representation of the target object;

[0075] The target feature representation is classified to identify the target object from the environment video stream.

[0076] In the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module, the target recognition model accurately identifies the target from the environmental video stream based on the above steps, which has the following technical benefits:

[0077] 1. Decompose the environmental video stream into a sequence of raw video frames

[0078] Refined processing: Decompose the continuous video stream into individual video frames so that each frame can be processed as an independent image. This refined processing provides the basis for subsequent feature extraction and recognition.

[0079] Reduce the amount of calculation: Although the video stream is continuous, not every frame contains useful information for recognition. By decomposing, the system can process key frames more efficiently and reduce unnecessary calculations.

[0080] 2. Preprocessing and key frame screening

[0081] De-redundancy: The preprocessing process can remove redundant information in the video frame, such as noise, blur, etc., to improve image quality.

[0082] Save resources: By filtering key frames, the system can focus on processing those frames that are important for identifying the target object, thereby saving computing resources and time.

[0083] Improve recognition efficiency: Keyframes usually contain more motion or change information, which is crucial for the recognition of objects. Therefore, filtering keyframes can significantly improve recognition efficiency.

[0084] 3. Light normalization

[0085] Reduce the impact of lighting: Video frames shot under different lighting conditions may have large differences in brightness and contrast, which will affect the accuracy of feature extraction. Lighting normalization can eliminate this effect, making images under different lighting conditions have similar brightness and contrast.

[0086] Improved robustness: Images that have undergone illumination normalization are more stable and less susceptible to changes in external lighting conditions, thereby improving the robustness of the target recognition model.

[0087] 4. Multi-scale feature extraction

[0088] Capturing details: Multi-scale feature extraction can capture image features at different scales, including global features and local details. This multi-scale processing approach enables the model to understand the image content more comprehensively.

[0089] Adapt to different targets: Targets of different sizes occupy different pixel areas in the image. Multi-scale feature extraction can adapt to targets of different sizes and improve recognition accuracy.

[0090] 5. Feature fusion and target classification

[0091] Comprehensive information: The feature fusion process combines features of different scales and levels to form a more comprehensive representation of the target object. This comprehensive information is crucial for subsequent target classification.

[0092] Improve classification accuracy: Classification based on the fused feature representation can more accurately identify the target object. At the same time, due to the comprehensiveness of the feature representation, the model also has a high accuracy in identifying targets in different scenes and angles.

[0093] Optionally, performing illumination normalization processing on the key video frame sequence includes:

[0094] Performing adaptive histogram equalization on each key frame in the key video frame sequence to obtain an adaptive equalized histogram;

[0095] Performing illumination compensation on the adaptive equalization histogram to obtain an illumination-compensated image frame;

[0096] Nonlinear mapping is performed on the illumination-compensated image frame to obtain an illumination-normalized image frame.

[0097] In the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module, the illumination normalization processing of key video frame sequences has the following technical benefits:

[0098] 1. Adaptive Histogram Equalization

[0099] Improve image contrast: Adaptive histogram equalization enhances image details by equalizing the histogram of each local area (such as a small block or window). This method is more adaptable to brightness changes in different areas of the image than global histogram equalization.

[0100] Reduce the impact of uneven lighting: In an environment with uneven lighting, some areas of the image may be too bright or too dark, resulting in loss of details. AHE reduces the impact of uneven lighting on image quality through local equalization, making the target objects in the image clearer and more visible.

[0101] 2. Light compensation

[0102] Further brightness adjustment: Lighting compensation is based on adaptive histogram equalization, which further adjusts the brightness level of the image based on the overall brightness of the image or the brightness distribution of a specific area. This helps to eliminate brightness differences caused by changes in the position, intensity or direction of the light source.

[0103] Enhanced image stability: Through illumination compensation, the system is able to generate more stable image sequences, maintaining the clarity and recognizability of objects in the image even when actual lighting conditions change.

[0104] 3. Nonlinear Mapping

[0105] Optimizing image quality: Nonlinear mapping is a technique for converting image data from one representation to another. In the context of illumination normalization, nonlinear mapping can be used to further optimize image quality, such as enhancing edge information, reducing noise, etc.

[0106] Improve recognition accuracy: Image frames processed by nonlinear mapping have more consistent lighting conditions and higher image quality, which helps improve the accuracy of the target recognition model. Because the model is usually trained on image data with consistent lighting conditions, in practical applications, the recognition performance of the model can be improved by making the image data closer to the training data through illumination normalization.

[0107] Optionally, based on Adaptive histogram equalization is performed on each key frame in the key video frame sequence to obtain an adaptive equalized histogram, where I adopt (x, y) represents the pixel value at the position (x, y) after adaptive histogram equalization, L represents the possible brightness level of the image, N represents the total number of pixels in the local area, g(k) represents the number of pixels at the kth brightness level in the local area, that is, the local histogram, and the local area is defined by a window in the image, which surrounds the current pixel (x, y) and includes the pixels around (x, y);

[0108] Based on I comp (x, y) = I adapt (x, y)-log(I illum (x, y)) performs illumination compensation on the adaptive equalization histogram to obtain an image frame after illumination compensation, wherein I comp (x, y) represents the pixel value after illumination compensation at the position (x, y), I illum (x, y) represents the estimated illumination component at the position (x, y), which can be estimated by analyzing the high brightness area in the image;

[0109] Based on I nl (x, y) = α·tanh(β·I comp (x, y)) performs nonlinear mapping on the image frame after illumination compensation to obtain a mapped image frame, wherein I nl (x, y) represents the pixel value at the position (x, y) after nonlinear mapping.

[0110] Tanh represents the hyperbolic tangent function, which is used to map the compensated pixel value into a preset range to enhance the contrast. α and β are used to adjust the intensity of the nonlinear mapping. α controls the range of the mapped value, and β controls the sensitivity of the mapping.

[0111] based on The mapping image frame is contrast enhanced to obtain a normalized illumination image frame, I lce (x, y) represents the pixel value after local contrast enhancement at the position (x, y), min(I nl ) and max(I nl ) represent the minimum and maximum values ​​of the image after nonlinear mapping.

[0112] Traditional histogram equalization is usually performed on the entire image, but the lighting conditions in industrial environments may be very uneven, and local areas may have brightness differences due to shadows, reflections or local light sources. For this reason, the above-mentioned adaptive histogram equalization adjusts the contrast of the image by considering the histogram of the local area, so that the contrast of these areas can be adjusted more accurately, so that the target object can be clearly identified under various lighting conditions.

[0113] In addition, industrial vehicles will encounter different lighting conditions during movement, such as sunlight, lights, etc. These lighting changes may affect the recognition of targets. Therefore, by estimating the lighting components in the image and removing these components from the image, the impact of lighting changes on target recognition and tracking can be reduced.

[0114] Furthermore, targets in industrial environments may have a large dynamic range in brightness. Nonlinear mapping can expand the details of dark and bright areas of the image, enhance the contrast of the image, make the features of the target more obvious, and help improve the accuracy of target recognition and tracking while avoiding noise amplification caused by over-enhancement.

[0115] Finally, in complex industrial environments, the target may have similar brightness to the background. Local contrast enhancement can further highlight the edges and contours of the target, which helps to distinguish the target from the background, making the target more prominent in the image and improving the recognizability of the target.

[0116] Optionally, the target tracking model is used to perform the following steps to track the identified target object:

[0117] Extracting a color histogram, edge features, and texture features from the environment video stream to locate a target object in the environment video stream;

[0118] The target object and the target object are predicted based on Calculating a cost function between the target object and the target object, and They represent the predicted position and speed of the target object i obtained by predicting the state of the target object; p x，j and v x，jis the predicted position and speed of the target object j obtained by predicting the state of the target object; σ p and σ v is the standard deviation of the position and velocity measurement noise, determined by describing the noise characteristics of the camera; w p and w v is a weight factor used to balance the importance of position and speed in cost calculation;

[0119] Based on the minimization of the cost function, the identified target object is tracked.

[0120] In the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module, the use of cost functions for target tracking has the following technical benefits:

[0121] (1) Improve tracking accuracy: The cost function can quantify the degree of match between the tracked target and the detected target by calculating the difference between the predicted position and the observed position, and between the predicted speed and the observed speed. This quantification method helps to select the best match, thereby improving tracking accuracy.

[0122] (2) Enhanced robustness: In complex industrial environments, targets may be affected by factors such as occlusion and illumination changes. The cost function takes into account the uncertainty of position and velocity (via σ p and σ v ), making the tracking algorithm more robust in these cases.

[0123] (3) Real-time performance: The 5G RedCap module provides low-latency communication capabilities, which, combined with the fast calculation of the cost function, can update the target status in real time, ensuring the real-time performance of the tracking system.

[0124] (4) Resource optimization: By optimizing the weight factor w in the cost function p and w v , the resource allocation of the algorithm can be adjusted according to the tracking requirements of different scenarios. For example, in some scenarios, more emphasis may be placed on position accuracy, while in other scenarios, more emphasis may be placed on speed estimation.

[0125] (5) Adaptability: The standard deviation of the measurement noise of position and velocity in the cost function σ p and σ v Adjustments can be made based on actual sensor characteristics and environmental conditions, allowing the tracking algorithm to adapt to different operating environments.

[0126] (6) Data association efficiency: In multi-target tracking, the cost function is used for data association, that is, to determine the correspondence between the observed targets and the tracked targets. By minimizing the cost function, the data association problem can be effectively solved and the tracking accuracy and efficiency can be improved.

[0127] Optionally, the collision risk identification model is used to predict whether there is a risk of collision between the target object and the industrial vehicle according to the real-time distance and the real-time relative moving speed between the target object and the industrial vehicle, and generate a risk warning strategy when the collision risk is predicted, so as to brake or decelerate the industrial vehicle based on Predict the remaining time before the collision occurs, d(t) is the distance between the target object and the vehicle at the current moment, v rel (t) is the relative speed; by setting a threshold TTC threshold To determine whether there is a collision risk, if TTC(t)<TTC threshold , it is considered that there is a collision risk, and based on Generate the corresponding warning strategy, Δv(t) is the deceleration that needs to be applied, k p and k d are the proportional and differential control coefficients used to adjust the braking intensity.

[0128] To this end, in the industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module, the implementation of the collision risk identification model can bring the following technical benefits:

[0129] 1. Improve forecast accuracy:

[0130] By calculating the distance \(d(t)\) and relative speed between the target and the industrial vehicle in real time

[0131] \(v_{rel}(t)\), and using this data to calculate the time to collision (Time-to-Collision, TTC), the system can more accurately predict potential collision risks. TTC is a key indicator that provides a quantitative measure of how much time it will take for the target object and the vehicle to collide at the current speed and direction.

[0132] 2. Real-time response capability:

[0133] The low latency of the 5G RedCap module enables the system to quickly respond to detected risks and generate risk warning strategies in a timely manner. This fast data processing and decision-making capability is critical for collision avoidance systems because it needs to take quick action in emergency situations.

[0134] 3. Optimization of braking or deceleration control:

[0135] Based on the risk assessment results, the system optimizes the braking or deceleration force to ensure that the vehicle can stop or decelerate within a safe distance. This optimized control strategy can reduce the instability and potential secondary accidents caused by sudden braking.

[0136] 4. Stability of closed-loop control system:

[0137] Through closed-loop control, the system can monitor the vehicle status and target status after braking or deceleration in real time, and continuously adjust the braking force until the risk is eliminated. This feedback mechanism improves the stability and robustness of the system, ensuring that it can work effectively in various environments and operating conditions.

[0138] 5. Improve the safety of industrial vehicles:

[0139] Through the above steps and mathematical models, the collision risk identification model can effectively predict and respond to potential collision risks, thereby improving the safety of industrial vehicles. This is of great significance for protecting workers and equipment and reducing accident losses.

[0140] The industrial vehicles in the above embodiments include but are not limited to forklifts. The distance sensors in the above embodiments include but are not limited to radars, infrared sensors, TOF sensors, and the like.

[0141] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0142] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0143] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. An industrial vehicle pedestrian collision avoidance detection and warning system based on 5G RedCap module, characterized in that: include: A camera, used for capturing a real-time video of the working environment of the industrial vehicle to form an environmental video stream; A distance sensor, used to measure the real-time distance and real-time relative moving speed between the tracked target and the industrial vehicle; an edge computing terminal, on which a target recognition model, a target tracking model, and an early warning model are deployed, wherein the target recognition model is used to recognize a target object from the environmental video stream, the target tracking model is used to track the recognized target object, and the collision risk recognition model is used to predict whether there is a risk of collision between the target object and the industrial vehicle according to the real-time distance and real-time relative moving speed between the target object and the industrial vehicle, and when the collision risk is predicted, generate a risk early warning strategy to brake or decelerate the industrial vehicle; The 5G RedCap module is used to send the environmental video stream, the real-time distance between the target object and the industrial vehicle, the real-time relative moving speed, the prediction result of the collision risk, and the risk warning strategy to the background server.

2. According to claim 1, an industrial vehicle pedestrian collision avoidance detection and warning system based on a 5G RedCap module is characterized in that: The number of the cameras is at least 4, so that at least one camera is installed in each of the four directions of the front, back, left, and right of the industrial vehicle to perform panoramic video capture of the working environment of the industrial vehicle.

3. According to claim 1, an industrial vehicle pedestrian collision avoidance detection and warning system based on a 5G RedCap module is characterized in that: The edge computing terminal is also used to generate an alarm signal when the collision risk is predicted.

4. According to claim 3, an industrial vehicle pedestrian collision avoidance detection and warning system based on a 5G RedCap module is characterized in that: Also includes: At least one of a warning light and a buzzer, wherein the warning light is used to generate a warning light signal under the drive of the alarm signal, and the buzzer is used to generate a warning sound signal under the drive of the alarm signal.

5. The industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module according to claim 3 is characterized in that: It also includes: a remote monitoring center connected to the backend server to receive the monitoring data sent by the backend server and perform real-time analysis.

6. The industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module according to claim 1 is characterized in that: The target recognition model is used to perform the following steps to recognize the target object from the environment video stream: Decomposing the environment video stream into single video frames to form an original video frame sequence; Preprocessing the original video frame sequence to filter out key frames and form a key video frame sequence; Performing illumination normalization processing on the key video frame sequence and then extracting multi-scale features to obtain a multi-scale feature map; Fusing the multi-scale feature maps to obtain a feature representation of the target object; The target feature representation is classified to identify the target object from the environment video stream.

7. The industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module according to claim 6 is characterized in that: The performing illumination normalization processing on the key video frame sequence includes: Performing adaptive histogram equalization on each key frame in the key video frame sequence to obtain an adaptive equalized histogram; Performing illumination compensation on the adaptive equalization histogram to obtain an illumination-compensated image frame; Nonlinear mapping is performed on the illumination-compensated image frame to obtain an illumination-normalized image frame.

8. The industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module according to claim 7 is characterized in that: based on Adaptive histogram equalization is performed on each key frame in the key video frame sequence to obtain an adaptive equalized histogram, where I adapt (x, y) represents the pixel value at the position (x, y) after adaptive histogram equalization, L represents the possible brightness level of the image, N represents the total number of pixels in the local area, g(k) represents the number of pixels at the kth brightness level in the local area, that is, the local histogram, and the local area is defined by a window in the image, which surrounds the current pixel (x, y) and includes the pixels around (x, y); Based on I comp (x, y) = I adapt (x, y)-log(I illum (x, y)) performs illumination compensation on the adaptive equalization histogram to obtain an image frame after illumination compensation, wherein I comp (x, y) represents the pixel value after illumination compensation at the position (x, y), I illum (x, y) represents the estimated illumination component at the position (x, y), which can be estimated by analyzing the high brightness area in the image; Based on I nl (x, y) = α·tanh(β·I comp (x, y)) performs nonlinear mapping on the image frame after illumination compensation to obtain a mapped image frame, wherein I nl (x, y) represents the pixel value after nonlinear mapping at the position (x, y); tanh represents the hyperbolic tangent function, which is used to map the compensated pixel value into a preset range to enhance the contrast; α and β are used to adjust the intensity of nonlinear mapping, α controls the range of the mapped value, and β controls the sensitivity of the mapping; based on The mapping image frame is contrast enhanced to obtain a normalized illumination image frame, I lce (x, y) represents the pixel value after local contrast enhancement at the position (x, y), min(I nl ) and max(I nl ) represent the minimum and maximum values ​​of the image after nonlinear mapping.

9. The industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module according to claim 1 is characterized in that: The target tracking model is used to perform the following steps to track the identified target: Extracting a color histogram, edge features, and texture features from the environment video stream to locate a target object in the environment video stream; The target object and the target object are predicted based on Calculating a cost function between the target object and the target object, and They represent the predicted position and speed of the target object i obtained by predicting the state of the target object; p x，j and v x，j is the predicted position and speed of the target object j obtained by predicting the state of the target object; σ p and σ v is the standard deviation of the position and velocity measurement noise, determined by describing the noise characteristics of the camera; w p and w v is a weight factor used to balance the importance of position and speed in cost calculation; Based on the minimization of the cost function, the identified target object is tracked.

10. The industrial vehicle pedestrian collision avoidance detection and warning system based on the 5G RedCap module according to claim 1 is characterized in that: The collision risk identification model is used to predict whether there is a collision risk between the target object and the industrial vehicle according to the real-time distance and the real-time relative moving speed between the target object and the industrial vehicle, and to generate a risk warning strategy when the collision risk is predicted, so as to brake or decelerate the industrial vehicle based on Predict the remaining time before the collision occurs, d(t) is the distance between the target object and the vehicle at the current moment, v rel (t) is the relative speed; by setting a threshold TTC threshold To determine whether there is a collision risk, if TTC(t)<TTC threshold , it is considered that there is a collision risk, and based on Generate the corresponding warning strategy, Δv(t) is the deceleration that needs to be applied, k p and k d are the proportional and differential control coefficients used to adjust the braking intensity.

Images