Highly intelligent and safe industrial vehicle ADAS system with personnel recognition function

Abstract

The present invention relates to the technical field of vehicle prediction collision, and specifically to a highly intelligent and safe industrial vehicle ADAS system with a personnel recognition function. The present invention obtains the predicted travel distance of the workers within the local preset range of the industrial vehicle's position at the current moment based on the position distribution and time interval of all workers with the same attributes at two adjacent analysis moments, and the time interval between the current moment and the adjacent analysis moments of the workers within the local preset range of the industrial vehicle's position at the current moment; according to the difference between the distance between the industrial vehicle's position at the current moment and the workers' position within its local preset range and the predicted travel distance, obtains the personnel presence index in the blind spot of the industrial vehicle at the current moment, and uses it to adjust the speed of the industrial vehicle. The present invention predicts the possibility of workers appearing in the blind spot of the industrial vehicle's field of vision, adjusts the speed of the industrial vehicle in advance, and effectively improves the obstacle avoidance effect of the industrial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle collision prediction, and in particular to a highly intelligent and safe industrial vehicle ADAS system with a personnel recognition function. Background Art

[0002] Advanced Driving Assistance System (ADAS) is a series of electronic systems designed to improve the driving safety and operating efficiency of industrial vehicles. These systems assist drivers in vehicle control and reduce operating accidents by integrating various sensors and algorithms. The personnel recognition system obtains facial features and body movements of personnel through video acquisition equipment, feeds them into the back-end recognition algorithm after preprocessing, and compares them with the personnel information database to quickly identify the real identity and movements of personnel.

[0003] The application of personnel identification system to industrial vehicle ADAS usually refers to the identification of personnel in the images captured by the camera of the industrial vehicle, and the prediction of the personnel's position and possible direction of movement in the camera image, so as to control the industrial vehicle to avoid obstacles. The existing technology can only avoid obstacles for personnel in the images captured by the camera, but because there are many facilities such as shelves and equipment in industrial plants, and there are many road intersections, it may cause the staff to suddenly appear in the blind spot of the video image, causing the industrial vehicle to fail to avoid obstacles in time, thus affecting work efficiency, and in serious cases, personnel safety accidents may occur. Summary of the invention

[0004] In order to solve the technical problem that workers suddenly appear in the blind spot of the industrial vehicle, resulting in poor obstacle avoidance effect of the industrial vehicle, the purpose of the present invention is to provide a highly intelligent and safe industrial vehicle ADAS system with personnel recognition function. The technical solution adopted is as follows:

[0005] The present invention proposes a highly intelligent and safe industrial vehicle ADAS system with a personnel recognition function, the system comprising:

[0006] The data acquisition module is used to record the time when the industrial vehicle recognizes each worker before the current time as the analysis time of the corresponding worker, and obtain the position of each worker and the industrial vehicle at the analysis time, as well as the attributes of the worker;

[0007] A personnel movement prediction module is used to obtain the predicted travel distance of each worker within the local preset range of the position of the industrial vehicle at the current moment based on the position distribution and time interval of all workers with the same attributes at two adjacent analysis moments, and the time interval between each worker at the current moment and the previous adjacent analysis moment within the local preset range of the position of the industrial vehicle at the current moment;

[0008] The vehicle speed adjustment module is used to obtain the presence index of personnel in the blind spot of the industrial vehicle at the current moment according to the difference between the distance between the position of the industrial vehicle at the current moment and the position of the personnel within its local preset range and the predicted travel distance; and adjust the speed of the industrial vehicle based on the presence index of personnel in the blind spot.

[0009] Furthermore, the step of obtaining the predicted travel distance of each worker within a local preset range of the position of the industrial vehicle at the current moment includes:

[0010] The set of all workers with the same attributes is recorded as a worker set; according to the position distribution and time interval of the workers in each worker set at two adjacent analysis moments, the predicted stay time and predicted travel speed of each worker set are obtained;

[0011] Acquire analysts of the industrial vehicle at the current moment; the position of the analyst at the previous analysis moment adjacent to the current moment is within a local preset range of the position of the industrial vehicle at the current moment; calculate the difference between the time interval between each analyst at the current moment and the previous analysis moment adjacent to the current moment and the predicted stay time of the personnel set to which each analyst belongs, and use the product of the difference and the predicted travel speed of the personnel set to which each analyst belongs as the predicted travel distance of each analyst of the industrial vehicle at the current moment.

[0012] Furthermore, obtaining the predicted stay time and predicted travel speed of each set of personnel includes:

[0013] Obtain a map of the industrial plant; use the roads in the map of the industrial plant as edges and the intersections of different roads as nodes to obtain a graph as a site map;

[0014] Mark the position of each staff member at each analysis time in the site map to obtain the position node of each staff member at each analysis time; obtain the shortest path between the position nodes of each staff member at two adjacent analysis times in the site map, and record it as the movement path of each staff member at two adjacent analysis times; obtain the length of the movement path as the movement distance of each staff member at two adjacent analysis times;

[0015] Obtaining the actual stay time of the staff between two adjacent analysis moments; and obtaining the actual travel speed of the staff between two adjacent analysis moments based on the moving distance and the actual stay time;

[0016] Constructing a travel speed function of a worker between two adjacent analysis moments; substituting the time interval and movement distance of the worker at two adjacent analysis moments into the travel speed function to obtain a travel speed fitting value of the worker between two adjacent analysis moments; the travel speed fitting value corresponds to a residence time fitting value;

[0017] Obtaining an error function for each personnel set according to the difference between the actual travel speed and the travel speed fitting value of the workers in each personnel set before all two adjacent analysis moments, and the difference between the actual stay time and the stay time fitting value;

[0018] The residence time fitting value and the travel speed fitting value corresponding to the minimum value of the error function are respectively used as the predicted residence time and the predicted travel speed of each personnel set.

[0019] Furthermore, the error function of each personnel set includes:

[0020] The tuples consisting of all two adjacent analysis moments of each staff member before the current moment are recorded as the moment pair of each staff member;

[0021] The error function of each personnel set is expressed as:

[0022] , ; In the formula, RSS is the error function of each personnel set; N is the total number of staff in each personnel set; is the total number of time pairs of the nth staff member in each staff set; The fitting value of the travel speed of the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the actual travel speed between the two analysis moments in the k-th moment pair of the n-th worker in each worker set; is the fitted value of the stay time between the two analysis moments in the k-th moment pair of the n-th worker in each staff set; is the actual stay time between the two analysis moments in the k-th moment pair of the n-th worker in each worker set; is the moving distance between the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the time interval between the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the absolute value function.

[0023] Furthermore, the step of obtaining the personnel presence index of the blind spot of the industrial vehicle at the current moment includes:

[0024] Obtain the shortest path between the position of the industrial vehicle at the current moment and the position of each analyst at the previous analysis moment adjacent to the current moment and the corresponding position nodes in the site map, and use the length of the shortest path as the vehicle distance between the industrial vehicle and each analyst at the current moment;

[0025] The cumulative sum of the absolute values ​​of the differences between the vehicle distances of all the analysts at the current moment of the industrial vehicle and the predicted travel distance is negatively correlated and normalized to obtain the presence index of personnel in the blind spot of the industrial vehicle at the current moment.

[0026] Furthermore, adjusting the speed of the industrial vehicle based on the blind spot personnel presence indicator includes:

[0027] Obtain the blind area of ​​the industrial vehicle in the video frame image at the current moment; obtain the distance between the position of the industrial vehicle at the current moment and the blind area of ​​the visual field in the video frame image, recorded as the blind area spacing of the industrial vehicle at the current moment;

[0028] If the blind spot distance of the industrial vehicle at the current moment is greater than the preset distance threshold, the current moment is updated until the blind spot distance of the industrial vehicle at the new current moment is equal to the preset distance threshold, the speed of the industrial vehicle at the new current moment is obtained, and the speed is adjusted using the blind spot personnel presence indicator of the industrial vehicle at the new current moment to obtain the corrected speed of the industrial vehicle at the new current moment; until there is no blind spot in the field of vision of the industrial vehicle in the video frame image at the new current moment, the industrial vehicle resumes its normal speed.

[0029] Furthermore, the method of adjusting the speed of the industrial vehicle by using the personnel presence index in the blind spot of the industrial vehicle at the new current moment to obtain the corrected speed of the industrial vehicle at the new current moment includes:

[0030] The difference between the speed of the industrial vehicle at the new current moment and a preset speed threshold is weighted by utilizing the personnel presence index in the blind spot of the industrial vehicle at the new current moment to obtain the correction degree of the industrial vehicle at the new current moment; the difference between the speed of the industrial vehicle at the new current moment and the correction degree is used as the corrected speed of the industrial vehicle at the new current moment.

[0031] Furthermore, obtaining the actual travel speed of the worker between two adjacent analysis moments includes:

[0032] The actual travel speed of the staff between the two adjacent analysis moments is obtained by taking the moving distance of the staff at two adjacent analysis moments as the numerator and the difference between the time interval between the two analysis moments and the actual stay time as the denominator.

[0033] Furthermore, the method for obtaining the shortest path between the position nodes of each staff member at two adjacent analysis moments in the site map is an ant colony algorithm.

[0034] Furthermore, the preset distance threshold is 10 meters.

[0035] The present invention has the following beneficial effects:

[0036] In an embodiment of the present invention, the action state of the staff between two adjacent analysis moments can be divided into two types: staying and moving. The movement conditions of all staff with the same attributes are relatively similar, that is, the travel speed and stay time in the time interval of adjacent analysis moments are relatively close; the ideal stay time and travel speed are obtained by the position distribution and time interval of all staff with the same attributes at two adjacent analysis moments, and combined with the time interval between the staff at the current moment and the previous analysis moment, the ideal distance that the staff moves from the position at the analysis moment to the position of the industrial vehicle is predicted to obtain the predicted travel distance; the distance between the position of the industrial vehicle at the current moment and the position of the staff within its local preset range represents the actual distance between the position of the staff at the previous analysis moment and the position of the industrial vehicle at the current moment; the difference between the two represents the ideal distance between the industrial vehicle and the staff at the current moment, which is used to measure the possibility of the staff appearing in the blind spot of the industrial vehicle's field of vision, and obtain the blind spot personnel presence index. When the industrial vehicle fails to recognize the staff, the speed of the industrial vehicle is adaptively adjusted in advance, and the braking time and obstacle avoidance time when the staff suddenly appear in the blind spot of the industrial vehicle's field of vision are increased, thereby improving the obstacle avoidance effect of the industrial vehicle and the safety in the industrial plant. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A system structure diagram of a highly intelligent and safe industrial vehicle ADAS system with a personnel recognition function provided by one embodiment of the present invention;

[0039] Figure 2 A structural diagram of a personnel movement prediction module provided by an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a computer device of a highly intelligent and safe industrial vehicle ADAS device with a personnel recognition function provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation, structure, features and effects of the highly intelligent and safe industrial vehicle ADAS system with personnel recognition function proposed by the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] The specific scheme of the highly intelligent and safe industrial vehicle ADAS system with personnel recognition function provided by the present invention is described in detail below with reference to the accompanying drawings.

[0044] Embodiment 1:

[0045] See also Figure 1 , which shows a system block diagram of a highly intelligent and safe industrial vehicle ADAS system with a personnel recognition function provided by an embodiment of the present invention. The system includes: a data acquisition module 110, a personnel movement prediction module 120, and a vehicle speed adjustment module 130.

[0046] The data collection module 110 is used to record the time when the industrial vehicle recognizes each worker before the current time as the analysis time of the corresponding worker, and obtain the position of each worker and the industrial vehicle at the analysis time, as well as the attributes of the worker.

[0047] Data of industrial plants are collected through surveying and mapping instruments, and input into map generation software to construct a two-dimensional map of industrial plants for reference in path planning and mobile obstacle avoidance of industrial vehicles. The edge devices on industrial vehicles collect video frame images at every moment in real time, and the position of industrial vehicles at every moment is obtained in real time through the vehicle-mounted global positioning system (GPS); edge devices refer to physical devices at the source of data generation, which have computing, storage and communication capabilities, can reduce data transmission and delays, and provide faster responses.

[0048] The face recognition algorithm is used to extract the face of the staff member in the video frame image at each moment, and then compare it with the ID photo of the staff member in the personnel database to identify the identity and job type of the staff member. The personnel database stores the ID photo and job type of the staff member.

[0049] The recognition of a worker by an industrial vehicle means that the worker appears in the image video frame collected by the edge device on the industrial vehicle; because the worker and the industrial vehicle are in motion and the edge device has a certain shooting range, the industrial vehicle recognizes different workers at different times. In the preset time period before the current moment, the moment when the industrial vehicle recognizes each worker is recorded as the analysis moment of the corresponding worker. The worker is equipped with a positioning device, and the positioning device is used to determine the position of the worker in the map of the industrial plant at the analysis moment. The positioning device is Bluetooth, Wi-Fi positioning or a mobile phone, etc.

[0050] In industrial plants, workers of different job types have different movement trajectories and movement frequencies; workers have multiple working states when working, for example, maintenance workers can be divided into patrol state and repair state. The movement of maintenance workers during patrol is relatively random and the movement route is relatively long. When repairing, maintenance workers will stay in place for a certain period of time, and the probability of appearing in other places later is low. Therefore, the attributes of workers in the embodiment of the present invention include: work type and work state; work types include: operators, managers and maintenance personnel, and work states include: tallying, transportation, unloading and patrolling.

[0051] The convolutional neural network is trained through action recognition technology to identify the working status of workers in video frame images; when there are workers in the video frame images of industrial vehicles at each analysis moment, the video frame images are input into the pre-trained convolutional neural network to output the working status of the workers at each analysis moment.

[0052] It should be noted that each staff member needs to include at least two analysis moments within the preset time period and the working status of all analysis moments must be the same; since the freight throughput of industrial plants is large and there are more staff members and industrial vehicles, the positions of industrial vehicles and staff members change frequently, which can enable the staff members to maintain the same working status for a certain period of time.

[0053] It should be noted that the surveying and mapping instrument can be a drone, a rangefinder or a total station; the map generation software can be AutoCAD (Auto Computer Aided Design) software, building information modeling software or ArcGIS software; the edge devices on industrial vehicles usually include infrared sensors and video cameras; face recognition algorithms include: convolutional neural networks, support vector machines and linear discriminant analysis algorithms, etc.

[0054] The personnel movement prediction module 120 is used to obtain the predicted travel distance of each worker within the local preset range of the industrial vehicle's position at the current moment based on the position distribution and time interval of all workers with the same attributes at two adjacent analysis moments, and the time interval between the current moment and the previous adjacent analysis moment of each worker within the local preset range of the industrial vehicle's position at the current moment.

[0055] The action states of the staff between two adjacent analysis moments can be divided into two types: staying and moving. The time that the staff is in the two action states is different, which easily affects the moving distance of the staff. The position distribution and time interval of all staff with the same attributes at two adjacent analysis moments are used to determine the optimal values ​​of the staff's stay time and travel speed between two adjacent analysis moments, and combined with the time interval between the staff at the current moment and the previous adjacent analysis moment, the travel distance of the staff within the time interval is predicted to obtain the predicted travel distance.

[0056] See also Figure 2 , which shows a structural diagram of a personnel movement prediction module provided by an embodiment of the present invention. The personnel movement prediction module includes: a time speed prediction unit 121 and a travel distance prediction unit 122.

[0057] The time-speed prediction unit 121 is used to record a set of all staff members with the same attributes as a staff member set; and obtain the predicted stay time and predicted travel speed of each staff member set according to the position distribution and time interval of the staff members in each staff member set at two adjacent analysis moments.

[0058] All staff members in each personnel set have the same work type and the same work status at all analysis moments within the preset time period. The analysis method of moving distance is as follows: obtain the map of the industrial plant; use the roads in the map of the industrial plant as edges and the intersections of different roads as nodes to obtain a graph as the site map; mark the position of each staff member at each analysis moment in the site map to obtain the position node of each staff member at each analysis moment; obtain the shortest path between the position nodes of each staff member at two adjacent analysis moments in the site map, and record it as the moving path of each staff member at two adjacent analysis moments; obtain the length of the moving path as the moving distance of each staff member at two adjacent analysis moments.

[0059] It should be noted that the field map is an undirected graph; in the embodiment of the present invention, the ant colony algorithm is used to obtain the shortest path between two nodes in the graph, and other embodiments may also use Dijkstra algorithm, A* search algorithm and other methods. Among them, the ant colony algorithm is a well-known technology for those skilled in the art and will not be described in detail here. Among them, the ant colony algorithm is a well-known technology for those skilled in the art and will not be described in detail here.

[0060] The error analysis method is as follows: obtain the actual stay time of the staff between two adjacent analysis moments; obtain the actual travel speed of the staff between two adjacent analysis moments based on the moving distance and the actual stay time; construct a travel speed function of the staff between two adjacent analysis moments; substitute the time interval and moving distance of the staff at two adjacent analysis moments into the travel speed function to obtain the travel speed fitting value of the staff between two adjacent analysis moments; the travel speed fitting value corresponds to a stay time fitting value; according to the difference between the actual travel speed of the staff in each personnel set before all two adjacent analysis moments and the travel speed fitting value, as well as the difference between the actual stay time and the stay time fitting value, obtain the error function of each personnel set.

[0061] Specifically, the action status of each staff member between two adjacent analysis moments can be divided into two types: staying and moving. During the time period between two adjacent analysis moments, the time the staff member stays still is directly collected using a stopwatch or mobile phone, and recorded as the actual stay time.

[0062] Since the staff did not move at the location of the actual stay time, the actual stay time needs to be removed when calculating the staff's actual travel speed; the method for obtaining the actual travel speed is: taking the staff's moving distance at two adjacent analysis moments as the numerator, and the difference between the time interval of the two analysis moments and the actual stay time as the denominator to obtain the ratio, which is the staff's actual travel speed between the two adjacent analysis moments.

[0063] It should be noted that if the position of the staff member at two adjacent analysis moments does not change, and it is meaningless to use the information between the two analysis moments to predict the staff member's actions, then the above two analysis moments will not participate in the subsequent analysis.

[0064] In industrial plants, due to the different job functions of workers of different work types, the routes of movement of workers of different work types in industrial plants are quite different; while the work conditions of workers of the same work type are the same, the routes of movement of these workers in industrial plants have strong regularity. The different working conditions of workers of the same work type will affect the movement of subsequent workers, and the movement of workers in the same working conditions is relatively close. Therefore, the information of workers of the same work type and the same working conditions, that is, workers in the same personnel set, is selected to train and predict the movement of workers of each work type in each working condition. The specific method is as follows:

[0065] The function of the worker's travel speed between two adjacent analysis moments is expressed as ; Where D is the moving distance of the staff at two adjacent analysis moments, is the time interval between two adjacent analysis moments of the staff, and Ts is the fitting value of the staff's residence time between two adjacent analysis moments. It is an unknown quantity and needs to be predicted using a neural network.

[0066] The tuples of all two adjacent analysis moments of each staff member before the current moment are recorded as the moment pairs of each staff member. It should be noted that there are at least two moment pairs of staff members in each staff set.

[0067] The average absolute error between the true value and the predicted value of the two indicators, travel speed and stay time, at the two analysis moments in the time pairs of the staff members in each personnel set is taken as the error function of each personnel set; taking the travel speed as an example, the true value and predicted value of the travel speed represent the actual travel speed and the travel speed fitting value respectively.

[0068] In a specific implementation of the embodiment of the present invention, the error function of each personnel set is expressed by the formula:

[0069]

[0070]

[0071] Where RSS is the error function of each personnel set; N is the total number of staff in each personnel set; is the total number of time pairs of the nth staff member in each staff set; The fitting value of the travel speed of the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the actual travel speed between the two analysis moments in the k-th moment pair of the n-th worker in each worker set; is the fitted value of the stay time between the two analysis moments in the k-th moment pair of the n-th worker in each staff set; is the actual stay time between the two analysis moments in the k-th moment pair of the n-th worker in each worker set; is the moving distance between the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the time interval between the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the absolute value function.

[0072] In other embodiments of the present invention, regression error indicators such as the mean square error, root mean square error and determination coefficient between the true value and the predicted value of the two indicators of travel speed and residence time at the two analysis moments in the time pair of the staff members in each personnel set can also be used as the error function.

[0073] Taking the mean square error as an example, the error function is expressed by the mean square error between the actual value and the predicted value of the two indicators of travel speed and stay time at the two analysis moments in the time pair of the staff in each personnel set: .

[0074] The method for data fitting in the embodiment of the present invention is the gradient descent method, and the minimum value of the error function of each personnel set is obtained by the gradient descent method. The residence time fitting value and the travel speed fitting value corresponding to the minimum value of the error function are respectively used as the predicted residence time and predicted travel speed of each personnel set. Among them, the gradient descent method is a well-known technology for those skilled in the art and will not be described in detail here. In other embodiments of the present invention, the least squares method and the maximum likelihood estimation method can also be used for data fitting.

[0075] Travel distance prediction unit 122: used to obtain the analyst of the industrial vehicle at the current moment; the position of the analyst at the previous analysis moment adjacent to the current moment is within the local preset range of the position of the industrial vehicle at the current moment; calculate the difference between the time interval between each analyst at the current moment and the previous analysis moment adjacent to the current moment and the predicted stay time of the personnel set to which each analyst belongs, and use the product of the difference and the predicted travel speed of the personnel set to which each analyst belongs as the predicted travel distance of each analyst of the industrial vehicle at the current moment.

[0076] This solution analyzes the possibility of a worker appearing in the blind spot of the industrial vehicle at the current moment, which is equivalent to predicting the worker's position at the current moment. It is necessary to analyze the distribution of the worker's position at the previous analysis moment and the position of the industrial vehicle at the current moment to determine the analyst of the industrial vehicle at the current moment.

[0077] The predicted travel speed and predicted stay time of the set of personnel to which the analyst belongs are used as the ideal travel speed and stay time of the staff at the current moment and the time interval between the current moment and the previous analysis moment respectively; the physical formula that the path is equal to time multiplied by the speed is used to obtain the travel distance of the staff in the above time interval and the predicted travel distance.

[0078] In the embodiment of the present invention, the position of the industrial vehicle at the current moment is taken as the origin, and a circular area with a radius of 50 meters is taken as the local preset range. The implementer can set the radius according to the specific situation.

[0079] The vehicle speed adjustment module 130 is used to obtain the blind spot personnel presence index of the industrial vehicle at the current moment according to the difference between the distance between the position of the industrial vehicle at the current moment and the position of the workers within its local preset range and the predicted travel distance; and adjust the speed of the industrial vehicle based on the blind spot personnel presence index.

[0080] The distance between the current position of the industrial vehicle and the position of the worker within its local preset range represents the actual distance between the worker's position at the previous analysis moment and the current position of the industrial vehicle; the predicted travel distance represents the ideal distance for the worker to move from the position at the previous analysis moment to the position of the industrial vehicle; the difference between the two represents the ideal distance between the industrial vehicle and the worker at the current moment. The smaller the ideal distance, the greater the possibility that the worker appears in the blind spot of the industrial vehicle, thus obtaining the blind spot personnel presence index.

[0081] Preferably, in some possible implementation modes of the embodiments of the present invention, the method for obtaining the blind spot personnel presence index includes: obtaining the shortest path between the position of the industrial vehicle at the current moment and the position of each analyst at the previous analysis moment adjacent to the current moment in the corresponding position nodes in the site map, and taking the length of the shortest path as the vehicle spacing of each analyst of the industrial vehicle at the current moment; negatively correlating and normalizing the cumulative sum of the absolute values ​​of the differences between the vehicle spacing of all analysts of the industrial vehicle at the current moment and the predicted travel distance, to obtain the blind spot personnel presence index of the industrial vehicle at the current moment.

[0082] The smaller the difference between the vehicle spacing and the predicted travel distance, the closer the analyst is to the industrial vehicle at the current moment, and the greater the possibility that the analyst appears in the blind spot of the industrial vehicle. Therefore, it is necessary to perform negative correlation processing on the cumulative sum of the absolute value of the difference between the vehicle spacing and the predicted travel distance.

[0083] It should be noted that the embodiment of the present invention uses an ant colony algorithm to obtain the shortest path between two nodes in the graph; this scheme does not consider the actual running direction of the staff, analyzes the relevant information of the staff moving towards the position of the industrial vehicle, considers the maximum possibility of the staff appearing in the field of view of the industrial vehicle, and obtains the blind spot personnel presence indicator.

[0084] In a specific implementation of the embodiment of the present invention, the blind spot person presence index is expressed by a formula:

[0085]

[0086] Where P is the personnel presence index in the blind spot of the industrial vehicle at the current moment; is the distance between industrial vehicles and the mth analyst at the current moment; is the predicted travel speed of the industrial vehicle at the current moment for the personnel set to which the mth analyst belongs; is the time interval between the current moment and the previous analysis moment of the mth analyst of the industrial vehicle at the current moment; is the predicted stay time of the industrial vehicle at the current moment for the set of personnel to which the mth analyst belongs; is the predicted travel distance of the mth analyst of the industrial vehicle at the current moment; exp is an exponential function with a natural constant as the base; is the absolute value function.

[0087] It should be noted that Must be greater than The mth staff member has the same work type as the staff members in the staff set to which he belongs, and the work status of the mth staff member at the previous analysis moment immediately before the current moment is the same as the work status of the staff members in the staff set to which he belongs at the analysis moment within the preset time period.

[0088] Obtain the blind spot of the industrial vehicle in the video frame image at the current moment; obtain the distance between the position of the industrial vehicle at the current moment and the blind spot in the video frame image, recorded as the blind spot spacing of the industrial vehicle at the current moment; if the blind spot spacing of the industrial vehicle at the current moment is greater than a preset distance threshold, update the current moment until the blind spot spacing of the industrial vehicle at the new current moment is equal to the preset distance threshold, obtain the speed of the industrial vehicle at the new current moment, adjust the speed using the personnel presence indicator of the blind spot of the industrial vehicle at the new current moment, and obtain the corrected speed of the industrial vehicle at the new current moment; until there is no blind spot in the video frame image of the industrial vehicle at the new current moment, the industrial vehicle resumes its normal speed.

[0089] When an industrial vehicle has a blind spot in the video frame image at time t, first determine the distance between the position of the industrial vehicle and the blind spot. If the distance is too large, that is, the blind spot spacing of the industrial vehicle at the current moment is greater than the preset distance threshold, in order to improve the working efficiency of the industrial vehicle, the industrial vehicle can maintain normal speed, and the distance between the industrial vehicle and the blind spot gradually decreases until the blind spot spacing between the industrial vehicle and the blind spot is equal to the preset distance threshold. Assuming that the blind spot spacing of the industrial vehicle at time t+4 is equal to the preset distance threshold, because the position of the industrial vehicle and the staff changes between time t and time t+4, it is necessary to obtain the blind spot personnel presence indicator of the industrial vehicle at time t+4. , and obtain the speed of the industrial vehicle at time t+4 , using the blind spot personnel presence indicator Adjustment , get the corrected vehicle speed at time t+4 Assuming that the industrial vehicle still has a blind spot in the video frame image at time t+9, but does not have a blind spot in the video frame image at time t+10, the industrial vehicle moves at the corrected speed between time t+4 and time t+10. Driving; industrial vehicles resume normal speed after t+10.

[0090] If there is a blind spot in the video frame image of the industrial vehicle at a time after time t+10, the method for adjusting the speed of the industrial vehicle is the same as the adjustment method at time t and time t+10 described above.

[0091] It should be noted that the embodiment of the present invention uses the YOLO network to obtain the blind spot in the video frame image. In other embodiments, the simultaneous localization and mapping (SLAM) technology and the deep learning model can also be used to obtain the blind spot in the image. This embodiment uses the vehicle-mounted GPS to collect the speed of the industrial vehicle, and can also use radar speed meters and laser speed meters and other equipment to collect the speed of the industrial vehicle. This solution needs to ensure that the initial value of the distance between the industrial vehicle and the same blind spot is greater than the preset distance threshold. The preset distance threshold takes the empirical value of 10 meters, and the implementer can set it according to the specific situation.

[0092] It should be noted that the method for obtaining the blind spot personnel presence index of industrial vehicles at each moment is the same as the blind spot presence index at the current moment; industrial vehicles usually travel at normal speed in industrial plants, and the normal speed is usually greater than the preset speed threshold.

[0093] The speed of industrial vehicles can be adaptively adjusted in advance using the presence indicator of people in the blind spot, increasing the braking time and obstacle avoidance time when workers suddenly appear in the blind spot of the industrial vehicle, thereby improving the safety in industrial plants.

[0094] Preferably, in some possible implementation modes of the embodiments of the present invention, the method for obtaining the corrected vehicle speed includes: utilizing the personnel presence index in the blind spot of the industrial vehicle at the new current moment, weighted processing the difference between the speed of the industrial vehicle at the new current moment and a preset speed threshold, and obtaining the correction degree of the industrial vehicle at the new current moment; using the difference between the speed of the industrial vehicle at the new current moment and the correction degree as the corrected speed of the industrial vehicle at the new current moment.

[0095] It should be noted that in order to improve the operating efficiency of industrial vehicles, industrial vehicles cannot stop, and the speed is adjusted by adding a speed threshold so that the minimum speed of the industrial vehicle is the preset speed threshold.

[0096] In a specific implementation of the embodiment of the present invention, the corrected speed of the industrial vehicle at the new current moment is expressed by the formula:

[0097]

[0098] In the formula, is the corrected speed of the industrial vehicle at the new current moment; is the speed of the industrial vehicle at the new current moment; is the personnel presence indicator of the industrial vehicle in the blind spot at the new current moment; VY is the preset speed threshold; The correction degree of the industrial vehicle at the new current moment.

[0099] It should be noted that when The larger the value, the more likely it is that there are workers in the blind spot of the industrial vehicle at the current moment. In order to ensure the safety of the workers, the speed of the industrial vehicle needs to be corrected to a greater extent. The smaller the speed, the longer the braking time and obstacle avoidance time when a worker suddenly appears in the blind spot of the industrial vehicle, thereby improving the safety in the industrial plant. In the embodiment of the present invention, the preset speed threshold is an empirical value of 10 kilometers per hour, and the implementer can set it according to the specific situation.

[0100] In other embodiments of the present invention, an exponential function with natural constants as base can be used to negatively map the blind spot personnel presence index of the industrial vehicle at the new current moment to obtain a speed correction coefficient; the product of the speed of the industrial vehicle at the new current moment and the speed correction coefficient is used as the corrected speed of the industrial vehicle at the new current moment. It should be noted that the opposite of the blind spot personnel presence index is first taken, and the opposite is used as the exponent of the function with natural constants as base, so as to realize the negative correlation mapping of the blind spot personnel presence index using the exponential function with natural constants as base; the value range of the speed correction coefficient is .

[0101] So far, the present invention is completed.

[0102] Embodiment 2:

[0103] Figure 3 A schematic diagram of a computer device of a highly intelligent and safe industrial vehicle ADAS device with a personnel recognition function provided by an embodiment of the present invention. Figure 3 As shown, the computer device includes: a memory 201, a processor 202, and a computer program 203 stored in the memory 201 and running on the processor 202, wherein when the processor 202 executes the computer program 203, the computer device can execute any of the highly intelligent and safe industrial vehicle ADAS systems with personnel recognition functions described above.

[0104] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute the highly intelligent and safe industrial vehicle ADAS system with personnel recognition function provided by the embodiment of the present application.

[0105] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0106] It should be understood that the device provided in this embodiment is used to execute the above-mentioned highly intelligent and safe industrial vehicle ADAS system with personnel recognition function, and thus can achieve the same effect as the above-mentioned implementation method.

[0107] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc.

[0108] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits disclosed in the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0109] Embodiment 3:

[0110] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement the highly intelligent and safe industrial vehicle ADAS system with personnel recognition function provided by the above-mentioned embodiment.

[0111] Among them, the device and computer-readable storage medium provided in this embodiment are used to execute the corresponding system provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding system provided above, and will not be repeated here.

[0112] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Highly intelligent and safe industrial vehicle ADAS system with personnel recognition function, characterized by: The system includes: The data acquisition module is used to record the time when the industrial vehicle recognizes each worker before the current time as the analysis time of the corresponding worker, and obtain the position of each worker and the industrial vehicle at the analysis time, as well as the attributes of the worker; A personnel movement prediction module is used to obtain the predicted travel distance of each worker within the local preset range of the position of the industrial vehicle at the current moment based on the position distribution and time interval of all workers with the same attributes at two adjacent analysis moments, and the time interval between each worker at the current moment and the previous adjacent analysis moment within the local preset range of the position of the industrial vehicle at the current moment; The vehicle speed adjustment module is used to obtain the presence index of personnel in the blind spot of the industrial vehicle at the current moment according to the difference between the distance between the position of the industrial vehicle at the current moment and the position of the personnel within its local preset range and the predicted travel distance; and adjust the speed of the industrial vehicle based on the presence index of personnel in the blind spot.

2. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 1 is characterized in that: The step of obtaining the predicted travel distance of each worker within a local preset range of the position of the industrial vehicle at the current moment includes: The set of all workers with the same attributes is recorded as a worker set; according to the position distribution and time interval of the workers in each worker set at two adjacent analysis moments, the predicted stay time and predicted travel speed of each worker set are obtained; Acquire analysts of the industrial vehicle at the current moment; the position of the analyst at the previous analysis moment adjacent to the current moment is within a local preset range of the position of the industrial vehicle at the current moment; calculate the difference between the time interval between each analyst at the current moment and the previous analysis moment adjacent to the current moment and the predicted stay time of the personnel set to which each analyst belongs, and use the product of the difference and the predicted travel speed of the personnel set to which each analyst belongs as the predicted travel distance of each analyst of the industrial vehicle at the current moment.

3. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 2 is characterized in that: The step of obtaining the predicted stay time and predicted travel speed of each set of personnel includes: Obtain a map of the industrial plant; use the roads in the map of the industrial plant as edges and the intersections of different roads as nodes to obtain a graph as a site map; Mark the position of each staff member at each analysis time in the site map to obtain the position node of each staff member at each analysis time; obtain the shortest path between the position nodes of each staff member at two adjacent analysis times in the site map, and record it as the movement path of each staff member at two adjacent analysis times; obtain the length of the movement path as the movement distance of each staff member at two adjacent analysis times; Obtaining the actual stay time of the staff between two adjacent analysis moments; and obtaining the actual travel speed of the staff between two adjacent analysis moments based on the moving distance and the actual stay time; Constructing a travel speed function of a worker between two adjacent analysis moments; substituting the time interval and movement distance of the worker at two adjacent analysis moments into the travel speed function to obtain a travel speed fitting value of the worker between two adjacent analysis moments; the travel speed fitting value corresponds to a residence time fitting value; Obtaining an error function for each personnel set according to the difference between the actual travel speed and the travel speed fitting value of the workers in each personnel set before all two adjacent analysis moments, and the difference between the actual stay time and the stay time fitting value; The residence time fitting value and the travel speed fitting value corresponding to the minimum value of the error function are respectively used as the predicted residence time and the predicted travel speed of each personnel set.

4. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 3 is characterized in that: The error function of each personnel set includes: The tuples consisting of all two adjacent analysis moments of each staff member before the current moment are recorded as the moment pair of each staff member; The error function of each personnel set is expressed as: , ; In the formula, RSS is the error function of each personnel set; N is the total number of staff in each personnel set; is the total number of time pairs of the nth staff member in each staff set; The fitting value of the travel speed of the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the actual travel speed between the two analysis moments in the k-th moment pair of the n-th worker in each worker set; is the fitted value of the stay time between the two analysis moments in the k-th moment pair of the n-th worker in each staff set; is the actual stay time between the two analysis moments in the k-th moment pair of the n-th worker in each worker set; is the moving distance between the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the time interval between the two analysis moments in the k-th moment pair of the n-th staff member in each staff set; is the absolute value function.

5. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 3 is characterized in that: The step of obtaining the personnel presence index of the blind spot of the industrial vehicle at the current moment includes: Obtain the shortest path between the position of the industrial vehicle at the current moment and the position of each analyst at the previous analysis moment adjacent to the current moment and the corresponding position nodes in the site map, and use the length of the shortest path as the vehicle distance between the industrial vehicle and each analyst at the current moment; The cumulative sum of the absolute values ​​of the differences between the vehicle distances of all the analysts at the current moment of the industrial vehicle and the predicted travel distance is negatively correlated and normalized to obtain the presence index of personnel in the blind spot of the industrial vehicle at the current moment.

6. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 1 is characterized in that: The adjusting the speed of the industrial vehicle based on the blind spot personnel presence indicator comprises: Obtain the blind area of ​​the industrial vehicle in the video frame image at the current moment; obtain the distance between the position of the industrial vehicle at the current moment and the blind area of ​​the visual field in the video frame image, recorded as the blind area spacing of the industrial vehicle at the current moment; If the blind spot distance of the industrial vehicle at the current moment is greater than the preset distance threshold, the current moment is updated until the blind spot distance of the industrial vehicle at the new current moment is equal to the preset distance threshold, the speed of the industrial vehicle at the new current moment is obtained, and the speed is adjusted using the blind spot personnel presence indicator of the industrial vehicle at the new current moment to obtain the corrected speed of the industrial vehicle at the new current moment; until there is no blind spot in the field of vision of the industrial vehicle in the video frame image at the new current moment, the industrial vehicle resumes its normal speed.

7. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 6 is characterized in that: The method of adjusting the speed of the industrial vehicle by using the personnel presence index in the blind spot of the industrial vehicle at the new current moment to obtain the corrected speed of the industrial vehicle at the new current moment includes: The difference between the speed of the industrial vehicle at the new current moment and a preset speed threshold is weighted by utilizing the personnel presence index in the blind spot of the industrial vehicle at the new current moment to obtain the correction degree of the industrial vehicle at the new current moment; the difference between the speed of the industrial vehicle at the new current moment and the correction degree is used as the corrected speed of the industrial vehicle at the new current moment.

8. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 3 is characterized in that: The obtaining of the actual travel speed of the worker between two adjacent analysis moments includes: The actual travel speed of the staff between the two adjacent analysis moments is obtained by taking the moving distance of the staff at two adjacent analysis moments as the numerator and the difference between the time interval between the two analysis moments and the actual stay time as the denominator.

9. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 3 is characterized in that: The method for obtaining the shortest path between the position nodes of each staff member at two adjacent analysis moments in the site map is an ant colony algorithm.

10. The highly intelligent and safe industrial vehicle ADAS system with personnel recognition function according to claim 6, characterized in that: The preset distance threshold is 10 meters.

Images