Man-vehicle shunting monitoring system and method based on machine vision
Through a human-vehicle diversion monitoring system based on machine vision, the trajectory of operators and vehicles in the garbage transfer station is identified, the overlap of human-vehicle trajectory is calculated and abnormal, which solves the problem of untimely and inability to monitor in a timely manner caused by relying on manual monitoring in the existing technology, and realizes safety monitoring and early warning in the garbage transfer station.
Patent Information
- Application Number
- CN202411674147.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-21
- Publication Date
- 2025-03-14
AI Technical Summary
The existing garbage transfer station monitoring system relies on manual monitoring, which is prone to inadequate monitoring due to fatigue or negligence, and cannot issue a warning in time before the accident occurs, and cannot achieve a timely warning.
It provides a human-vehicle diversion monitoring system based on machine vision, obtains monitoring video stream data through a camera, draws frames to form a frame image set, recognizes operators and vehicle trajectories, calculates the overlap of human-vehicle trajectories, judges trajectories abnormalities and sends early warnings.
Accurate, automated and timely monitoring and early warning of operators and vehicles in the garbage transfer station, and can promptly detect potential safety hazards and warn them to avoid accidents.
Smart Images
Figure CN119625828A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of garbage transfer station monitoring, and in particular to a human-vehicle diversion monitoring system and method based on machine vision. Background Art
[0002] In high-risk working environments such as garbage transfer stations, the collaborative work of operators and vehicles in the station is crucial. Garbage transfer stations are usually located on the edge of cities or in industrial areas. In high-risk working environments such as garbage transfer stations, operators and vehicles need to move frequently in a small space, resulting in a large flow of people and vehicles in the garbage transfer station. Therefore, the normal operation of the garbage transfer station, in which the coordination between operators and vehicles in the station is crucial. However, due to the complexity of the working environment and the frequent activities of people and vehicles, operators may be hit by moving vehicles due to obstructed vision or distracted attention when carrying garbage or performing other operations. Therefore, monitoring and early warning of garbage transfer stations are very important.
[0003] The existing monitoring system of garbage transfer stations mainly relies on manual monitoring and post-analysis. Although it can achieve the monitoring effect within the garbage transfer station, it still has the following problems:
[0004] Manual monitoring requires operators to focus on multiple monitoring screens for a long time, which may lead to untimely monitoring due to fatigue or negligence. In addition, manual monitoring relies on the operator's experience and judgment, which may easily lead to false alarms or missed alarms, especially in high-density pedestrian and vehicle traffic environments.
[0005] Usually, only post-analysis can be conducted after an accident occurs, and timely warnings cannot be issued before the accident occurs, thus missing the best time for intervention and failing to provide timely warnings.
[0006] There is an urgent need for a human-vehicle diversion monitoring system and method based on machine vision, which can realize accurate, automated and timely monitoring and early warning of the trajectories corresponding to operators and vehicles in a garbage transfer station. Summary of the invention
[0007] One of the purposes of the present invention is to provide a human-vehicle diversion monitoring system and method based on machine vision, which can realize accurate, automatic and timely monitoring and early warning of the trajectories corresponding to operators and vehicles in a garbage transfer station.
[0008] In order to achieve the above purpose, a human-vehicle separation monitoring system based on machine vision is provided, including a service end and a management end;
[0009] The server also includes:
[0010] The monitoring video acquisition module is used to conduct all-round monitoring of the garbage transfer station through cameras at preset monitoring points in the garbage transfer station to generate corresponding monitoring video stream data;
[0011] A frame extraction module is used to extract frame images from the corresponding monitoring video stream data to form a frame image set;
[0012] An image recognition module is used to identify the operators and vehicles in the station involved in each sub-frame image in the frame image set according to the formed frame image set, so as to form a set of personnel frame images corresponding to each operator and a set of vehicle frame images corresponding to each vehicle in the station;
[0013] A trajectory analysis module is used to input the personnel frame image set and the vehicle frame image set into the corresponding trajectory recognition models, and output the personnel trajectory data and vehicle trajectory data corresponding to the operating personnel and the vehicles in the station respectively;
[0014] The overlap calculation module is used to calculate the overlap between each operator and each vehicle in the station based on the personnel trajectory data and the vehicle trajectory data and the preset personnel trajectory overlap calculation strategy;
[0015] A judgment module is used to judge whether the overlap degree of each operator's and vehicle's trajectory is greater than or equal to a preset overlap threshold according to the overlap degree of each operator's and vehicle's trajectory in each station. If so, it is judged that the trajectory of the corresponding operator and vehicle in the station is abnormal; otherwise, it is judged that there is no abnormality in the trajectory of the corresponding operator and vehicle in the station;
[0016] The early warning module is used to retrieve a sub-frame image with the smallest time interval with the current time from the frame image set corresponding to the operator when the judgment result is that the trajectory of the corresponding operator and the vehicle in the station is abnormal, identify the identity information and location information of the operator based on the retrieved sub-frame image, and send an alarm message to the management end, wherein the alarm information is associated with the identity information and location information of the operator.
[0017] Technical principles and effects of this solution: In this solution, firstly, the garbage transfer station is monitored in all directions through health cameras installed at various preset monitoring points, so as to obtain the monitoring video stream data in the transfer station.
[0018] By extracting frames from the surveillance video stream data, a corresponding frame image set is formed. Then, by identifying the content in each sub-frame image in the frame image set, the vehicle frame image set of each vehicle in the station and the personnel frame image set corresponding to each operator can be mined and identified.
[0019] Afterwards, according to the personnel frame image and the vehicle frame image, they are input into the corresponding trajectory recognition model to realize the recognition of the vehicle trajectory data and personnel trajectory data corresponding to the vehicles in the station and the operators, and then the preset human-vehicle trajectory overlap calculation strategy is used to calculate the human-vehicle trajectory overlap between each operator and each vehicle in the station. Finally, based on the human-vehicle trajectory overlap between each operator and each vehicle in the station, it is judged whether the human-vehicle trajectory overlap is greater than or equal to the preset overlap threshold, so as to judge whether there is any abnormality in the trajectory between the operator and the vehicle in the station. Once an abnormality is found, the identity information and location information of the corresponding operator will be immediately identified to send an alarm message. In this way, the management personnel on the management end can know the identity information and location information of the operator involved in the first time when receiving the alarm message, and then quickly remind and inform the operator to ensure the safety of the life and property of the operator.
[0020] 1. Through all-round monitoring video stream data, we can grasp the dynamic situation in the garbage transfer station in real time and discover potential safety hazards in time.
[0021] 2. Based on the calculation of the overlap between the trajectory of people and vehicles, it can quickly identify the abnormal proximity between the operator and the vehicle in the station, and warn of possible dangerous situations in advance. When an abnormality is detected, the system can retrieve the frame image closest to the current time, accurately identify the identity information and location information of the operator, and provide a basis for taking timely measures, thereby realizing the diversion of people and vehicles in the garbage transfer station.
[0022] Furthermore, the preset human-vehicle trajectory overlap calculation strategy includes the following steps:
[0023] S100, initializing the coincidence point counter;
[0024] S100, selecting a vehicle from each of the vehicles in the station, and judging whether there is a corresponding personnel trajectory point on the vehicle trajectory point of each vehicle in the station according to the vehicle trajectory point corresponding to the selected vehicle in the station and the personnel trajectory point corresponding to each operator based on preset restriction conditions, if so, the counter is incremented by 1 until the judgment of the selected vehicle in the station is completed, and S100 is executed again;
[0025] The restrictions are:
[0026]
[0027] Where d is the spatial distance between the vehicle trajectory point of the vehicle in the station and the operator's trajectory point, (x ci ,y ci ,t ci) is the coordinates and time corresponding to the i-th trajectory point of a certain operator, (x rj ,y rj ,t rj ) is the coordinates and time corresponding to the jth vehicle trajectory point of the vehicle in the station, m is the total number of personnel trajectory points of a certain operator, n is the total number of vehicle trajectory points of vehicles in a certain station, d threshold is the spatial distance threshold, t threshold is the time interval threshold;
[0028] S100, after completing the determination of each vehicle in the station, based on a preset human-vehicle trajectory coincidence calculation formula, calculating the human-vehicle trajectory coincidence degree between the selected vehicle in the station and other operators;
[0029] The calculation formula of the human-vehicle trajectory overlap is as follows:
[0030]
[0031] Where R is the overlap of human and vehicle trajectories, and C is the count value of the counter corresponding to the vehicle in the station after the judgment is completed.
[0032] Beneficial effects: In this solution, the positional relationship between the numerous points corresponding to the vehicles and other operators in the station is determined by setting restrictions on the vehicles and other operators, so as to achieve accurate processing of multiple data volumes very quickly. That is, by analyzing the overlap of vehicle and pedestrian trajectories, potential collision risk points can be identified in advance, and timely measures can be taken to avoid accidents.
[0033] Furthermore, the server also includes:
[0034] A helmet recognition module is used to detect whether each operator is wearing a helmet based on the set of personnel frame images corresponding to each operator and the preset helmet target detection model, and output the helmet wearing data corresponding to each operator at each time point;
[0035] A work clothes detection module is used to detect whether each operator is wearing a work clothes according to the set of personnel frame images corresponding to each operator and based on a preset work clothes detection model, and output the work clothes wearing data corresponding to each operator at each time point;
[0036] The early warning module is also used to send an early warning reminder message to the management end when the safety helmet wearing data shows that the safety helmet is not worn or the work clothes wearing data shows that the work clothes are worn.
[0037] Beneficial effects: By real-time monitoring of operators' helmet and work clothes, it is possible to promptly detect situations where safety equipment is not worn as required, thus reducing potential safety hazards. Once an abnormality is detected, the system can immediately send an early warning reminder to the management end, so that management personnel can quickly take measures to correct violations and avoid accidents.
[0038] By setting up the safety helmet target detection model and the work clothes detection model, automatic detection and early warning of whether the operator is wearing a safety helmet and work clothes can be achieved, reducing the workload of manual inspection and improving management efficiency.
[0039] Furthermore, the server also includes:
[0040] The model construction training module is used to construct a personnel trajectory recognition model corresponding to the operator based on a set of personnel frame images with annotated content in the historical database, and to train and verify the constructed trajectory recognition model. When the verification conditions are met, the corresponding trajectory recognition model is judged to be a feasible model, and the trajectory recognition model corresponding to the operator is output.
[0041] Beneficial effects: In this solution, a large amount of annotated data in the historical database is used for model training, which enables the model to learn more diverse features and improve the accuracy and robustness of the model. Through diversified training data, the model can better generalize to unseen data and reduce overfitting.
[0042] Furthermore, the model building training module includes:
[0043] A data retrieval module is used to retrieve a set of person frame images with annotated content from a historical database, and divide the corresponding person frame image set into a training set and a test set according to a preset division ratio;
[0044] A construction module, used to construct a personnel trajectory recognition model corresponding to the operator;
[0045] The first training module is used to randomly generate a number of model parameter groups based on the constructed personnel trajectory recognition model, and input the personnel frame image set with annotated content in the training set into the personnel trajectory recognition model corresponding to each model parameter group, and output the model output result corresponding to each model parameter group;
[0046] A rationality calculation module is used to calculate the rationality of parameter settings corresponding to each model parameter group based on the model output results corresponding to each model parameter group and the annotation content corresponding to each model output result, based on a preset parameter setting rationality calculation formula;
[0047] The calculation formula for the rationality of parameter settings is:
[0048]
[0049] In the formula, H is the rationality of parameter setting, x lk ,y lk is the coordinate data in the model output result corresponding to the model parameter group, is the annotation coordinate data in the annotation content corresponding to each model output result, K is the total number of model output results; ∝, γ is the corresponding weighting coefficient;
[0050] A selection module is used to select a preset number of model parameter groups based on a preset parameter group selection strategy according to the rationality of the parameter settings corresponding to each model parameter group;
[0051] An iterative update module is used to randomly exchange parameter values corresponding to the same parameter type in each model parameter group according to each selected model parameter group to form a new model parameter group;
[0052] The second training module is used to input the set of personnel frame images with annotated content in the training set into the personnel trajectory recognition model corresponding to the new model parameter group according to the newly formed model parameter group, and output the model output result corresponding to the new model parameter group;
[0053] An evaluation module is used to calculate the error evaluation value corresponding to each new model parameter group according to the model output result corresponding to the new model parameter group, and select the new model parameter group corresponding to the error evaluation value less than the preset error evaluation threshold, and re-execute the iterative update module until the number of iterations meets the preset number of iterations, at which time the model parameter group with the smallest corresponding error evaluation value is output, and a personnel trajectory recognition model corresponding to the corresponding model parameter group is formed;
[0054] The verification module is used to input the set of personnel frame images with annotated content in the verification set as input data into the corresponding model according to the personnel trajectory recognition model formed by the evaluation module, output the corresponding verification output result, and judge whether the corresponding personnel trajectory recognition model is feasible based on the verification output result and the corresponding annotation content. If so, the final personnel trajectory recognition model is output; if not, the selection module is re-executed.
[0055] Beneficial effects: In this solution, by retrieving a set of human frame images with annotated content from the historical database, the diversity and representativeness of the model training data are ensured. This not only improves the generalization ability of the model, but also enhances the robustness of the model in the face of complex and changing environments.
[0056] Through multiple rounds of parameter optimization and iterative updates, the performance of the model is gradually improved. Each iteration is based on the previous optimization results to ensure that the model can accurately identify the operator's trajectory in different scenarios. That is, in this solution, the first step is to screen the model parameter group with a large amount of data by using the rationality of parameter settings. Then, through the selected model parameter groups, the parameter values of the same parameter type in each model parameter group are randomly exchanged to form a new model parameter group, and then the subsequent error evaluation value is calculated. In this way, multiple iterations are performed to generate the optimal model parameter group.
[0057] From data preparation, model building, parameter optimization to model evaluation, the entire process is highly automated, reducing manual intervention and improving modeling efficiency. This not only saves time and resources, but also reduces the possibility of human error.
[0058] Furthermore, the preset parameter group selection strategy is:
[0059] According to the rationality of parameter settings corresponding to each model parameter group, the parameter settings are sorted from small to large to form a rationality ranking table;
[0060] According to the formed rationality ranking table, the rationality of the parameter setting ranked in the middle of the rationality ranking table is selected, and it is determined whether the rationality of the parameter setting is less than a preset first threshold value. If so, a preset number of model parameter groups are selected in order from front to back;
[0061] Otherwise, a random number between 0 and 1 is randomly generated. If the random number is greater than the random threshold, a preset number of model parameter groups are first selected in order from the front to the back, and based on the preset number, a corresponding selection ratio is determined. According to the determined selection ratio, in order from the back to the front, the corresponding model parameter group is selected, and the twice selected model parameter groups are merged to form the final selected model parameters;
[0062] If the random number is less than or equal to the random threshold, no selection is made this time and the first training module is executed again.
[0063] Beneficial effects: Through the rationality ranking table and multi-stage selection strategy, the optimal model parameter group can be selected in different situations. This strategy not only improves the flexibility of selection, but also enhances the efficiency of model optimization. According to the rationality of parameter settings and the judgment of random numbers, the selection strategy is dynamically adjusted to ensure that the best model parameter group can be found in different stages.
[0064] By judging whether the rationality of the parameter setting ranked in the middle of the rationality ranking table is less than the preset first threshold, it is ensured that the selected model parameter group has a certain rationality and stability. Introduction of random numbers: Introducing random numbers for dynamic adjustment increases the randomness and diversity of model optimization, helps avoid local optimal solutions, and improves the global optimization ability of the model.
[0065] The present invention also provides a method for monitoring the flow of people and vehicles based on machine vision, using the above-mentioned monitoring system for monitoring the flow of people and vehicles based on machine vision, comprising the following steps:
[0066] S1. Conduct all-round monitoring of the garbage transfer station through cameras at preset monitoring points in the garbage transfer station to generate corresponding monitoring video stream data;
[0067] S2, extracting frame images from the corresponding monitoring video stream data to form a frame image set;
[0068] S3, according to the formed frame image set, identifying the operators and vehicles in the station involved in each sub-frame image in the frame image set, to form a personnel frame image set corresponding to each operator, and a vehicle frame image set corresponding to each vehicle in the station;
[0069] S4, inputting the personnel frame image set and the vehicle frame image set into the corresponding trajectory recognition models, and outputting the personnel trajectory data and the vehicle trajectory data corresponding to the operator and the vehicle in the station respectively;
[0070] S5. Calculate the overlap between each operator and each vehicle in the station based on the personnel trajectory data and the vehicle trajectory data and the preset personnel trajectory overlap calculation strategy;
[0071] S6. According to the overlap degree between the trajectory of each operator and each vehicle in the station, it is determined whether the overlap degree between each operator and each vehicle in the station is greater than or equal to a preset overlap threshold. If so, it is determined that the trajectory of the corresponding operator and the vehicle in the station is abnormal. Otherwise, it is determined that there is no abnormality in the trajectory of the corresponding operator and the vehicle in the station.
[0072] S7. When the judgment result is that the trajectory of the corresponding operator and the vehicle in the station is abnormal, a sub-frame image with the smallest time interval with the current time is retrieved from the frame image set corresponding to the operator, and the identity information and location information of the operator are identified based on the retrieved sub-frame image, and an alarm message is sent to the management end, and the alarm message is associated with the identity information and location information of the operator. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1This is a logic block diagram of a human-vehicle separation monitoring system based on machine vision in Embodiment 1 of the present invention;
[0074] Figure 2 This is a flow chart of a human-vehicle separation monitoring method based on machine vision in Embodiment 1 of the present invention. DETAILED DESCRIPTION
[0075] The following is further described in detail through specific implementation methods:
[0076] Embodiment 1
[0077] A human-vehicle separation monitoring system based on machine vision is basically as follows Figure 1 As shown, it includes the server and the management side;
[0078] The server also includes:
[0079] The monitoring video acquisition module is used to monitor the garbage transfer station in all directions through the cameras at the preset monitoring points in the garbage transfer station to form corresponding monitoring video stream data; in this embodiment, the preset monitoring points are set in all directions, for example, multiple points are set in the unloading hall, the transfer hall, the hoisting area square, etc. Different areas have different numbers of points.
[0080] A frame extraction module is used to extract frame images from the corresponding monitoring video stream data to form a frame image set;
[0081] An image recognition module is used to identify the operators and vehicles in the station involved in each sub-frame image in the frame image set according to the formed frame image set, so as to form a set of personnel frame images corresponding to each operator and a set of vehicle frame images corresponding to each vehicle in the station;
[0082] A trajectory analysis module is used to input the personnel frame image set and the vehicle frame image set into the corresponding trajectory recognition models, and output the personnel trajectory data and vehicle trajectory data corresponding to the operating personnel and the vehicles in the station respectively;
[0083] The overlap calculation module is used to calculate the overlap between each operator and each vehicle in the station based on the personnel trajectory data and the vehicle trajectory data and the preset personnel trajectory overlap calculation strategy;
[0084] The preset human-vehicle trajectory overlap calculation strategy includes the following steps:
[0085] S100, initializing the coincidence point counter;
[0086] S100, selecting a vehicle from each of the vehicles in the station, and judging whether there is a corresponding personnel trajectory point on the vehicle trajectory point of each vehicle in the station according to the vehicle trajectory point corresponding to the selected vehicle in the station and the personnel trajectory point corresponding to each operator based on preset restriction conditions, if so, the counter is incremented by 1 until the judgment of the selected vehicle in the station is completed, and S100 is executed again;
[0087] The restrictions are:
[0088]
[0089] Where d is the spatial distance between the vehicle trajectory point of the vehicle in the station and the operator's trajectory point, (x ci ,y ci ,t ci ) is the coordinates and time corresponding to the i-th trajectory point of a certain operator, (x rj ,y rj ,t rj ) is the coordinates and time corresponding to the jth vehicle trajectory point of the vehicle in the station, m is the total number of personnel trajectory points of a certain operator, n is the total number of vehicle trajectory points of vehicles in a certain station, d threshold is the spatial distance threshold, t threshold is the time interval threshold;
[0090] S100, after completing the determination of each vehicle in the station, based on a preset human-vehicle trajectory coincidence calculation formula, calculating the human-vehicle trajectory coincidence degree between the selected vehicle in the station and other operators;
[0091] The calculation formula of the human-vehicle trajectory overlap is as follows:
[0092]
[0093] Where R is the overlap of human and vehicle trajectories, and C is the count value of the counter corresponding to the vehicle in the station after the judgment is completed.
[0094] A judgment module is used to judge whether the overlap degree of each operator's and vehicle's trajectory is greater than or equal to a preset overlap threshold according to the overlap degree of each operator's and vehicle's trajectory in each station. If so, it is judged that the trajectory of the corresponding operator and vehicle in the station is abnormal; otherwise, it is judged that there is no abnormality in the trajectory of the corresponding operator and vehicle in the station;
[0095] The early warning module is used to retrieve a sub-frame image with the smallest time interval with the current time from the frame image set corresponding to the operator when the judgment result is that the trajectory of the corresponding operator and the vehicle in the station is abnormal, identify the identity information and location information of the operator based on the retrieved sub-frame image, and send an alarm message to the management end, wherein the alarm message is associated with the identity information and location information of the operator. In this embodiment, when the management personnel at the management end view the alarm information, the identity information and location information of the operator associated with the alarm information will be retrieved at the first time, so that the management personnel can view the relevant information in time and carry out dissuasion and warning as soon as possible.
[0096] The server also includes:
[0097] The safety helmet recognition module is used to detect whether each operator is wearing a safety helmet based on the set of personnel frame images corresponding to each operator and based on the preset safety helmet target detection model, and output the safety helmet wearing data corresponding to each operator at each time point; in this embodiment, the safety helmet wearing data not only includes whether the safety helmet is worn, but also identifies the type of the safety helmet after the safety helmet is worn.
[0098] A work clothes detection module is used to detect whether each operator is wearing a work clothes according to the set of personnel frame images corresponding to each operator and based on a preset work clothes detection model, and output the work clothes wearing data corresponding to each operator at each time point;
[0099] The early warning module is also used to send an early warning reminder message to the management end when the safety helmet wearing data shows that the safety helmet is not worn or the work clothes wearing data shows that the work clothes are worn.
[0100] The server also includes:
[0101] The model construction training module is used to construct a personnel trajectory recognition model corresponding to the operator based on a set of personnel frame images with annotated content in the historical database, and to train and verify the constructed trajectory recognition model. When the verification conditions are met, the corresponding trajectory recognition model is judged to be a feasible model, and the trajectory recognition model corresponding to the operator is output.
[0102] The model building training module includes:
[0103] A data retrieval module is used to retrieve a set of person frame images with annotated content from a historical database, and divide the corresponding person frame image set into a training set and a test set according to a preset division ratio;
[0104] The construction module is used to construct the personnel trajectory recognition model corresponding to the operator; the personnel trajectory recognition model, the vehicle trajectory recognition model, the helmet target detection model and the work clothes detection model can all adopt BP neural network or neural network based on multiple convolutional layers. Of course, the YOLOV7 network or target detection technology can also be used to construct the model. In this embodiment, the personnel trajectory recognition model adopts BP neural network technology. Among them, the helmet target detection model and the work clothes detection model adopt the existing target detection technology to construct the corresponding model.
[0105] The first training module is used to randomly generate a number of model parameter groups based on the constructed personnel trajectory recognition model, and input the personnel frame image set with annotated content in the training set into the personnel trajectory recognition model corresponding to each model parameter group, and output the model output result corresponding to each model parameter group;
[0106] A rationality calculation module is used to calculate the rationality of parameter settings corresponding to each model parameter group based on the model output results corresponding to each model parameter group and the annotation content corresponding to each model output result, based on a preset parameter setting rationality calculation formula;
[0107] The calculation formula for the rationality of parameter settings is:
[0108]
[0109] In the formula, H is the rationality of parameter setting, x lk ,y lk is the coordinate data in the model output result corresponding to the model parameter group, is the annotation coordinate data in the annotation content corresponding to each model output result, K is the total number of model output results; ∝, γ is the corresponding weighting coefficient;
[0110] A selection module is used to select a preset number of model parameter groups based on a preset parameter group selection strategy according to the rationality of the parameter settings corresponding to each model parameter group;
[0111] The preset parameter group selection strategy is:
[0112] According to the rationality of parameter settings corresponding to each model parameter group, the parameter settings are sorted from small to large to form a rationality ranking table;
[0113] According to the formed rationality ranking table, the rationality of the parameter setting ranked in the middle in the rationality ranking table is selected to determine whether the rationality of the parameter setting is less than the preset first threshold. If so, a preset number of model parameter groups are selected in order from front to back; in this embodiment, if the total number of the rationality ranking table is an even number, then the one ranked in the middle is one of the two middle numbers in the rationality ranking table, which is randomly selected.
[0114] Otherwise, a random number between 0 and 1 is randomly generated. If the random number is greater than the random threshold, a preset number of model parameter groups are first selected in order from the front to the back, and based on the preset number, a corresponding selection ratio is determined. According to the determined selection ratio, in order from the back to the front, the corresponding model parameter group is selected, and the twice selected model parameter groups are merged to form the final selected model parameters;
[0115] If the random number is less than or equal to the random threshold, no selection is made this time and the first training module is executed again.
[0116] An iterative update module is used to randomly exchange parameter values corresponding to the same parameter type in each model parameter group according to each selected model parameter group to form a new model parameter group;
[0117] The second training module is used to input the set of personnel frame images with annotated content in the training set into the personnel trajectory recognition model corresponding to the new model parameter group according to the newly formed model parameter group, and output the model output result corresponding to the new model parameter group;
[0118] The evaluation module is used to calculate the error evaluation value corresponding to each new model parameter group according to the model output result corresponding to the new model parameter group, and select the new model parameter group corresponding to the error evaluation value less than the preset error evaluation threshold, and re-execute the iterative update module until the number of iterations meets the preset number of iterations. At this time, the model parameter group with the smallest corresponding error evaluation value is output, and a personnel trajectory recognition model corresponding to the corresponding model parameter group is formed; in this embodiment, the error evaluation value is calculated by the error mean square error.
[0119] The verification module is used to input the set of personnel frame images with annotated content in the verification set as input data into the corresponding model according to the personnel trajectory recognition model formed by the evaluation module, output the corresponding verification output result, and judge whether the corresponding personnel trajectory recognition model is feasible based on the verification output result and the corresponding annotation content. If so, the final personnel trajectory recognition model is output; if not, the selection module is re-executed.
[0120] like Figure 2As shown, this embodiment also discloses a method for monitoring the flow of people and vehicles based on machine vision, using the above-mentioned system for monitoring the flow of people and vehicles based on machine vision, including the following steps:
[0121] S1. Conduct all-round monitoring of the garbage transfer station through cameras at preset monitoring points in the garbage transfer station to generate corresponding monitoring video stream data;
[0122] S2, extracting frame images from the corresponding monitoring video stream data to form a frame image set;
[0123] S3, according to the formed frame image set, identifying the operators and vehicles in the station involved in each sub-frame image in the frame image set, to form a personnel frame image set corresponding to each operator, and a vehicle frame image set corresponding to each vehicle in the station;
[0124] S4, inputting the personnel frame image set and the vehicle frame image set into the corresponding trajectory recognition models, and outputting the personnel trajectory data and the vehicle trajectory data corresponding to the operator and the vehicle in the station respectively;
[0125] S5. Calculate the overlap between each operator and each vehicle in the station based on the personnel trajectory data and the vehicle trajectory data and the preset personnel trajectory overlap calculation strategy;
[0126] S6. According to the overlap degree between the trajectory of each operator and each vehicle in the station, it is determined whether the overlap degree between each operator and each vehicle in the station is greater than or equal to a preset overlap threshold. If so, it is determined that the trajectory of the corresponding operator and the vehicle in the station is abnormal. Otherwise, it is determined that there is no abnormality in the trajectory of the corresponding operator and the vehicle in the station.
[0127] S7. When the judgment result is that the trajectory of the corresponding operator and the vehicle in the station is abnormal, a sub-frame image with the smallest time interval with the current time is retrieved from the frame image set corresponding to the operator, and the identity information and location information of the operator are identified based on the retrieved sub-frame image, and an alarm message is sent to the management end, and the alarm message is associated with the identity information and location information of the operator.
[0128] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is described too much here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can know all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.
Claims
1. A human-vehicle separation monitoring system based on machine vision, characterized by: Including server and management side; The server also includes: The monitoring video acquisition module is used to conduct all-round monitoring of the garbage transfer station through cameras at preset monitoring points in the garbage transfer station to generate corresponding monitoring video stream data; A frame extraction module is used to extract frame images from the corresponding monitoring video stream data to form a frame image set; An image recognition module is used to identify the operators and vehicles in the station involved in each sub-frame image in the frame image set according to the formed frame image set, so as to form a set of personnel frame images corresponding to each operator and a set of vehicle frame images corresponding to each vehicle in the station; A trajectory analysis module is used to input the personnel frame image set and the vehicle frame image set into the corresponding trajectory recognition models, and output the personnel trajectory data and vehicle trajectory data corresponding to the operating personnel and the vehicles in the station respectively; The overlap calculation module is used to calculate the overlap between each operator and each vehicle in the station based on the personnel trajectory data and the vehicle trajectory data and the preset personnel trajectory overlap calculation strategy; A judgment module is used to judge whether the overlap degree of each operator's and vehicle's trajectory is greater than or equal to a preset overlap threshold according to the overlap degree of each operator's and vehicle's trajectory in each station. If so, it is judged that the trajectory of the corresponding operator and vehicle in the station is abnormal; otherwise, it is judged that there is no abnormality in the trajectory of the corresponding operator and vehicle in the station; The early warning module is used to retrieve a sub-frame image with the smallest time interval with the current time from the frame image set corresponding to the operator when the judgment result is that the trajectory of the corresponding operator and the vehicle in the station is abnormal, identify the identity information and location information of the operator based on the retrieved sub-frame image, and send an alarm message to the management end, wherein the alarm information is associated with the identity information and location information of the operator.
2. The machine vision-based monitoring system for separating people and vehicles according to claim 1 is characterized by: The preset human-vehicle trajectory overlap calculation strategy includes the following steps: S100, initializing the coincidence point counter; S100, selecting a vehicle from each of the vehicles in the station, and judging whether there is a corresponding personnel trajectory point on the vehicle trajectory point of each vehicle in the station according to the vehicle trajectory point corresponding to the selected vehicle in the station and the personnel trajectory point corresponding to each operator based on preset restriction conditions, if so, the counter is incremented by 1 until the judgment of the selected vehicle in the station is completed, and S100 is executed again; The restrictions are: Where d is the spatial distance between the vehicle trajectory point of the vehicle in the station and the operator's trajectory point, (x ci ,y ci ,t ci ) is the coordinates and time corresponding to the i-th trajectory point of a certain operator, (x rj ,y rj ,t rj ) is the coordinates and time corresponding to the jth vehicle trajectory point of the vehicle in the station, m is the total number of personnel trajectory points of a certain operator, n is the total number of vehicle trajectory points of vehicles in a certain station, d threshold is the spatial distance threshold, t threshold is the time interval threshold; S100, after completing the determination of each vehicle in the station, based on a preset human-vehicle trajectory coincidence calculation formula, calculating the human-vehicle trajectory coincidence degree between the selected vehicle in the station and other operators; The calculation formula of the human-vehicle trajectory overlap is as follows: Where R is the overlap of human and vehicle trajectories, and C is the count value of the counter corresponding to the vehicle in the station after the judgment is completed.
3. The machine vision-based monitoring system for separating people and vehicles according to claim 2 is characterized by: The server also includes: A helmet recognition module is used to detect whether each operator is wearing a helmet based on the set of personnel frame images corresponding to each operator and the preset helmet target detection model, and output the helmet wearing data corresponding to each operator at each time point; A work clothes detection module is used to detect whether each operator is wearing a work clothes according to the set of personnel frame images corresponding to each operator and based on a preset work clothes detection model, and output the work clothes wearing data corresponding to each operator at each time point; The early warning module is also used to send an early warning reminder message to the management end when the safety helmet wearing data shows that the safety helmet is not worn or the work clothes wearing data shows that the work clothes are worn.
4. The machine vision-based monitoring system for separating people and vehicles according to claim 1 is characterized by: The server also includes: The model construction training module is used to construct a personnel trajectory recognition model corresponding to the operator based on a set of personnel frame images with annotated content in the historical database, and to train and verify the constructed trajectory recognition model. When the verification conditions are met, the corresponding trajectory recognition model is judged to be a feasible model, and the trajectory recognition model corresponding to the operator is output.
5. The machine vision-based monitoring system for separating people and vehicles according to claim 4 is characterized by: The model building training module includes: A data retrieval module is used to retrieve a set of person frame images with annotated content from a historical database, and divide the corresponding person frame image set into a training set and a test set according to a preset division ratio; A construction module, used to construct a personnel trajectory recognition model corresponding to the operator; The first training module is used to randomly generate a number of model parameter groups based on the constructed personnel trajectory recognition model, and input the personnel frame image set with annotated content in the training set into the personnel trajectory recognition model corresponding to each model parameter group, and output the model output result corresponding to each model parameter group; A rationality calculation module is used to calculate the rationality of parameter settings corresponding to each model parameter group based on the model output results corresponding to each model parameter group and the annotation content corresponding to each model output result, based on a preset parameter setting rationality calculation formula; The calculation formula for the rationality of parameter settings is: In the formula, H is the rationality of parameter setting, x lk ,y lk is the coordinate data in the model output result corresponding to the model parameter group, is the annotation coordinate data in the annotation content corresponding to each model output result, K is the total number of model output results; ∝, γ is the corresponding weighting coefficient; A selection module is used to select a preset number of model parameter groups based on a preset parameter group selection strategy according to the rationality of the parameter settings corresponding to each model parameter group; An iterative update module is used to randomly exchange parameter values corresponding to the same parameter type in each model parameter group according to each selected model parameter group to form a new model parameter group; The second training module is used to input the set of personnel frame images with annotated content in the training set into the personnel trajectory recognition model corresponding to the new model parameter group according to the newly formed model parameter group, and output the model output result corresponding to the new model parameter group; An evaluation module is used to calculate the error evaluation value corresponding to each new model parameter group according to the model output result corresponding to the new model parameter group, and select the new model parameter group corresponding to the error evaluation value less than the preset error evaluation threshold, and re-execute the iterative update module until the number of iterations meets the preset number of iterations, at which time the model parameter group with the smallest corresponding error evaluation value is output, and a personnel trajectory recognition model corresponding to the corresponding model parameter group is formed; The verification module is used to input the set of personnel frame images with annotated content in the verification set as input data into the corresponding model according to the personnel trajectory recognition model formed by the evaluation module, output the corresponding verification output result, and judge whether the corresponding personnel trajectory recognition model is feasible based on the verification output result and the corresponding annotation content. If so, the final personnel trajectory recognition model is output; if not, the selection module is re-executed.
6. The machine vision-based monitoring system for separating people and vehicles according to claim 5 is characterized by: The preset parameter group selection strategy is: According to the rationality of parameter settings corresponding to each model parameter group, the parameter settings are sorted from small to large to form a rationality ranking table; According to the formed rationality ranking table, the rationality of the parameter setting ranked in the middle of the rationality ranking table is selected, and it is determined whether the rationality of the parameter setting is less than a preset first threshold value. If so, a preset number of model parameter groups are selected in order from front to back; Otherwise, a random number between 0 and 1 is randomly generated. If the random number is greater than the random threshold, a preset number of model parameter groups are first selected in order from the front to the back, and based on the preset number, a corresponding selection ratio is determined. According to the determined selection ratio, in order from the back to the front, the corresponding model parameter group is selected, and the twice selected model parameter groups are merged to form the final selected model parameters; If the random number is less than or equal to the random threshold, no selection is made this time and the first training module is executed again.
7. A method for monitoring the separation of vehicles and pedestrians based on machine vision, using a monitoring system for monitoring the separation of vehicles and pedestrians based on machine vision according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1. Conduct all-round monitoring of the garbage transfer station through cameras at preset monitoring points in the garbage transfer station to generate corresponding monitoring video stream data; S2, extracting frame images from the corresponding monitoring video stream data to form a frame image set; S3, according to the formed frame image set, identifying the operators and vehicles in the station involved in each sub-frame image in the frame image set, to form a personnel frame image set corresponding to each operator, and a vehicle frame image set corresponding to each vehicle in the station; S4, inputting the personnel frame image set and the vehicle frame image set into the corresponding trajectory recognition models, and outputting the personnel trajectory data and the vehicle trajectory data corresponding to the operating personnel and the vehicles in the station respectively; S5. Calculate the overlap between each operator and each vehicle in the station based on the personnel trajectory data and the vehicle trajectory data and the preset personnel trajectory overlap calculation strategy; S6. According to the overlap degree between the trajectory of each operator and each vehicle in the station, it is determined whether the overlap degree between each operator and each vehicle in the station is greater than or equal to a preset overlap threshold. If so, it is determined that the trajectory of the corresponding operator and the vehicle in the station is abnormal. Otherwise, it is determined that there is no abnormality in the trajectory of the corresponding operator and the vehicle in the station. S7. When the judgment result is that the trajectory of the corresponding operator and the vehicle in the station is abnormal, a sub-frame image with the smallest time interval with the current time is retrieved from the frame image set corresponding to the operator, and the identity information and location information of the operator are identified based on the retrieved sub-frame image, and an alarm message is sent to the management end, and the alarm message is associated with the identity information and location information of the operator.