A pedestrian recognition method and system based on image and motion behavior prediction

Abstract

The present invention discloses a pedestrian recognition method and system based on image and motion behavior prediction, and relates to the field of computer vision technology. The method includes: collecting and analyzing environmental and pedestrian information to obtain target environmental and pedestrian information; predicting the motion behavior trajectory of pedestrians based on the GCN network and DeepSort algorithm in combination with the target environmental and pedestrian information, and generating pedestrian trajectory prediction results; according to the pedestrian trajectory prediction results, the user's motion trajectory is formulated based on the RRT algorithm and the PRM algorithm; according to the pedestrian trajectory prediction results and the user's motion trajectory, the motion trajectory is simulated based on the multi-layer LSTM model and the Seq2Seq model to obtain simulation results; and safety warning information is generated and sent according to the simulation results. The present invention can simulate the scenarios that occur after the pedestrian motion prediction, improve the reliability of the motion route; can analyze and capture information in complex scenarios; and can also warn pedestrians and users during the motion process.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a pedestrian recognition method and system based on image and motion behavior prediction. Background Art

[0002] The research on pedestrian recognition technology began in the mid-1990s. With the rise of deep learning technology, the detection accuracy and speed have been significantly improved. Pedestrian recognition technology covers pedestrian detection and pedestrian re-identification. It is the process of automatically identifying pedestrians from images or video sequences using computer vision technology. Pedestrian detection is to analyze the pixels in the image through machine learning or deep learning algorithms, extract human features such as shape, size and color, and then determine whether there are pedestrians. Pedestrian re-identification is a technology to determine whether there are specific pedestrians in the image or video sequence. It is a sub-problem of image retrieval, which aims to make up for the visual limitations of fixed cameras and is widely used in intelligent video surveillance and intelligent security. Motion behavior prediction uses the observed historical trajectory and environmental information to predict the future position of pedestrians. Motion behavior prediction is important for fields such as autonomous driving, robot navigation and video surveillance. It can significantly reduce the risk of collision and improve system efficiency. Traditional model-driven methods are difficult to cope with complex and high-dynamic scenes, and the prediction effect is poor. Relying on large-scale data collection and data-driven methods can better capture complex pedestrian interactions and achieve accurate prediction.

[0003] Pedestrians appear in complex backgrounds, such as crowded crowds, cluttered or occluded backgrounds, and conventional pedestrian detection algorithms find it difficult to accurately locate and identify pedestrian targets. The size and angle of pedestrians in the image will change due to factors such as shooting distance and angle, causing the appearance and shape of the target to change, resulting in pedestrian detection algorithms usually requiring a large amount of computing resources and time. At the same time, pedestrians are occluded and overlapped in the display scene. Especially in multi-target detection, pedestrian detection algorithms find it difficult to accurately detect target boundaries and positions. Pedestrian movements are flexible and changeable, making it difficult to establish a reasonable dynamic model. The motion trajectory is affected by the pedestrian's intentions and the surrounding environment, making it difficult to predict.

[0004] The existing technology cannot simulate the scenarios that occur after the pedestrian movement is predicted, cannot analyze and capture information in complex scenarios, cannot filter the collected information, and cannot warn pedestrians and users during movement, resulting in collision accidents with pedestrians and jams during movement, wasting time, difficulty in capturing and identifying information that affects one's own movement route in complex scenarios, the collected information is cumbersome and difficult to analyze, privacy protection, distraction leading to collisions with pedestrians, and jams due to improper handling. Summary of the invention

[0005] In order to overcome the above problems or at least partially solve the above problems, the present invention provides a pedestrian recognition method and system based on image and motion behavior prediction, which can simulate the scenarios that occur after pedestrian motion prediction and improve the reliability of motion routes; can analyze and capture information in complex scenarios; and can also realize the function of warning pedestrians and users during motion.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] In a first aspect, the present invention provides a method for pedestrian recognition based on image and motion behavior prediction, comprising the following steps:

[0008] Collect and analyze environmental and pedestrian information to obtain target environmental and pedestrian information;

[0009] Based on the GCN network and DeepSort algorithm, the pedestrian's motion behavior trajectory is predicted by combining the target environment and pedestrian information to generate pedestrian trajectory prediction results;

[0010] According to the pedestrian trajectory prediction results, the user's motion trajectory is proposed based on the RRT algorithm and PRM algorithm;

[0011] According to the pedestrian trajectory prediction results and the user's motion trajectory, the motion trajectory simulation is performed based on the multi-layer LSTM model and Seq2Seq model to obtain the simulation results;

[0012] Generate and send safety warning information based on the simulation results.

[0013] The present invention can simulate the scenarios that occur after pedestrian movement prediction, can analyze and capture information in complex scenarios, can filter the collected information, and can warn pedestrians and users during movement.

[0014] Based on the first aspect, further, the method for collecting and analyzing environment and pedestrian information to obtain target environment and pedestrian information includes the following steps:

[0015] Collect environmental and pedestrian information;

[0016] Feature selection and feature extraction are performed on the environment and pedestrian information to obtain the target environment and pedestrian information.

[0017] Based on the first aspect, further, the pedestrian recognition method based on image and motion behavior prediction further includes the following steps:

[0018] Analyze the simulation results and optimize the proposed user motion trajectory.

[0019] Based on the first aspect, further, the pedestrian recognition method based on image and motion behavior prediction further includes the following steps:

[0020] Collect environmental and pedestrian information in real time;

[0021] According to the real-time collected environment and pedestrian information, the pedestrian's movement behavior is predicted based on the LSTM model, GCN network and DeepSort algorithm to generate real-time prediction results;

[0022] Formulate the user's movement trajectory based on real-time prediction results;

[0023] When the user moves along the set motion trajectory, the external environment and pedestrian information are collected in real time to update and adjust the motion trajectory.

[0024] Based on the first aspect, further, the method for generating and sending safety warning information according to the simulation results includes the following steps:

[0025] When the simulation results show that there is a risk of collision between the user and the pedestrian, a safety warning message is generated and sent to the user and / or the pedestrian.

[0026] In a second aspect, the present invention provides a pedestrian recognition system based on image and motion behavior prediction, comprising an information collection module, a pedestrian trajectory prediction module, a user trajectory planning module, a motion trajectory simulation module and a safety warning module, wherein:

[0027] An information collection module is used to collect and analyze environmental and pedestrian information to obtain target environmental and pedestrian information;

[0028] The pedestrian trajectory prediction module is used to predict the pedestrian's motion behavior trajectory based on the GCN network and DeepSort algorithm combined with the target environment and pedestrian information to generate pedestrian trajectory prediction results;

[0029] The user trajectory prediction module is used to predict the user's motion trajectory based on the RRT algorithm and the PRM algorithm according to the pedestrian trajectory prediction results;

[0030] The motion trajectory simulation module is used to simulate the motion trajectory based on the multi-layer LSTM model and Seq2Seq model according to the pedestrian trajectory prediction results and the user's motion trajectory to obtain the simulation results;

[0031] The safety warning module is used to generate and send safety warning information according to the simulation results.

[0032] Through the cooperation of multiple modules such as information collection module, pedestrian trajectory prediction module, user trajectory planning module, motion trajectory simulation module and safety warning module, this system can simulate the scenarios that occur after pedestrian movement prediction, analyze and capture information in complex scenarios, filter the collected information, and warn pedestrians and users during movement.

[0033] Based on the second aspect, further, the pedestrian recognition system based on image and motion behavior prediction also includes a trajectory optimization module for analyzing the simulation results and optimizing the proposed user motion trajectory.

[0034] Based on the second aspect, further, the above-mentioned information acquisition module includes a visual sensor, a force / torque sensor, a tactile sensor and an inertial measurement unit.

[0035] The present invention has at least the following advantages or beneficial effects:

[0036] 1. The present invention realizes the function of simulating the scenarios that occur after the pedestrian movement prediction, solves the problems of collision accidents between users and pedestrians and time wasting due to congestion during the movement, can avoid collisions between users and the movement trajectory of pedestrians during the movement, improves the safety performance of users, reduces the possibility of accidents, and improves the reliability of the movement route;

[0037] 2. The present invention realizes the function of analyzing and capturing information in complex scenes, solves the problem that it is difficult to capture and identify information that affects one's own movement route in complex scenes, can accurately capture specific information, reduce unnecessary information capture, speed up the prediction of pedestrian movement behavior, save time and cost, and improve work efficiency;

[0038] 3. The present invention realizes the function of filtering the collected information by screening the collected information, solves the problem that the collected information is complicated and difficult to analyze and protect privacy, can reduce information leakage, improve the efficiency of data collection, and improve the accuracy and reliability of information collection;

[0039] 4. The present invention realizes the function of warning pedestrians and users during movement by warning pedestrians and users, solves the problems of collision with pedestrians caused by distraction and congestion caused by improper handling, can avoid accidents caused by user distraction, improves safety and reliability, and reduces the possibility of accidents and the occurrence of congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A flowchart of a pedestrian recognition method based on image and motion behavior prediction according to an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of simulation and planned route in a pedestrian recognition method based on image and motion behavior prediction according to an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a flow chart of real-time adjustment of motion trajectory in a pedestrian recognition method based on image and motion behavior prediction according to an embodiment of the present invention;

[0044] Figure 4 A functional block diagram of a pedestrian recognition system based on image and motion behavior prediction according to an embodiment of the present invention;

[0045] Figure 5 A structural block diagram of an electronic device provided by an embodiment of the present invention.

[0046] Explanation of the reference numerals: 100, information collection module; 200, pedestrian trajectory prediction module; 300, user trajectory planning module; 400, motion trajectory simulation module; 500, safety warning module; 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0050] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0051] In the description of the embodiments of the present invention, "plurality" means at least 2.

[0052] Example:

[0053] like Figure 1-Figure 2 As shown, in a first aspect, an embodiment of the present invention provides a pedestrian recognition method based on image and motion behavior prediction, comprising the following steps:

[0054] S1. Collect and analyze the environment and pedestrian information to obtain target environment and pedestrian information;

[0055] Furthermore, it includes: collecting environment and pedestrian information; performing feature selection and feature extraction on the environment and pedestrian information to obtain target environment and pedestrian information.

[0056] In some embodiments of the present invention, after collecting information about the external environment and pedestrians, the collected information is screened through feature selection and feature extraction to filter out unimportant information, and the information is transmitted to a simulation module. The simulation module analyzes the received information and removes unnecessary information again, thereby realizing the function of filtering the collected information, solving the problem of the collected information being cumbersome and difficult to analyze and the problem of privacy protection, reducing information leakage, improving the efficiency of data collection, and improving the accuracy and reliability of information collection.

[0057] S2, based on the GCN network and DeepSort algorithm, the pedestrian's motion behavior trajectory is predicted in combination with the target environment and pedestrian information to generate the pedestrian trajectory prediction result;

[0058] S3, according to the pedestrian trajectory prediction results, the user's motion trajectory is formulated based on the RRT algorithm and the PRM algorithm;

[0059] S4. According to the pedestrian trajectory prediction results and the user's motion trajectory, a motion trajectory simulation is performed based on a multi-layer LSTM model and a Seq2Seq model to obtain a simulation result;

[0060] Furthermore, it also includes: analyzing the simulation results and optimizing the proposed user movement trajectory.

[0061] S5. Generate and send safety warning information according to the simulation results;

[0062] Furthermore, it includes: when the simulation result shows that there is a risk of collision between the user and the pedestrian, a safety warning message is generated and sent to the user and / or the pedestrian.

[0063] In some embodiments of the present invention, the motion trajectory of a pedestrian is predicted and the motion trajectory of the user is optimized. When the user moves according to the optimized motion trajectory, external information is collected in real time. When the motion trajectory of the pedestrian collides with the motion trajectory given by the user, the user is prompted, and the pedestrian is warned when approaching the pedestrian. The function of warning pedestrians and users during movement is realized, which solves the problems of collision with pedestrians due to distraction and congestion caused by improper handling, can avoid accidents caused by user distraction, improves safety and reliability, and reduces the possibility of accidents and the occurrence of congestion.

[0064] The present invention is implemented based on a pre-built simulation module, which predicts pedestrian movement behavior based on three algorithms: LSTM model, GCN network and DeepSort algorithm, proposes an action route for the user based on the prediction results, and optimizes the user's movement route after secondary simulation of the prediction results and the proposed action route, wherein the objects of the secondary simulation include the initial pedestrian movement behavior prediction results, the initial proposed user movement route and environmental changes.

[0065] The simulation module receives the information filtered by the information acquisition module, and predicts the pedestrian's movement behavior trajectory based on the GCN network and DeepSort algorithm. The simulation module proposes the user's movement trajectory based on the RRT algorithm and the PRM algorithm. The simulation module simulates the first predicted pedestrian movement trajectory and the proposed user movement trajectory by constructing a multi-layer LSTM model and a Seq2Seq model. The simulation module analyzes the simulation results and optimizes the proposed user movement trajectory. It realizes the function of simulating the scenario after the pedestrian movement prediction, solves the problem of collision accidents between users and pedestrians and congestion and time waste during movement, and can avoid collisions between users and pedestrians during movement. The movement trajectory of pedestrians improves the safety performance of users, reduces the possibility of accidents, and improves the reliability of movement routes.

[0066] The present invention can simulate the scenarios that occur after pedestrian movement prediction, can analyze and capture information in complex scenarios, can filter the collected information, and can warn pedestrians and users during movement.

[0067] like Figure 3 As shown, based on the first aspect, further, the pedestrian recognition method based on image and motion behavior prediction further includes the following steps:

[0068] Collect environmental and pedestrian information in real time;

[0069] According to the real-time collected environment and pedestrian information, the pedestrian's movement behavior is predicted based on the LSTM model, GCN network and DeepSort algorithm to generate real-time prediction results;

[0070] Formulate the user's movement trajectory based on real-time prediction results;

[0071] When the user moves along the set motion trajectory, the external environment and pedestrian information are collected in real time to update and adjust the motion trajectory.

[0072] In some embodiments of the present invention, after the simulation module receives the information about the external environment and pedestrians transmitted by the information collection module, the simulation module predicts the movement behavior of the pedestrians based on the LSTM model, the GCN network and the DeepSort algorithm. According to the predicted movement behavior of the pedestrians, the simulation module formulates the movement trajectory of the user, and the user moves according to the formulated movement trajectory. The simulation module receives external information in real time and adjusts the movement trajectory, and the user moves according to the real-time updated movement trajectory, thereby realizing the function of real-time update of the user's correct movement trajectory, solving the problem of not being able to respond in time when an unexpected situation occurs when moving along the set route, and can improve the safety of users and pedestrians during movement, improve the reliability of the product, and reduce the possibility of accidents.

[0073] like Figure 4 As shown, in the second aspect, an embodiment of the present invention provides a pedestrian recognition system based on image and motion behavior prediction, including an information acquisition module 100, a pedestrian trajectory prediction module 200, a user trajectory planning module 300, a motion trajectory simulation module 400 and a safety warning module 500, wherein:

[0074] The information collection module 100 is used to collect and analyze the environment and pedestrian information to obtain the target environment and pedestrian information;

[0075] The pedestrian trajectory prediction module 200 is used to predict the movement behavior trajectory of pedestrians based on the GCN network and the DeepSort algorithm combined with the target environment and pedestrian information to generate a pedestrian trajectory prediction result;

[0076] The user trajectory planning module 300 is used to plan the user's motion trajectory based on the pedestrian trajectory prediction result and the RRT algorithm and the PRM algorithm;

[0077] The motion trajectory simulation module 400 is used to simulate the motion trajectory based on the multi-layer LSTM model and the Seq2Seq model according to the pedestrian trajectory prediction result and the user's motion trajectory to obtain a simulation result;

[0078] The safety warning module 500 is used to generate and send safety warning information according to the simulation results.

[0079] This system is implemented based on a pre-built simulation module. The simulation module predicts pedestrian movement behavior based on three algorithms: LSTM model, GCN network and DeepSort algorithm, proposes an action route for the user based on the prediction results, and optimizes the user's movement route after a secondary simulation of the prediction results and the proposed action route. The objects of the secondary simulation include the initial pedestrian movement behavior prediction results, the initial proposed user movement route and environmental changes.

[0080] Through the cooperation of multiple modules such as the information collection module 100, the pedestrian trajectory prediction module 200, the user trajectory planning module 300, the motion trajectory simulation module 400 and the safety warning module 500, the system can simulate the scenarios that occur after the pedestrian movement prediction, analyze and capture information in complex scenarios, filter the collected information, and warn pedestrians and users during the movement.

[0081] Based on the second aspect, further, the pedestrian recognition system based on image and motion behavior prediction also includes a trajectory optimization module for analyzing the simulation results and optimizing the proposed user motion trajectory.

[0082] Based on the second aspect, further, the information acquisition module 100 includes a visual sensor, a force / torque sensor, a tactile sensor and an inertial measurement unit.

[0083] In some embodiments of the present invention, the visual sensor is connected to the analog module through signal transmission, the force / torque sensor is connected to the analog module through signal transmission, the tactile sensor is connected to the analog module through signal transmission, and the inertial measurement unit is connected to the analog module through signal transmission. The visual sensor is internally provided with a light source, a lens, an image sensor, an analog / digital converter, an image processor and an image memory. Based on optical imaging and image processing technology, the light source illuminates the object to be measured, the lens focuses the image, the image sensor captures the image, and the object to be measured is converted into a digital signal. The image processor processes the digital signal, including feature extraction, edge detection and shape recognition, compares the processed image with the reference feature stored in the memory, makes analysis and judgment, and transmits the information to the analog module.

[0084] The force / torque sensor is equipped with a piezoelectric element, an elastic sensitive element, a charge amplifier and a measurement circuit. Based on the piezoelectric effect, when the piezoelectric element is subjected to an external force, electric polarization occurs inside the piezoelectric element, and charges with opposite signs are generated on the surface of the piezoelectric element. When the external force is removed, the surface of the piezoelectric element returns to a non-charged state. When the direction of the external force changes, the polarity of the charge changes. After the charge is amplified, converted and processed by the charge amplifier and the measurement circuit, the size of the external force is measured and the information is transmitted to the analog module.

[0085] The tactile sensor is equipped with a permanent magnet, an induction coil, a magnetic flux changing element and a processing circuit. The permanent magnet and the induction coil are fixed. The magnetic flux changing element changes the size of the air gap in the measuring magnetic circuit. When the air gap decreases, the magnetic resistance increases, and the magnetic flux decreases. When the air gap increases, the magnetic resistance decreases, and the magnetic flux increases. The processing circuit collects information by measuring the changes in magnetic resistance and magnetic flux, and transmits the information to the analog module.

[0086] The inertial measurement unit is internally provided with a three-axis accelerometer, a three-axis gyroscope and a solution circuit. The three-axis accelerometer is used to measure the linear acceleration in three directions. In the state of rest and uniform linear motion, the three-axis accelerometer can measure the components of gravity acceleration on three axes, thereby calculating the inclination angle of the object. When the three-axis gyroscope rotates around one axis, a force perpendicular to the lower rotation axis is generated. The generated force causes the sensitive element inside the three-axis gyroscope to deflect, thereby measuring the magnitude and direction of the angular velocity. After the data is processed by the solution circuit, the data is transmitted to the simulation module. The function of analyzing and capturing information in complex scenes is realized, and the problem of difficulty in capturing and identifying information that affects the movement route of the vehicle itself is solved in complex scenes. It can accurately capture specific information, reduce unnecessary information capture, speed up the prediction of pedestrian movement behavior, save time and cost, and improve work efficiency.

[0087] In other embodiments of the present invention, after analyzing the filtered information, the simulation module performs further screening to extract key information, and predicts the movement behavior of key pedestrians based on the GCN network and the DeepSort algorithm. The simulation module transmits the range of extracted key information to the information acquisition module 100. After the information acquisition module 100 receives the range, in the subsequent information collection process, it slightly expands the range to collect and filter information, thereby realizing the functions of efficient information collection and rapid pedestrian recognition, solving the problem that the collected information is huge and complex to process, making it difficult to quickly recognize pedestrians, improving the work efficiency of the product, reducing energy consumption, improving the accuracy of pedestrian recognition, and avoiding interference from the external environment on the recognition results.

[0088] like Figure 5 As shown, in a third aspect, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs and a processor 102. When the one or more programs are executed by the processor 102, any method in the first aspect described above is implemented.

[0089] The memory 101, the processor 102 and the communication interface 103 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0090] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0091] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0092] In the embodiments provided in the present application, it should be understood that the disclosed method and system can also be implemented in other ways. The method and system embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the method and system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0094] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by the processor 102, a method as in any one of the first aspects described above is implemented. If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0095] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0096] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A pedestrian recognition method based on image and motion behavior prediction, characterized in that: The following steps are involved: Collect and analyze environmental and pedestrian information to obtain target environmental and pedestrian information; Based on the LSTM model, GCN network and DeepSort algorithm, the pedestrian's motion behavior trajectory is predicted in combination with the target environment and pedestrian information to generate pedestrian trajectory prediction results; According to the pedestrian trajectory prediction results, the user's motion trajectory is proposed based on the RRT algorithm and PRM algorithm; According to the pedestrian trajectory prediction results and the user's motion trajectory, the motion trajectory simulation is performed based on the multi-layer LSTM model and Seq2Seq model to obtain the simulation results; Generate and send safety warning information based on the simulation results.

2. The method for pedestrian recognition based on image and motion behavior prediction according to claim 1, characterized in that: The method for collecting and analyzing environment and pedestrian information to obtain target environment and pedestrian information includes the following steps: Collect environmental and pedestrian information; Feature selection and feature extraction are performed on the environment and pedestrian information to obtain the target environment and pedestrian information.

3. The pedestrian recognition method based on image and motion behavior prediction according to claim 1, characterized in that: The following steps are also included: Analyze the simulation results and optimize the proposed user motion trajectory.

4. The method for pedestrian recognition based on image and motion behavior prediction according to claim 1, characterized in that: The following steps are also included: Collect environmental and pedestrian information in real time; According to the real-time collected environment and pedestrian information, the pedestrian's movement behavior is predicted based on the LSTM model, GCN network and DeepSort algorithm to generate real-time prediction results; Formulate the user's movement trajectory based on real-time prediction results; When the user moves along the set motion trajectory, the external environment and pedestrian information are collected in real time to update and adjust the motion trajectory.

5. The pedestrian recognition method based on image and motion behavior prediction according to claim 1, characterized in that: The method for generating and sending safety warning information according to the simulation result comprises the following steps: When the simulation results show that there is a risk of collision between the user and the pedestrian, a safety warning message is generated and sent to the user and / or the pedestrian.

6. A pedestrian recognition system based on image and motion behavior prediction, characterized in that: It includes information collection module, pedestrian trajectory prediction module, user trajectory planning module, motion trajectory simulation module and safety warning module, among which: An information collection module is used to collect and analyze environmental and pedestrian information to obtain target environmental and pedestrian information; The pedestrian trajectory prediction module is used to predict the pedestrian's motion behavior trajectory based on the LSTM model, GCN network and DeepSort algorithm combined with the target environment and pedestrian information to generate pedestrian trajectory prediction results; The user trajectory prediction module is used to predict the user's motion trajectory based on the RRT algorithm and the PRM algorithm according to the pedestrian trajectory prediction results; The motion trajectory simulation module is used to simulate the motion trajectory based on the multi-layer LSTM model and Seq2Seq model according to the pedestrian trajectory prediction results and the user's motion trajectory to obtain the simulation results; The safety warning module is used to generate and send safety warning information according to the simulation results.

7. A pedestrian recognition system based on image and motion behavior prediction according to claim 6, characterized in that: It also includes a trajectory optimization module for analyzing the simulation results and optimizing the proposed user motion trajectory.

8. The pedestrian recognition system based on image and motion behavior prediction according to claim 6, characterized in that: The information acquisition module includes a visual sensor, a force / torque sensor, a tactile sensor and an inertial measurement unit.

Images