A fire rescue training scene simulation method, device and system

Abstract

The present invention relates to the field of image analysis technology, and in particular to a method, device and system for simulating a fire rescue training scene. The method obtains the key points of the reference environment that the trainee must pass through in the topological triangulation corresponding to the fire rescue training scene; according to the position distribution of the key points of the reference environment in the topological triangulation, the positional relationship between the key points of the environment in the current scene and the key points of the human body in the character image, and the action of each key point of the human body in the current scene, the attention level of each key point of the human body in the current scene is obtained, and then the next operation and scene of the trainee is predicted. The present invention accurately obtains the attention level of each key point of the human body in the current scene, and then accurately predicts the next operation and scene of the current trainee, caches data in advance, ensures that the trainee's operation is responded to in a timely manner, and at the same time increases the refresh frequency of the scene, effectively avoids the jamming of the simulation screen, and improves the simulation effect of the trainee.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a fire rescue training scene simulation method, device and system. Background Art

[0002] The high resolution of VR devices can provide clearer images and reduce visual blur, the high refresh rate can make the image smoother and reduce stuttering and flickering, and the precise visual tracking can help adjust the image in time according to the user's perspective changes to reduce visual delays and discomfort. Therefore, it is necessary to improve the high resolution, high refresh rate and visual tracking accuracy of VR devices to enhance the user experience.

[0003] Conventional VR devices refresh the corresponding scenes at a high frequency according to the user's movement speed, direction, etc. However, when firefighters are training through fire rescue training scene simulation, they need to extinguish fires, save people and escape quickly within a limited time. Compared with ordinary users, firefighters' operation speed is faster and the scenes are more complex. When simulating fire rescue training scenes through conventional VR devices, it is difficult to perform timely scene refreshes and action responses according to the firefighters' operation speed and direction changes, resulting in poor simulation training effects for firefighters. Summary of the invention

[0004] In order to solve the technical problem that conventional VR equipment is difficult to perform timely scene refresh and action response according to the operation speed and direction changes of firefighters, which affects the simulation training effect of firefighters, the purpose of the present invention is to provide a fire rescue training scene simulation method, device and system. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for simulating a fire rescue training scenario, the method comprising the following steps:

[0006] Obtain key points of the reference environment that the trainee will pass through in the topological triangulation network corresponding to the fire rescue training scene; obtain the human body image of the trainee in the current scene;

[0007] According to the position distribution of the reference environment key points in the topological triangulation network, the position relationship between the environment key points in the current scene and the human body key points in the character image, and the action of each human body key point in the current scene, the attention level of each human body key point in the current scene is obtained;

[0008] Based on the attention level and the prediction model, the trainee's next operation and scenario are predicted.

[0009] Furthermore, the method for obtaining the key points of the reference environment is:

[0010] For any trainee, the trainee's best forward route is used as the standard route, and the small triangles that the standard route passes through in the topological triangulation network are used as reference triangles; wherein, the environmental key points in all fire rescue training scenes are obtained in advance, and all environmental key points are processed by the Delaunay triangulation algorithm to obtain the topological triangulation network corresponding to the fire rescue training scene, and each vertex in the topological triangulation network is the environmental key point;

[0011] The area formed by the reference triangle in the topological triangulation network and the small triangles directly connected to the reference triangle is used as the standard path area for the trainer;

[0012] The key environmental points in the standard route area are used as the reference key environmental points that the trainee will pass through.

[0013] Furthermore, the method for obtaining the degree of attention is:

[0014] According to the position distribution of the key points of the reference environment in the topological triangulation network, the environmental complexity of the trainer's path is obtained;

[0015] According to the positional relationship between the key points of the environment in the current scene and the key points of the human body in the character image, the degree of limitation of the trainee in the current scene is obtained;

[0016] The product of the environmental complexity and the degree of limitation is used as the degree of influence of the trainee in the current scene;

[0017] According to the action status and the affected degree of each key point of the human body in the current scene, the attention degree of each key point of the human body in the current scene is obtained.

[0018] Furthermore, the method for obtaining the environmental complexity is:

[0019] For any trainee, the adjacent small triangles belonging to the same area in the standard pathway area of ​​the trainee are merged and all are regarded as the pathway local area of ​​the trainee;

[0020] Obtain the degree of each reference environment key point in each path local area as the first eigenvalue of the corresponding reference environment key point;

[0021] Obtain the shortest distance between each key point of the reference environment of the trainee and the standard route as the second eigenvalue of the corresponding key point of the reference environment;

[0022] According to the first eigenvalue and the second eigenvalue of each reference environment key point, the local environment complexity of each reference environment key point is obtained; wherein the first eigenvalue is positively correlated with the local environment complexity, and the second eigenvalue is negatively correlated with the local environment complexity;

[0023] The sum of the local environmental complexity of all key points of the trainee's reference environment is taken as the environmental complexity of the trainee's path.

[0024] Furthermore, the method for obtaining the degree of limitation is:

[0025] Get the Euclidean distance between each environmental key point and each human key point in the current scene and arrange them in ascending order to obtain a distance sequence;

[0026] The distance sequence is divided into multiple segments by a multi-threshold segmentation algorithm; the intersection and union ratio of the corresponding environmental key points in each segment and the trainee's reference environmental key points is obtained as the reference degree of each segment;

[0027] Classify the segments whose reference degree is greater than 0 into the first segment category, and classify the segments whose reference degree is equal to 0 into the second segment category;

[0028] The human key points in the current scene and the corresponding environmental key points in the first segment category are divided into the same cluster as the first target clustering cluster;

[0029] The corresponding environmental key points in the second segment category are divided into the same cluster as the second target clustering cluster;

[0030] Cluster the environmental key points and human key points in the current scene according to the position coordinates to obtain multiple reference clustering clusters;

[0031] The difference between the inter-class variance of the reference cluster and the target cluster is negatively correlated and normalized as the degree of limitation of the trainer in the current scenario.

[0032] Furthermore, the method for obtaining the degree of attention is:

[0033] For any human body key point in the current scene, the movement frequency and movement amplitude of the human body key point are obtained, and the attention degree of the human body key point is obtained according to the movement frequency and movement amplitude of the human body key point and the affected degree; wherein the movement frequency and the affected degree are both positively correlated with the attention degree, and the movement amplitude is negatively correlated with the attention degree.

[0034] Furthermore, the method for predicting the trainee's next operation and scenario is:

[0035] A prediction model is trained using machine learning algorithms, decision trees, and support vector machines. The input of the prediction model is the action state, attention level, and scene information of each key point of the human body in the scene, and the output is the prediction of the trainee's next action and scene.

[0036] The action state, attention level and current scene information of each key point of the human body in the current scene are input into the trained prediction model, and the output is the predicted next operation and scene of the trainee.

[0037] Furthermore, the topological triangulation network is obtained by a Delaunay triangulation algorithm.

[0038] In a second aspect, another embodiment of the present invention provides a fire rescue training scene simulation system, the system comprising:

[0039] The parameter acquisition module is used to obtain the key points of the reference environment that the trainee must pass through in the topological triangulation network corresponding to the fire rescue training scene; and obtain the human body image of the trainee in the current scene;

[0040] The attention degree acquisition module is used to obtain the attention degree of each human key point in the current scene according to the position distribution of the reference environment key points in the topological triangulation network, the position relationship between the environment key points in the current scene and the human key points in the character image, and the action of each human key point in the current scene;

[0041] The prediction module is used to predict the trainee's next operation and scenario based on the attention level and the prediction model.

[0042] In the third aspect, another embodiment of the present invention provides a fire rescue training scene simulation device, which includes: a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above methods are implemented.

[0043] The present invention has the following beneficial effects:

[0044] The present invention firstly analyzes the complexity of the environment that the trainee passes through according to the position distribution of the key points of the reference environment that the trainee is to pass through in the topological triangulation network, and preliminarily analyzes the degree of attention that needs to be paid to the trainee; in order to accurately obtain the attention paid to the trainee, which is conducive to the subsequent accurate prediction of the trainee's behavior, the position relationship between the key points of the environment in the current scene and the key points of the human body in the character image is further analyzed to accurately reflect the limitations of the trainee's operation in the current scene, and further determine the degree of attention that needs to be paid to the trainee; in combination with the degree of attention that needs to be paid to the trainee and the action of each key point of the human body in the current scene, the degree of attention of each key point of the human body in the current scene is accurately obtained, and then the loss of each key point of the human body in the corresponding behavior prediction network is accurately set according to the degree of attention, so that the next operation and scene of the trainee can be predicted more accurately, data caching is performed in advance, it is ensured that the trainee's operation is responded to in time, and at the same time the refresh frequency of the scene is improved, the simulation screen is effectively avoided from being stuck, and the fluency of the simulation screen of the fire rescue training scene is improved, thereby effectively improving the simulation effect of the trainee. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic flow chart of a fire rescue training scenario simulation method provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a topological triangulated network provided by an embodiment of the present invention;

[0048] Figure 3 A flow chart of a method for obtaining a degree of attention provided by an embodiment of the present invention;

[0049] Figure 4 A structural diagram of a fire rescue training scenario simulation system provided by an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a fire rescue training scene simulation method, device and system proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0052] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0053] The following is a detailed description of a fire rescue training scene simulation method, device and system provided by the present invention in conjunction with the accompanying drawings.

[0054] Embodiment 1:

[0055] The present invention proposes a fire rescue training scene simulation method, please refer to Figure 1 , which shows a schematic flow chart of a fire rescue training scene simulation method provided by an embodiment of the present invention, the method comprising the following steps:

[0056] Step S1: obtaining key points of the reference environment that the trainee will pass through in the topological triangulation network corresponding to the fire rescue training scene; obtaining a human body image of the trainee in the current scene.

[0057] Specifically, this embodiment takes a trainer as an example for analysis. It should be noted that the trainers that appear later all refer to this trainer. The trainer performs corresponding rescue operations according to the scene of the fire rescue training simulation from his own perspective. During the entire training process, simulation training needs to be carried out through a fire rescue training scene simulation device, which is mainly composed of a computer system, a display device, and a sensor system. Among them, the computer system, as the core computing unit, is responsible for rendering the simulation scene, scheduling computing tasks, and storing training data. It usually requires a powerful processor and graphics card to support efficient three-dimensional simulation; the display device includes a virtual reality (VR) helmet, augmented reality (AR) glasses, and a multi-screen display system to provide an immersive training experience. The trainer can observe the virtual scene in real time through the display device; the sensor system includes a position sensor and a motion capture sensor, etc., which are used to track the behavior and position of the trainer in the virtual scene, record the trainer's movements, and then feed back to the fire rescue training scene simulation system in real time.

[0058] The purpose of this embodiment is to predict the trainee’s next operation and scene, and to cache data in advance to avoid a lack of timely interaction in the scene when the trainee makes the corresponding operation, resulting in poor picture fluency, which in turn causes visual fatigue of the trainee and affects the training effect. In order to better predict the simulation scene for the trainee, this embodiment first obtains the environmental key points in all fire rescue training scenes in advance according to the preset fire rescue training scene model, such as the key support points of the building structure, the key points of the fire source location, and the key points of the escape passage, etc., that is, all the environmental key points in the fire rescue training scene are obtained in advance, and the trainee’s best forward route can be obtained in advance as a standard route. All environmental key points are processed by the Delaunay triangulation algorithm to obtain the topological triangulation network corresponding to the fire rescue training scene, wherein the Delaunay triangulation algorithm is a well-known technology and will not be described in detail. It should be noted that the triangle vertices in the topological triangulation network are environmental key points, such as Figure 2 The figure shows a schematic diagram of a topological triangulation network. In order to determine the key environmental points that the trainee must pass through, the small triangles that the standard route passes through in the topological triangulation network are used as reference triangles; considering that the trainee's route may have slight deviations in actual situations, the area formed by the reference triangle in the topological triangulation network and the small triangles directly connected to the reference triangle are used as the standard route area for the trainee; the key environmental points in the standard route area are used as the reference key environmental points that the trainee must pass through.

[0059] At the same time, the motion capture devices in the sensor system, such as optical cameras and inertial measurement units, are used to capture the trainee's human motion data in real time, and obtain the trainee's human image, acceleration, angular velocity, etc. in the current scene; the real-time position coordinates of the trainee in the scene are obtained through the position sensor. It should be noted that the human image is input into the OpenPose model to generate a confidence map and a partial affinity field, and the key points of the trainee's human body in the current scene are obtained, so as to accurately analyze the trainee's posture. Among them, the generation of confidence maps and partial affinity fields are both well-known technologies and will not be repeated here.

[0060] Step S2: Obtain the attention level of each human key point in the current scene based on the position distribution of the reference environment key points in the topological triangulation network, the positional relationship between the environment key points in the current scene and the human key points in the character image, and the action of each human key point in the current scene.

[0061] The known reference environment key points are the environment key points that the trainee will pass through. The greater the complexity of the area formed by the reference environment key points, the greater the difficulty for the trainee to respond to the complex environment and make corresponding rescue operations during rescue. In order to improve the operation response in the simulation scene, more attention needs to be paid to the trainee so as to reflect the trainee's operation in time. At the same time, if the current trainee is in a small space, in order to rescue in time, the trainee's operation amplitude needs to be small and fast. At this time, more attention needs to be paid to the human body key points corresponding to the trainee, which is helpful to better predict the trainee's next behavior. Therefore, this embodiment obtains the attention level of each human body key point in the current scene according to the position distribution of the reference environment key points in the topological triangulation network, the position relationship between the environment key points in the current scene and the human body key points in the character image, and the action of each human body key point in the current scene. It is conducive to accurately predicting the trainee's next operation through the attention level in the future, caching data in advance, and improving the fluency of the simulation picture.

[0062] Preferably, in one possible implementation of this embodiment, the method for obtaining the degree of attention is as follows: Figure 3 , which shows a flow chart of a method for obtaining the degree of attention provided by this embodiment, the method comprising the following steps:

[0063] Step S301: Obtain the environmental complexity of the trainer's path according to the position distribution of the reference environment key points in the topological triangulation network.

[0064] Since the fire rescue training scene is set in advance, it is possible to know in advance which small triangles in the standard path area belong to the same area. Therefore, for any trainee, the adjacent small triangles belonging to the same area in the trainee's standard path area are merged and used as the trainee's path local area; then the degree of each reference environment key point in each path local area is obtained as the first eigenvalue of the corresponding reference environment key point; wherein, the closer the degree of the reference environment key point is to the middle of the path local area, the greater the environment in the corresponding path local area is for the trainee. Therefore, the larger the first eigenvalue is, the more complex the environment that the trainee passes through is indirectly shown. Among them, the method of obtaining the degree of the reference environment key point is a well-known technology and will not be repeated.

[0065] In order to more accurately analyze the complexity of the environment passed by the trainee based on the reference environment key points, and then obtain the shortest distance between each reference environment key point of the trainee and the standard route, as the second eigenvalue of the corresponding reference environment key point, wherein the smaller the second eigenvalue is, the more indirectly it indicates that the corresponding reference environment key point is closer to the middle position of the local area of ​​the route, and the more complex the environment passed by the trainee is; wherein, the method for obtaining the shortest distance from a point to a line is a well-known technology and will not be repeated here.

[0066] Then, according to the first eigenvalue and the second eigenvalue of each key point of the reference environment, the local environment complexity of each key point of the reference environment is obtained; wherein, the first eigenvalue is positively correlated with the local environment complexity, and the second eigenvalue is negatively correlated with the local environment complexity; in order to comprehensively analyze the environmental complexity of the trainee's path, the sum of the local environment complexity of all key points of the trainee's reference environment is taken as the environmental complexity of the trainee's path. The greater the environmental complexity, the more attention should be paid to the trainee's behavior to ensure that the trainee feels a smoother picture.

[0067] The calculation formula of environmental complexity is: ; In the formula, is the environmental complexity of the i-th trainer path; is the total number of key points of the reference environment that the i-th trainee must pass through; is the first eigenvalue of the nth key point of the reference environment that the i-th trainee will pass through; e is a natural constant; is the second eigenvalue of the nth key point of the reference environment that the i-th trainee is going to pass through; is the local environment complexity of the nth key point of the reference environment that the i-th trainee has to pass through.

[0068] Step S302: Obtain the degree of limitation of the trainee in the current scene according to the positional relationship between the environmental key points in the current scene and the human body key points in the character image.

[0069] If the space where the trainee is located in the current scene is small, the trainee's operation in the current scene will be limited. In order to rescue in time, the trainee's operation frequency will be faster. In order to ensure timely response to the trainee's operation in the simulation scene, more attention needs to be paid to the trainee. In order to analyze whether the space where the trainee is located in the current scene is small, the degree of limitation of the trainee in the current scene is obtained according to the positional relationship between the key points of the environment in the current scene and the key points of the human body in the character image. The greater the degree of limitation, the smaller the space where the trainee is located in the current scene.

[0070] Preferably, in a method that can be implemented in this embodiment, the method for obtaining the degree of limitation is: obtaining the Euclidean distance between each environmental key point and each human key point in the current scene and arranging them in ascending order to obtain a distance sequence; wherein, the method for obtaining the Euclidean distance is a well-known technology and will not be repeated. The distance sequence is divided into multiple segmentation segments by a multi-threshold segmentation algorithm; wherein, the multi-threshold segmentation algorithm is a well-known technology and will not be repeated. If the distance between the reference environmental key point and the human key point in the current scene is closer, and the distance between the environmental key point of the non-reference environmental key point and the human key point is farther, it means that the trainee is in a smaller space in the current scene, and the greater the limitation of the trainee when performing the rescue operation, the trainee needs to increase the operation speed at this time, and in order to better predict the trainee's next operation, more attention needs to be paid to the trainee. In order to indirectly analyze the relationship between the reference environmental key point and the human key point, the present embodiment obtains the intersection and union ratio of the corresponding environmental key point in each segment and the reference environmental key point of the trainee as the reference degree of each segment; wherein, the acquisition of the intersection and union ratio is a well-known technology and will not be repeated. The segments with a reference degree greater than 0 are classified into the first segment category, and the segments with a reference degree equal to 0 are classified into the second segment category; therefore, the reference environment key points in the current scene are all distributed in the corresponding environment key points in the first segment category;

[0071] In order to analyze the limitation of the trainee in the current scene, the human key points in the current scene and the corresponding environmental key points in the first segment category are divided into the same cluster as the first target clustering cluster; the corresponding environmental key points in the second segment category are divided into the same cluster as the second target clustering cluster; at the same time, the environmental key points and human key points in the current scene are clustered according to the position coordinates by the DBSCAN density clustering algorithm to obtain multiple reference clustering clusters; wherein the DBSCAN density clustering algorithm is a well-known technology and will not be described in detail;

[0072] The total variance is obtained through the coordinates of all environmental key points and all human key points in the current scene, and the total variance is subtracted from the intra-class variance corresponding to the coordinates of the key points in the first target clustering cluster, and then the intra-class variance corresponding to the coordinates of the environmental key points in the second target clustering cluster is subtracted, and the result is used as the inter-class variance of the target clustering cluster; the result of subtracting the sum of the intra-class variances corresponding to the coordinates of the key points in all reference clustering clusters from the total variance is used as the inter-class variance of the reference clustering cluster; the smaller the inter-class variance of the target clustering cluster is compared with the inter-class variance of the reference clustering cluster, the less the division of the first target clustering cluster and the second target clustering cluster does not conform to the standard division, the farther the reference environmental key points in the current scene should be from the human key points, and the smaller the limitations of the trainee's operation in the current scene should be. Then, the difference between the inter-class variances of the reference clustering cluster and the target clustering cluster is negatively correlated and normalized, and the result is used as the degree of limitation of the trainee in the current scene. It should be noted that in this embodiment, The difference between the inter-class variance of the reference cluster and the target cluster is negatively correlated and normalized, where exp is an exponential function with a natural constant as the base, and x represents the difference between the inter-class variance of the reference cluster and the target cluster.

[0073] Step S303: taking the product of the environmental complexity and the degree of limitation as the degree of influence of the trainee in the current scene.

[0074] When the environmental complexity and the degree of limitation are greater, it means that the trainee's operation in the current scene is more affected by the environment, and thus this embodiment uses the product of the environmental complexity and the degree of limitation as the degree of influence of the trainee in the current scene. The greater the degree of influence, the more attention needs to be paid to the trainee's operation in the current scene, which is conducive to accurately predicting the trainee's next operation.

[0075] Step S304: Obtain the attention level of each human body key point in the current scene according to the action status of each human body key point in the current scene and the affected degree.

[0076] It is known that the greater the degree of influence, the more attention should be paid to the trainee's operation in the current scene; at the same time, if the action frequency of a certain key point of the human body is greater and the action amplitude is smaller in the current scene, it means that the key point of the human body should be given more attention to ensure that the action of the key point of the human body responds in time in the simulation picture to ensure the smoothness of the simulation picture. In addition, this embodiment obtains the degree of attention of each key point of the human body in the current scene according to the action situation of each key point of the human body in the current scene and the degree of influence.

[0077] It should be noted that the acceleration and angular velocity of the trainee are first obtained through the inertial measurement unit, and combined with the position information of the key points of the human body, the movement amplitude of each key point of the human body in the current scene can be accurately calculated. For example, in a rotational action, the rotation angle can be determined by the angular velocity, and the movement amplitude of each key point of the human body in the rotational action can be calculated by combining the position change of the key points of the human body; then the movement frequency of each key point of the trainee's body is obtained through the motion capture device. The frequency of the motion capture device refers to the number of frames output per second.

[0078] Then, for any human key point in the current scene, the action frequency and action amplitude of the human key point are obtained, and the attention degree of the human key point is obtained according to the action frequency and action amplitude of the human key point and the degree of influence; wherein the action frequency and the degree of influence are positively correlated with the degree of attention, and the action amplitude is negatively correlated with the degree of attention. The calculation formula of the degree of attention is: ; In the formula, is the attention level of the ath human key point in the current scene; c is the influence level of the trainee in the current scene; is the action frequency of the ath key point of the human body in the current scene; is the motion amplitude of the ath key point of the human body in the current scene; e is a natural constant.

[0079] At this point, the attention level of each key point of the human body in the current scene is obtained.

[0080] Step S3: Based on the attention level and the prediction model, predict the trainee's next operation and scenario.

[0081] Specifically, a prediction model is trained using a machine learning algorithm, a decision tree, and a support vector machine; wherein the input of the prediction model is the action state, attention level, and scene information of each key point of the human body in the scene, and the output is the prediction of the next operation and scene of the trainee; the action state, attention level, and current scene information of each key point of the human body in the current scene are input into the trained prediction model, and the output is the predicted next operation and scene of the trainee. The construction and training of the prediction model are both well-known technologies and will not be described in detail.

[0082] The data corresponding to the predicted next operation and scene of the trainee are preloaded and cached, and stored in the cache of the computer system, so as to realize the advance caching of the environment required for the trainee's next operation. When the operation occurs, the scene can interact in time to avoid the simulation screen from freezing. At the same time, the refresh frequency of the corresponding scene is increased to avoid the scene not responding in time after the trainee makes the corresponding operation, which affects the training effect.

[0083] In summary, this embodiment obtains the reference environment key points that the trainee must pass through in the topological triangulation corresponding to the fire rescue training scene; according to the position distribution of the reference environment key points in the topological triangulation, the positional relationship between the environment key points in the current scene and the human body key points in the character image, and the action of each human body key point in the current scene, the attention level of each human body key point in the current scene is obtained, and then the trainee's next operation and scene are predicted. The present invention accurately obtains the attention level of each human body key point in the current scene, and then accurately predicts the next operation and scene of the current trainee, caches data in advance, ensures that the trainee's operation is responded to in a timely manner, and at the same time increases the refresh frequency of the scene, effectively avoids the simulation screen from freezing, and improves the trainee's simulation effect.

[0084] Embodiment 2:

[0085] The present invention also proposes a fire rescue training scene simulation system, please refer to Figure 4 , which shows a structure diagram of a fire rescue training scenario simulation system provided by an embodiment of the present invention. The system includes: a parameter acquisition module 10, a concern degree acquisition module 20 and a prediction module 30.

[0086] The parameter acquisition module 10 is used to acquire key points of the reference environment that the trainee will pass through in the topological triangulation network corresponding to the fire rescue training scene; and to acquire the human body image of the trainee in the current scene.

[0087] The attention level acquisition module 20 is used to obtain the attention level of each human key point in the current scene based on the position distribution of the reference environment key points in the topological triangulation network, the position relationship between the environment key points in the current scene and the human key points in the character image, and the action status of each human key point in the current scene.

[0088] The prediction module 30 is used to predict the trainee's next operation and scenario based on the attention level and the prediction model.

[0089] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the fire rescue training scene simulation system and the fire rescue training scene simulation method embodiment provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0090] Embodiment 3:

[0091] The present invention also proposes a fire rescue training scene simulation device, which includes a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to perform a fire rescue training scene simulation method provided in an embodiment of the present application. The device can be a chip, a component or a module, and the chip can include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a fire rescue training scene simulation method provided in the above embodiment.

[0092] In addition, the present application embodiment also protects a computer device, see Figure 5 The computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program 403, the computer device can execute any one of the fire rescue training scene simulation methods introduced above.

[0093] Embodiment 4:

[0094] This embodiment also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a fire rescue training scene simulation method provided by the above embodiment.

[0095] Embodiment 5:

[0096] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a fire rescue training scene simulation method provided by the above embodiment.

[0097] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0098] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A fire rescue training scene simulation method, characterized in that: The method comprises the following steps: Obtain key points of the reference environment that the trainee will pass through in the topological triangulation network corresponding to the fire rescue training scene; obtain the human body image of the trainee in the current scene; According to the position distribution of the reference environment key points in the topological triangulation network, the position relationship between the environment key points in the current scene and the human body key points in the character image, and the action of each human body key point in the current scene, the attention level of each human body key point in the current scene is obtained; Based on the attention level and the prediction model, predict the trainee's next operation and scenario; The method for obtaining the degree of attention is: According to the position distribution of the key points of the reference environment in the topological triangulation network, the environmental complexity of the trainer's path is obtained; According to the positional relationship between the key points of the environment in the current scene and the key points of the human body in the character image, the degree of limitation of the trainee in the current scene is obtained; The product of the environmental complexity and the degree of limitation is used as the degree of influence of the trainee in the current scene; According to the action status and the affected degree of each key point of the human body in the current scene, the attention degree of each key point of the human body in the current scene is obtained.

2. A fire rescue training scene simulation method as claimed in claim 1, characterized in that: The method for obtaining the key points of the reference environment is: For any trainee, the trainee's best forward route is used as the standard route, and the small triangles that the standard route passes through in the topological triangulation network are used as reference triangles; wherein, the environmental key points in all fire rescue training scenes are obtained in advance, and all environmental key points are processed by the Delaunay triangulation algorithm to obtain the topological triangulation network corresponding to the fire rescue training scene, and each vertex in the topological triangulation network is the environmental key point; The area formed by the reference triangle in the topological triangulation network and the small triangles directly connected to the reference triangle is used as the standard path area for the trainer; The key environmental points in the standard route area are used as the reference key environmental points that the trainee will pass through.

3. A fire rescue training scene simulation method as claimed in claim 1, characterized in that: The method for obtaining the environmental complexity is: For any trainee, the adjacent small triangles belonging to the same area in the standard pathway area of ​​the trainee are merged and all are regarded as the pathway local area of ​​the trainee; Obtain the degree of each key point of the reference environment in each local area of ​​the pathway as the first eigenvalue of the corresponding key point of the reference environment; Obtain the shortest distance between each key point of the reference environment of the trainee and the standard route as the second eigenvalue of the corresponding key point of the reference environment; According to the first eigenvalue and the second eigenvalue of each reference environment key point, the local environment complexity of each reference environment key point is obtained; wherein the first eigenvalue is positively correlated with the local environment complexity, and the second eigenvalue is negatively correlated with the local environment complexity; The sum of the local environmental complexity of all key points of the trainee's reference environment is taken as the environmental complexity of the trainee's path.

4. A fire rescue training scene simulation method as claimed in claim 1, characterized in that: The method for obtaining the degree of limitation is: Get the Euclidean distance between each environmental key point and each human key point in the current scene and arrange them in ascending order to obtain a distance sequence; The distance sequence is divided into multiple segments by a multi-threshold segmentation algorithm; the intersection and union ratio of the corresponding environmental key points in each segment and the trainee's reference environmental key points is obtained as the reference degree of each segment; Classify the segments whose reference degree is greater than 0 into the first segment category, and classify the segments whose reference degree is equal to 0 into the second segment category; The human key points in the current scene and the corresponding environmental key points in the first segment category are divided into the same cluster as the first target clustering cluster; The corresponding environmental key points in the second segment category are divided into the same cluster as the second target clustering cluster; Cluster the environmental key points and human key points in the current scene according to the position coordinates to obtain multiple reference clustering clusters; The difference between the inter-class variance of the reference cluster and the target cluster is negatively correlated and normalized as the degree of limitation of the trainer in the current scenario.

5. A fire rescue training scene simulation method as claimed in claim 1, characterized in that: The method for obtaining the degree of attention is: For any human body key point in the current scene, the movement frequency and movement amplitude of the human body key point are obtained, and the attention degree of the human body key point is obtained according to the movement frequency and movement amplitude of the human body key point and the affected degree; wherein the movement frequency and the affected degree are both positively correlated with the attention degree, and the movement amplitude is negatively correlated with the attention degree.

6. A fire rescue training scene simulation method as claimed in claim 1, characterized in that: The method for predicting the trainee's next operation and scenario is: A prediction model is trained using machine learning algorithms, decision trees, and support vector machines. The input of the prediction model is the action state, attention level, and scene information of each key point of the human body in the scene, and the output is the prediction of the trainee's next action and scene. The action state, attention level and current scene information of each key point of the human body in the current scene are input into the trained prediction model, and the output is the predicted next operation and scene of the trainee.

7. A fire rescue training scene simulation method as claimed in claim 1, characterized in that: The topological triangulation network is obtained by using a Delaunay triangulation algorithm.

8. A fire rescue training scene simulation system, characterized in that: The system comprises: The parameter acquisition module is used to obtain the key points of the reference environment that the trainee must pass through in the topological triangulation network corresponding to the fire rescue training scene; and obtain the human body image of the trainee in the current scene; The attention degree acquisition module is used to obtain the attention degree of each human key point in the current scene according to the position distribution of the reference environment key points in the topological triangulation network, the position relationship between the environment key points in the current scene and the human key points in the character image, and the action of each human key point in the current scene; A prediction module, used to predict the next operation and scenario of the trainee based on the attention level and the prediction model; The method for obtaining the degree of attention is: According to the position distribution of the key points of the reference environment in the topological triangulation network, the environmental complexity of the trainer's path is obtained; According to the positional relationship between the key points of the environment in the current scene and the key points of the human body in the character image, the degree of limitation of the trainee in the current scene is obtained; The product of the environmental complexity and the degree of limitation is used as the degree of influence of the trainee in the current scene; According to the action status and the affected degree of each key point of the human body in the current scene, the attention degree of each key point of the human body in the current scene is obtained.

9. A fire rescue training scene simulation device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When executing the computer program, the processor implements the steps of a fire rescue training scene simulation method as described in any one of claims 1 to 7.

Images