Construction method and system based on vehicle-mounted virtual character animal model motion

Abstract

The invention discloses a construction method and system based on vehicle-mounted virtual character animal model motion, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining the image information of an automatic driving scene, obtaining the information of a character animal model based on the data scene reconstruction according to the image information of the automatic driving scene, and obtaining the motion of the vehicle-mounted virtual character animal model according to the information of the character animal model. And based on morphological analysis, obtaining model morphological data. According to the method, the target recognition efficiency is improved through frame selection of the character and animal targets, the number of nodes required by image recognition is dynamically adjusted through the distance adjustment coefficient and the node adjustment coefficient, the image recognition efficiency is improved on the basis of ensuring the target recognition precision, and the image recognition efficiency is improved through character and animal model information. A data basis is provided for construction of the character and animal data model, the motion state of the target is accurately evaluated through the character and animal data model and the model form data, and the stability and reliability of character and animal model motion construction are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and particularly to a construction method and system for the movement of vehicle-mounted virtual human and animal models. Background Art

[0002] With the rapid development of intelligent networked vehicle technology, the application demand for vehicle-mounted virtual scenarios is increasing day by day, covering fields such as autonomous driving testing, in-vehicle entertainment interaction, and intelligent cockpit design. In these scenarios, the dynamic simulation of human and animal models is one of the key technologies. Its core goal is to build a high-fidelity and real-time virtual traffic participant model on the vehicle platform to further improve autonomous driving technology. For example, through the dynamic behavior simulation of virtual pedestrians and animals, the safety and robustness of autonomous driving algorithms in complex scenarios are verified.

[0003] Currently, there are still problems in the construction of the movement of vehicle-mounted virtual human and animal models. It is impossible to accurately identify the humans and animals in the image through the autonomous driving scene image, and often directly use conventional algorithm technologies for identification. It is impossible to adjust the recognition criteria according to the actual situation of the image. The human recognition algorithm often needs to detect corresponding points to identify humans, but due to the influence of resolution, etc., the detection time of some points will be greatly extended, reducing the image recognition efficiency, and it is impossible to set appropriate data model construction criteria according to the recognition results, affecting the construction efficiency of the movement of the human and animal models. Summary of the Invention

[0004] To solve the above technical problems, a construction method and system for the movement of vehicle-mounted virtual human and animal models are provided. The technical solution solves the problems proposed in the above background art, that is, it is impossible to accurately identify the humans and animals in the image through the autonomous driving scene image, and often directly use conventional algorithm technologies for identification. It is impossible to adjust the recognition criteria according to the actual situation of the image. The human recognition algorithm often needs to detect corresponding points to identify humans, but due to the influence of resolution, etc., the detection time of some points will be greatly extended, reducing the image recognition efficiency, and it is impossible to set appropriate data model construction criteria according to the recognition results, affecting the construction efficiency of the movement of the human and animal models.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: A construction method for the movement of vehicle-mounted virtual human and animal models, comprising: Obtaining autonomous driving scene image information, where the autonomous driving scene image information represents the scene image during autonomous driving; Based on the autonomous driving scene image information, obtaining human and animal model information through data-based scene reconstruction; Based on the human and animal model information, obtaining model morphology data through morphological analysis; Construct a data model and model skeleton based on the information of the human / animal model, and obtain the human / animal data model; Analyze the autonomous driving scene image according to the human / animal data model and model morphology data, and obtain the human / animal movement information; Obtain the key frame information of the movement according to the human / animal movement information; Perform motion simulation on the human / animal data model with the key frame information of the movement to complete the construction of the model movement.

[0006] Preferably, the obtaining of the human / animal model information based on the reconstruction of the digital scene according to the autonomous driving scene image information specifically includes: According to the autonomous driving scene image information, obtain the reference scene image based on image preprocessing, and the image preprocessing includes image grayscale conversion and image noise reduction processing; Taking the reference scene image as the reference, based on the YOLO object detection algorithm, frame the human / animal target to obtain the model bounding box information, and the model bounding box is a rectangular border; Obtain the target key node information and standard resolution information based on the requirements of the OpenPose model detection; Obtain the scene image resolution information based on the reference scene image; Take the ratio of the scene image resolution to the standard resolution as the node adjustment coefficient; Obtain the standard height information of the model bounding box based on the scene image resolution information; Obtain the height information of the model bounding box according to the model bounding box information; Obtain the distance adjustment coefficient according to the height information of the model bounding box and the standard height information of the model bounding box; Obtain the human / animal model information according to the distance adjustment coefficient, node adjustment coefficient and target key node information.

[0007] Preferably, the obtaining of the distance adjustment coefficient according to the height information of the model bounding box and the standard height information of the model bounding box specifically includes: According to the scene image resolution information, test the framing of the human / animal target, and frame the test target with the YOLO object detection algorithm to obtain the test model bounding box information; Based on the OpenPose model, identify the target key nodes of the test target to obtain the test target key node identification information; Taking the test target key node identification information as the reference, adjust the height of the test model bounding box until all target key nodes are detected; On the basis of detecting all target key nodes, gradually reduce the height of the test model bounding box of the test target in the benchmark scene image until all target key nodes cannot be recognized, and use the height of the test model bounding box at this time as the standard height of the model bounding box; According to the standard height of the model bounding box, use the ratio of the height of the model bounding box to the standard height of the model bounding box as the distance adjustment coefficient.

[0008] Preferably, the obtaining of the human / animal model information according to the distance adjustment coefficient, the node adjustment coefficient and the target key node information specifically includes: According to the motion detection requirements of the human / animal target, divide the target key nodes to obtain the first target key nodes and the second target key nodes. The first target key nodes include the head node, shoulder node, elbow node, wrist node, hip node, knee node and ankle node; Classify the second target key nodes based on the human / animal target parts to obtain the classification information of the second target key nodes. The classification information of the second target key nodes includes the second head node, the second trunk node, the second upper limb node and the second lower limb node; According to the classification information of the second target key nodes, obtain the number of nodes of each type of second target key nodes; Use the ratio of the number of nodes of each type of second target key nodes to the total number of second target key nodes as the weight coefficient of this type of second target key nodes; According to the distance adjustment coefficient, the node adjustment coefficient and the second target key nodes, obtain the required number of the second target key nodes of the model; According to the classification information of the second target key nodes, use the product of the weight coefficient of each type of second target key nodes and the required number of the second target key nodes of the model as the recognition number of this type of second target key nodes, and obtain the recognition threshold information of the second target key nodes; Based on the first target key nodes, identify and connect the benchmark scene image in the model bounding box to obtain the basic human model; Based on the basic human model, identify the second target key nodes until the number of the identified second target key nodes meets the recognition threshold of the second target key nodes; Based on the identified second target key nodes and the basic human model, connect the nodes of the basic human model to obtain the human / animal model information.

[0009] Preferably, the constructing of the data model and the model skeleton based on the human / animal model information to obtain the human / animal data model specifically includes: Based on the construction of the human / animal data model, conduct a system stress test to obtain the maximum number of model vertices; Obtain the second target key node information of the model according to the character and animal model information, where the second target key node of the model represents the second target key node in the character and animal model; Based on the requirements for building a character and animal data model, obtain the joint structure wiring information, where the joint structure wiring includes a node loop composed of model nodes; According to the joint structure wiring information, using the second target key node as a reference, take the second target key node corresponding to each joint structure wiring as the wiring second target key node; According to the joint structure wiring information and the second target key node information of the model, obtain the second target key node of the model corresponding to each joint structure wiring in the character and animal model; According to the joint structure wiring information, take the ratio of the second target key node to the wiring second target key node in each joint structure wiring as the wiring node difference coefficient of each joint structure wiring; Multiply the maximum value of the wiring node difference coefficient by the maximum number of model vertices to obtain the number of model vertices; Obtain the number of model faces according to the number of model vertices; Based on the number of model vertices and the number of model faces, construct a basic data model; Based on forward dynamics and inverse dynamics, using the model vertices in the basic data model as a reference, construct a model skeleton; Bind the basic data model to the model skeleton to obtain a character and animal data model; Among them, the number of model faces is specifically: In the formula, b is the number of model faces, and a is the number of model vertices.

[0010] Preferably, the obtaining of the motion key frame generation information according to the character and animal motion information specifically includes: According to the character and animal data model information, based on morphological analysis, obtain model morphological data, where the model morphological data includes the model's head-to-body ratio; According to the model morphological data, construct motion speed standard information, where the motion speed standard information includes normal walking speed, normal running speed, and fast running speed; Analyze the autonomous driving scene image according to the character and animal data model to obtain motion speed information; Compare the motion speed information with the motion speed standard information to obtain motion state information; Based on the motion state information, obtain motion key frame information.

[0011] Furthermore, a construction system based on the motion of a vehicle-mounted virtual character and animal model is proposed, which is used to implement the above construction method, including: The main control module is used to obtain the number of model vertices according to the human / animal model information, obtain the number of model faces according to the number of model vertices, construct a basic data model based on the number of model vertices and the number of model faces, construct a model skeleton based on forward dynamics and inverse dynamics with the model vertices in the basic data model as the reference, bind the basic data model to the model skeleton to obtain a human / animal data model, obtain model morphology data based on morphological analysis according to the human / animal data model information, construct motion speed standard information according to the model morphology data, obtain motion speed information according to the human / animal data model, compare the motion speed information with the motion speed standard information to obtain motion state information, obtain motion key frame information based on the motion state information, and perform motion simulation on the human / animal data model with the motion key frame information; The information acquisition module is used to obtain autonomous driving scenario image information, obtain a reference scenario image based on image preprocessing according to the autonomous driving scenario image information, box the human / animal target based on the YOLO object detection algorithm with the reference scenario image as the reference to obtain model bounding box information, obtain target key node information and standard resolution information based on the OpenPose model detection requirements, and obtain scenario image resolution information based on the reference scenario image; The image recognition module is used to test the boxing of the human / animal target according to the scenario image resolution information, box the test target with the YOLO object detection algorithm to obtain test model bounding box information, obtain the standard height of the model bounding box according to the test model bounding box information, use the ratio of the model bounding box height to the standard height of the model bounding box as the distance adjustment coefficient, divide the target key nodes according to the human / animal target motion detection requirements to obtain the first target key node and the second target key node, identify and connect the reference scenario image in the model bounding box with the first target key node as the reference to obtain a basic human model, identify the second target key node with the basic human model as the reference, and connect the nodes of the basic human model with the identified second target key node and the basic human model as the reference to obtain human / animal model information; The display module interacts with the main control module and is used to output and display the autonomous driving scenario image information, human / animal model information, human / animal data model, and motion key frame information.

[0012] Optionally, the main control module specifically includes: A control unit, which is used to obtain model shape data based on shape analysis according to the human / animal data model information, construct motion speed standard information according to the model shape data, obtain motion speed information according to the human / animal data model, compare the motion speed information with the motion speed standard information to obtain motion state information, and obtain motion key frame information based on the motion state information, and perform motion simulation on the human / animal data model with the motion key frame information; An information receiving unit, which interacts with the information acquisition module and the image recognition module, and is used to receive data and transmit it to the data model unit; A data model unit, which is used to obtain the number of model vertices according to the human / animal model information, obtain the number of model faces according to the number of model vertices, construct a basic data model based on the number of model vertices and the number of model faces, construct model bones based on the forward dynamics and inverse dynamics with the model vertices in the basic data model as the benchmark, bind the basic data model with the model bones, and obtain the human / animal data model.

[0013] Optionally, the information acquisition module specifically includes: A first acquisition unit, which is used to obtain the automatic driving scene image information, obtain the reference scene image based on image preprocessing according to the automatic driving scene image information, and perform frame selection on the human / animal target based on the YOLO object detection algorithm with the reference scene image as the benchmark to obtain the model bounding box information; A second acquisition unit, which is used to obtain the target key node information and the standard resolution information based on the OpenPose model detection requirements, and obtain the scene image resolution information based on the reference scene image.

[0014] Optionally, the image recognition module specifically includes: A distance evaluation unit, which is used to test the frame selection of the human / animal target according to the scene image resolution information, perform frame selection on the test target with the YOLO object detection algorithm to obtain the test model bounding box information, obtain the standard height of the model bounding box according to the test model bounding box information, and use the ratio of the model bounding box height to the standard height of the model bounding box as the distance adjustment coefficient; An image model unit, which is used to divide target key nodes according to the motion detection requirements of human and animal targets, obtain a first target key node and a second target key node, identify and connect the reference scene image in the model bounding box based on the first target key node to obtain a basic human model, identify the second target key node based on the basic human model, and connect the nodes of the basic human model based on the identified second target key node and the basic human model to obtain human and animal model information.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention proposes a construction method and system for vehicle-mounted virtual human and animal model motion. By selecting the human and animal targets by frame, the recognition efficiency of human and animal targets is improved. Through the distance adjustment coefficient and the node adjustment coefficient, the number of nodes required for image recognition is dynamically adjusted, and on the basis of ensuring the target recognition accuracy, the image recognition efficiency is improved. Through the human and animal model information, a data basis is provided for the construction of the human and animal data model. Through the human and animal data model and the model form data, the motion state of the target is accurately evaluated, ensuring the stability and reliability of the construction of the human and animal model motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a construction method for vehicle-mounted virtual human and animal model motion proposed by the present invention; Figure 2 It is a flowchart for obtaining human and animal model information in the present invention; Figure 3 It is a flowchart for obtaining the distance adjustment coefficient in the present invention; Figure 4 It is a flowchart for obtaining the human and animal data model in the present invention; Figure 5 It is a schematic diagram of the target key nodes of the human and animal data model in the present invention; Figure 6 It is a block diagram of the structure of a construction system for vehicle-mounted virtual human and animal model motion proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations.

[0018] Refer to Figure 1 - Figure 5 As shown, a construction method for vehicle-mounted virtual human and animal model motion in an embodiment of the present invention includes: Obtain the image information of the autonomous driving scenario, where the image information of the autonomous driving scenario represents the scenario image during autonomous driving; According to the image information of the autonomous driving scenario, based on the data-driven scenario reconstruction, obtain the information of the human and animal models; Specifically, according to the image information of the autonomous driving scenario, based on the data-driven scenario reconstruction, obtaining the information of the human and animal models specifically includes: According to the image information of the autonomous driving scenario, based on image preprocessing, obtain the reference scenario image, where the image preprocessing includes image grayscale conversion and image noise reduction processing; Taking the reference scenario image as a reference, based on the YOLO object detection algorithm, select the human and animal targets by bounding boxes to obtain the model bounding box information, where the model bounding box is a rectangular border; Based on the detection requirements of the OpenPose model, obtain the target key node information and the standard resolution information; Based on the reference scenario image, obtain the scenario image resolution information; Take the ratio of the scenario image resolution to the standard resolution as the node adjustment coefficient; Based on the scenario image resolution information, obtain the standard height information of the model bounding box; According to the model bounding box information, obtain the height information of the model bounding box; According to the height information of the model bounding box and the standard height information of the model bounding box, obtain the distance adjustment coefficient; According to the distance adjustment coefficient, the node adjustment coefficient and the target key node information, obtain the information of the human and animal models.

[0019] In this solution, through the preprocessing combination of "image grayscale conversion + noise reduction processing", the light interference (such as backlight, shadow) and sensor noise in the autonomous driving scenario are effectively filtered, the image contrast and clarity are improved, and a high-quality data foundation is laid for subsequent object detection. For example, the noise reduction processing can reduce the blurred influence of noises such as raindrops and road surface reflections on the human contour, ensure that the YOLO algorithm can accurately capture the target boundary, avoid misjudgment or missed detection of the model bounding box caused by image quality defects, quickly select the rectangular bounding box of the human and animal, meet the real-time requirements of the in-vehicle system (such as 60FPS high-speed detection), ensure the continuous tracking of the target in the dynamic scenario, and the OpenPose extracts the key nodes (such as human joints, animal limb joint points), endows the model with pose information (such as walking, running actions), solves the limitation that the traditional bounding box can only describe the target position and cannot represent the motion characteristics. The two work together to enable the virtual model to not only locate the spatial coordinates of the target in the scenario but also simulate the real motion pose, significantly improving the perception training accuracy of the autonomous driving system for complex behaviors such as "pedestrian waving" and "animal jumping"; It is understandable that the maximum number of key nodes in OpenPose can reach 25. However, in the actual recognition process, affected by factors such as the resolution of the captured image and the distance of the person's position, if all key nodes or a fixed number of nodes are to be accurately recognized, not only does the recognition time take longer, but the accuracy of the recognized nodes also varies. Therefore, by introducing the "node adjustment coefficient" and the "distance adjustment coefficient", a unified modeling standard across resolutions and scales is constructed, improving the image recognition efficiency.

[0020] Specifically, according to the height information of the model bounding box and the standard height information of the model bounding box, the distance adjustment coefficient is obtained, which specifically includes: According to the resolution information of the scene image, the person / animal target is selected for testing. Using the YOLO object detection algorithm, the test target is selected, and the test model bounding box information is obtained. Based on the OpenPose model, the target key nodes of the test target are recognized, and the test target key node recognition information is obtained. Based on the test target key node recognition information, the height of the test model bounding box is adjusted until all target key nodes are detected. On the basis of detecting all target key nodes, gradually reduce the height of the test model bounding box of the test target in the reference scene image until all target key nodes cannot be recognized. At this time, the height of the test model bounding box is used as the standard height of the model bounding box. According to the standard height of the model bounding box, the ratio of the model bounding box height to the standard height of the model bounding box is used as the distance adjustment coefficient.

[0021] In this solution, by "gradually adjusting the height of the test model bounding box until all key nodes are detected", a direct association between the bounding box size and the recognition ability of target key nodes (such as human joints, animal limb endpoints) is established. For example, when a pedestrian target is far from a vehicle, the height of its bounding box in the image is small. If the height is lower than the "standard height of the model bounding box", the OpenPose model may miss details nodes such as wrists and ankles due to insufficient pixel information, resulting in the distortion of the virtual model's pose (such as being unable to recognize the action of "the pedestrian raising the hand to signal"). Therefore, by using the ratio of the model bounding box height to the standard height of the model bounding box as the distance adjustment coefficient, the loss of pose information caused by the target being far and small is avoided, providing complete pose data support for subsequent motion trajectory simulation.

[0022] Specifically, according to the distance adjustment coefficient, the node adjustment coefficient, and the target key node information, the person / animal model information is obtained, which specifically includes: According to the requirements of target motion detection for human and animal targets, the target key nodes are divided to obtain the first target key nodes and the second target key nodes. The first target key nodes include the head node, shoulder node, elbow node, wrist node, hip node, knee node, and ankle node; The second target key nodes are classified based on the target parts of human and animal targets to obtain the classification information of the second target key nodes. The classification information of the second target key nodes includes the second head node, the second trunk node, the second upper limb node, and the second lower limb node; According to the classification information of the second target key nodes, the number of nodes of each type of the second target key nodes is obtained; The ratio of the number of nodes of each type of the second target key nodes to the total number of the second target key nodes is used as the weight coefficient of this type of the second target key nodes; According to the distance adjustment coefficient, the node adjustment coefficient, and the second target key nodes, the required number of the second target key nodes of the model is obtained; According to the classification information of the second target key nodes, the product of the weight coefficient of each type of the second target key nodes and the required number of the second target key nodes of the model is used as the recognition number of this type of the second target key nodes, and the recognition threshold information of the second target key nodes is obtained; Based on the first target key nodes, the reference scene image in the model bounding box is recognized and connected to obtain the basic human model; Based on the basic human model, the second target key nodes are recognized until the number of the recognized second target key nodes meets the recognition threshold of the second target key nodes; Based on the recognized second target key nodes and the basic human model, the nodes of the basic human model are connected to obtain the human and animal model information.

[0023] In this solution, by dividing the target key nodes into the first target key nodes (core key nodes) and the second target key nodes (part subdivision nodes), a hierarchical modeling system of "basic skeleton → detailed features" is constructed. The first key nodes (head, shoulder, elbow, etc.) form the basic motion skeleton of the human body / animal, ensuring the physical rationality of the core posture of the model in a dynamic scene (such as the joint linkage logic when walking), and avoiding the distortion of the motion trajectory caused by the missing key nodes (such as the robotic arm-like motion of "no elbow joint"). Then, the second key nodes are classified by part (head, trunk, etc.), and the modeling resources are differentially allocated in combination with the weight coefficient (the proportion of the number of nodes). For example, the weight of the lower limb nodes is higher in the walking scene, and the motion accuracy of the leg joints is preferentially guaranteed, so that the model is closer to the real biomechanical characteristics in complex actions (such as running and turning).

[0024] It can be understood that the number of nodes that can be accurately recognized during the image recognition process with different precisions is also very different. Therefore, a basic human model is first constructed by recognizing nodes such as the head, shoulders, and elbows, and then each type of second target key node is recognized until the number of recognized second target key nodes of each type exceeds the recognition threshold of that type of second target key node. The recognition threshold is the product of the weight coefficient of each type of second target key node and the required number of second target key nodes of the model.

[0025] It should be noted that the classification information of the second target key nodes includes the second head node, the second torso node, the second upper limb node, and the second lower limb node, and specifically: The head nodes include the nose, the left eye, the right eye, the left ear, and the right ear; The torso nodes include the left shoulder, the right shoulder, the left hip, the right hip, and the neck (implied): the position of the neck is calculated through the midpoint of the shoulders and hips (not an explicit key point, the OpenPose 18-point version includes the neck node); The upper limb nodes include the left elbow, the left wrist, the right elbow, the right wrist, and the fingers (optional, included in the 25-point version): the left thumb, the left little finger, the right thumb, and the right little finger; The lower limb nodes include the left knee, the left ankle, the right knee, and the right ankle.

[0026] According to the human and animal model information, based on morphological analysis, the model morphological data is obtained; Based on the human and animal model information, a data model and a model skeleton are constructed to obtain a human and animal data model; Specifically, based on the human and animal model information, a data model and a model skeleton are constructed to obtain a human and animal data model, which specifically includes: Based on the construction of the human and animal data model, a system stress test is carried out to obtain the maximum number of model vertices; According to the human and animal model information, the information of the second target key nodes of the model is obtained, and the second target key nodes of the model represent the second target key nodes in the human and animal model; Based on the requirements for constructing the human and animal data model, the joint structure wiring information is obtained, and the joint structure wiring includes a node loop composed of model nodes; According to the joint structure wiring information, with the second target key nodes as the reference, the second target key nodes corresponding to each joint structure wiring are used as the wiring second target key nodes; According to the joint structure wiring information and the information of the second target key nodes of the model, the second target key nodes of the model corresponding to each joint structure wiring in the human and animal model are obtained; According to the joint structure wiring information, the ratio of the second target key nodes of the model to the wiring second target key nodes in each joint structure wiring is used as the wiring node difference coefficient of each joint structure wiring; Take the product of the maximum value of the wiring node difference coefficient and the maximum number of model vertices as the number of model vertices; Obtain the number of model faces according to the number of model vertices; Construct a basic data model based on the number of model vertices and the number of model faces; Based on forward dynamics and inverse dynamics, construct a model skeleton with the model vertices in the basic data model as the reference; Bind the basic data model to the model skeleton to obtain a human / animal data model; Among them, the number of model faces is specifically: In the formula, b is the number of model faces, and a is the number of model vertices.

[0027] In this solution, the maximum number of model vertices is determined through system pressure testing. Combining the wiring node difference coefficient (the matching degree between the actual nodes of the joint structure and the standard nodes), the vertex scale is dynamically calculated, and an "performance - detail" adaptive adjustment mechanism is constructed. Key wiring nodes are screened through the difference coefficient to ensure the synchronization of the model surface movement during skeleton driving (such as the neck skin wrinkles being driven when the head rotates), solving the problem of the traditional model's "separation of skeleton and epidermis", providing consistent visual and geometric features for multi - sensor fusion testing (such as the matching of camera and lidar point clouds), constructing a high - precision data model that not only conforms to the real biological movement law but also adapts to real - time computing constraints, and providing a reliable virtual test carrier for the perception and decision - making algorithms of the autonomous driving system.

[0028] Analyze the autonomous driving scene image according to the human / animal data model and the model morphology data to obtain the human / animal movement information; Obtain the key frame information of the movement according to the human / animal movement information; Specifically, according to the human / animal movement information, obtain the key frame generation information of the movement, which specifically includes: Based on morphological analysis, obtain the model morphology data according to the human / animal data model information, and the model morphology data includes the model's head - to - body ratio; Construct the movement speed standard information according to the model morphology data, and the movement speed standard information includes normal walking speed, normal running speed, and fast running speed; Analyze the autonomous driving scene image according to the human / animal data model to obtain the movement speed information; Compare the movement speed information with the movement speed standard information to obtain the movement state information; Obtain the key frame information of the movement based on the movement state information.

[0029] In this solution, a motion speed standard is constructed through model morphological data (such as the head-to-body ratio) to make the motion characteristics of the virtual model conform to the real biomechanical laws. The motion state (such as walking, running fast) is generated by comparing the motion speed information with the standard, and a closed-loop response mechanism of "environment perception → motion decision → key frame generation" is constructed. The accurate extraction of motion key frame information solves the pain point of "fixed key frame library cannot cover dynamic scenes" in traditional methods.

[0030]

[0031] In this embodiment, the project frame rate is set to 24fps (24 frames per second), and the time axis unit is frame (Frame). A. Static action (non-cyclic, single pose design) Core principle: The foot controller is fixed, and the upper body (waist, shoulders, head) has slight dynamic changes to avoid stiffness. The key frames are:

[0032] Note: The static action does not need to be strictly cyclic. A slight sense of breathing or fine adjustment of the center of gravity can be designed within 0 - 24 frames to avoid stiffness throughout the process.

[0033] B. Normal walking action (24 frames / cycle, double-foot alternation) Core cycle: 24 frames complete a full stride (left foot steps out → right foot steps out → return to the initial posture), which conforms to the real walking rhythm of a person (about 1.2m / s). The key frames are: The posture of the 24th frame is symmetric to the 0th frame (left and right feet are exchanged). When exporting the loop animation, the "Cycle Offset" needs to be checked.

[0034] C. Normal running action (16 frames / cycle, including the airborne phase, faster rhythm) Core cycle: 16 frames complete a running cycle (both feet are airborne → left and right feet land alternately), which conforms to the real running rhythm of a person (about 3m / s). The key frames are: Using the motion key frame information, motion simulation is performed on the character and animal data model to complete the construction of the model motion.

[0035] Refer to Figure 6 As shown, further, in combination with the above-mentioned construction method for the motion of an in-vehicle virtualized character and animal model, a construction system for the motion of an in-vehicle virtualized character and animal model is proposed, including: The main control module is used to obtain the number of model vertices according to the human / animal model information, obtain the number of model faces based on the number of model vertices, construct a basic data model based on the number of model vertices and the number of model faces, construct a model skeleton based on forward and inverse dynamics with the model vertices in the basic data model as the reference, bind the basic data model to the model skeleton to obtain a human / animal data model, obtain model morphology data based on morphological analysis according to the human / animal data model information, construct motion speed standard information based on the model morphology data, obtain motion speed information according to the human / animal data model, compare the motion speed information with the motion speed standard information to obtain motion state information, obtain motion key frame information based on the motion state information, and perform motion simulation on the human / animal data model with the motion key frame information; The information acquisition module is used to obtain autonomous driving scenario image information, obtain a reference scenario image based on image preprocessing according to the autonomous driving scenario image information, frame the human / animal target based on the YOLO object detection algorithm with the reference scenario image as the reference to obtain model bounding box information, obtain target key node information and standard resolution information based on the requirements of the OpenPose model detection, and obtain scenario image resolution information based on the reference scenario image; The image recognition module is used to test the framing of the human / animal target according to the scenario image resolution information, frame the test target with the YOLO object detection algorithm to obtain test model bounding box information, obtain the standard height of the model bounding box according to the test model bounding box information, use the ratio of the height of the model bounding box to the standard height of the model bounding box as the distance adjustment coefficient, divide the target key nodes according to the motion detection requirements of the human / animal target to obtain the first target key node and the second target key node, identify and connect the reference scenario image in the model bounding box with the first target key node as the reference to obtain a basic human model, identify the second target key node based on the basic human model, and connect the nodes of the basic human model with the identified second target key node and the basic human model as the reference to obtain human / animal model information; The display module interacts with the main control module and is used to output and display autonomous driving scenario image information, human / animal model information, human / animal data model, and motion key frame information.

[0036] The main control module specifically includes: A control unit, which is used to obtain model morphology data based on morphology analysis according to the human / animal data model information, construct motion speed standard information according to the model morphology data, obtain motion speed information according to the human / animal data model, compare the motion speed information with the motion speed standard information to obtain motion state information, and obtain motion key frame information based on the motion state information, and perform motion simulation on the human / animal data model with the motion key frame information; An information receiving unit, which interacts with the information acquisition module and the image recognition module, and is used to receive data and transmit it to the data model unit; A data model unit, which is used to obtain the number of model vertices according to the human / animal model information, obtain the number of model faces according to the number of model vertices, construct a basic data model based on the number of model vertices and the number of model faces, construct model bones based on the forward dynamics and inverse dynamics with the model vertices in the basic data model as the reference, bind the basic data model and the model bones, and obtain the human / animal data model.

[0037] The information acquisition module specifically includes: A first acquisition unit, which is used to obtain the image information of the autonomous driving scene, obtain the reference scene image based on image preprocessing according to the image information of the autonomous driving scene, and perform frame selection on the human / animal target based on the YOLO target detection algorithm with the reference scene image as the reference to obtain the model bounding box information; A second acquisition unit, which is used to obtain the target key node information and the standard resolution information based on the detection requirements of the OpenPose model, and obtain the scene image resolution information based on the reference scene image.

[0038] The image recognition module specifically includes: A distance evaluation unit, which is used to test the frame selection of the human / animal target according to the scene image resolution information, perform frame selection on the test target with the YOLO target detection algorithm to obtain the test model bounding box information, obtain the standard height of the model bounding box according to the test model bounding box information, and use the ratio of the height of the model bounding box to the standard height of the model bounding box as the distance adjustment coefficient; An image model unit, which is used to divide target key nodes according to the motion detection requirements of human and animal targets, obtain a first target key node and a second target key node, identify and connect the reference scene image in the model bounding box based on the first target key node to obtain a basic human model, identify the second target key node based on the basic human model, and connect the nodes of the basic human model based on the identified second target key node and the basic human model to obtain human and animal model information.

[0039] In summary, the advantages of the present invention are as follows: By selecting the human and animal targets to obtain the information of the model bounding box, the recognition efficiency of the human and animal targets is improved, providing a basis for subsequent image recognition. Through the distance adjustment coefficient and the node adjustment coefficient, the number of nodes required for image recognition is dynamically adjusted, improving the image recognition efficiency on the premise of ensuring the target recognition accuracy. By constructing the data model and the model skeleton based on the human and animal model information, a data basis is provided for the construction of the human and animal data model, improving the data model construction efficiency. By analyzing the autonomous driving scene image through the human and animal data model and the model morphology data, the motion state of the target is accurately evaluated, so as to accurately select the key frames and complete the motion simulation of the human and animal data model with the key frames.

[0040] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A construction method based on the movement of vehicle-mounted virtual human and animal models, characterized in that Including: Obtain the image information of the autonomous driving scenario, where the image information of the autonomous driving scenario represents the scenario image during autonomous driving; Based on the image information of the autonomous driving scenario and the data-based scenario reconstruction, obtain the information of the human and animal models; Based on the information of the human and animal models and the morphological analysis, obtain the morphological data of the models; Construct a data model and model skeleton based on the information of the human and animal models to obtain the human and animal data model; Analyze the autonomous driving scenario image according to the human and animal data model and the morphological data of the models to obtain the movement information of the human and animal; Obtain the key frame information of the movement according to the movement information of the human and animal; Perform motion simulation on the human and animal data model with the key frame information of the movement to complete the construction of the model movement.

2. The construction method based on the movement of the in-vehicle virtual human and animal models according to claim 1, characterized in that, The step of obtaining the information of the human and animal models based on the data-based scenario reconstruction according to the image information of the autonomous driving scenario specifically includes: Based on the image information of the autonomous driving scenario and through image preprocessing, obtain the reference scenario image, where the image preprocessing includes image grayscale conversion and image noise reduction processing; Based on the reference scenario image and using the YOLO object detection algorithm, perform box selection on the human and animal targets to obtain the model bounding box information, where the model bounding box is a rectangular border; Based on the detection requirements of the OpenPose model, obtain the target key node information and the standard resolution information; Based on the reference scenario image, obtain the scenario image resolution information; Take the ratio of the scenario image resolution to the standard resolution as the node adjustment coefficient; Based on the scenario image resolution information, obtain the standard height information of the model bounding box; According to the model bounding box information, obtain the height information of the model bounding box; According to the height information of the model bounding box and the standard height information of the model bounding box, obtain the distance adjustment coefficient; According to the distance adjustment coefficient, the node adjustment coefficient, and the target key node information, obtain the information of the human and animal models.

3. The construction method based on the movement of in-vehicle virtual human and animal models according to claim 2, wherein, The step of obtaining the distance adjustment coefficient according to the height information of the model bounding box and the standard height information of the model bounding box specifically includes: Based on the scenario image resolution information, test the box selection of the human and animal targets, and use the YOLO object detection algorithm to perform box selection on the test targets to obtain the test model bounding box information; Based on the OpenPose model, identify the target key nodes of the test targets to obtain the test target key node identification information; Based on the test target key node identification information, adjust the height of the test model bounding box until all target key nodes are detected; On the basis of detecting all target key nodes, gradually reduce the height of the test model bounding box of the test targets in the reference scenario image until all target key nodes cannot be recognized, and take the height of the test model bounding box at this time as the standard height of the model bounding box; According to the standard height of the model bounding box, take the ratio of the height of the model bounding box to the standard height of the model bounding box as the distance adjustment coefficient.

4. A construction method based on the movement of vehicle-mounted virtual human and animal models according to claim 2, characterized in that, The step of obtaining the information of the human and animal models according to the distance adjustment coefficient, the node adjustment coefficient, and the target key node information specifically includes: For the demand of target motion detection of human and animal targets, the key target nodes are divided to obtain the first target key nodes and the second target key nodes. The first target key nodes include the head node, shoulder node, elbow node, wrist node, hip node, knee node, and ankle node; The second target key nodes are classified based on the target parts of human and animal targets to obtain the classification information of the second target key nodes. The classification information of the second target key nodes includes the second head node, the second trunk node, the second upper limb node, and the second lower limb node; According to the classification information of the second target key nodes, the number of nodes of each type of the second target key nodes is obtained; The ratio of the number of nodes of each type of the second target key nodes to the total number of the second target key nodes is used as the weight coefficient of this type of the second target key nodes; According to the distance adjustment coefficient, the node adjustment coefficient, and the second target key nodes, the required number of the second target key nodes of the model is obtained; According to the classification information of the second target key nodes, the product of the weight coefficient of each type of the second target key nodes and the required number of the second target key nodes of the model is used as the recognition number of this type of the second target key nodes, and the recognition threshold information of the second target key nodes is obtained; Based on the first target key nodes, the reference scene image in the model bounding box is recognized and connected to obtain the basic human model; Based on the basic human model, the second target key nodes are recognized until the number of the recognized second target key nodes meets the recognition threshold of the second target key nodes; Based on the recognized second target key nodes and the basic human model, the nodes of the basic human model are connected to obtain the human and animal model information.

5. A construction method based on the movement of an in-vehicle virtual human and animal model according to claim 1, characterized in that, Constructing a data model and a model skeleton based on the human and animal model information to obtain the human and animal data model specifically includes: Based on the construction of the human and animal data model, a system stress test is performed to obtain the maximum number of model vertices; According to the human and animal model information, the information of the second target key nodes of the model is obtained. The second target key nodes of the model represent the second target key nodes in the human and animal model; Based on the requirements of the construction of the human and animal data model, the joint structure wiring information is obtained. The joint structure wiring includes a node loop composed of model nodes; According to the joint structure wiring information, with the second target key nodes as the reference, each second target key node corresponding to each joint structure wiring is used as the second target key node for wiring; According to the joint structure wiring information and the information of the second target key nodes of the model, the second target key nodes of the model corresponding to each joint structure wiring in the human and animal model are obtained; According to the joint structure wiring information, the ratio of the second target key nodes of the model to the second target key nodes for wiring in each joint structure wiring is used as the wiring node difference coefficient of each joint structure wiring; The product of the maximum value of the wiring node difference coefficient and the maximum number of model vertices is used as the number of model vertices; According to the number of model vertices, the number of model faces is obtained; Based on the number of model vertices and the number of model faces, a basic data model is constructed; Based on forward dynamics and inverse dynamics, with the model vertices in the basic data model as the reference, a model skeleton is constructed; Bind the basic data model to the model skeleton to obtain the human / animal data model; Among them, the number of model faces is specifically: In the formula, b is the number of model faces, and a is the number of model vertices.

6. The construction method based on the movement of in-vehicle virtual human and animal models according to claim 1, wherein, The obtaining of the motion key frame generation information according to the human / animal motion information specifically includes: Based on the human / animal data model information and morphological analysis, obtain the model morphological data, and the model morphological data includes the model head-to-body ratio; According to the model morphological data, construct the motion speed standard information, and the motion speed standard information includes normal walking speed, normal running speed, and fast running speed; Analyze the autonomous driving scene image according to the human / animal data model to obtain the motion speed information; Compare the motion speed information with the motion speed standard information to obtain the motion state information; Based on the motion state information, obtain the motion key frame information.

7. A construction system based on the movement of vehicle-mounted virtual human and animal models, for implementing the construction method according to any one of claims 1-6, characterized in that, It includes: The main control module is used to obtain the number of model vertices according to the human / animal model information, obtain the number of model faces according to the number of model vertices, construct the basic data model based on the number of model vertices and the number of model faces, construct the model skeleton based on the forward dynamics and inverse dynamics with the model vertices in the basic data model as the benchmark, bind the basic data model to the model skeleton to obtain the human / animal data model, obtain the model morphological data based on the human / animal data model information and morphological analysis, construct the motion speed standard information according to the model morphological data, obtain the motion speed information according to the human / animal data model, compare the motion speed information with the motion speed standard information to obtain the motion state information, obtain the motion key frame information based on the motion state information, and perform motion simulation on the human / animal data model with the motion key frame information; The information acquisition module is used to acquire the autonomous driving scene image information, based on the autonomous driving scene image information and image preprocessing, obtain the reference scene image, based on the reference scene image, frame the human / animal target based on the YOLO object detection algorithm to obtain the model bounding box information, obtain the target key node information and standard resolution information based on the OpenPose model detection requirements, and obtain the scene image resolution information based on the reference scene image; An image recognition module, which is used to test the bounding of human and animal targets according to the scene image resolution information, use the YOLO object detection algorithm to perform bounding on the test targets, obtain the bounding box information of the test model, obtain the standard height of the model bounding box according to the bounding box information of the test model, and use the ratio of the height of the model bounding box to the standard height of the model bounding box as the distance adjustment coefficient. For the motion detection requirements of human and animal targets, divide the target key nodes to obtain the first target key node and the second target key node. Based on the first target key node, identify and connect the reference scene image in the model bounding box to obtain the basic human model. Based on the basic human model, identify the second target key node, and based on the identified second target key node and the basic human model, perform node connection on the basic human model to obtain the human and animal model information; A display module, which interacts with the main control module and is used to output and display the information of the autonomous driving scene image, the human and animal model information, the human and animal data model, and the motion key frame information.

8. A construction system based on the movement of in-vehicle virtual human and animal models according to claim 7, characterized in that, The main control module specifically includes: A control unit, which is used to obtain the model morphology data based on morphology analysis according to the human and animal data model information, construct the standard motion speed information according to the model morphology data, obtain the motion speed information according to the human and animal data model, compare the motion speed information with the standard motion speed information to obtain the motion state information, and obtain the motion key frame information based on the motion state information. Based on the motion key frame information, perform motion simulation on the human and animal data model; An information receiving unit, which interacts with the information acquisition module and the image recognition module and is used to receive data and transmit it to the data model unit; A data model unit, which is used to obtain the number of model vertices according to the human and animal model information, obtain the number of model faces according to the number of model vertices, construct a basic data model based on the number of model vertices and the number of model faces, construct model bones based on the model vertices in the basic data model based on forward dynamics and inverse dynamics, and bind the basic data model to the model bones to obtain the human and animal data model.

9. A construction system based on the movement of vehicle-mounted virtual human and animal models according to claim 7, characterized in that, The information acquisition module specifically includes: A first acquisition unit, which is used to obtain the information of the autonomous driving scene image, obtain the reference scene image based on image preprocessing according to the information of the autonomous driving scene image, and perform bounding on the human and animal targets based on the reference scene image and the YOLO object detection algorithm to obtain the bounding box information of the model; A second acquisition unit, which is used to obtain the target key node information and the standard resolution information based on the OpenPose model detection requirements, and obtain the scene image resolution information based on the reference scene image.

10. A construction system based on the movement of in-vehicle virtual human and animal models according to claim 7, characterized in that, The image recognition module specifically includes: Distance evaluation unit, which is used to test the box selection of human and animal targets according to the scene image resolution information, use the YOLO target detection algorithm to box select the test targets, obtain the bounding box information of the test model, obtain the standard height of the model bounding box according to the bounding box information of the test model, and use the ratio of the height of the model bounding box to the standard height of the model bounding box as the distance adjustment coefficient; Image model unit, which is used to divide the target key nodes according to the motion detection requirements of human and animal targets, obtain the first target key node and the second target key node, identify and connect the reference scene image in the model bounding box based on the first target key node to obtain the basic human model, identify the second target key node based on the basic human model, and connect the nodes of the basic human model based on the identified second target key node and the basic human model to obtain the human and animal model information.

Images