Simulation driving posture evaluation method based on binocular camera and simulation driving system

Abstract

The present application belongs to the technical field of simulation driving training, and particularly relates to a simulation driving posture evaluation method based on a binocular camera and a simulation driving system. The method first captures the driving training area through two cameras arranged at different positions to obtain two driving training area images and construct a three-dimensional space position calculation model. Then, the driver and the gesture area of the driver in the two driving training area images are identified respectively. Next, the human body feature points are identified, and the positions of the human body feature points in the three-dimensional space are calculated in combination with the three-dimensional space position calculation model. Finally, the positions of the human body feature points in the three-dimensional space are used to determine whether the hand of the driver making the gesture is the left hand or the right hand and the gesture direction. The method can realize the automatic evaluation of the simulation driving posture of the trainee during the simulation driving training process, the result is more objective, and the efficiency of the simulation driving training evaluation is improved.

Description

Technical Field

[0001] This invention belongs to the field of simulated driving training technology, specifically relating to a simulated driving posture evaluation method and a simulated driving system based on a binocular camera. Background Technology

[0002] Motor vehicles have become an essential means of transportation. Statistics show that most road traffic accidents and congestion are caused by poor driving behavior. Therefore, it is necessary to impose strict operational requirements on locomotive / train crew members or trainees to cultivate good and standardized driving habits. During simulated driving training, trainees must strictly adhere to the standard requirements of a single crew operation, performing actions based on visual observation and hand gestures. This helps trainees develop correct operating habits and demonstrate good driving posture in actual driving situations.

[0003] In current motor vehicle driver training, instructors are often used to guide trainees to perform the required operations. The evaluation of whether the trainees' actions are standard also depends on the instructor. Without an instructor present, trainees cannot confirm whether their behavior is compliant, which affects the quality and effectiveness of training and results in low training efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for evaluating simulated driving posture based on a binocular camera, in order to solve the problem of low training efficiency caused by relying on instructors to evaluate the simulated driving actions of trainees in the prior art.

[0005] To address the aforementioned technical problems, this invention provides a method for simulating driving posture evaluation based on a binocular camera, comprising the following steps:

[0006] 1) The driving training area is photographed by two cameras set in different positions to obtain two images of the driving training area; the two cameras are calibrated and positioned, and a three-dimensional spatial position calculation model is constructed using the calibration and positioning information of the two cameras. The three-dimensional spatial position calculation model is used to calculate the position of a point in three-dimensional space based on the image positions of a point in the two driving training area images respectively.

[0007] 2) Identify the driver and the driver's gesture area in the two driving training area images respectively, determine the position of the gesture area in the two driving training area images respectively, and determine the position of the gesture area in three-dimensional space by combining the three-dimensional spatial position calculation model;

[0008] 3) Detect the driver's human feature points in the two driving training area images respectively, determine the position of the human feature points in the corresponding driving training area images, and combine the three-dimensional spatial position calculation model to determine the position of the human feature points in three-dimensional space.

[0009] 4) Determine whether the driver's hand making the gesture is the left or right hand based on the three-dimensional spatial relationship between human feature points and the gesture area;

[0010] 5) Determine the direction of the driver's gesture based on the position of the feature points of the hand making the gesture in three-dimensional space, and evaluate the driver's simulated driving posture based on the gesture direction.

[0011] The beneficial effects are as follows: This invention utilizes two cameras positioned at different locations to capture images of the driving training area, and calibrates and positions the two cameras to construct a three-dimensional spatial position calculation model. This model can calculate the position coordinates of a point in three-dimensional space based on its position coordinates in the image. Using this three-dimensional spatial position calculation model, the positions of the driver's human body feature points in three-dimensional space can be calculated. Furthermore, by combining the positions of these human body feature points in three-dimensional space, it is possible to analyze whether the driver's hand is the left or right hand making a gesture, and to determine the direction of the gesture. Based on the direction of the gesture, the simulated driving posture can be evaluated to determine whether the trainee is making the correct gesture as required. This method enables automated evaluation of the trainee's simulated driving posture during simulated driving training, resulting in more objective results and improving the efficiency of simulated driving training evaluation.

[0012] Furthermore, the human body feature points include the left wrist feature point and the right wrist feature point. In step 4), the following method is used to determine whether the driver's hand making the gesture is the left hand or the right hand: determine the position of the left wrist feature point and the right wrist feature point in three-dimensional space; determine the position of the center point of the gesture area in three-dimensional space; calculate the three-dimensional distance between the left wrist feature point and the center point of the gesture area, and the three-dimensional distance between the right wrist feature point and the center point of the gesture area. The two distances are denoted as LDistance and RDistance respectively. If LDistance < RDistance, then the hand making the gesture is determined to be the left hand. If LDistance ≥ RDistance, then the hand making the gesture is determined to be the right hand.

[0013] Its beneficial effects are: it transforms human body feature points into three-dimensional space, and combines the distance between the wrist feature points and the center point of the gesture area in three-dimensional space to determine whether the hand making the gesture is the left or right hand, thus meeting the requirements for assessment accuracy.

[0014] Furthermore, the human body feature points include the left wrist feature points and the right wrist feature points. In step 4), when determining the driver's gesture direction, it is necessary to determine whether the driver's gesture direction is to the left, to the center, or to the right. The following method is used to determine whether the driver's gesture direction is to the left, to the center, or to the right: Determine which spatial region of the three-dimensional space the wrist feature point of the hand making the gesture is located in: the left spatial region, the center spatial region, or the right spatial region. If it is located in the left spatial region, the driver's gesture direction is determined to be to the left; if it is located in the center spatial region, the driver's gesture direction is determined to be to the center; if it is located in the right spatial region, the driver's gesture direction is determined to be to the right. The left spatial region, the center spatial region, and the right spatial region are three regions that are pre-defined in three-dimensional space.

[0015] Its beneficial effect is that it can directly set up three areas in three-dimensional space to determine whether the driver's hand gesture is to the left, center or right. The method is simple but highly accurate.

[0016] Furthermore, in step 4), when determining the driver's gesture direction, it is also necessary to determine whether the driver's gesture direction is upward or downward. The following method is used to determine whether the driver's gesture direction is upward or downward: determine whether the wrist feature point of the hand making the gesture is located in the upper region or the lower region of the spatial area. If it is located in the upper region, the driver's gesture direction is determined to be upward; if it is located in the lower region, the driver's gesture direction is determined to be downward. Each spatial area is divided into two regions using its corresponding plane. In three-dimensional space, the upper region is the upper region, and the lower region is the lower region.

[0017] Its beneficial effects are: it directly fits a plane into the three regions to divide the three regions into upper and lower regions, and combines the wrist feature points in the upper and lower regions to accurately determine whether the driver's hand gesture is upward or downward. The method is simple but highly accurate.

[0018] Furthermore, the human body feature points include left wrist feature points and right wrist feature points. In step 4), it is necessary to determine whether the driver's gesture direction is upward or downward, and the following method is used to determine whether the driver's gesture direction is upward or downward: the position of the wrist feature point of the hand making the gesture in three-dimensional space is used as the model input and input into the trained gesture up and down classification model to obtain the driver's gesture direction classification result, which is upward or downward; the gesture up and down classification model is trained using the position of the wrist feature point of the hand making the gesture in three-dimensional space and the corresponding gesture direction classification result as the training set.

[0019] Its beneficial effects are: it uses a machine learning-based gesture up-down classification model to determine whether the driver's gesture is up or down, and the gesture up-down classification model is trained with a large amount of accurate historical data, so the accuracy is high.

[0020] Furthermore, the gesture up-and-down classification model is an SVM classifier.

[0021] Its advantages are: using an SVM classifier, the method is simple and easy to implement.

[0022] Further, in step 2), the driver in the driving training area image is identified using the following method: the driving training area image is input into a trained target behavior detection model to identify the personnel area and the behavior of the personnel in the personnel area, wherein the personnel behavior includes standing and sitting; the target behavior detection model is trained using an image set labeled with personnel and personnel behavior as a training set; it is determined whether the personnel area corresponding to the person whose behavior is sitting is located in the designated driver area, and if it is located in the designated driver area, the person is identified as the driver; wherein, the designated driver area is the area in the image.

[0023] Its beneficial effects are: it uses a machine learning-based target behavior detection model to identify personnel areas and the behavior of personnel in personnel areas, and the target behavior detection model is trained using a large amount of accurate historical data, so it has high accuracy.

[0024] Furthermore, the driver's gesture region is identified using the following method: after identifying the driver, the image of the person region corresponding to the driver is input into the trained gesture region recognition model to identify the driver's gesture region; the gesture region recognition model is trained using an image set marked with gesture regions as the training set.

[0025] Its beneficial effect is that by using a trained machine learning model to find the gesture area, it is easier to accurately locate the driver's gesture area.

[0026] Furthermore, in step 3), the OpenPose algorithm is used to detect the driver's human feature points.

[0027] Its beneficial effects are: the OpenPose algorithm is used to detect human feature points, and the method is simple and easy to implement.

[0028] To address the aforementioned technical problems, this invention also provides a binocular camera-based simulated driving system, comprising a simulated cockpit, a simulated driving control panel, a driver's seat, two cameras positioned at different locations, and a main control device. The main control device is connected to the two cameras. The main control device includes a memory and a processor. The processor executes computer instructions stored in the memory to implement the binocular camera-based simulated driving posture evaluation method described above, achieving the same beneficial effects as the method. Attached Figure Description

[0029] Figure 1 This is a flowchart of the simulated driving posture evaluation method based on a binocular camera according to the present invention;

[0030] Figure 2 This is a schematic diagram showing the spatial relationship between the two cameras used in this invention;

[0031] Figure 3 This is a flowchart illustrating the process of determining left and right hands in the binocular camera-based simulated driving posture evaluation method of the present invention. Detailed Implementation

[0032] This invention employs two cameras in a simulated driving system, positioned at different locations but both capturing images of the simulated driving training area. Both cameras are calibrated and positioned, and their calibration and positioning information allows the construction of a three-dimensional spatial position calculation model. This model calculates the position of a point in three-dimensional space based on its image positions in the two driving training area images. Using the image information from the two cameras and the three-dimensional spatial position calculation model, the driver's actual position in three-dimensional space can be calculated. Based on this information, the driver's hand gestures can be identified, and the direction of those gestures can be determined. The driver's simulated driving posture can then be evaluated based on the identification results.

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0034] Example of a binocular camera-based simulated driving system:

[0035] An embodiment of the simulated driving system based on binocular cameras according to the present invention includes a simulated driving cockpit. Inside the cockpit are a simulated driving control panel, a driver's seat, and other equipment for simulated driving. Furthermore, two cameras are mounted on the inner surface of the cockpit, with different positions. Typically, one camera is positioned on the upper left side of the driver's seat (hereinafter referred to as the left camera), and the other camera is positioned on the upper right side of the driver's seat (hereinafter referred to as the right camera). Both cameras simultaneously capture images of the driving training area covering the driver's seat.

[0036] The system also includes a main control unit, which can be connected to two cameras via hardwired or wireless means to acquire images captured by the two cameras. The main control unit processes these images to implement the binocular camera-based simulated driving posture evaluation method of this invention, thus completing the assessment of the student's driving posture. The main control unit includes a memory, a processor, and an internal bus. The memory and processor communicate and interact with each other via the internal bus. The processor can be a microprocessor (MCU), a programmable logic device (FPGA), or other processing devices. The memory can be various types of memory that store information using electrical energy, such as RAM and ROM; it can also be various types of memory that store information using magnetic energy, such as hard disks, floppy disks, magnetic tapes, magnetic core memory, bubble memory, and USB flash drives; it can also be various types of memory that store information using optical methods, such as CDs and DVDs; and of course, it can also be other types of memory, such as quantum memory and graphene memory.

[0037] The system also includes a display device, which is connected to the main control device via hard wire or wireless means, to visually display the simulated driving posture evaluation results of the trainee.

[0038] The following section provides a detailed introduction to the binocular camera-based simulated driving posture evaluation method implemented in this system. The overall process is as follows: Figure 1 As shown.

[0039] Step 1: Establish a three-dimensional spatial position calculation model. This model is used to calculate the position (i.e., object coordinates) of a point in three-dimensional space based on the image position (i.e., image point coordinates) of a point in two driving training area images.

[0040] like Figure 2 The image shows a schematic diagram of 3D reconstruction of spatial points in binocular stereo vision. Let the left camera be... Located at the origin of the world coordinate system and without rotation, the image coordinate system is... The effective focal length is The coordinate system of the right camera is... The effective focal length is From the camera perspective transformation model, we have:

[0041]

[0042]

[0043] and coordinate system and The relative positions of coordinate systems can be determined by a spatial transformation matrix. To indicate, for:

[0044]

[0045]

[0046] In the formula, , , respectively coordinate system and Rotation matrix between coordinate systems and translation vector between the origin.

[0047] From the above formula, it can be seen that for The correspondence between spatial points in the coordinate system and image plane points of two cameras is as follows:

[0048]

[0049] Therefore, the three-dimensional coordinates of a point in space can be expressed as:

[0050]

[0051] Therefore, given the focal length , Given the image coordinates of a point in space within the left and right cameras, we only need to determine the rotation matrix. Translation vector This allows us to obtain the three-dimensional spatial coordinates of the points on the object being measured.

[0052] The left and right cameras were calibrated using Zhang Zhengyou's calibration algorithm, and the camera intrinsic parameter matrix was calculated according to the following formula. extrinsic parameter matrix ,in, , Indicates the focal length of the camera. Represents the coordinates of the image center. , express The rotation matrix, express The translation vector.

[0053] Assuming that it is known , and , Let the left and right cameras be positioned relative to the world coordinate system, respectively. Then, the positional relationship between the two cameras can be expressed by the following formula:

[0054] ,

[0055] Therefore, the same calibration template is used to calibrate the two cameras simultaneously, obtain the intrinsic and extrinsic parameters of the two cameras respectively, and calibrate the structural parameters of the binocular vision system, thereby constructing a three-dimensional spatial position calculation model.

[0056] Step two: Use two cameras, one on the left and one on the right, to capture images of the driving training area, resulting in two images of the driving training area.

[0057] Step 3: Construct a target behavior detection model. Input the two driving training area images into the target behavior detection model to identify the personnel area and the behavior of the personnel in the personnel area, including standing and sitting.

[0058] In this embodiment, the target behavior detection model uses the YOLO algorithm, trained using an image set labeled with individuals and their corresponding behaviors as the training set. The main purpose of this model is to delineate the individual regions within the entire driver training area image and to identify the behaviors of each individual. Of course, other machine learning-based target detection models from existing technologies can also be used.

[0059] Generally, the person sitting is most likely the driver, while the person standing is most likely an unrelated person, not the driver.

[0060] Step 4: Based on prior knowledge or region fitting, accurately obtain the set driver region driver_rect. Determine whether the personnel region corresponding to the identified seated person is located within the set driver region driver_rect. If it is located within the region, it indicates that the person is the driver; otherwise, it is determined that the driver has left their post. Directly return to the driver left-post status and repeat the identification in Step 3 until the final judgment result is whether the personnel region corresponding to the identified seated person is located within the set driver region, thus identifying the driver.

[0061] It should be noted that in this step, both driver training area images can identify the corresponding personnel area, and are judged against the corresponding set driver area. When the personnel area in both driver training area images is located in the corresponding set driver area, the person is identified as the driver.

[0062] Step 5: Construct a gesture region recognition model. Input the identified driver's corresponding personnel region into the gesture region recognition model to identify the driver's gesture region, which includes the gesture regions identified from the two driver training area images.

[0063] In this embodiment, the gesture region recognition model also uses the YOLO algorithm, trained using an image set labeled with gesture regions as the training set. The main purpose of this model is to delineate the gesture region to help determine which hand is making the gesture. Of course, other machine learning-based object detection models in the prior art can also be used.

[0064] Step Six: Activate the OpenPose algorithm to detect the driver's human body feature points. This step can detect a wide variety of feature points, including wrist, elbow, shoulder, and nose features. Calculate the number of human body feature points (point_num) for each person that fall within the designated driver region (driver_rect). The person with the most points (point_num) is identified as the driver. This step serves to corroborate the driver identification results in Step Four. In practice, this step can be skipped and Step Seven can be executed directly. Similarly, this step processes the individuals in the two driver training area images separately, and the final result is only determined if the results of the separate processing are consistent.

[0065] Step seven, determine whether the driver is using their left or right hand to make the gesture; the overall process is as follows: Figure 3 As shown.

[0066] 1. Determine the center point coordinates PointL(x,y) of the gesture area in the driver training area image captured by the left camera and the center point coordinates PointR(x,y) of the gesture area in the driver training area image captured by the right camera. Based on the three-dimensional spatial position calculation model constructed in step one, calculate the three-dimensional spatial position coordinates CenterPoint(x,y,z) of the center point of the gesture area.

[0067] 2. Extract the feature points of the left and right wrists; determine the position coordinates of the left wrist feature point in the driving training area image captured by the left camera, and the position coordinates of the left wrist feature point in the driving training area image captured by the right camera. Using the 3D spatial position calculation model constructed in step one, calculate the 3D spatial position coordinates LPoint(x,y,z) of the left wrist feature point; similarly, determine the position coordinates of the right wrist feature point in the driving training area image captured by the left camera, and the position coordinates of the right wrist feature point in the driving training area image captured by the right camera. Using the 3D spatial position calculation model constructed in step one, calculate the 3D spatial position coordinates of the right wrist feature point. .

[0068] 3. According to and Calculate the actual distance LDistance between the center point of the gesture area and the feature point of the left wrist; based on and Calculate the actual distance RDistance between the center point of the gesture area and the feature point of the right wrist.

[0069] 4. If LStistance is less than RStistance, the hand making the gesture is determined to be the left hand; otherwise, the hand making the gesture is determined to be the right hand.

[0070] Step 8: Determine the direction of the driver's hand gesture. Specifically, first determine whether the gesture is pointing left, center, or right; then determine whether it is pointing up or down; and finally determine whether the gesture is pointing to the upper left, lower left, upper center, lower center, upper right, or lower right.

[0071] 1. Based on prior knowledge, define three spatial regions in three-dimensional space: the left spatial region (3D-left_rect), the middle spatial region (3D-middle_rect), and the right spatial region (3D-right_rect).

[0072] 2. Determine which spatial region in the previous step the wrist feature point of the hand making the gesture is located in: If the wrist feature point is located in the set left spatial region 3D-left_rect, the gesture direction is determined to be to the left; if the wrist feature point is located in the set middle spatial region 3D-middle_rect, the gesture direction is determined to be to the center; if the wrist feature point is located in the set right spatial region 3D-right_rect, the gesture direction is determined to be to the right.

[0073] 3. Use either of the following two methods to determine whether the gesture is upward or downward:

[0074] Method 1: By using the three-dimensional spatial coordinates of a large number of wrist feature points of the hand making the gesture, a plane is fitted to each of the three spatial regions introduced in the previous step. , , These three planes divide each of the three spatial regions into an upper region and a lower region. The system determines whether the wrist feature point of the gesturing hand is located above or below the fitted plane in its spatial region in three-dimensional space: if it is above the fitted plane, the gesture direction is determined to be upward; if it is below the fitted plane, the gesture direction is determined to be downward.

[0075] Method 2: A large number of 3D spatial coordinates of wrist feature points of the gesturing hand are used as training data for an SVM classifier, resulting in a trained SVM classifier with classification results including "up" and "down". The 3D spatial coordinates of the driver's gesturing hand are then input into the trained SVM classifier to obtain the gesture direction of the driver's hand.

[0076] Step nine: After determining the direction of the driver's hand gestures, evaluate the driver's simulated driving posture based on the direction of the driver's hand gestures to determine whether the driver has made the corresponding hand gestures as required.

[0077] In summary, this invention utilizes human body feature points and transforms these points into a three-dimensional space to judge the trainee's hand gestures, thereby determining whether the trainee has made the correct gestures as required. This method enables automated evaluation of the trainee's simulated driving posture during simulated driving training, resulting in more objective results and improving the efficiency of simulated driving training evaluation.

[0078] Example of a simulated driving posture evaluation method based on binocular cameras:

[0079] An embodiment of the simulated driving posture evaluation method based on a binocular camera according to the present invention, the overall process of which is as follows: Figure 1 As shown, the method for evaluating simulated driving posture based on a binocular camera is consistent with the method described in the embodiment of the simulated driving system based on a binocular camera, and will not be repeated here.

Claims

1. A method for evaluating simulated driving posture based on a binocular camera, characterized in that, include: 1) Two cameras positioned at different locations are used to capture images of the driver training area, resulting in two images of the training area; Two cameras have been located and calibrated using the same calibration template. A three-dimensional spatial position calculation model is constructed using the calibration and positioning information. This calculation model is used to calculate the position of a point in three-dimensional space based on the image positions of a point in the two training area images, given the effective focal lengths of the two cameras and the obtained rotation matrix and translation vector. 2) Identify the driver and the driver's gesture regions in the two training area images respectively, determine the positions of the gesture regions in the two training area images respectively, and determine the positions of the gesture regions in three-dimensional space in combination with the calculation model; 3) Detect the positions of human feature points in the corresponding training area images, and determine the positions of human feature points in three-dimensional space based on the calculation model; 4) Determine the positions of the left wrist feature point, right wrist feature point, and the center point of the gesture area in three-dimensional space; calculate the three-dimensional distances from the left wrist feature point and right wrist feature point to the center point of the gesture area, and denot the two distances as LDistance and RDistance respectively; compare the two distances to determine whether the driver's hand is the left or right hand: if LDistance < RDistance, then it is determined to be the left hand, otherwise it is determined to be the right hand; Based on the position of the feature points of the hand making the gesture in three-dimensional space, the direction of the driver's gesture is determined, and the driver's simulated driving posture is evaluated based on the gesture direction.

2. The method for evaluating simulated driving posture based on a binocular camera according to claim 1, characterized in that, In step 4), when determining the direction of the driver's hand gesture, it is necessary to determine whether the driver's hand gesture is pointing to the left, center, or right, and the following method is used to determine whether the driver's hand gesture is pointing to the left, center, or right: Determine which of the three-dimensional spatial regions the wrist feature point of the hand making the gesture is located in: a designated left spatial region, a designated middle spatial region, or a designated right spatial region. If it is located in the designated left spatial region, the driver's gesture direction is determined to be left; if it is located in the designated middle spatial region, the driver's gesture direction is determined to be center; and if it is located in the designated right spatial region, the driver's gesture direction is determined to be right. The designated left spatial region, the designated middle spatial region, and the designated right spatial region are three pre-defined regions in three-dimensional space.

3. The method for evaluating simulated driving posture based on a binocular camera according to claim 2, characterized in that, In step 4), when determining the direction of the driver's hand gesture, it is also necessary to determine whether the driver's hand gesture is pointing upwards or downwards, and the following method is used to determine whether the driver's hand gesture is pointing upwards or downwards: The system determines whether the wrist feature point of the hand making the gesture is located in the upper or lower region of the spatial area. If it is located in the upper region, the driver's gesture direction is determined to be upward; if it is located in the lower region, the driver's gesture direction is determined to be downward. Each spatial area is divided into two regions using its corresponding plane. In three-dimensional space, the upper region is the upper region, and the lower region is the lower region.

4. The simulated driving posture evaluation method based on a binocular camera according to claim 1, characterized in that, Human body feature points include left wrist feature points and right wrist feature points. In step 4), it is necessary to determine whether the driver's hand gesture direction is upward or downward, and the following method is used to determine whether the driver's hand gesture direction is upward or downward: The position of the wrist feature point of the hand making the gesture in three-dimensional space is used as the model input and input into the trained gesture up and down classification model to obtain the driver's gesture direction classification result, which is up or down; The gesture up-down classification model is trained using the position of the wrist feature points of the hand making the gesture in three-dimensional space and the corresponding gesture direction classification results as the training set.

5. The method for evaluating simulated driving posture based on a binocular camera according to claim 4, characterized in that, The gesture classification model is an SVM classifier.

6. The method for evaluating simulated driving posture based on a binocular camera according to claim 1, characterized in that, In step 2), the driver in the driver training area image is identified using the following method: The driver training area image is input into the trained target behavior detection model to identify the personnel area and the behavior of the personnel in the personnel area, including standing and sitting; the target behavior detection model is trained using an image set labeled with personnel and personnel behavior as the training set; Determine whether the area corresponding to the person whose behavior is sitting is located in the designated driver area. If it is located in the designated driver area, then the person is identified as the driver. The designated driver area is the area in the image.

7. The method for evaluating simulated driving posture based on a binocular camera according to claim 6, characterized in that, The following method is used to identify the driver's gesture area: After identifying the driver, the image of the person area corresponding to the driver is input into the trained gesture region recognition model to identify the driver's gesture region; the gesture region recognition model is trained using an image set marked with gesture regions as the training set.

8. The method for evaluating simulated driving posture based on a binocular camera according to any one of claims 1 to 7, characterized in that, In step 3), the OpenPose algorithm is used to detect the driver's human feature points.

9. A driving simulation system based on a binocular camera, characterized in that, The system includes a simulated cockpit, a simulated driving console, a driver's seat, two cameras positioned at different locations, and a main control device. The main control device is connected to the two cameras via hardwired or wireless means to acquire images captured by the two cameras. The main control device includes a memory and a processor, the processor being used to execute computer instructions stored in the memory to implement the simulated driving posture evaluation method based on binocular cameras as described in claim 1.

10. The binocular camera-based simulated driving system according to claim 9, characterized in that, In step 4), when determining the direction of the driver's hand gesture, it is necessary to determine whether the driver's hand gesture is pointing to the left, center, or right, and the following method is used to determine whether the driver's hand gesture is pointing to the left, center, or right: Determine which of the three-dimensional spatial regions the wrist feature point of the hand making the gesture is located in: a designated left spatial region, a designated middle spatial region, or a designated right spatial region. If it is located in the designated left spatial region, the driver's gesture direction is determined to be left; if it is located in the designated middle spatial region, the driver's gesture direction is determined to be center; and if it is located in the designated right spatial region, the driver's gesture direction is determined to be right. The designated left spatial region, the designated middle spatial region, and the designated right spatial region are three pre-defined regions in three-dimensional space.

11. The binocular camera-based simulated driving system according to claim 10, characterized in that, In step 4), when determining the direction of the driver's hand gesture, it is also necessary to determine whether the driver's hand gesture is pointing upwards or downwards, and the following method is used to determine whether the driver's hand gesture is pointing upwards or downwards: The system determines whether the wrist feature point of the hand making the gesture is located in the upper or lower region of the spatial area. If it is located in the upper region, the driver's gesture direction is determined to be upward; if it is located in the lower region, the driver's gesture direction is determined to be downward. Each spatial area is divided into two regions using its corresponding plane. In three-dimensional space, the upper region is the upper region, and the lower region is the lower region.

12. The binocular camera-based simulated driving system according to claim 9, characterized in that, Human body feature points include left wrist feature points and right wrist feature points. In step 4), it is necessary to determine whether the driver's hand gesture direction is upward or downward, and the following method is used to determine whether the driver's hand gesture direction is upward or downward: The position of the wrist feature point of the hand making the gesture in three-dimensional space is used as the model input and input into the trained gesture up and down classification model to obtain the driver's gesture direction classification result, which is up or down; The gesture up-down classification model is trained using the position of the wrist feature points of the hand making the gesture in three-dimensional space and the corresponding gesture direction classification results as the training set.

13. The binocular camera-based simulated driving system according to claim 12, characterized in that, The gesture classification model is an SVM classifier.

14. The binocular camera-based simulated driving system according to claim 9, characterized in that, In step 2), the driver in the driver training area image is identified using the following method: The driver training area image is input into the trained target behavior detection model to identify the personnel area and the behavior of the personnel in the personnel area, including standing and sitting; the target behavior detection model is trained using an image set labeled with personnel and personnel behavior as the training set; Determine whether the area corresponding to the person whose behavior is sitting is located in the designated driver area. If it is located in the designated driver area, then the person is identified as the driver. The designated driver area is the area in the image.

15. The binocular camera-based simulated driving system according to claim 14, characterized in that, The following method is used to identify the driver's gesture area: After identifying the driver, the image of the person area corresponding to the driver is input into the trained gesture region recognition model to identify the driver's gesture region; the gesture region recognition model is trained using an image set marked with gesture regions as the training set.

16. The binocular camera-based simulated driving system according to any one of claims 9 to 15, characterized in that, In step 3), the OpenPose algorithm is used to detect the driver's human feature points.

Images