Man-machine visual field analysis method and system

Abstract

The invention relates to the technical field of visual field analysis, in particular to a man-machine visual field analysis method and system. The method comprises the following steps: acquiring view data of an operator of the engineering vehicle based on 360-degree view video acquisition equipment on the engineering vehicle; analyzing the collected view data, and determining a view range and a blind area of the operator; according to an analysis result, proposing an optimization suggestion of a cockpit design and an operation process; according to the method, the effect of the optimization suggestion is evaluated through simulation verification, and the safety, efficiency and comfort of engineering vehicle operation can be remarkably improved through comprehensive application of view data collection, data analysis, optimization suggestion and simulation verification. The method not only optimizes the design of the cockpit and reduces the blind area of the visual field, but also provides a safer and more efficient working environment for operators through an intelligent auxiliary system and a real-time monitoring function. Meanwhile, data support is provided for equipment management and maintenance, and efficient operation of the engineering vehicle is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of visual field analysis, and particularly to a human-machine visual field analysis method and system. Background Art

[0002] Engineering vehicles, such as bulldozers, excavators, and loaders, etc., are an important part of the equipment industry. During the operation of engineering vehicles, the human-machine visual field is one of the key indicators for evaluating the human-machine performance of construction machinery, and its performance directly affects the operation efficiency and safety. Therefore, it is crucial to quantitatively analyze the human-machine visual field of the driver's cab space of construction machinery, which not only helps to compare the differences in human-machine visual field performance between similar but different configurations of construction machinery, but also provides a scientific basis for subsequent optimization design.

[0003] Currently, the human-machine visual field analysis methods in the prior art usually adopt the following method: A tester holds a benchmark and moves within a specific area outside the construction machinery, while another driver observes and records data in the cab. The two need to record a series of data according to the predetermined operation steps, and then analyze the visual field blind area through these data. However, this method has certain defects: The testing process is time-consuming and laborious, with low efficiency; the test results are greatly affected by human factors, which easily leads to non-standard and inaccurate data recording, thus affecting the reliability of the analysis results. To solve this technical problem, a human-machine visual field analysis method and system are proposed. Summary of the Invention

[0004] To solve the above technical problems existing in the prior art, the present invention provides a human-machine visual field analysis method and system.

[0005] To achieve the above object, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, in an embodiment provided by the present invention, a human-machine visual field analysis method is provided, and the method includes the following steps:

[0007] Collect the visual field data of the operator of the engineering vehicle based on the 360° visual field video acquisition device on the engineering vehicle;

[0008] Analyze the collected visual field data to determine the visual field range and blind area of the operator;

[0009] According to the analysis results, put forward optimization suggestions for the cab design and operation process;

[0010] Evaluate the effect of the optimization suggestions through simulation verification.

[0011] As a further solution of the present invention, the analyzing the collected visual field data to determine the visual field range and blind area of the operator includes:

[0012] Use image processing algorithms to analyze obstacles and key areas in the field of view data;

[0013] Automatically identify blind spots in the field of view.

[0014] As a further aspect of the present invention, the use of image processing algorithms to analyze obstacles and key areas in the field of view data includes:

[0015] Use a depth camera to obtain a depth image, and detect obstacles through threshold processing and region segmentation algorithms;

[0016] Extract key areas based on the ROI segmentation algorithm.

[0017] As a further aspect of the present invention, using a depth camera to obtain a depth image and detecting obstacles through threshold processing and region segmentation algorithms includes: obtaining depth image data, and performing format conversion and calibration; setting a distance threshold, and marking pixels with a depth value less than the threshold as obstacles; segmenting and classifying obstacles through connected region analysis or clustering algorithms.

[0018] As a further aspect of the present invention, the extraction of key areas based on the ROI segmentation algorithm includes: dividing the image into foreground and background by setting a threshold; using Sobel or Canny operators to extract edges, and then segmenting out key areas; extracting key areas through morphological operations such as dilation and erosion.

[0019] As a further aspect of the present invention, the use of the analysis results to propose optimization suggestions for the cockpit design and operation process includes:

[0020] Use CAD software integrated with the YOLOv model to propose optimization suggestions for the cockpit design and operation process based on the analysis results.

[0021] As a further aspect of the present invention, the optimization suggestions include adjusting the seat position, optimizing the dashboard layout, adding auxiliary display devices, and optimizing the front and rear views of the cab.

[0022] As a further aspect of the present invention, using a depth camera to obtain a depth image and detecting obstacles through threshold processing and region segmentation algorithms includes:

[0023] Efficiently read depth image data using the SDK or API of a depth camera; perform denoising and filtering on the depth image; dynamically adjust the threshold according to the ambient light and the scene; combine multiple threshold ranges to detect near and far obstacles respectively; use an efficient connected component analysis algorithm to quickly segment the obstacle area; adopt the K-Means clustering algorithm to quickly cluster the depth data; set a reasonable area threshold; quickly calculate the bounding box of the obstacle and extract the depth value of the center point for judging the distance of the obstacle; display the detected obstacles on the user interface in real time; dynamically adjust the detection parameters in combination with user feedback to optimize the detection effect; use a convolutional neural network or a YOLO series model to detect obstacles in the depth image and automatically learn the features of the obstacles; combine the data of the depth camera and the ordinary camera, and utilize the texture of the visual information and the accuracy of the depth information; convert the depth image into point cloud data and use the point cloud library for filtering and clustering.

[0024] As a further solution of the present invention, the extraction of the key area based on the ROI segmentation algorithm includes:

[0025] Perform preprocessing on the image data to be processed, and select the region of interest by defining a rectangular area; automatically identify the key area in the image, and map the region of interest from the original image to the feature map, usually by calculating the proportional relationship between the coordinates of the region of interest and the size of the feature map; divide the region of interest into a fixed number of sub-regions, and the number and size of the sub-regions are determined by subsequent processing requirements; when the size of the region of interest cannot be evenly divided into the specified number of sub-regions, adopt the strategy of rounding up or down; perform max pooling or average pooling operations on each sub-region to obtain a feature representation of a fixed size; map regions of interest of different sizes to the same-sized feature map; perform segmentation and classification on the extracted region of interest to identify the key area; use a deep learning model to perform instance segmentation on the region of interest.

[0026] In a second aspect, in another embodiment provided by the present invention, a human-machine vision analysis system is provided, and the system includes: a data acquisition module, a vision data analysis module, an analysis and optimization module, and a verification and evaluation module.

[0027] The data acquisition module is used to collect the vision data of the operator of the engineering vehicle based on the 360° vision video acquisition device on the engineering vehicle.

[0028] The vision data analysis module is used to analyze the collected vision data to determine the vision range and blind area of the operator.

[0029] The analysis and optimization module is used to propose optimization suggestions for the cockpit design and operation process according to the analysis results.

[0030] The verification and evaluation module is used to evaluate the effect of the optimization suggestions through simulation verification.

[0031] The technical solution provided by the present invention has the following beneficial effects:

[0032] The human-machine vision analysis method and system provided by the present invention are based on a 360° vision video acquisition device on an engineering vehicle to collect vision data of an engineering vehicle operator; analyze the collected vision data to determine the operator's vision range and blind area; according to the analysis results, propose optimization suggestions for the cockpit design and operation process; evaluate the effect of the optimization suggestions through simulation verification. By comprehensively applying vision data collection, data analysis, optimization suggestions, and simulation verification, the present invention can significantly improve the safety, efficiency, and comfort of engineering vehicle operation. This method not only optimizes the cockpit design and reduces the vision blind area, but also provides a safer and more efficient working environment for the operator through an intelligent assistance system and real-time monitoring function. At the same time, it also provides data support for equipment management and maintenance to ensure the efficient operation of engineering vehicles.

[0033] These aspects or other aspects of the present invention will be more clearly understood in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these drawings.

[0035] Figure 1 It is a flowchart of the human-machine vision analysis method according to an embodiment of the present invention.

[0036] Figure 2 It is a specific flowchart of step S20 in the human-machine vision analysis method according to an embodiment of the present invention.

[0037] Figure 3 It is a specific flowchart of step S201 in the human-machine vision analysis method according to an embodiment of the present invention.

[0038] Figure 4 It is a structural block diagram of the human-machine vision analysis system according to an embodiment of the present invention.

[0039] In the figure: data acquisition module - 100, vision data analysis module - 200, analysis and optimization module - 300, verification and evaluation module - 400. Detailed Implementation Modes

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] The flowchart shown in the accompanying drawings is only an example illustration, and does not necessarily include all the contents and operations / steps, nor does it necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, combined, or partially merged, so the actual execution order may be changed according to the actual situation.

[0042] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0043] Specifically, the embodiments of the present invention will be further elaborated below in conjunction with the accompanying drawings.

[0044] Please refer to Figure 1 , Figure 1 which is a flowchart of a human-machine vision analysis method provided by an embodiment of the present invention. As Figure 1 shown, the human-machine vision analysis method includes steps S10 to step S40.

[0045] S10. Based on the 360° vision video acquisition device on the engineering vehicle, collect the vision data of the engineering vehicle operator.

[0046] S20. Analyze the collected vision data to determine the operator's vision range and blind area.

[0047] Please refer to Figure 2 , in the embodiment of the present invention, the step S20 of analyzing the collected vision data to determine the operator's vision range and blind area includes steps S201 and S202.

[0048] S201. Use an image processing algorithm to analyze the obstacles and key areas in the vision data.

[0049] Please refer to Figure 3 , the step S201 of using an image processing algorithm to analyze the obstacles and key areas in the vision data includes steps S2011 and S2012.

[0050] S2011. Obtain a depth image using a depth camera, and detect obstacles through threshold processing and region segmentation algorithms;

[0051] Specifically, obtaining a depth image using a depth camera and detecting obstacles through threshold processing and region segmentation algorithms includes: obtaining depth image data, and performing format conversion and calibration; setting a distance threshold, and marking pixels with depth values less than the threshold as obstacles; segmenting and classifying obstacles through connected component analysis or clustering algorithms (such as K-Means). The clustering algorithm can be K-Means.

[0052] A clustering algorithm is an unsupervised learning method used to divide a set of data points into several clusters, such that data points within the same cluster are similar to each other, while data points in different clusters are quite different. The goal of clustering is to reveal the internal structure and patterns in the data, and it is widely applied in fields such as data mining, image processing, market analysis, recommendation systems, etc.

[0053] Furthermore, obtaining a depth image using a depth camera and detecting obstacles through threshold processing and region segmentation algorithms includes: efficiently reading depth image data using the SDK or API of the depth camera, and ensuring that the data format is suitable for subsequent processing; performing denoising and filtering on the depth image to reduce the impact of noise on obstacle detection; dynamically adjusting the threshold according to the environmental light and scene to avoid the limitations of a fixed threshold in different scenarios; combining multiple threshold ranges to detect close-range and long-range obstacles respectively, improving the flexibility of detection; using an efficient connected component analysis algorithm (such as an algorithm based on breadth-first search) to quickly segment the obstacle area; using clustering algorithms such as K-Means to quickly cluster the depth data, reducing the computational complexity; setting a reasonable area threshold to quickly filter out small-area noise regions, reducing false detections; quickly calculating the bounding box of the obstacle and extracting the center point depth value for judging the distance of the obstacle; classifying obstacles by combining depth information and other features (such as texture, color) to improve the detection accuracy; displaying the detected obstacles on the user interface in real time for easy and quick response by the operator; dynamically adjusting the detection parameters in combination with user feedback to optimize the detection effect; using a convolutional neural network (CNN) or YOLO series models to detect obstacles in the depth image and automatically learn the features of the obstacles; adopting a lightweight network structure (such as MobileNet or Tiny-YOLO) to improve the real-time performance of the model; combining the data of the depth camera and the ordinary camera, and using the texture of the visual information and the accuracy of the depth information to improve the robustness of obstacle detection; converting the depth image into point cloud data and using the Point Cloud Library (PCL) for filtering and clustering to further optimize obstacle detection.

[0054] In the present invention, obstacle detection can also be performed through OpenCV or by using a pre-trained deep learning model.

[0055] S2012. Extract the key region based on the ROI segmentation algorithm.

[0056] The ROI segmentation algorithm can be a convolutional neural network, YOLO series models, or Mask R-CNN models trained with a large number of labeled data. The YOLO series models have high efficiency and real-time performance. Mask R-CNN can generate precise pixel boundaries of the target object and is suitable for ROI segmentation in complex scenarios.

[0057] The extraction of the key region based on the ROI segmentation algorithm includes: dividing the image into foreground and background by setting a threshold; extracting edges using Sobel or Canny operators, and then segmenting the key region; extracting the key region through morphological operations such as dilation and erosion.

[0058] Further, the extraction of the key region based on the ROI segmentation algorithm includes:

[0059] Preprocess the image data to be processed, select the region of interest by defining a rectangular area; automatically identify the key region in the image, map the region of interest from the original image to the feature map, usually by calculating the proportional relationship between the coordinates of the region of interest and the size of the feature map; divide the region of interest into a fixed number of sub-regions (bins), and the number and size of the sub-regions are determined by subsequent processing requirements; when the size of the region of interest cannot be evenly divided into the specified number of sub-regions, adopt the strategy of rounding up or down; perform max pooling or average pooling operations on each sub-region to obtain a fixed-size feature representation; map regions of interest of different sizes to the same-size feature map for subsequent network processing; segment and classify the extracted regions of interest to identify the key region; perform instance segmentation on the regions of interest using a deep learning model (such as MaskR-CNN).

[0060] S202. Automatically identify the blind spots in the field of vision.

[0061] S30. According to the analysis results, propose optimization suggestions for the cockpit design and operation process.

[0062] The S30, using the analysis results, propose optimization suggestions for the cockpit design and operation process, including:

[0063] Use CAD software integrated with the YOLOv8 model to propose optimization suggestions for the cockpit design and operation process according to the analysis results.

[0064] In the embodiments of the present invention, the optimization suggestions include adjusting the seat position, optimizing the dashboard layout, adding auxiliary display devices, and optimizing the front and rear views of the cab, etc.

[0065] S40. Through simulation verification, evaluate the effect of the optimization suggestions.

[0066] By analyzing the vision data, the present invention can accurately identify the vision blind spots of the operator, issue early warnings in a timely manner, reduce the accident risk caused by blind spots, and based on the vision analysis results, combined with machine learning algorithms, provide intelligent operation path planning for engineering vehicles and optimize the operation process; through the intelligent assistance system, reduce the operation burden of the operator in complex environments and improve the overall operation experience; through the analysis of vision data, extract key operation indicators to provide decision-making support for management; through simulation verification, evaluate the actual effect of the optimization suggestions to ensure the effectiveness of the optimization measures. By comprehensively applying vision data collection, data analysis, optimization suggestions, and simulation verification, the present invention can significantly improve the safety, efficiency, and comfort of engineering vehicle operation. This method not only optimizes the cab design and reduces vision blind spots, but also provides a safer and more efficient working environment for the operator through the intelligent assistance system and real-time monitoring function. At the same time, it also provides data support for equipment management and maintenance to ensure the efficient operation of engineering vehicles.

[0067] It should be understood that although the above is described in a certain order, these steps are not necessarily executed in the above order. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, a part of the steps in this embodiment may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0068] In one embodiment, see Figure 3 As shown, in the embodiments of the present invention, a human-machine vision analysis system is also provided. The system includes a data collection module 100, a vision data analysis module 200, an analysis and optimization module 300, and a verification and evaluation module 400.

[0069] The data collection module 100 is used to collect the vision data of the operator of the engineering vehicle based on the 360° vision video collection device on the engineering vehicle.

[0070] The vision data analysis module 200 is used to analyze the collected vision data and determine the vision range and blind spots of the operator.

[0071] The analysis and optimization module 300 is configured to propose optimization suggestions for the cockpit design and operation process based on the analysis results.

[0072] The verification and evaluation module 400 is configured to evaluate the effect of the optimization suggestions through simulation verification.

[0073] By comprehensively applying vision data collection, data analysis, optimization suggestions, and simulation verification, the present invention can significantly improve the safety, efficiency, and comfort of the operation of engineering vehicles. This method not only optimizes the cockpit design and reduces the vision blind area, but also provides a safer and more efficient working environment for operators through the intelligent assistance system and real-time monitoring function. At the same time, it also provides data support for equipment management and maintenance to ensure the efficient operation of engineering vehicles.

[0074] In one embodiment, an embodiment of the present invention further provides a computer device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus.

[0075] The memory is used to store a computer program.

[0076] When the processor is used to execute the computer program stored on the memory, it executes the above-mentioned human-machine vision analysis method, and when the processor executes the instructions, it implements the steps in the above-mentioned method embodiments:

[0077] S10. Based on the 360° vision video acquisition device on the engineering vehicle, collect the vision data of the engineering vehicle operator.

[0078] S20. Analyze the collected vision data to determine the vision range and blind area of the operator.

[0079] S30. According to the analysis results, propose optimization suggestions for the cockpit design and operation process.

[0080] S40. Evaluate the effect of the optimization suggestions through simulation verification.

[0081] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0082] The communication interface is used for communication between the above terminal and other devices.

[0083] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0084] The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0085] The computer device includes a user device and a network device. Among them, the user device includes, but is not limited to, a computer, a smart phone, a PDA, etc.; the network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on Cloud Computing. Among them, Cloud Computing is a type of distributed computing, consisting of a super virtual computer formed by a group of loosely coupled computer sets. Among them, the computer device can run independently to implement the present invention, or can be connected to a network and implement the present invention through interactive operations with other computer devices in the network. Among them, the network where the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, etc.

[0086] It should also be understood that the term "and / or" used in the specification and appended claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0087] In an embodiment of the present invention, a storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented:

[0088] S10. Based on the 360° vision video acquisition device on the engineering vehicle, acquire the vision data of the engineering vehicle operator;

[0089] S20. Analyze the collected field of view data to determine the operator's field of view range and blind spots;

[0090] S30. Based on the analysis results, propose optimization suggestions for the cockpit design and operation process;

[0091] S40. Through simulation verification, evaluate the effect of the optimization suggestions.

[0092] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include at least one of non-volatile and volatile memories.

[0093] It should be understood that, as used herein, unless the context clearly supports exceptions, the singular form "a" is also intended to include the plural form. It should also be understood that the "and / or" used herein refers to any and all possible combinations of one or more of the associated listed items. The serial numbers of the disclosed embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0094] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including the claims) of the disclosed embodiments of the present invention is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other variations in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention shall be included in the protection scope of the embodiments of the present invention.

Claims

1. A human-machine vision analysis method, characterized in that, The method includes: Collecting the vision data of the operator of the engineering vehicle based on the 360° vision video acquisition device on the engineering vehicle; Analyzing the collected vision data to determine the operator's vision range and blind area; Proposing optimization suggestions for the cockpit design and operation process according to the analysis results; Evaluating the effect of the optimization suggestions through simulation verification.

2. The human-machine vision analysis method according to claim 1, wherein, The analyzing the collected vision data to determine the operator's vision range and blind area includes: Using image processing algorithms to analyze obstacles and key areas in the vision data; Automatically identifying the blind areas in the vision.

3. The human-machine vision analysis method according to claim 1, characterized in that, The using image processing algorithms to analyze obstacles and key areas in the vision data includes: Obtaining a depth image using a depth camera and detecting obstacles through threshold processing and region segmentation algorithms; Extracting key areas based on the ROI segmentation algorithm.

4. The human-machine vision analysis method according to claim 2, characterized in that Obtaining a depth image using a depth camera and detecting obstacles through threshold processing and region segmentation algorithms includes: obtaining depth image data and performing format conversion and calibration; setting a distance threshold and marking pixels with depth values less than the threshold as obstacles; segmenting and classifying obstacles through connected region analysis or clustering algorithms.

5. The human-machine vision analysis method according to claim 2, characterized in that, The extracting key areas based on the ROI segmentation algorithm includes: dividing the image into foreground and background by setting a threshold; using Sobel or Canny operators to extract edges and then segmenting the key areas; extracting key areas through morphological operations such as dilation and erosion.

6. The human-machine vision analysis method according to claim 1, characterized in that The proposing optimization suggestions for the cockpit design and operation process according to the analysis results includes: Using CAD software integrated with the YOLOv8 model to propose optimization suggestions for the cockpit design and operation process according to the analysis results.

7. The human-machine vision analysis method according to claim 1, characterized in that The optimization suggestions include adjusting the seat position, optimizing the dashboard layout, adding auxiliary display devices, and optimizing the front and rear vision of the cab.

8. The human-machine vision analysis method according to claim 2, characterized in that, Obtaining a depth image using a depth camera and detecting obstacles through threshold processing and region segmentation algorithms includes: Efficiently reading depth image data using the SDK or API of the depth camera; denoising and filtering the depth image; dynamically adjusting the threshold according to the ambient light and the scene; combining multiple threshold ranges to detect close-range and long-range obstacles respectively; using an efficient connected region analysis algorithm to quickly segment the obstacle area; using the K-Means clustering algorithm to quickly cluster the depth data; setting a reasonable area threshold; quickly calculating the bounding box of the obstacle and extracting the center point depth value for judging the distance of the obstacle; displaying the detected obstacles on the user interface in real time; dynamically adjusting the detection parameters in combination with user feedback to optimize the detection effect; using a convolutional neural network or YOLO series models to detect obstacles in the depth image and automatically learn the features of the obstacles; combining the data of the depth camera and the ordinary camera and utilizing the texture of the visual information and the accuracy of the depth information; converting the depth image into point cloud data and using the point cloud library for filtering and clustering.

9. The human-machine vision analysis method according to claim 2, wherein, The extracting key areas based on the ROI segmentation algorithm includes: Preprocess the image data to be processed, and select the region of interest by defining a rectangular area; automatically identify the key regions in the image, and map the region of interest from the original image to the feature map, usually achieved by calculating the proportional relationship between the coordinates of the region of interest and the size of the feature map; divide the region of interest into a fixed number of sub-regions, and the number and size of the sub-regions are determined by subsequent processing requirements; when the size of the region of interest cannot be evenly divided into the specified number of sub-regions, adopt the strategy of rounding up or down; perform max pooling or average pooling operations on each sub-region to obtain a feature representation of a fixed size; map regions of interest of different sizes to the same-sized feature map; segment and classify the extracted regions of interest to identify key regions; use a deep learning model to perform instance segmentation on the regions of interest.

10. A human-machine vision analysis system, characterized in that, The system includes: a data acquisition module, a field of view data analysis module, an analysis and optimization module, and a verification and evaluation module. The data acquisition module is used to collect the field of view data of the engineering vehicle operator based on the 360° field of view video acquisition device on the engineering vehicle. The field of view data analysis module is used to analyze the collected field of view data to determine the operator's field of view range and blind area. The analysis and optimization module is used to propose optimization suggestions for the cockpit design and operation process according to the analysis results. The verification and evaluation module is used to evaluate the effect of the optimization suggestions through simulation verification.

Images