Obstacle detection model training method, obstacle avoidance detection method and related equipment

Abstract

The obstacle detection model training method, obstacle avoidance detection method, and related equipment proposed in this application include: acquiring collected video data and acquiring generated video data based on a video expert model; inputting the collected video data into an initial detection model to extract obstacle features, obtaining collected obstacle features, and inputting the generated video data into the initial detection model to extract obstacle features, obtaining generated obstacle features; obtaining a distribution alignment error based on the statistical difference between the collected obstacle features and the generated obstacle features; obtaining an overall target error based on the distribution alignment error, classification error, and detection error; training the initial detection model based on the overall target error, and obtaining an obstacle detection model based on the trained initial detection model, thereby meeting the high-precision obstacle recognition and avoidance requirements in large-scale, complex, and variable scenarios.

Description

Technical Field

[0001] This application relates to the field of visual data processing technology, and in particular to obstacle detection model training methods, obstacle avoidance detection methods, and related equipment. Background Technology

[0002] Obstacle detection, a crucial research area in computer vision, plays a vital role in autonomous driving in automobiles and aircraft, as well as intelligent monitoring in security scenarios. Currently, deep learning-based obstacle detection algorithms have made some progress, but their performance is highly dependent on high-quality training data. In related technologies, training data primarily consists of visual data collected on-site in different typical scenarios, either directly or indirectly online.

[0003] However, this traditional method of acquiring training data is not only costly, but also limited by on-site conditions such as weather and lighting. The acquired training data often lacks diversity and the types of obstacles are relatively simple. As a result, the detection models trained on this training data cannot meet the accurate recognition needs of large-scale, complex and ever-changing scenarios in practical applications. Summary of the Invention

[0004] This application provides an obstacle detection model training method that can solve the above-mentioned technical problems.

[0005] To achieve the above objectives, a first aspect of this application proposes an obstacle detection model training method, the method comprising: Acquire captured video data and acquired generated video data based on a video expert model, wherein a portion of the captured video data contains real obstacles and the generated video data contains virtual obstacles; The acquired video data is input into the initial detection model to extract obstacle features, thereby obtaining the acquired obstacle features. The generated video data is then input into the initial detection model to extract obstacle features, thereby obtaining the generated obstacle features. Based on the statistical difference between the collected obstacle features and the generated obstacle features, the distribution alignment error is obtained; The classification error and detection error generated by the initial detection model when extracting obstacle features from the acquired video data and the generated video data are obtained respectively, and the overall target error is obtained based on the distribution alignment error, the classification error and the detection error; The initial detection model is trained based on the overall target error, and an obstacle detection model is obtained based on the trained initial detection model.

[0006] In some embodiments, obtaining the distribution alignment error based on the statistical difference between the acquired obstacle features and the generated obstacle features includes: Calculate the first feature mean and first feature standard deviation of the collected obstacle features, and the second feature mean and second feature standard deviation of the generated obstacle features, respectively; The mean alignment error term is obtained based on the square of the difference between the mean of the first feature and the mean of the second feature; The standard deviation alignment error term is obtained based on the square of the difference between the first feature standard deviation and the second feature standard deviation. The distribution alignment error is obtained by summing the mean alignment error term and the standard deviation alignment error term.

[0007] In some embodiments, obtaining the generated video data based on the video expert model includes: Acquire obstacle-free video data, obstacle images, and obstacle addition instructions; The obstacle-free video data, the obstacle image, and the obstacle addition instruction are input into the video expert model for data processing to obtain candidate obstacle video data. Image frame alignment extraction is performed on the candidate obstacle video data and the obstacle-free video data to obtain image sequence pairs; The image sequence pairs are scored using a visual language model and a discriminator model to obtain a joint score. When the joint score is greater than a preset score threshold, the generated video data is obtained based on the candidate obstacle video data.

[0008] In some embodiments, the video expert model includes a variational autoencoder, a multimodal feature encoding network, and a diffusion generation network. The step of inputting the obstacle-free video data, the obstacle image, and the obstacle addition instruction into the video expert model for data processing to obtain candidate obstacle video data includes: The first latent space video features are obtained by performing feature encoding on the obstacle-free video data using the variational autoencoder. The obstacle image and the obstacle addition instruction are feature-encoded using the multimodal feature coding network to obtain a multimodal feature vector; The first latent space video features and the multimodal feature vector are input into the diffusion generation network to predict diffusion features and obtain predicted obstacle features. The predicted obstacle features are decoded by the variational autoencoder to obtain the candidate obstacle video data.

[0009] In some embodiments, the step of calculating a joint score by using a visual language model and a discriminator model to score the image sequence pairs includes: The visual language model is used to score the consistency of unmodified regions, image quality, and physical consistency in the image sequence pair, and then the average value is calculated to obtain the first filtering score. When the first filtering score is greater than or equal to a preset score threshold, the image sequence pair is input into multiple discriminator models to obtain the discrimination score output by each discriminator model, and the average of all the discrimination scores is calculated to obtain the second fine screening score. The joint score is obtained based on the first filtering score and the second fine screening score.

[0010] In some embodiments, obtaining the obstacle detection model based on the trained initial detection model includes: An updated detection model is obtained based on the trained initial detection model, and the importance score of each convolutional kernel in the updated detection model to the model's predicted output is determined. The convolutional kernels in the updated detection model whose importance scores are lower than the preset retention scores are removed to obtain the pruned model; The floating-point model weights in the pruning model are converted into precision integer model weights to obtain the obstacle detection model.

[0011] To achieve the above objectives, a second aspect of this application provides an obstacle avoidance detection method, the method comprising: Acquire the image data of the target detection scene; The image data to be tested is input into the obstacle detection model obtained by the obstacle detection model training method as described in the first aspect for feature extraction, so as to obtain the detection result characterizing the existence state and position of the obstacle in the image data to be tested.

[0012] To achieve the above objectives, a third aspect of this application provides an obstacle detection model training device, the device comprising: The acquisition module is used to acquire captured video data and generated video data based on a video expert model, wherein some of the captured video data contains real obstacles and the generated video data contains virtual obstacles. The feature extraction module is used to input the acquired video data into the initial detection model to extract obstacle features, thereby obtaining the acquired obstacle features, and to input the generated video data into the initial detection model to extract obstacle features, thereby obtaining the generated obstacle features; The distribution error determination module is used to obtain the distribution alignment error based on the statistical difference between the acquired obstacle features and the generated obstacle features; The overall error determination module is used to obtain the classification error and detection error generated by the initial detection model when extracting obstacle features for the acquired video data and the generated video data, respectively, and to obtain the overall target error based on the distribution alignment error, the classification error and the detection error; The model training module is used to train the initial detection model based on the overall target error, and to obtain the obstacle detection model based on the trained initial detection model.

[0013] To achieve the above objectives, a fourth aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the obstacle detection model training method as described in the first aspect or the obstacle avoidance detection method as described in the second aspect.

[0014] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the obstacle detection model training method described in the first aspect or the obstacle avoidance detection method described in the second aspect.

[0015] The obstacle detection model training method, obstacle avoidance detection method, and related equipment proposed in this application include: First, acquiring captured video data and generated video data based on a video expert model, wherein some captured video data contains real obstacles and generated video data contains virtual obstacles; then, inputting the captured video data into an initial detection model for obstacle feature extraction to obtain captured obstacle features, and inputting the generated video data into the initial detection model for obstacle feature extraction to obtain generated obstacle features; next, obtaining a distribution alignment error based on the statistical difference between the captured obstacle features and the generated obstacle features; then, acquiring the classification error and detection error generated by the initial detection model when extracting obstacle features from the captured video data and the generated video data respectively, and obtaining an overall target error based on the distribution alignment error, classification error, and detection error; finally, training the initial detection model based on the overall target error, and obtaining an obstacle detection model based on the trained initial detection model. This application's embodiments introduce a large number of diverse training samples at a low cost by acquiring captured video data containing real obstacles and generated video data containing virtual obstacles based on a video expert model. This effectively alleviates the problems of high cost, limited acquisition conditions, and single obstacle type in traditional manual on-site acquisition methods. Furthermore, during model training, obstacle features are extracted by inputting captured video data and generated video data into the initial detection model, respectively. A distribution alignment error is introduced based on the statistical difference between these two types of features. This distribution alignment error is combined with classification error and detection error to construct an overall target error for joint network training. This allows the model to align and narrow the differences in feature space distribution between generated video features and naturally captured video features while learning target classification and location detection. As a result, the obstacle detection model trained in this way can improve the detection capability of diverse data scene types and enhance the model's generalization ability in real environments, thereby meeting the high-precision obstacle recognition and avoidance requirements in large-scale, complex, and variable scenarios.

[0016] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0017] Figure 1 This is a flowchart of an obstacle detection model training method provided in an embodiment of this application.

[0018] Figure 2 yes Figure 1 The flowchart for step 101.

[0019] Figure 3 yes Figure 2 The flowchart for step 202.

[0020] Figure 4 yes Figure 2 The flowchart for step 204.

[0021] Figure 5 yes Figure 1 The flowchart for step 103.

[0022] Figure 6 This is a flowchart of the pruning process for an updated detection model provided in one embodiment of this application.

[0023] Figure 7 This is a flowchart of an obstacle avoidance detection method provided in an embodiment of this application.

[0024] Figure 8 This is a schematic diagram of a detection model training and application process provided in one embodiment of this application.

[0025] Figure 9 This is a schematic diagram of the structure of an obstacle detection model training device provided in an embodiment of this application.

[0026] Figure 10 This is a schematic diagram of the hardware structure of a system provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0030] Obstacle detection, a core research area in computer vision, plays a crucial role in scenarios such as autonomous driving in automobiles and aircraft, and intelligent monitoring in security. To ensure the safety and reliability of the system in unstructured and complex environments, deep learning-based detection models require training with large-scale, high-quality sample data to extract accurate obstacle feature information. In related technologies, the production of training data primarily relies on on-site collection by professionals in different typical scenarios, or on secondary processing of the collected data using image processing tools.

[0031] However, data production based on manual on-site collection faces numerous constraints. Firstly, the uncontrollability of environmental conditions makes the data collection process highly susceptible to natural factors such as geographical location, weather, lighting, and obstructions. For example, in the field of aviation obstacle avoidance, specialized personnel must be dispatched to different airports to photograph specific obstacles, resulting in a lengthy overall production cycle and high costs. Secondly, when detection algorithms need to adapt to new hardware platforms (such as replacing camera modules with different specifications), it is usually necessary to restart a large-scale data collection and annotation process, which significantly extends the algorithm's development and delivery cycle.

[0032] Furthermore, while existing technologies attempt to use image processing tools (such as Photoshop) to simulate filters or overlay obstacles on the collected obstacle-free data, aiming to enhance sample diversity through human intervention, these methods still require deep involvement from relatively specialized technical personnel and are difficult to use in batches to automatically produce video samples with high physical consistency and natural motion characteristics. Limited by on-site data collection conditions and manual processing efficiency, the resulting datasets often exhibit relatively homogeneous obstacle types, and there are significant differences in feature distribution between the generated data and the real data.

[0033] The aforementioned defects collectively result in limited generalization ability of the trained detection model in practical applications, making it difficult to achieve accurate identification and reliable obstacle avoidance when faced with large-scale, unstructured, or sudden complex obstacle avoidance scenarios.

[0034] Based on this, the embodiments of this application introduce a large number of diverse training samples at a low cost by acquiring video data containing real obstacles and generated video data containing virtual obstacles based on video expert models. This effectively alleviates the problems of high cost, limited acquisition conditions, and single obstacle type in traditional manual on-site acquisition methods. In addition, during model training, obstacle features are extracted by inputting the acquired video data and generated video data into the initial detection model respectively. A distribution alignment error is introduced based on the statistical difference between the two features. This distribution alignment error is combined with the classification error and detection error to construct an overall target error for joint network training. This allows the model to align and narrow the differences in feature space distribution between generated video features and naturally acquired video features while learning target classification and location detection. As a result, the obstacle detection model trained in this way can improve the detection capability of diverse data scene types and enhance the model's generalization ability in real environments. This can meet the high-precision obstacle recognition and avoidance requirements in large-scale, complex and variable scenarios.

[0035] The obstacle detection model training method, obstacle avoidance detection method, and related equipment provided in this application will be further described below. (Refer to...) Figure 1 This is an optional flowchart of the obstacle detection model training method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps 101 to 105. It is also understood that this embodiment... Figure 1 The order of steps 101 to 105 is not specifically limited; the order of steps can be adjusted or certain steps can be added or removed according to actual needs. The method provided in this application can be applied to any server or intelligent processing terminal, etc.

[0036] Step 101: Acquire the captured video data and the generated video data based on the video expert model. Some of the captured video data contains real obstacles, while the generated video data contains virtual obstacles.

[0037] Step 101 will be described in detail below.

[0038] In step 101, to properly train the initial detection model, the data source required for training in this application includes two parts: one part is the video data collected manually on-site, some of which contains obstacles in real-world scenes; the other part is the generated video data containing virtual obstacles, automatically produced by a pre-trained video expert model by inputting obstacle-free videos, obstacle images, and addition instructions. Compared to traditional acquisition methods that require collecting tens of thousands of videos, this application only requires collecting about a few hundred videos, with lower requirements for the acquisition scene and the level of expertise required from the acquisition personnel. Thus, by combining virtual generated data with real acquired data, a large-scale and diverse sample foundation is provided for the subsequent training of the initial detection model. The following will further describe how to obtain the generated video data.

[0039] Reference Figure 2 The process of obtaining generated video data based on a video expert model includes the following steps 201 to 205.

[0040] Step 201: Obtain obstacle-free video data, obstacle images, and obstacle addition instructions.

[0041] Step 202: Input the obstacle-free video data, obstacle images, and obstacle addition instructions into the video expert model for data processing to obtain candidate obstacle video data.

[0042] Steps 201 to 202 are described in detail below.

[0043] In step 201, to obtain the generated video data using a video expert model, it is first necessary to acquire obstacle-free video data and obstacle images. The obstacle-free video data can be simple scene videos collected manually in advance (e.g., open-air scene videos taken by drones at different airports) or obtained from various open-source online sources. The obstacle images can be images of vehicles, roadblocks, etc., from open-source data. To enrich the generation conditions, corresponding obstacle addition instructions also need to be obtained. Considering that some open-source category information may be incomplete, this application also uses a visual language model (such as ChatGPT) to supplement detailed information such as obstacle color and vehicle type, thereby generating complete obstacle addition instructions.

[0044] In step 202, to achieve large-scale data augmentation, multiple (e.g., 5 to 20) obstacle images and their corresponding obstacle addition instructions can be randomly paired for each acquired obstacle-free video. These combinations are then input into a pre-trained video expert model for inference generation, thereby batch-outputting corresponding candidate video data with added virtual obstacles. This forms multiple sets of corresponding video pairs. The video expert model's structure includes four modules: the first is a variational autoencoder (VAE codec), the second is a multimodal feature encoding network (VLM multimodal feature encoder), the third is a diffusion generation network (DiT diffusion generation model), and the fourth is a discriminator module. The following section will further describe how to use the video expert model for video data generation.

[0045] Reference Figure 3 The obstacle-free video data, obstacle images, and obstacle addition instructions are input into the video expert model for data processing to obtain candidate obstacle video data, including the following steps 301 to 304.

[0046] Step 301: Use a variational autoencoder to encode the features of the obstacle-free video data to obtain the first latent space video features.

[0047] Step 302: Encode the obstacle image and obstacle addition instructions using a multimodal feature encoding network to obtain a multimodal feature vector.

[0048] Step 303: Input the first latent space video features and multimodal feature vectors into the diffusion generation network to predict the diffusion features and obtain the predicted obstacle features.

[0049] Step 304: Perform feature decoding on the predicted obstacle features using a variational autoencoder to obtain candidate obstacle video data.

[0050] Steps 301 to 304 are described in detail below.

[0051] In step 301, the variational autoencoder (VAE codec) in the video expert model is used as the basic feature processing module. The acquired obstacle-free video is input into the variational autoencoder to transform and compress it from the high-dimensional pixel space to the low-dimensional latent space through encoding operations. The encoded first latent space video features are output. The first latent space video features are used as the basic noise floor or conditional input of the subsequent generative network. The part with obstacles is used as the fitting target of the diffusion generative network (i.e., DiT model), and the part without obstacles is used as the input data of the diffusion generative network (i.e., DiT model).

[0052] In step 302, a multimodal feature encoder, such as a visual language model (VLM), is used to jointly extract and spatially map the obstacle reference image of the visual modality (e.g., an image of a red A brand car) and the descriptive instruction of the text modality (e.g., "Add a red A brand a model car to the video") to output a multimodal feature vector that integrates visual and textual intent.

[0053] In step 303, the diffusion generation network (i.e., the DiT diffusion generation model) takes the obtained latent space features of the obstacle-free video and the multimodal feature vector as joint inputs. Under the conditional constraints and guidance of the multimodal feature vector, it performs feature prediction and denoising in the latent space to output the predicted obstacle features with obstacles after fusing obstacle information.

[0054] In step 304, the predicted obstacle features output by the diffusion generation network are transmitted back to the decoder part of the variational autoencoder (VAE model) so that the latent space features with added obstacle information can be remapped back to the video pixel space through the decoding operation, thereby outputting candidate video data with added virtual obstacles.

[0055] In addition, the input to the discriminator module is the video data decoded after being generated by the DiT model, as well as some additional auxiliary data (such as the original obstacle-free video, obstacle reference map, etc.). Different discriminators have different combinations of auxiliary data.

[0056] In addition, the loss function in the pre-training process of the video expert model includes three parts: diffusion loss function, latent space regularization loss, generative perception loss, and GAN loss (composed of multiple discriminators), as described below.

[0057] The diffusion loss function is used to train the model's ability to fit the target distribution during the diffusion generation network's diffusion processing, giving it a basic ability to add obstacles to the video (where p is the DiT prediction and y is the predicted target). The diffusion loss function is shown in the following formula.

[0058]

[0059] The latent space regularization loss is used to further enhance the ability to add obstacles in the latent space. Specifically, it is a contrastive loss that makes the conditional prediction (the output of the DiT diffusion generation model when the input of the VLM multimodal feature encoder is an obstacle image and the corresponding instruction) and the unconditional prediction (the output of the DiT diffusion generation model when the input of the VLM multimodal feature encoder is empty) as far apart as possible, thereby strengthening the guidance ability of the conditions and avoiding the failure of the model to add obstacles. The latent space regularization loss is shown in the following formula.

[0060]

[0061] The generation perception loss is applied to the decoding output space, including quality perception losses such as LPIPS and SSIM, to improve the overall generation quality. The generation perception loss is shown in the following formula.

[0062]

[0063] The GAN loss consists of three different discriminator losses, trained using the GAN Loss formula. The first discriminator is a consistency discriminator, which takes the video data decoded from the DiT model and the corresponding obstacle-free video data as input. It is used to judge the consistency of the video before and after generation, and the output is the classification loss. The second discriminator is an obstacle consistency discriminator, which takes the video data decoded from the DiT model and the corresponding obstacle image data as input, and the output is the classification loss. The third is an optical flow naturalness discriminator, which judges the naturalness of the video by taking the optical flow information of the generated video as input, and the output is the classification loss. The overall GAN ​​Loss consists of the losses of the three discriminators as shown in the following formula.

[0064]

[0065] Through steps 301 to 304 above, a video expert model is constructed that includes a variational autoencoder, a multimodal feature encoding network, and a diffusion generative network. This transfers the complex video generation process to the latent space, reducing the computational resource overhead of long-term video generation. Furthermore, through deep fusion of multimodal features, precise control over the appearance attributes of obstacles and the intention to add them is achieved during the generation process. This makes the generated candidate video data more reliable for subsequent initial retrieval model training.

[0066] Step 203: Perform image frame alignment extraction on the candidate obstacle video data and the obstacle-free video data to obtain image sequence pairs.

[0067] Step 204: Use the visual language model and discriminator model to calculate the score of the image sequence pairs and obtain the joint score.

[0068] Steps 203 to 204 are described in detail below.

[0069] In step 203, in order to facilitate the subsequent refined evaluation of the quality of the generated video, the present application adopts a strategy that combines uniform frame extraction and I-frame extraction to perform synchronous frame extraction processing on the generated candidate obstacle-containing video and the original obstacle-free video to obtain image sequence pairs (i.e., multiple image pairs), and finally produce two final filtering scores, and calculate the average.

[0070] Since common filtering methods that rely solely on Visual Language Models (VLMs) often result in limited consistency between the generated video and the original video, a multi-stage joint scoring mechanism is adopted in step 204: First, the Visual Language Model is used to score the consistency of the unmodified regions, image quality, and physical consistency in the image sequence pairs for initial screening (e.g., scoring each within the range of 1-5 points); for the data that passes the initial screening, multiple discriminator models are used for further fine screening, and the corresponding joint score is finally calculated by combining the above processes, as described below.

[0071] Reference Figure 4 The image sequence pairs are scored using a visual language model and a discriminator model to obtain a joint score, including the following steps 401 to 403.

[0072] Step 401: Use a visual language model to calculate scores for consistency of unmodified regions, image quality, and physical consistency in image sequence pairs, and then calculate the mean to obtain the first filtering score.

[0073] Step 402: When the first filtering score is greater than or equal to the preset score threshold, the image sequence pair is input into multiple discriminator models to obtain the discrimination score output by each discriminator model, and the average of all discrimination scores is calculated to obtain the second fine screening score.

[0074] Step 403: Obtain the joint score based on the first filtering score and the second fine screening score.

[0075] Steps 401 to 403 are described in detail below.

[0076] In step 401, considering that data directly generated by conventional methods often has poor consistency with the original video, this application's solution utilizes a Visual Language Model (VLM) to perform multi-dimensional objective scoring on the image pairs obtained from frame extraction. Specifically, it scores the consistency of the unmodified area before and after generation, the overall image clarity and quality, and the consistency of the generated obstacles with the physical laws of the original scene environment (e.g., using an evaluation scale of 1 to 5 points) to obtain the corresponding score. Then, the scores from these three dimensions are averaged to obtain the first filtering score, which represents the initial filtering quality. .

[0077] In step 402, to compensate for the insufficient filtering capability of a single visual language model in terms of video temporal consistency, the proposed solution judges the calculated first filtering score. When the score reaches a preset passing score (e.g., greater than 3 points, i.e., S>3), the corresponding image pair is determined to have passed the initial screening. Then, the image sequence pairs that pass the passing score are input into multiple trained dedicated discriminators (mainly including a consistency discriminator, an obstacle consistency discriminator, and an optical flow naturalness discriminator) for secondary judgment. Each discriminator scores the temporal changes and the naturalness of optical flow motion of the data (e.g., each discriminator outputs a discrimination score of 0 to 1). The average or summation of the outputs of all discriminators is then performed to obtain the second fine screening score used to control the generation quality. .

[0078] In step 403, after obtaining the scores for the initial screening and the fine screening, the first filtering score, which focuses on the static semantics and the physical logic of the image, will be output based on the visual language model. Compared with the second fine screening score based on the output of multiple discriminator models, which focuses on dynamic temporal consistency and motion naturalness, A comprehensive calculation (such as averaging) is performed to obtain a joint score that can characterize the quality of candidate videos.

[0079] Through steps 401 to 403 above, the dual evaluation filtering combining the visual language model and multiple discriminators overcomes the flaws that are prone to occur in the traditional data augmentation generation process, such as the disconnect between the foreground and background images and the violation of physical motion. This ensures that the final output virtual sample has a high degree of realism and scene fit, thereby improving the training accuracy and reliability of the subsequent initial detection model training.

[0080] Step 205: When the joint score is greater than the preset score threshold, the generated video data is obtained based on the candidate obstacle video data.

[0081] Step 205 will be described in detail below.

[0082] In step 205, after obtaining the joint score of each candidate obstacle video data, it is compared with a preset score threshold (e.g., 1.8 points), and the candidate obstacle video data with a joint score higher than the preset score threshold are retained to obtain generated video data that can be used for subsequent model training.

[0083] Through steps 201 to 205 above, by combining multimodal instruction completion with video expert models, large-scale automated data generation and augmentation based on a small number of obstacle-free videos is achieved, effectively improving data diversity; and through a multi-level filtering mechanism, the generated video data finally used for model training is ensured to have high realism and physical consistency, thereby improving the training accuracy and reliability of the subsequent initial detection model training.

[0084] Step 102: Input the acquired video data into the initial detection model to extract obstacle features, and input the generated video data into the initial detection model to extract obstacle features, and obtain the generated obstacle features.

[0085] Step 102 is described in detail below.

[0086] In step 102, the proposed solution employs an efficient detection CNN architecture as the initial detection model, such as the Faster R-CNN architecture. In this solution, the acquired video data and generated video data are input into the initial detection model, and feature extraction is performed through the convolutional neural network (CNN) layers within the model to obtain the acquired obstacle features and generated obstacle features corresponding to the naturally acquired samples and artificially generated samples, respectively.

[0087] In this application, the input of the initial detection model is an image, and the output consists of two parts: 1. Whether there is an obstacle; 2. If there is an obstacle, the bbox position of the obstacle.

[0088] Step 103: Based on the statistical difference between the collected obstacle features and the generated obstacle features, obtain the distribution alignment error.

[0089] Step 103 will be described in detail below.

[0090] In step 103, in order to address the potential distribution differences between the virtual generated data and the natural real data, the present application introduces a distribution alignment loss. Specifically, by calculating the mean square error (MSE) of the mean and variance of the generated obstacle features and the naturally collected obstacle features in the feature space, a distribution alignment error is obtained to measure the similarity of the distribution features of the two types of data, as described below.

[0091] Reference Figure 5 Based on the statistical difference between the collected obstacle features and the generated obstacle features, the distribution alignment error is obtained, including the following steps 501 to 504.

[0092] Step 501: Calculate the first feature mean and first feature standard deviation of the collected obstacle features, and the second feature mean and second feature standard deviation of the generated obstacle features.

[0093] Step 502: Based on the square of the difference between the first feature mean and the second feature mean, obtain the mean alignment error term.

[0094] Step 503: Based on the square of the difference between the first feature standard deviation and the second feature standard deviation, obtain the standard deviation alignment error term.

[0095] Step 504: Summate the mean alignment error term and the standard deviation alignment error term to obtain the distribution alignment error.

[0096] Steps 501 to 504 are described in detail below.

[0097] In step 501, based on the last layer of features extracted by the initial detection model, statistical parameters are extracted from the naturally acquired obstacle video features and the obstacle video features generated using a multimodal video expert model, respectively. This is achieved by calculating the mean of the spatial distribution of these two types of input features (including the mean of the first feature). Second characteristic mean ) and standard deviation (i.e., including the first characteristic standard deviation) Second characteristic standard deviation ).

[0098] In step 502, mean square error (MSE) is calculated for the distribution center of the features. The mean alignment error term is obtained by taking the square of the difference between the first feature mean of the naturally acquired video features and the second feature mean of the generated video features, as shown in the following formula.

[0099] .

[0100] In step 503, the dispersion of the features is further calculated. The standard deviation alignment error term is obtained by taking the square of the difference between the first feature standard deviation of the naturally acquired video features and the second feature standard deviation of the generated video features, as shown in the following formula.

[0101]

[0102] In step 504, the mean alignment error term and the standard deviation alignment error term are summed to obtain the final distribution alignment error. As shown in the formula below.

[0103]

[0104] This distribution alignment error is then incorporated into the overall loss function along with the classification loss and detection loss, guiding the iterative optimization of the entire detection model to make the feature spaces of the generated video and the naturally captured video as close as possible, and to align them as much as possible.

[0105] Through steps 501 to 504 above, by calculating the mean and standard deviation of the generated video features and the naturally acquired video features respectively, constraints are imposed during the model training phase to make the generated video feature space as close as possible to the naturally acquired video feature space. This bridges the feature gap between virtual synthetic data and real acquired data, ensuring that the detection model can fully utilize massive generated data to expand the sample equivalent without deviating from the real physical application distribution.

[0106] Step 104: Obtain the classification error and detection error generated by the initial detection model when extracting obstacle features from the acquired video data and the generated video data respectively, and obtain the overall target error based on the distribution alignment error, classification error and detection error.

[0107] Step 104 is described in detail below.

[0108] In step 104, during the training process, the initial detection model not only needs to learn how to align feature distributions, but also needs to complete basic classification and localization tasks. Therefore, in this application's scheme, the classification error between the model output and the true label is also obtained. and detection error And these two errors are compared with the obtained distribution alignment error. By performing weighted or summative processing, the overall target error is constructed as shown in the following formula.

[0109]

[0110] Step 105: Train the initial detection model based on the overall target error, and obtain the obstacle detection model based on the trained initial detection model.

[0111] Step 105 is described in detail below.

[0112] In step 105, optimizers such as AdamW are used on multi-GPU servers (e.g., GPU servers with 8 A100 GPUs) based on the overall target error. The model parameters of the initial detection model are iteratively updated to minimize the overall target error. This allows the initial detection model to improve detection accuracy while enabling the feature space of the generated video to converge towards the feature space of the natural video, until the loss function no longer decreases and tends to converge, thus obtaining an obstacle detection model with strong generalization performance.

[0113] In addition, in order to meet the requirements of real-world application scenarios regarding model size and running speed, the proposed solution also performs lightweight operations such as pruning and quantization on the trained obstacle detection algorithm, as described below.

[0114] Reference Figure 6 Obtaining an obstacle detection model based on the trained initial detection model includes the following steps 601 to 603.

[0115] Step 601: Obtain an updated detection model based on the trained initial detection model, and determine the importance score of each convolutional kernel in the updated detection model to the model's predicted output.

[0116] Step 602: Remove convolutional kernels in the updated detection model whose importance scores are lower than the preset retention scores to obtain the pruned model.

[0117] Step 603: Convert the floating-point model weights in the pruning model to precision integer model weights to obtain the obstacle detection model.

[0118] Steps 601 to 603 are described in detail below.

[0119] In step 601, the proposed solution uses the trained initial detection model as the updated detection model, and then employs a structured pruning method based on sensitivity analysis to evaluate each network layer of the updated detection model. This quantitatively evaluates the influence of each convolutional kernel or channel on the final model's predicted output, thereby determining the importance score corresponding to its sensitivity.

[0120] In step 602, after obtaining the importance score of each convolutional kernel in the updated training detection, the present application scheme removes convolutional kernels or channels that have little impact on the model output and whose importance score is lower than the set retention threshold in order of scores from low to high, until the size of the updated detection model reaches the expected target (such as a reduction of about 40%), thereby outputting a more compact pruned model.

[0121] In step 603, based on structured pruning, the present application further employs post-training quantization (such as post-training quantization) technology to compress the model, so as to convert the original floating-point weights in the pruned model into integer weights with target precision such as 8 bits (INT8) by minimizing the quantization error, thereby obtaining an obstacle detection model that can be used for edge hardware deployment.

[0122] In this application, training simulations were conducted to verify the reliability of the pruning and compression operation. Experimental results show that after pruning, the number of model parameters is reduced by approximately 40%, while the detection accuracy decreases by less than 1%. Regarding quantization, this application employs post-training quantization technology, converting floating-point weights into 8-bit integers to further compress the model size and improve inference speed. This minimizes quantization error to ensure that model performance is not significantly affected. Experimental results show that the quantized model improves inference speed by approximately 30% while reducing model size by nearly 75%.

[0123] Furthermore, when deploying the lightweight obstacle detection algorithm to different application scenarios, the limitations of the hardware and software environments must be fully considered. Taking autonomous vehicles as an example, due to their high requirements for real-time performance and reliability, this application's solution selects a high-performance GPU as the inference acceleration hardware and optimizes the data flow processing logic of model input and output to reduce computational latency. In addition, for the application scenario of intelligent monitoring equipment, considering its relatively limited hardware resources, this application's solution uses a low-power CPU as the inference platform and further reduces computational overhead by optimizing the model's memory access mode. At the software level, this paper develops a unified interface layer to support multiple operating systems and inference frameworks (such as TensorRT and OpenVINO), thereby improving the algorithm's compatibility and scalability. Experimental results show that the adapted algorithm exhibits excellent performance in both autonomous vehicles and intelligent monitoring equipment, meeting the needs of practical applications.

[0124] Through steps 601 to 603 above, by introducing a lightweight processing mechanism that combines structured pruning based on sensitivity analysis with post-training quantization, the number of model parameters and size are reduced while improving the model's inference speed, ensuring that the impact on model detection performance is negligible. This enables the high-performance obstacle detection algorithm to run stably on the target application platform with lower latency and memory overhead.

[0125] Through steps 101 to 105 above, by introducing distribution alignment error, the statistical distribution of virtual generated data and naturally collected data in the feature space is narrowed, eliminating the feature gap between AIGC generated data and real physical scene data. This enables the detection model to fully utilize the diversity advantages brought by low-cost, large-scale generated data, while improving its accurate recognition ability and generalization performance in real application scenarios. This solves the problem of limited recognition performance caused by insufficient training data or virtual-real differences when facing new scenarios or new hardware platforms.

[0126] Reference Figure 7 This is an optional flowchart of the obstacle avoidance detection method provided in the embodiments of this application. Figure 7 The method may include, but is not limited to, steps 701 to 702. It is also understood that this embodiment... Figure 7 The order of steps 701 to 702 is not specifically limited; the order of steps can be adjusted or certain steps can be added or removed according to actual needs. The method provided in this application can be applied to any server or intelligent processing terminal, etc.

[0127] Step 701: Obtain the image data to be tested in the target detection scene.

[0128] Step 702: Input the image data to be tested into the obstacle detection model obtained by the obstacle detection model training method for feature extraction, and obtain the detection results that characterize the existence state and position of obstacles in the image data to be tested.

[0129] Steps 701 to 702 are described in detail below.

[0130] In step 701, the terminal (such as an autonomous vehicle, an airplane, or an intelligent security monitoring device) uses its onboard visual sensors (such as a camera module) to collect real-time image data of the current environment during actual operation, so as to reflect the real scene information on the terminal's driving or monitoring path.

[0131] In step 702, the collected image data to be tested is fed into a pre-trained and lightweight obstacle detection model to perform deep feature extraction on the image through a convolutional neural network, thereby outputting the result of whether an obstacle exists, and the specific bounding box (BBox) position coordinates when an obstacle exists, so as to provide the terminal with environmental perception information and thus better execute the corresponding obstacle avoidance decision.

[0132] Through steps 701 to 702 above, by applying a lightweight model based on distribution alignment training and after pruning and quantization, the inference latency and computational overhead of the algorithm on the target terminal are significantly reduced while ensuring detection accuracy. This also improves the accuracy of obstacle recognition in various complex and unstructured environments, thereby ensuring the safety and reliability of autonomous driving and intelligent monitoring systems in practical applications.

[0133] Reference Figure 8 This is a schematic diagram illustrating a detection model training and application process provided in an embodiment of this application. Figure 8As shown, the process begins with "Data Acquisition and Generation," where initial environmental videos and basic image pairs are manually collected to provide training start-up data for the system. Then, in the "Cross-Modal Multi-View Image Generation" stage, high-quality video or multi-view image data containing virtual targets (such as obstacles) is automatically and on a large scale by utilizing a multi-modal video expert model combined with text instructions and visual references. Next, in the "Multi-View Image Understanding and Damage Estimation" stage, a high-performance detection algorithm trained with high-quality, virtual-real aligned data is used to extract deep features and assess the target state of the image under test (in different specific business scenarios, this image understanding task can be specifically manifested as the aforementioned obstacle detection, or extended to downstream applications such as target damage estimation). Finally, in the "Model Deployment and Application" stage, the target model, after lightweight processing such as structured pruning and quantization, is adapted and deployed to corresponding hardware terminals (such as autonomous vehicles or intelligent security drones) to support real-time reasoning and action decision-making in real-world environments.

[0134] This application embodiment also provides an obstacle detection model training device, which can implement the above-described obstacle detection model training method, referring to... Figure 9 The device 900 includes: The acquisition module 910 is used to acquire captured video data and generated video data based on a video expert model. Some of the captured video data contains real obstacles, while the generated video data contains virtual obstacles. The feature extraction module 920 is used to input the acquired video data into the initial detection model to extract obstacle features, thereby obtaining the acquired obstacle features, and to input the generated video data into the initial detection model to extract obstacle features, thereby obtaining the generated obstacle features. The distribution error determination module 930 is used to obtain the distribution alignment error based on the statistical difference between the collected obstacle features and the generated obstacle features; The overall error determination module 940 is used to obtain the classification error and detection error generated by the initial detection model for obstacle feature extraction on the acquired video data and the generated video data respectively, and to obtain the overall target error based on the distribution alignment error, classification error and detection error; The model training module 950 is used to train the initial detection model based on the overall target error, and to obtain the obstacle detection model based on the trained initial detection model.

[0135] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, the specific implementation of the obstacle detection model training device is basically the same as the specific implementation of the obstacle detection model training method described above, and will not be repeated here.

[0136] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the target detection method described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0137] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1002 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the target detection method of the embodiments of this application. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0138] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described target detection method.

[0139] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0140] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0141] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0144] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0145] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0147] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0149] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for training an obstacle detection model, characterized in that, The method includes: Acquire captured video data and acquired generated video data based on a video expert model, wherein a portion of the captured video data contains real obstacles and the generated video data contains virtual obstacles; The acquired video data is input into the initial detection model to extract obstacle features, thereby obtaining the acquired obstacle features. The generated video data is then input into the initial detection model to extract obstacle features, thereby obtaining the generated obstacle features. Based on the statistical difference between the collected obstacle features and the generated obstacle features, the distribution alignment error is obtained; The classification error and detection error generated by the initial detection model when extracting obstacle features from the acquired video data and the generated video data are obtained respectively, and the overall target error is obtained based on the distribution alignment error, the classification error and the detection error; The initial detection model is trained based on the overall target error, and an obstacle detection model is obtained based on the trained initial detection model.

2. The obstacle detection model training method according to claim 1, characterized in that, The distribution alignment error is obtained based on the statistical difference between the acquired obstacle features and the generated obstacle features, including: Calculate the first feature mean and first feature standard deviation of the collected obstacle features, and the second feature mean and second feature standard deviation of the generated obstacle features, respectively; The mean alignment error term is obtained based on the square of the difference between the mean of the first feature and the mean of the second feature; The standard deviation alignment error term is obtained based on the square of the difference between the first feature standard deviation and the second feature standard deviation. The distribution alignment error is obtained by summing the mean alignment error term and the standard deviation alignment error term.

3. The obstacle detection model training method according to claim 1, characterized in that, The acquisition of generated video data based on the video expert model includes: Acquire obstacle-free video data, obstacle images, and obstacle addition instructions; The obstacle-free video data, the obstacle image, and the obstacle addition instruction are input into the video expert model for data processing to obtain candidate obstacle video data. Image frame alignment extraction is performed on the candidate obstacle video data and the obstacle-free video data to obtain image sequence pairs; The image sequence pairs are scored using a visual language model and a discriminator model to obtain a joint score. When the joint score is greater than a preset score threshold, the generated video data is obtained based on the candidate obstacle video data.

4. The obstacle detection model training method according to claim 3, characterized in that, The video expert model includes a variational autoencoder, a multimodal feature encoding network, and a diffusion generation network. The process of inputting the obstacle-free video data, the obstacle image, and the obstacle addition instructions into the video expert model for data processing to obtain candidate obstacle video data includes: The first latent space video features are obtained by performing feature encoding on the obstacle-free video data using the variational autoencoder. The obstacle image and the obstacle addition instruction are feature-encoded using the multimodal feature coding network to obtain a multimodal feature vector; The first latent space video features and the multimodal feature vector are input into the diffusion generation network to predict diffusion features and obtain predicted obstacle features. The predicted obstacle features are decoded by the variational autoencoder to obtain the candidate obstacle video data.

5. The obstacle detection model training method according to claim 3, characterized in that, The step of using a visual language model and a discriminator model to score the image sequence pairs and obtain a joint score includes: The visual language model is used to score the consistency of unmodified regions, image quality, and physical consistency in the image sequence pair, and then the average value is calculated to obtain the first filtering score. When the first filtering score is greater than or equal to a preset score threshold, the image sequence pair is input into multiple discriminator models to obtain the discrimination score output by each discriminator model, and the average of all the discrimination scores is calculated to obtain the second fine screening score. The joint score is obtained based on the first filtering score and the second fine screening score.

6. The obstacle detection model training method according to claim 1, characterized in that, The step of obtaining the obstacle detection model based on the trained initial detection model includes: An updated detection model is obtained based on the trained initial detection model, and the importance score of each convolutional kernel in the updated detection model to the model's predicted output is determined. The convolutional kernels in the updated detection model whose importance scores are lower than the preset retention scores are removed to obtain the pruned model; The floating-point model weights in the pruning model are converted into precision integer model weights to obtain the obstacle detection model.

7. An obstacle avoidance detection method, characterized in that, The method includes: Acquire the image data of the target detection scene; The image data to be tested is input into the obstacle detection model obtained by the obstacle detection model training method according to any one of claims 1 to 6 for feature extraction, so as to obtain the detection result characterizing the existence state and position of the obstacle in the image data to be tested.

8. An obstacle detection model training device, characterized in that, The device includes: The acquisition module is used to acquire captured video data and generated video data based on a video expert model, wherein some of the captured video data contains real obstacles and the generated video data contains virtual obstacles. The feature extraction module is used to input the acquired video data into the initial detection model to extract obstacle features, thereby obtaining the acquired obstacle features, and to input the generated video data into the initial detection model to extract obstacle features, thereby obtaining the generated obstacle features. The distribution error determination module is used to obtain the distribution alignment error based on the statistical difference between the acquired obstacle features and the generated obstacle features; The overall error determination module is used to obtain the classification error and detection error generated by the initial detection model when extracting obstacle features for the acquired video data and the generated video data, respectively, and to obtain the overall target error based on the distribution alignment error, the classification error and the detection error; The model training module is used to train the initial detection model based on the overall target error, and to obtain the obstacle detection model based on the trained initial detection model.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the obstacle detection model training method according to any one of claims 1 to 6 or the obstacle avoidance detection method according to claim 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the obstacle detection model training method according to any one of claims 1 to 6 or the obstacle avoidance detection method according to claim 7.

Images