Obstacle detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an obstacle detection method and device, electronic equipment and a storage medium, and belongs to the technical field of sweeping robots. The method comprises the following steps: acquiring an initial obstacle detection model in a normal illumination environment; acquiring a plurality of dark light images in the dark light environment through the camera; multiple rounds of screening are carried out according to the dark light images and the initial obstacle detection model so as to acquire a plurality of dark light obstacle training sets, and each dark light obstacle training set comprises a plurality of target pseudo labels; performing multiple rounds of training on the initial obstacle detection model according to each dark light obstacle training set to obtain a target obstacle detection model; and detecting the dark light environment through the target obstacle detection model, and determining an obstacle in the dark light environment. According to the scheme provided by the invention, the recognition accuracy of the target obstacle detection model on the obstacle in the dark light environment can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of sweeping robots, and in particular to an obstacle detection method, device, electronic device and storage medium. Background Art

[0002] With the popularity of sweeping robots in home environments, the importance of obstacle detection functions of sweeping robots has become increasingly prominent. Traditional obstacle detection methods usually rely on large-scale annotated data sets for model training. However, most of these annotated data sets are collected under standard lighting conditions. When sweeping robots operate in low-light environments, their detection performance will drop significantly. In order to achieve accurate obstacle detection under low-light conditions, it is usually necessary to collect a large amount of dark light environment data and manually annotate it, which requires high labor costs and poor recognition accuracy. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present application provide an obstacle detection method, device, electronic device and storage medium.

[0004] In a first aspect, an embodiment of the present application provides an obstacle detection method, the method comprising:

[0005] Obtain the initial obstacle detection model under normal lighting environment;

[0006] Collect multiple dark-light images in a dark-light environment through a camera;

[0007] Performing multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets including multiple target pseudo labels;

[0008] Performing multiple rounds of training on the initial obstacle detection model according to each of the dark light obstacle training sets to obtain a target obstacle detection model;

[0009] The dark-light environment is detected by using the target obstacle detection model to determine obstacles in the dark-light environment.

[0010] In one embodiment, a classification system for each obstacle is defined, and a training data set is generated according to each of the classification systems;

[0011] The preset deep learning model is optimized according to the training data set to generate the initial obstacle detection model.

[0012] In one embodiment, a preset screening condition is set, a plurality of initial pseudo labels are generated according to each of the dark-light images by the initial obstacle detection model, and a screening operation is performed on the plurality of initial pseudo labels according to the preset screening condition to obtain a plurality of initial pseudo labels screened out in the first round;

[0013] In the i-th round, the step of generating a plurality of initial pseudo labels according to each of the dark-light images by using the initial obstacle detection model is repeated, and the preset screening condition is adjusted according to the detection accuracy of the initial obstacle detection model, wherein i≥2;

[0014] A screening operation is performed on the multiple initial pseudo-labels generated in the i-th round according to the preset screening conditions adjusted in the i-1th round to obtain the multiple initial pseudo-labels screened in the i-th round, until the screening number i reaches the preset screening number threshold, and the multiple initial pseudo-labels screened in the last round are determined as the multiple target pseudo-labels, and the multiple target pseudo-labels are used as a dark light obstacle training set.

[0015] In one embodiment, each of the dark light images is detected by the initial obstacle detection model to obtain a plurality of detection results, wherein each of the detection results includes: a detection category, a detection coordinate, and a detection confidence;

[0016] Each of the detection results is used as an initial pseudo label of each of the dark-light images.

[0017] In one embodiment, the preset screening condition includes: a confidence screening condition or a pseudo-label quantity screening condition;

[0018] According to the confidence screening condition, multiple initial pseudo-labels whose detection confidence is greater than a preset confidence threshold are retained among the multiple initial pseudo-labels, or the initial pseudo-labels are sorted in descending order according to the detection confidence, the sorting order of the initial pseudo-labels is determined, and the initial pseudo-labels whose sorting order is within the preset sorting ratio threshold are retained according to the pseudo-label quantity screening condition.

[0019] In one embodiment, in the first round of training, the model parameters of the initial obstacle detection model are adjusted according to the first dark light obstacle training set;

[0020] In the jth round of training, the model parameters of the obstacle detection model after the j-1th round of training are adjusted according to the jth dark-light obstacle training set until the number of training times j reaches a preset training times threshold, and the obstacle detection model after the last round of adjustment is determined as the target obstacle detection model, where j≥2.

[0021] In one embodiment, a plurality of obstacle images to be detected in the dark light environment are obtained, and a plurality of target detection results of each of the obstacle images to be detected are obtained by using the target obstacle detection model;

[0022] The target pseudo labels of the obstacle image to be detected are determined according to the multiple target detection results, and the detection category and detection coordinates of each obstacle are determined according to the target pseudo labels.

[0023] In a second aspect, an embodiment of the present application provides an obstacle detection device, the obstacle detection device comprising:

[0024] A first acquisition module is used to acquire an initial obstacle detection model under a normal lighting environment;

[0025] A collection module, used for collecting multiple dark-light images in a dark-light environment through a camera;

[0026] a second acquisition module, configured to perform multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets including multiple target pseudo labels;

[0027] A training module, used to perform multiple rounds of training on the initial obstacle detection model according to each of the dark light obstacle training sets to obtain a target obstacle detection model;

[0028] The detection module is used to detect the dark light environment through the target obstacle detection model to determine the obstacles in the dark light environment.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is run by the processor, the obstacle detection method provided in the first aspect is executed.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which executes the obstacle detection method provided in the first aspect when running on a processor.

[0031] The obstacle detection method, device, electronic device and storage medium provided by the present application include: obtaining an initial obstacle detection model in a normal lighting environment; collecting multiple dark-light images in a dark-light environment through a camera; performing multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets includes multiple target pseudo labels; performing multiple rounds of training on the initial obstacle detection model according to each of the dark-light obstacle training sets to obtain a target obstacle detection model; detecting the dark-light environment through the target obstacle detection model to determine obstacles in the dark-light environment. The present application improves the detection accuracy of the target obstacle detection model for the dark-light environment by using the initial obstacle detection model trained in a normal lighting environment and performing multiple rounds of training on the initial obstacle detection model through the acquired dark-light obstacle training set. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and should not be regarded as limiting the scope of protection of the present application. In each of the drawings, similar components are numbered similarly.

[0033] Figure 1 A schematic diagram of a process of an obstacle detection method provided in an embodiment of the present application is shown;

[0034] Figure 2 Another schematic diagram of the process of the obstacle detection method provided by the embodiment of the present application is shown;

[0035] Figure 3 A schematic diagram of the structure of an obstacle detection device provided in an embodiment of the present application is shown;

[0036] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.

[0037] Icons: 300 - obstacle detection device, 301 - first acquisition module, 302 - acquisition module, 303 - second acquisition module, 304 - training module, 305 - detection module, 400 - electronic device, 401 - transceiver, 402 - processor, 403 - memory. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0039] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0040] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations of the foregoing items.

[0041] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0042] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0043] With the popularity of sweepers in households, obstacle detection has become an important part of the intelligence of sweepers. Traditional obstacle detection methods usually rely on a large amount of data labeled under standard lighting to train the target detection model. However, when the sweeper is operating in a dark environment, the performance of the detection model will drop significantly, and it is difficult to accurately identify obstacles. The distribution difference caused by this change in lighting makes it impossible for existing target detection models to directly adapt to dark environments. In order to adapt the model to dark environments, it is usually necessary to collect a large amount of dark data and manually label it. This process is costly and difficult to implement in practical applications. Therefore, how to make the obstacle detection algorithm able to efficiently adapt to dark environments without increasing the labeling cost is an urgent problem to be solved in the current sweeper detection technology.

[0044] Therefore, this application proposes an obstacle detection method, which can improve the model's adaptability to new environments without increasing the annotation cost, thereby effectively improving the obstacle detection performance in dark environments.

[0045] Example 1

[0046] The present application embodiment provides an obstacle detection method. Figure 1 The obstacle detection method includes: S101, obtaining an initial obstacle detection model under a normal lighting environment.

[0047] It should be noted that by pre-training the initial obstacle detection model under normal lighting conditions, it can help the model establish relatively stable initial parameters. In the supervised learning process, the initial obstacle detection model is trained with a large amount of normal lighting image data, and its internal weights, biases and other parameters are continuously adjusted to achieve better performance in obstacle detection under normal lighting.

[0048] When faced with dark-light environment data in the future, the initial obstacle detection model is adjusted and optimized based on the training parameters through unsupervised domain adaptation algorithms, which speeds up the adaptation process of the initial obstacle detection model in dark-light environments and improves the training efficiency of the initial obstacle detection model.

[0049] It is further explained that the normal light environment and the dark light environment can be distinguished by setting a light threshold. In the embodiment of the present application, the light environment greater than 250 lux is regarded as the normal light environment, and the light environment less than 50 lux is regarded as the dark light environment. The specific light threshold can be set according to actual needs, and this application does not limit it.

[0050] In one embodiment, if Figure 2 As shown, S1011, define a category system for each obstacle, and generate a training data set according to each category system; S1012, optimize a preset deep learning model according to the training data set to generate the initial obstacle detection model.

[0051] It should be noted that under normal circumstances, it is necessary to first define a category system, build a training data set based on the category system, and train the preset deep learning model based on this. In the process of building the data set, the collected images need to be annotated. In the process of training the preset deep learning model, the preset deep learning model will learn the relationship between the image and the annotation information. After the training is completed, the initial obstacle detection model can automatically detect the target of the input image.

[0052] S102, collecting a plurality of dark-light images in a dark-light environment through a camera.

[0053] In an embodiment, an initial obstacle detection model trained under normal lighting conditions is applied to a dark-light environment. A plurality of dark-light images in the dark-light environment are acquired through a camera installed on a sweeping robot. Each acquired dark-light image is provided as an input to the initial obstacle detection model. The initial obstacle detection model detects each dark-light image through a training data set.

[0054] S103, performing multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets including multiple target pseudo labels.

[0055] It should be noted that in each round, multiple initial pseudo labels of dark-light images are generated by the initial obstacle detection model, and a screening operation is performed on each initial pseudo label to obtain multiple target pseudo labels, and each target pseudo label is used as a dark-light obstacle training set.

[0056] In one embodiment, a preset screening condition is set, a plurality of initial pseudo labels are generated respectively according to each of the dark-light images by the initial obstacle detection model, and a screening operation is performed on the plurality of initial pseudo labels according to the preset screening condition to obtain a plurality of initial pseudo labels screened out in the first round; the step of generating a plurality of initial pseudo labels respectively according to each of the dark-light images by the initial obstacle detection model is repeated in the i-th round, and the preset screening condition is adjusted according to the detection accuracy of the initial obstacle detection model, wherein i≥2; a screening operation is performed on the plurality of initial pseudo labels generated in the i-th round according to the preset screening condition adjusted in the i-1th round to obtain a plurality of initial pseudo labels screened out in the i-th round, until the screening number i reaches the preset screening number threshold, and the plurality of initial pseudo labels screened out in the last round are determined as the plurality of target pseudo labels, and the plurality of target pseudo labels are used as a dark-light obstacle training set.

[0057] It should be understood that this application proposes a dynamic threshold refining strategy for the initially generated pseudo-labels. By performing multiple rounds of screening on the initial pseudo-labels, the preset screening conditions are adjusted in each round of screening based on the detection accuracy of the current initial obstacle detection model, so that the initial pseudo-labels retained in each round are more accurate, thereby gradually improving the quality of the initial pseudo-labels.

[0058] It should be noted that as the screening proceeds, the detection accuracy of the initial obstacle detection model will change. If the detection accuracy of the initial obstacle detection model improves during a round of screening, it means that the current pseudo-label quality is good and the initial obstacle detection model can learn effective features from it. At this time, the confidence threshold and / or pseudo-label quantity screening threshold in the preset screening conditions can be appropriately increased to further screen out more accurate pseudo-labels, so that the pseudo-labels used in the next round of training are of higher quality. On the contrary, if the detection accuracy does not improve significantly or decreases, it may mean that the threshold is set too high, resulting in too few available pseudo-labels, affecting the learning effect of the model. At this time, it is necessary to appropriately lower the confidence threshold and / or pseudo-label quantity screening threshold in the preset screening conditions to increase the number of pseudo-labels used for training.

[0059] As the screening progresses, the initial obstacle detection model can gradually adapt to the dark light environment, and the detection accuracy continues to improve. High-quality pseudo-labels can provide the model with more refined and accurate learning targets, further improving the detection performance of the model, enabling it to more accurately identify obstacles in dark light environments, thereby achieving a gradual optimization process from initial rough prediction to later high-precision detection.

[0060] Optionally, the preset screening condition includes: a confidence screening condition or a pseudo-label quantity screening condition; according to the confidence screening condition, multiple initial pseudo-labels whose detection confidence is greater than a preset confidence threshold are retained, or, each of the initial pseudo-labels is sorted in descending order according to the detection confidence, the sorting order of each of the initial pseudo-labels is determined, and the initial pseudo-labels whose sorting order is within a preset sorting ratio threshold are retained according to the pseudo-label quantity screening condition.

[0061] It should be noted that the preset sorting ratio threshold β is an indicator for controlling the number of all detection results, which is presented in the form of a percentage, where 0<β<100.

[0062] All the detection results obtained by the initial obstacle detection model after detecting all dark-light images are sorted according to the confidence score, and the top β% of the prediction results are selected. For example, if the model generates a total of 1000 detection results and the preset sorting ratio threshold β is 20, then the top 1000×20% prediction results with the highest confidence scores, that is, 200 prediction results, will be selected.

[0063] It is further explained that the confidence reflects the credibility of the initial obstacle detection model's prediction results of the detection category, and its value range is generally between 0 and 1.

[0064] This application refines and screens the initial pseudo labels to ensure that there are enough pseudo labels to support training in the early stage of unsupervised model training, and ensures that the initial obstacle detection model can further improve the detection performance in the later training process based on high-quality pseudo labels.

[0065] In one embodiment, each of the dark-light images is detected by the initial obstacle detection model to obtain multiple detection results, wherein each of the detection results includes: a detection category, a detection coordinate, and a detection confidence; and each of the detection results is used as an initial pseudo-label for each of the dark-light images.

[0066] In this embodiment, an initial pseudo label corresponds to multiple detection results. Each dark-light image is detected by the initial obstacle detection model. Each dark-light image has multiple corresponding detection results. One detection result includes a detection frame. If N detection results are obtained, N corresponding detection frames are generated, where N≥0. When N is 0, it means that the initial obstacle detection model has not detected an obstacle in the dark-light image. When N>0, it means that the initial obstacle detection model has detected N possible obstacles.

[0067] It should be noted that the detection box reflects the prediction of the initial obstacle detection model for the dark light image. For example, the detection box predicts that the obstacle in a dark light image is a "table" with a confidence of 0.8, which means that the initial obstacle detection model believes that the probability of the obstacle being a "table" is 80%. A higher confidence indicates that the model is 80% confident in the accuracy of the prediction result, while a lower confidence indicates that the prediction result may have a certain degree of uncertainty, and further verification or screening may be required in subsequent processing.

[0068] S104, performing multiple rounds of training on the initial obstacle detection model according to each of the dark light obstacle training sets to obtain a target obstacle detection model.

[0069] It should be noted that each round of training includes multiple rounds of screening of initial pseudo-labels and updating of the initial obstacle detection model, so that the initial obstacle detection model can gradually adapt to the data characteristics of the dark-light environment in each round of training, which can effectively enable the initial obstacle detection model to continuously learn and optimize in the dark-light environment and gradually improve its detection performance.

[0070] In one embodiment, in the first round of training, the model parameters of the initial obstacle detection model are adjusted according to the first dark-light obstacle training set; in the jth round of training, the model parameters of the obstacle detection model after the j-1th round of training are adjusted according to the jth dark-light obstacle training set, until the number of training times j reaches a preset training times threshold, and the obstacle detection model after the last round of adjustment is determined as the target obstacle detection model, where j≥2.

[0071] It should be noted that the dark-light obstacle training set is used to continue training the initial obstacle detection model in a dark-light environment. The dark-light obstacle training set is used as the supervision signal in a normal lighting environment. In multiple rounds of training, the parameters of the initial obstacle detection model are adjusted according to the error between the pseudo-label and the model prediction result to reduce this error, until the number of training times reaches the preset training times threshold, and the target obstacle detection model is obtained.

[0072] For example, 100 dark-light images are collected by the camera, and 1000 detection frames are generated by the initial obstacle detection model. Each detection frame has a detection category, detection coordinates, and detection confidence. The confidence threshold is set to 0.8, and the detection frames with a confidence higher than 0.8 are selected as pseudo labels for each dark-light image. Assuming that 800 of the 1000 detection frames meet the requirements, these 800 detection frames are used as target pseudo labels.

[0073] The dark-light images and detection box information corresponding to the 800 target pseudo labels are used as training data to input into the initial obstacle detection model. The initial obstacle detection model predicts the position and category of obstacles in the dark-light images and obtains multiple prediction results. The error between each prediction result and each target pseudo label is calculated, and the parameters of the initial obstacle detection model are updated according to the error to generate the target obstacle detection model.

[0074] It is further explained that during multiple rounds of training, each training will generate new target pseudo labels and update the initial obstacle detection model parameters. Each round of training will gradually improve the detection performance of the initial obstacle detection model in a dark light environment. After multiple rounds of training, without increasing the annotation cost, the detection accuracy of the initial obstacle detection model in a dark light environment is gradually improved, and the data distribution difference between the dark light environment and the normal light environment is gradually bridged, so that the trained target obstacle detection model can maintain stable obstacle detection performance in a dark light environment.

[0075] S105: Detect the dark-light environment using the target obstacle detection model to determine obstacles in the dark-light environment.

[0076] It should be understood that by deploying the target obstacle detection model in the sweeping robot system, the sweeping robot can effectively identify obstacles in a dark environment, thereby achieving the goal of cross-domain adaptation.

[0077] It should be noted that by obtaining the IoU (Intersection over Union) and setting the IoU threshold, if the IoU value between the predicted box and the true box is greater than or equal to the IoU threshold, the predicted position is considered valid. If the IoU value is less than the IoU threshold, the predicted box position is considered inaccurate.

[0078] It is further explained that IoU is the intersection-over-union ratio between the predicted box obtained by the target obstacle detection model and the real box in the training data set, that is, the ratio of the intersection area to the union area of ​​the two. The intersection is the area of ​​the overlap of the two boxes, and the union is the total area covered by the two boxes. The IoU value range is 0-1. When the two boxes completely overlap, IoU is 1, and when the two boxes do not overlap at all, IoU is 0.

[0079] It is further explained that, by obtaining the IoU value, the present application can accurately determine whether the target obstacle detection model can effectively identify obstacles in a dark light environment, and perform relevant operations on the target obstacle detection model based on the judgment result.

[0080] In one embodiment, a plurality of obstacle images to be detected in the dark light environment are obtained, and a plurality of target detection results of each of the obstacle images to be detected are obtained through the target obstacle detection model; target pseudo labels of the obstacle images to be detected are determined according to the plurality of target detection results, and detection categories and detection coordinates of each of the obstacles are determined according to the target pseudo labels.

[0081] It should be noted that the target obstacle detection model is used to detect the obstacle image to be tested. If the detection result is greater than 0, it is determined that there is an obstacle in the obstacle image to be tested. Each detection result is used as the target pseudo label of the obstacle image to be tested. The detection result with the highest confidence is obtained from each detection result, and the detection category and detection coordinates corresponding to the detection result with the highest confidence are obtained.

[0082] The obstacle detection method provided in this embodiment includes: obtaining an initial obstacle detection model in a normal lighting environment; collecting multiple dark-light images in a dark-light environment through a camera; performing multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets includes multiple target pseudo labels; performing multiple rounds of training on the initial obstacle detection model according to each of the dark-light obstacle training sets to obtain a target obstacle detection model; detecting the dark-light environment through the target obstacle detection model to determine obstacles in the dark-light environment. The present application improves the detection accuracy of the target obstacle detection model for the dark-light environment by using the initial obstacle detection model trained in a normal lighting environment and performing multiple rounds of training on the initial obstacle detection model through the acquired dark-light obstacle training set.

[0083] Example 2

[0084] In addition, an embodiment of the present application provides an obstacle detection device.

[0085] like Figure 3 As shown, the obstacle detection device 300 includes:

[0086] A first acquisition module 301 is used to acquire an initial obstacle detection model under a normal lighting environment;

[0087] The acquisition module 302 is used to acquire multiple dark-light images in a dark-light environment through a camera;

[0088] A second acquisition module 303 is used to perform multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets including multiple target pseudo labels;

[0089] A training module 304 is used to perform multiple rounds of training on the initial obstacle detection model according to each of the dark light obstacle training sets to obtain a target obstacle detection model;

[0090] The detection module 305 is used to detect the dark light environment by using the target obstacle detection model to determine obstacles in the dark light environment.

[0091] The obstacle detection device 300 provided in this embodiment can implement the obstacle detection method provided in Embodiment 1, which will not be described again to avoid repetition.

[0092] The obstacle detection device provided in this embodiment uses an initial obstacle detection model trained in a normal lighting environment and performs multiple rounds of training on the initial obstacle detection model using an acquired dark light obstacle training set, thereby improving the detection accuracy of the target obstacle detection model in a dark light environment.

[0093] Example 3

[0094] In addition, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the obstacle detection method provided in Embodiment 1 is executed.

[0095] For details, see Figure 4The electronic device 400 includes: a transceiver 401, a bus interface and a processor 402, wherein the processor 402 is used to: obtain an initial obstacle detection model in a normal light environment; collect multiple dark-light images in a dark-light environment through a camera; perform multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets includes multiple target pseudo labels; perform multiple rounds of training on the initial obstacle detection model according to each of the dark-light obstacle training sets to obtain a target obstacle detection model; detect the dark-light environment through the target obstacle detection model to determine obstacles in the dark-light environment.

[0096] In one embodiment, the processor 402 is further used to define a category system for each obstacle, generate a training data set according to each category system; optimize a preset deep learning model according to the training data set to generate the initial obstacle detection model.

[0097] In one embodiment, the processor 402 is further configured to set a preset screening condition, generate a plurality of initial pseudo labels respectively according to each of the dark-light images by using the initial obstacle detection model, and perform a screening operation on the plurality of initial pseudo labels according to the preset screening condition to obtain a plurality of initial pseudo labels screened out in the first round; repeatedly perform the step of generating a plurality of initial pseudo labels respectively according to each of the dark-light images by using the initial obstacle detection model in the i-th round, and adjust the preset screening condition according to the detection accuracy of the initial obstacle detection model, wherein i≥2; perform a screening operation on the plurality of initial pseudo labels generated in the i-th round according to the preset screening condition adjusted in the i-1th round to obtain a plurality of initial pseudo labels screened out in the i-th round, until the screening number i reaches the preset screening number threshold, and determine the plurality of initial pseudo labels screened out in the last round as the plurality of target pseudo labels, and use the plurality of target pseudo labels as a dark-light obstacle training set.

[0098] In one embodiment, the processor 402 is further used to detect each of the dark-light images through the initial obstacle detection model to obtain multiple detection results, wherein each of the detection results includes: detection category, detection coordinates and detection confidence; and each of the detection results is used as an initial pseudo-label for each of the dark-light images.

[0099] In one embodiment, the processor 402 is also used for the preset screening conditions to include: confidence screening conditions or pseudo-label quantity screening conditions; according to the confidence screening conditions, retaining multiple initial pseudo-labels whose detection confidence is greater than a preset confidence threshold among the multiple initial pseudo-labels, or sorting each of the initial pseudo-labels in order from large to small according to the detection confidence, determining the sorting order of each of the initial pseudo-labels, and retaining the initial pseudo-labels whose sorting order is within the preset sorting ratio threshold according to the pseudo-label quantity screening conditions.

[0100] In one embodiment, the processor 402 is further configured to adjust the model parameters of the initial obstacle detection model according to the first dark light obstacle training set in the first round of training;

[0101] In the jth round of training, the model parameters of the obstacle detection model after the j-1th round of training are adjusted according to the jth dark-light obstacle training set until the number of training times j reaches a preset training times threshold, and the obstacle detection model after the last round of adjustment is determined as the target obstacle detection model, where j≥2.

[0102] In one embodiment, the processor 402 is further used to obtain multiple obstacle images to be detected in the dark light environment, and obtain multiple target detection results of each of the obstacle images to be detected through the target obstacle detection model; determine the target pseudo-label of the obstacle image to be detected according to the multiple target detection results, and determine the detection category and detection coordinates of each obstacle according to the target pseudo-label.

[0103] In the embodiment of the present application, the electronic device 400 further includes a memory 403. Figure 4 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically connecting various circuits of one or more processors represented by processor 402 and memory represented by memory 403. The bus architecture may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 401 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 402 is responsible for managing the bus architecture and general processing, and the memory 403 may store data used by the processor 402 when performing operations.

[0104] The electronic device 400 provided in the embodiment of the present application can execute the steps of the obstacle detection method provided in the above method embodiment 1, which will not be described again here to avoid repetition.

[0105] The electronic device provided in this embodiment uses the initial obstacle detection model trained in a normal lighting environment and performs multiple rounds of training on the initial obstacle detection model using the acquired dark light obstacle training set, thereby improving the detection accuracy of the target obstacle detection model in a dark light environment.

[0106] Example 4

[0107] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the obstacle detection method provided in Example 1 is implemented.

[0108] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0109] The computer-readable storage medium provided in this embodiment can implement the obstacle detection method provided in Embodiment 1, and will not be described again here to avoid repetition.

[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0112] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An obstacle detection method, characterized in that: The method comprises: Obtain the initial obstacle detection model under normal lighting environment; Collect multiple dark-light images in a dark-light environment through a camera; Performing multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets including multiple target pseudo labels; Performing multiple rounds of training on the initial obstacle detection model according to each of the dark light obstacle training sets to obtain a target obstacle detection model; The dark-light environment is detected by using the target obstacle detection model to determine obstacles in the dark-light environment.

2. The obstacle detection method according to claim 1, characterized in that: The obtaining of the initial obstacle detection model under a normal lighting environment includes: Defining a classification system for each obstacle, and generating a training data set according to each of the classification systems; The preset deep learning model is optimized according to the training data set to generate the initial obstacle detection model.

3. The obstacle detection method according to claim 1, characterized in that: Perform multiple rounds of screening based on the dark-light images and the initial obstacle detection model to obtain a dark-light obstacle training set, including: Setting a preset screening condition, generating a plurality of initial pseudo labels respectively according to each of the dark-light images by using the initial obstacle detection model, and performing a screening operation on the plurality of initial pseudo labels according to the preset screening condition to obtain a plurality of initial pseudo labels screened out in the first round; In the i-th round, the step of generating a plurality of initial pseudo labels according to each of the dark-light images by using the initial obstacle detection model is repeated, and the preset screening condition is adjusted according to the detection accuracy of the initial obstacle detection model, wherein i≥2; A screening operation is performed on the multiple initial pseudo-labels generated in the i-th round according to the preset screening conditions adjusted in the i-1th round to obtain the multiple initial pseudo-labels screened in the i-th round, until the screening number i reaches the preset screening number threshold, and the multiple initial pseudo-labels screened in the last round are determined as the multiple target pseudo-labels, and the multiple target pseudo-labels are used as a dark light obstacle training set.

4. The obstacle detection method according to claim 3, characterized in that: The generating a plurality of initial pseudo labels according to each of the dark-light images by using the initial obstacle detection model comprises: Detecting each of the dark-light images using the initial obstacle detection model to obtain a plurality of detection results, wherein each of the detection results includes: a detection category, a detection coordinate, and a detection confidence; Each of the detection results is used as an initial pseudo label of each of the dark-light images.

5. The obstacle detection method according to claim 4, characterized in that: The preset screening conditions include: confidence screening conditions or pseudo-label quantity screening conditions; Performing a screening operation on the plurality of initial pseudo labels according to a preset screening condition to obtain a plurality of initial pseudo labels screened in the first round includes: According to the confidence screening condition, retaining multiple initial pseudo labels whose detection confidence is greater than a preset confidence threshold among the multiple initial pseudo labels, or, The initial pseudo labels are sorted in descending order according to the detection confidence, the sorting order of the initial pseudo labels is determined, and the initial pseudo labels whose sorting order is within a preset sorting ratio threshold are retained according to the pseudo label quantity screening condition.

6. The obstacle detection method according to claim 1, characterized in that: The initial obstacle detection model is trained for multiple rounds according to each of the dark light obstacle training sets to obtain a target obstacle detection model, including: In the first round of training, adjusting the model parameters of the initial obstacle detection model according to the first dark light obstacle training set; In the jth round of training, the model parameters of the obstacle detection model after the j-1th round of training are adjusted according to the jth dark-light obstacle training set until the number of training times j reaches a preset training times threshold, and the obstacle detection model after the last round of adjustment is determined as the target obstacle detection model, where j≥2.

7. The obstacle detection method according to claim 1, characterized in that: The detecting the dark light environment by using the target obstacle detection model to determine obstacles in the dark light environment includes: Acquire a plurality of obstacle images to be detected in the dark light environment, and acquire a plurality of target detection results of each of the obstacle images to be detected by using the target obstacle detection model; The target pseudo labels of the obstacle image to be detected are determined according to the multiple target detection results, and the detection category and detection coordinates of each obstacle are determined according to the target pseudo labels.

8. An obstacle detection device, characterized in that: The device comprises: A first acquisition module is used to acquire an initial obstacle detection model under a normal lighting environment; A collection module, used for collecting multiple dark-light images in a dark-light environment through a camera; a second acquisition module, configured to perform multiple rounds of screening according to each of the dark-light images and the initial obstacle detection model to obtain multiple dark-light obstacle training sets, each of the dark-light obstacle training sets including multiple target pseudo labels; A training module, used to perform multiple rounds of training on the initial obstacle detection model according to each of the dark light obstacle training sets to obtain a target obstacle detection model; The detection module is used to detect the dark light environment through the target obstacle detection model to determine the obstacles in the dark light environment.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run by the processor, the obstacle detection method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: The computer program stores a computer program, which executes the obstacle detection method according to any one of claims 1 to 7 when running on a processor.

Images