Image cleaning methods, apparatus, electronic devices and computer-readable media

Abstract

This disclosure discloses an image cleaning method, apparatus, electronic device, and computer-readable medium. One specific embodiment of the method includes: acquiring data from speed limit signs to obtain a speed limit sign image sequence; selecting sample speed limit sign image sequences; and performing a determination step: inputting the data into an initial target detection model to obtain a probability value sequence; determining a target time point; creating an initial variable window; generating at least one image sequence; cleaning the at least one image sequence to obtain a processed speed limit sign image sequence; determining the ratio of the number of sequences greater than or equal to a preset probability threshold to the number of detected probability value sequences; determining the initial variable window as a target variable window; and using the processed speed limit sign image sequence as a sample speed limit sign image sequence, and performing the determination step again. This embodiment utilizes an adjustable-size variable window and a target detection model to reduce the false negative rate and improve the efficiency of image cleaning.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, to an image cleaning method and device, an electronic device and a computer readable medium. BACKGROUND

[0002] Image detection technology is the basis of automatic driving visual perception technology and is widely used in intelligent visual systems in various fields, and image cleaning is an important step to improve image detection technology. For image cleaning, the commonly used way is: first, cleaning the image by using an existing target detection model to detect the image, and then evaluating the image cleaning effect by using the accuracy of the existing target detection model.

[0003] However, the inventors have found that when the above-mentioned way is used to clean the image, the following technical problems often exist:

[0004] First, the accuracy of the target detection model trained by open source data for speed limit plate detection is low, and the miss detection rate for some edge scene images is high, resulting in low image cleaning efficiency and thus reducing the safety of automatic driving.

[0005] Second, only spatial-temporal domain image filter or frequency domain image filter is used to filter the collected image data in bad weather, resulting in unclear edge information of objects in the image, and thus reducing the accuracy and safety of automatic driving vehicle recognition.

[0006] The above information disclosed in the background section is only intended to enhance the understanding of the background of the present inventive concept, and therefore, it can include information that does not form the prior art known to those of ordinary skill in the art in the country. SUMMARY

[0007] The summary section is provided to introduce the concepts briefly in a simplified form, which will be described in detail in the specific embodiments section. The summary section is not intended to identify key or essential features of the claimed technology nor is it intended to be used to limit the scope of the claimed technology.

[0008] Some embodiments of the present disclosure propose an image cleaning method, device, electronic device and computer readable medium to solve one or more of the technical problems mentioned in the background section.

[0009] In a first aspect, some embodiments of this disclosure provide an image cleaning method, including: controlling a data acquisition vehicle to acquire data on speed limit signs, obtaining a sequence of speed limit sign images; selecting a sample speed limit sign image sequence from the speed limit sign image sequence, and performing the following determination steps: sequentially inputting the sample speed limit sign image sequence into an initial target detection model to obtain a probability value sequence, wherein the probability values ​​in the probability value sequence represent the probability that the speed limit sign exists in the sample speed limit sign image; determining a target time point based on the probability value sequence; creating an initial variable window based on the target time point, wherein the initial variable window is used to determine the window of the sample speed limit sign image to be cleaned; and based on the initial variable window... Generate at least one image sequence; perform cleaning processing on the at least one image sequence to obtain a processed speed limit sign image sequence; determine the ratio of the number of probability values ​​greater than or equal to a preset probability threshold in the probability value sequence to the total number of probability values ​​in the probability value sequence; in response to determining that the ratio is greater than or equal to the preset threshold, determine the initial variable window as the target variable window; in response to determining that the ratio is less than the preset threshold, adjust the initial variable window and the initial target detection model, use the processed speed limit sign image sequence as the sample speed limit sign image sequence, determine the adjusted initial variable window and the adjusted initial target detection model as the initial variable window and the initial target detection model respectively, and perform the determination step again.

[0010] Secondly, some embodiments of this disclosure provide an image cleaning apparatus, including: a control unit configured to control a data acquisition vehicle to acquire data from speed limit signs to obtain a sequence of speed limit sign images; and an execution unit configured to select a sample speed limit sign image sequence from the speed limit sign image sequence, and to perform the following determination steps: sequentially inputting the sample speed limit sign image sequence into an initial target detection model to obtain a probability value sequence, wherein the probability values ​​in the probability value sequence represent the probability that the speed limit sign exists in the sample speed limit sign image; determining a target time point based on the probability value sequence; and creating an initial variable window based on the target time point, wherein the initial variable window is used to determine the window of the sample speed limit sign image to be cleaned; and determining the target time point based on the initial variable window. The system generates at least one image sequence by changing the window size; it then cleans the at least one image sequence to obtain a processed speed limit sign image sequence; it determines the ratio of the number of probability values ​​greater than or equal to a preset probability threshold in the probability value sequence to the total number of probability values ​​in the probability value sequence; in response to determining that the ratio is greater than or equal to the preset threshold, it determines the initial variable window as the target variable window; and it adjusts the initial variable window and the initial target detection model in response to determining that the ratio is less than the preset threshold, using the processed speed limit sign image sequence as the sample speed limit sign image sequence, determining the adjusted initial variable window and the adjusted initial target detection model as the initial variable window and the initial target detection model, and then performing the determination step again.

[0011] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0012] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0013] The above embodiments of this disclosure have the following beneficial effects: the image cleaning methods of some embodiments of this disclosure utilize adjustable-size variable windows and object detection models to reduce the false negative rate and improve the efficiency of image cleaning. Specifically, the reason for the reduced safety of related autonomous driving is that manual cleaning is inefficient, and the cleaning cycle and cleaning cost are high. Furthermore, the accuracy of using pre-trained object detection models for speed limit sign detection is low, and the false negative rate is high for images in some edge scenes, resulting in low image cleaning efficiency. Based on this, the image cleaning methods of some embodiments of this disclosure can first control a data acquisition vehicle to collect data on speed limit signs, obtaining a speed limit sign image sequence. Here, the speed limit sign image sequence facilitates the use of an initial variable window to reduce the false negative rate and facilitates subsequent adjustment of the variable window and object detection model. Then, a sample speed limit sign image sequence is selected from the above speed limit sign image sequence, and the following determination steps are performed: the sample speed limit sign image sequence is sequentially input into the initial object detection model to obtain a probability value sequence, wherein the probability value in the probability value sequence represents the probability value of the presence of the speed limit sign in the sample speed limit sign image. Here, the obtained probability numerical sequence facilitates the subsequent determination of the accuracy of the initial target detection model. Based on the aforementioned probability numerical sequence, the target time point is determined. This target time point facilitates the creation of an initial variable window and reduces the false negative rate in cleaning the sample speed limit sign image sequence. Obtaining the edge image data of the speed limit signs is beneficial for improving the model's detection rate. Based on the aforementioned target time point, an initial variable window is created, which is used to determine the window of the sample speed limit sign images to be cleaned. Creating an initial variable window allows for the acquisition of edge images in the sample speed limit sign image sequence, improving the false negative rate in cleaning the sample speed limit sign image sequence. Based on the aforementioned initial variable window, at least one image sequence is generated. Obtaining at least one image sequence facilitates subsequent cleaning processing to obtain edge images. Subsequently, the at least one image sequence is cleaned to obtain a processed speed limit sign image sequence. Cleaning at least one image sequence can improve the detection accuracy of the target detection model and reduce the cleaning cycle and cost of the sample speed limit sign image sequence. The ratio of the number of probability values ​​greater than or equal to a preset probability threshold in the aforementioned probability value sequence to the total number of probability values ​​in the aforementioned probability value sequence is determined. In response to determining that the ratio is greater than or equal to the preset threshold, the initial variable window is determined as the target variable window. Finally, in response to determining that the ratio is less than the preset threshold, the initial variable window and the initial target detection model are adjusted. The processed speed limit sign image sequence is used as the sample speed limit sign image sequence. The adjusted initial variable window and the adjusted initial target detection model are determined as the initial variable window and the initial target detection model, respectively, and the determination step is performed again.Here, the false negative rate of cleaning sample speed limit sign image sequences can be reduced based on the target detection model and variable window. Adjusting the initial variable window and initial target detection model can improve the efficiency of the model and image cleaning, and reduce the cleaning cycle and cleaning cost. Therefore, this image cleaning method, by utilizing an adjustable variable window and target detection model, can reduce the false negative rate of image cleaning, improve the efficiency of image cleaning, and enhance the detection efficiency of the target detection model. Attached Figure Description

[0014] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0015] Figure 1 This is a flowchart of some embodiments of the image cleaning method according to the present disclosure;

[0016] Figure 2 This is a schematic diagram of the composition of a channel image according to the image cleaning method of this disclosure;

[0017] Figure 3 These are schematic diagrams illustrating the structure of some embodiments of the image cleaning apparatus according to this disclosure;

[0018] Figure 4 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] Figure 1 A flow 100 of some embodiments of an image cleaning method according to the present disclosure is shown. The image cleaning method includes the following steps:

[0026] Step 101: Control the data acquisition vehicle to collect data from the speed limit signs and obtain a sequence of speed limit sign images.

[0027] In some embodiments, the entity executing the image cleaning method described above (e.g., an electronic device) can control a data acquisition vehicle to acquire data from speed limit signs, obtaining a sequence of speed limit sign images. This speed limit sign image data sequence can be an image sequence ordered according to the time sequence in which the data acquisition vehicle collects the speed limit signs and stores them on a disk. Storage on disk can be the process of storing the acquired speed limit sign image sequence on a disk. The storage time can be the time when the speed limit sign images are stored on the disk. The data acquisition vehicle can be a vehicle that collects data from the speed limit signs. For example, the data acquisition vehicle can be an autonomous vehicle.

[0028] In some optional implementations of certain embodiments, the control and data acquisition vehicle described above, for collecting data from speed limit signs to obtain a sequence of speed limit sign images, may include the following steps:

[0029] The first step is to control the aforementioned data collection vehicle to collect data from the speed limit sign, obtaining a sequence of images. This sequence can be composed of images captured from the speed limit sign itself. The order of the collected images can be determined by sorting them according to the time sequence of the disc's placement.

[0030] The second step involves dividing the acquired image sequence into channels to obtain multiple channel image sequences. Each channel image in these multiple channel image sequences can be an image containing only one color channel. For example, the multiple channel image sequences could include: a red channel image sequence, a green channel image sequence, and a blue channel image sequence.

[0031] The third step involves performing the following determination steps for each channel image in the above multiple channel image sequences:

[0032] Sub-step 1 involves smoothing the aforementioned channel image to obtain a smoothed image. This smoothing process can involve removing high-frequency pixels from the channel image. In practice, the executing entity can first use a Fourier transform to convert the channel image to the frequency domain, obtaining a frequency domain image. This frequency domain image can be obtained by performing a Fourier transform on the channel image located in the spatial domain to the frequency domain. The frequency domain can characterize the grayscale changes of the channel image. The spatial domain can be a two-dimensional planar coordinate system. Then, a low-pass filter is used to denoise the frequency domain image, resulting in a smoothed image sequence.

[0033] Sub-step 2 involves performing a frequency domain transformation on the smoothed image to obtain a spatial domain image. This spatial domain image can be an image where the pixels are located in a two-dimensional plane coordinate system. In practice, the executing entity can utilize inverse Fourier transform to perform the frequency domain transformation on the smoothed image to obtain the spatial domain image.

[0034] Sub-step 3: Determine the reflectivity of the aforementioned spatial domain image. This reflectivity can be the ratio of the energy of light reflected by the object itself to the total energy of light projected onto the object. For example... Figure 2 As shown, a spatial domain image is an image composed of the product of incident light and reflectance. I(x, y) represents the spatial domain image. L(x, y) represents the incident light. R(x, y) represents the reflectance. The incident light can be external lighting when photographing a speed limit sign. In practice, the execution entity can first normalize the pixel values ​​in the spatial domain image to obtain normalized values. Then, it can perform Gaussian convolution filtering on the normalized values ​​to obtain the reflectance of the spatial domain image.

[0035] The fourth step involves channel fusion of the obtained multiple reflectance sequences to obtain a fused image sequence. This fused image sequence can be a speed limit sign image sequence obtained by fusing the reflectance values ​​of three color channels.

[0036] The fifth step is to perform equalization processing on the above-mentioned fused image sequence to obtain the speed limit sign image sequence. In practice, the above-mentioned execution entity can use the histogram equalization method to perform equalization processing on the above-mentioned fused image sequence to obtain the speed limit sign image sequence.

[0037] The above-described technical solution and its related content, as an inventive point of this disclosure, solve the second technical problem mentioned in the background: "Filtering image data collected under adverse weather conditions using only spatiotemporal domain image filters or frequency domain image filters results in unclear edge information of objects in the image, leading to lower accuracy and safety of autonomous vehicle recognition." Factors contributing to lower accuracy and safety of autonomous vehicle recognition often include: filtering image data collected under adverse weather conditions using only spatiotemporal domain image filters or frequency domain image filters results in unclear edge information of objects in the image, leading to lower accuracy and safety of autonomous vehicle recognition. Solving these factors can improve the clarity of edge information of objects in the image and reduce overall brightness differences in the image. To achieve this effect, this disclosure first controls the aforementioned data acquisition vehicle to collect data from speed limit signs, obtaining a sequence of acquired images. Secondly, the acquired image sequence is divided into channels to obtain multiple channel image sequences. Here, channel division allows for fine-grained image enhancement, resulting in clearer, noise-free images. Subsequently, the channel images are smoothed to obtain smoothed images. Here, smoothing helps reduce noise in the image and highlights important information. The smoothed image is then transformed in the frequency domain to obtain a spatial domain image. This transformation from the frequency domain to the spatial domain is more beneficial for determining the reflectance of subsequent images. The reflectance of the spatial domain image is determined. Then, multiple reflectance sequences are fused to obtain a fused image sequence. Determining the reflectance enhances the defogging effect in adverse weather conditions, improving image clarity and edge information. Finally, the fused image sequence is equalized to obtain a speed limit sign image sequence. This equalization process increases image contrast and clarity, and reduces noise. Therefore, this technical solution, combining frequency and spatial domain denoising methods, can improve the clarity and edge information of images acquired under adverse weather conditions, thereby improving the accuracy and safety of autonomous vehicle recognition.

[0038] Step 102: Select a sample speed limit sign image sequence from the speed limit sign image sequence, and perform the following determination steps:

[0039] Step 1021: Input the sample speed limit sign image sequence into the initial target detection model in sequence to obtain the probability value sequence.

[0040] In some embodiments, the execution entity may sequentially input the sample speed limit sign image sequence into an initial object detection model to obtain a probability value sequence. The probability values ​​in the probability value sequence represent the probability that the speed limit sign exists in the sample speed limit sign image. The sample speed limit sign image sequence may be an image sequence used to train the initial object detection model. The initial object detection model may include, but is not limited to, the YOLO (You Only Look Once) model and the SSD (Single Shot MultiBox Detector) model.

[0041] Step 1022: Determine the target time point based on the probability value sequence.

[0042] In some embodiments, the execution entity can determine a target time point based on the probability value sequence. The target time point can be the disk placement time point corresponding to a probability value in the probability value sequence that is greater than or equal to a preset probability threshold and is located at an initial position. The initial position can be the first position.

[0043] As an example, the aforementioned execution entity can first compare each probability value in the probability value sequence with a preset probability threshold to generate a comparison result sequence. Secondly, it can determine the comparison results in the sequence that are greater than or equal to the preset probability threshold as the target result sequence. Finally, it can determine the collection time point corresponding to the probability value of the comparison result at the initial position in the target result sequence as the target time point.

[0044] In some optional implementations of certain embodiments, determining the target time point based on the aforementioned probability numerical sequence may include the following steps:

[0045] The first step is to filter out probability values ​​greater than or equal to the preset probability threshold from the above probability value sequence to obtain the target probability value sequence. The preset probability threshold can be 0.75.

[0046] The second step is to determine the sample speed limit sign image sequence corresponding to the above target probability numerical sequence as the target sample speed limit sign image sequence.

[0047] The third step is to determine the acquisition time point corresponding to the target sample speed limit sign image located at the initial position in the above target sample speed limit sign image sequence as the target time point.

[0048] Step 1023: Create an initial variable window based on the target time point.

[0049] In some embodiments, the executing entity may create an initial variable window based on the target time point. This initial variable window is used to determine the window containing the sample speed limit sign image to be cleaned.

[0050] As an example, the aforementioned execution entity can create an initial variable window for the sample speed limit sign image corresponding to the aforementioned target time point.

[0051] In some optional implementations of certain embodiments, creating an initial variable window based on the target time point may include the following steps:

[0052] The first step is to determine the starting time point as the time point whose difference from the target time point falls within a first preset time range. For example, the first preset time range can be 1 to 3 seconds. The difference from the target time point can be 3 seconds.

[0053] The second step is to select the sample speed limit sign image corresponding to the same acquisition time point as the starting time point from the above sample speed limit sign image sequence, and use it as the starting sample speed limit sign image. The acquisition time point can be the time when the acquisition vehicle acquires the speed limit sign and stores it in memory.

[0054] The third step is to determine the termination time point as the time point whose corresponding difference from the target time point falls within a second preset time range. For example, the second preset time range can be 1 to 2 seconds. The corresponding difference from the target time point can be 1 second.

[0055] The fourth step is to select the sample speed limit sign image corresponding to the same collection time point as the above termination time point from the above sample speed limit sign image sequence, and use it as the termination sample speed limit sign image.

[0056] The fifth step is to determine the sample speed limit sign images located in the above sample speed limit sign image sequence, from the above starting sample speed limit sign image to the above ending sample speed limit sign image, as the sample speed limit sign image sequence included in the above initial variable window.

[0057] Step 1024: Generate at least one image sequence based on the initial variable window.

[0058] In some embodiments, the execution entity may generate at least one image sequence based on the initial variable window. The at least one image sequence may be a sequence of sample speed limit sign images included in at least one initial variable window.

[0059] As an example, the aforementioned execution entity can create an initial variable window for the probability values ​​of the aforementioned sample speed limit sign image sequences that are greater than or equal to a preset probability threshold, thereby obtaining at least one image sequence.

[0060] In some optional implementations of certain embodiments, generating at least one image sequence based on the aforementioned initial variable window may include the following steps:

[0061] The first step, based on the first probability numerical sequence, is to perform the following sequence determination steps:

[0062] Sub-step 1: In response to determining that the initial variable window does not include the sample speed limit sign image sequence corresponding to the same acquisition time point as the sequence termination time point, the initial variable window is expanded to obtain an expanded variable window and an extended window. The extended window can be the window corresponding to the portion of the expanded variable window that is larger than the initial variable window. The expanded variable window can be a window obtained by expanding the time point corresponding to the termination position of the initial variable window. The first probability value sequence is the probability value sequence from the target time point to the sequence termination time point, and the sequence termination time point can be the time point corresponding to the probability value located at the termination position of the probability value sequence.

[0063] Sub-step 2, in response to determining that there is no probability value greater than or equal to the preset probability threshold in the probability value sequence corresponding to the sample speed limit sign image sequence included in the extended window, adds the sample speed limit sign image sequence included in the extended variable window to at least one preset image sequence. The at least one preset image sequence may be a pre-defined image sequence that includes at least one image sequence.

[0064] Sub-step 3, in response to determining that there is a second probability value greater than or equal to the preset probability threshold in the second probability value sequence, filters out second probability values ​​greater than or equal to the preset probability value from the second probability value sequence to obtain a target filtered probability value sequence. Here, the second probability value sequence is the probability value sequence located between the extended probability value and the probability value corresponding to the sequence termination time point in the previous probability value sequence. The extended probability value is the probability value corresponding to the sample speed limit sign image sequence located at the termination position in the obtained extended variable window.

[0065] Sub-step 4 involves creating a variable sample window for the speed limit sign images corresponding to the probability values ​​at the starting position in the aforementioned target screening probability value sequence. This variable sample window can be used to determine the window of the speed limit sign images to be cleaned.

[0066] Sub-step 5: In response to determining that the above-mentioned sample variable window includes the sample speed limit sign image corresponding to the same acquisition time point as the above-mentioned sequence termination time point, the above-mentioned at least one preset image sequence is determined as the above-mentioned at least one image sequence.

[0067] The second step involves determining that the sample variable window does not include the sample speed limit sign image corresponding to the same acquisition time point as the aforementioned sequence termination time point. This involves defining the probability value sequence located within the aforementioned probability value sequence and the probability value sequence between the window termination time point and the aforementioned sequence termination time point as the first probability value sequence, and then repeating the aforementioned sequence determination step. Here, the aforementioned window termination time point is the time point corresponding to the sample speed limit sign image located at the termination position of the sample variable window.

[0068] In some optional implementations of certain embodiments, the above-described initial variable window is extended to obtain an extended variable window and an extended window, including:

[0069] First, based on the initial variable window described above, perform the following extended steps:

[0070] Sub-step 1 involves determining the probability values ​​corresponding to the sample speed limit sign images included between the target time point and the termination time point as a window probability value sequence. This window probability value sequence can be the probability values ​​corresponding to the sample speed limit sign image sequence included between the target time point and the termination time point.

[0071] Sub-step 2, in response to determining that there is a window probability value greater than or equal to the preset probability threshold in the above window probability value sequence, determines at least one window probability value greater than or equal to the preset probability threshold as the target window probability value sequence. The target window probability value sequence may be a probability value sequence located in the above window probability value sequence that is greater than or equal to the preset probability threshold.

[0072] Sub-step 3: Determine the target window probability value located at the end position of the above target window probability value sequence, and use it as the target termination window probability value.

[0073] Sub-step 4: Determine the collection time point of the above target termination window probability value as the adjustment time point.

[0074] Sub-step 5 involves expanding the initial variable window based on the adjusted time point, resulting in an expanded variable window and the expanded window itself. The expanded variable window can be a window that extends the time point corresponding to the end position of the initial variable window by a preset time. For example, the preset time could be 1 second.

[0075] As an example, the aforementioned execution entity takes the aforementioned adjustment time point as the starting time point and extends the time point corresponding to the end position of the aforementioned initial variable window by 1 second to obtain the extended variable window.

[0076] Sub-step 6: In response to determining that there is no probability value greater than or equal to the preset probability threshold in the probability value sequence corresponding to the sample speed limit sign image sequence included in the above extended window, the above extended step ends.

[0077] Optionally, the aforementioned implementing entity may also perform the following steps:

[0078] In response to the determination that there is a probability value greater than or equal to the preset probability threshold in the probability value sequence corresponding to the sample speed limit sign image sequence included in the expanded window, the acquisition time point of the sample speed limit sign image located at the termination position in the initial variable window is determined as the target time point, the expanded variable window is determined as the initial variable window, and the above expansion steps are executed again.

[0079] Optionally, after determining that the initial variable window does not include the sample speed limit sign image sequence corresponding to the same acquisition time point as the sequence termination time point, and expanding the initial variable window to obtain an expanded variable window and an expanded window, the method further includes:

[0080] In response to determining that the initial variable window includes a sample speed limit sign image sequence corresponding to the same acquisition time point as the sequence termination time point, the sample speed limit sign image sequence included in the initial variable window is added to the at least one preset image sequence.

[0081] Step 1025: Clean at least one image sequence to obtain a processed speed limit sign image sequence.

[0082] In some embodiments, the execution entity may perform cleaning processing on the at least one image sequence to obtain a processed speed limit sign image sequence. The processed speed limit sign image sequence may be an image sequence whose probability threshold is less than a preset probability threshold, but which includes a speed limit sign. The cleaning processing may include, but is not limited to: cleaning the at least one image sequence using statistical methods, or manual detection cleaning processing.

[0083] Step 1026: Determine the ratio of the number of probability values ​​in the probability value sequence that are greater than or equal to a preset probability threshold to the total number of probability values ​​in the probability value sequence.

[0084] In some embodiments, the executing entity may determine the ratio of the number of probability values ​​in the probability value sequence that are greater than or equal to a preset probability threshold to the total number of probability values ​​in the probability value sequence. The preset probability threshold may characterize the completion of training of the initial object detection model. For example, the preset probability threshold may be 0.75.

[0085] Step 1027: In response to determining that the ratio is greater than or equal to a preset threshold, the initial variable window is determined as the target variable window.

[0086] In some embodiments, the executing entity may determine the initial variable window as a target variable window in response to determining that the ratio is greater than or equal to a preset threshold. The target variable window may be a window corresponding to the initial target detection model. The target variable window may be a window whose length decreases as the accuracy of the initial target model increases.

[0087] Step 103: In response to the determination that the ratio is less than a preset threshold, the initial variable window and the initial target detection model are adjusted, the processed speed limit sign image sequence is used as the sample speed limit sign image sequence, the adjusted initial variable window and the adjusted initial target detection model are determined as the initial variable window and the initial target detection model respectively, and the determination step is performed again.

[0088] In some embodiments, the execution entity may adjust the initial variable window and the initial target detection model in response to the determination that the ratio is less than the preset threshold, take the processed speed limit sign image sequence as the sample speed limit sign image sequence, determine the adjusted initial variable window and the adjusted initial target detection model as the initial variable window and the initial target detection model respectively, and perform the determination step again.

[0089] The above embodiments of this disclosure have the following beneficial effects: the image cleaning methods of some embodiments of this disclosure utilize adjustable-size variable windows and object detection models to reduce the false negative rate and improve the efficiency of image cleaning. Specifically, the reason for the reduced safety of related autonomous driving is that manual cleaning is inefficient, and the cleaning cycle and cleaning cost are high. Furthermore, the accuracy of using pre-trained object detection models for speed limit sign detection is low, and the false negative rate is high for images in some edge scenes, resulting in low image cleaning efficiency. Based on this, the image cleaning methods of some embodiments of this disclosure can first control a data acquisition vehicle to collect data on speed limit signs, obtaining a speed limit sign image sequence. Here, the speed limit sign image sequence facilitates the use of an initial variable window to reduce the false negative rate and facilitates subsequent adjustment of the variable window and object detection model. Then, a sample speed limit sign image sequence is selected from the above speed limit sign image sequence, and the following determination steps are performed: the sample speed limit sign image sequence is sequentially input into the initial object detection model to obtain a probability value sequence, wherein the probability value in the probability value sequence represents the probability value of the presence of the speed limit sign in the sample speed limit sign image. Here, the obtained probability numerical sequence facilitates the subsequent determination of the accuracy of the initial target detection model. Based on the aforementioned probability numerical sequence, the target time point is determined. This target time point facilitates the creation of an initial variable window and reduces the false negative rate in cleaning the sample speed limit sign image sequence. Obtaining the edge image data of the speed limit signs is beneficial for improving the model's detection rate. Based on the aforementioned target time point, an initial variable window is created, which is used to determine the window of the sample speed limit sign images to be cleaned. Creating an initial variable window allows for the acquisition of edge images in the sample speed limit sign image sequence, improving the false negative rate in cleaning the sample speed limit sign image sequence. Based on the aforementioned initial variable window, at least one image sequence is generated. Obtaining at least one image sequence facilitates subsequent cleaning processing to obtain edge images. Subsequently, the at least one image sequence is cleaned to obtain a processed speed limit sign image sequence. Cleaning at least one image sequence can improve the detection accuracy of the target detection model and reduce the cleaning cycle and cost of the sample speed limit sign image sequence. The ratio of the number of probability values ​​greater than or equal to a preset probability threshold in the aforementioned probability value sequence to the total number of probability values ​​in the aforementioned probability value sequence is determined. In response to determining that the ratio is greater than or equal to the preset threshold, the initial variable window is determined as the target variable window. Finally, in response to determining that the ratio is less than the preset threshold, the initial variable window and the initial target detection model are adjusted. The processed speed limit sign image sequence is used as the sample speed limit sign image sequence. The adjusted initial variable window and the adjusted initial target detection model are determined as the initial variable window and the initial target detection model, respectively, and the determination step is performed again.Here, the false negative rate of cleaning sample speed limit sign image sequences can be reduced based on the target detection model and variable window. Adjusting the initial variable window and initial target detection model can improve the efficiency of the model and image cleaning, and reduce the cleaning cycle and cleaning cost. Therefore, this image cleaning method, by utilizing an adjustable variable window and target detection model, can reduce the false negative rate of image cleaning, improve the efficiency of image cleaning, and enhance the detection efficiency of the target detection model.

[0090] Further reference Figure 3 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an image cleaning apparatus, which are similar to... Figure 1 Corresponding to the method embodiments shown, this image cleaning apparatus can be specifically applied to various electronic devices.

[0091] like Figure 3 As shown, an image cleaning device 300 includes: an acquisition unit 301, an execution unit 302, and an adjustment unit 303. The acquisition unit 301 is configured to control an acquisition vehicle to acquire data from speed limit signs, obtaining a sequence of speed limit sign images. The execution unit 302 is configured to: select a sample speed limit sign image sequence from the aforementioned speed limit sign image sequence, and perform the following determination steps: sequentially input the sample speed limit sign image sequence into an initial target detection model to obtain a probability value sequence, wherein the probability values ​​in the probability value sequence represent the probability values ​​of the existence of the speed limit sign in the sample speed limit sign image; determine a target time point based on the probability value sequence; create an initial variable window based on the target time point, wherein the initial variable window is used to determine the window of the sample speed limit sign image to be cleaned; generate at least one image sequence based on the initial variable window; perform cleaning processing on the at least one image sequence to obtain a processed speed limit sign image sequence; determine the ratio of the number of probability values ​​in the probability value sequence that are greater than or equal to a preset probability threshold to the number of probability values ​​in the probability value sequence; in response to determining that the ratio is greater than or equal to the preset threshold, determine the initial variable window as a target variable window. The adjustment unit 303 is configured to: adjust the initial variable window and the initial target detection model in response to the determination ratio being less than the preset threshold, take the processed speed limit sign image sequence as the sample speed limit sign image sequence, determine the adjusted initial variable window and the adjusted initial target detection model as the initial variable window and the initial target detection model respectively, and perform the determination step again.

[0092] It is understandable that the units described in the image cleaning apparatus 300 are similar to those in the reference. Figure 1The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the image cleaning apparatus 300 and the units contained therein, and will not be repeated here.

[0093] The following is for reference. Figure 4 It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 400 suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0094] like Figure 4 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. Processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 305 is also connected to bus 404.

[0095] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 4 Each box shown can represent a device or multiple devices as needed.

[0096] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0097] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0098] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0099] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: control a data acquisition vehicle to acquire data from speed limit signs, obtaining a sequence of speed limit sign images; select sample speed limit sign image sequences from the aforementioned speed limit sign image sequence, and perform the following determination steps: sequentially input the sample speed limit sign image sequences into an initial target detection model to obtain a probability value sequence, wherein the probability values ​​in the probability value sequence represent the probability that the speed limit sign exists in the sample speed limit sign image; determine a target time point based on the aforementioned probability value sequence; create an initial variable window based on the aforementioned target time point, wherein the initial variable window is used to determine the window of the sample speed limit sign images to be cleaned; and so on. Based on the aforementioned initial variable window, at least one image sequence is generated; the at least one image sequence is cleaned to obtain a processed speed limit sign image sequence; the ratio of the number of probability values ​​greater than or equal to a preset probability threshold in the probability value sequence to the total number of probability values ​​in the probability value sequence is determined; in response to determining that the ratio is greater than or equal to the preset threshold, the initial variable window is determined as a target variable window; in response to determining that the ratio is less than the preset threshold, the initial variable window and the initial target detection model are adjusted, the processed speed limit sign image sequence is used as a sample speed limit sign image sequence, the adjusted initial variable window and the adjusted initial target detection model are determined as the initial variable window and the initial target detection model, respectively, and the determination step is performed again.

[0100] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0102] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a data acquisition unit, an execution unit, and an adjustment unit. The names of these units do not necessarily limit the specific unit; for example, a data acquisition unit may also be described as "a unit that controls a data acquisition vehicle to acquire data from speed limit signs and obtain a sequence of speed limit sign images."

[0103] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0104] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An image cleaning method, comprising: controlling a collection vehicle to collect data for speed limit signs to obtain a sequence of speed limit sign images; selecting a sample sequence of speed limit sign images from the sequence of speed limit sign images, and performing the following determination steps: inputting the sample sequence of speed limit sign images into an initial target detection model in sequence to obtain a sequence of probability values, wherein a probability value in the sequence of probability values represents a probability value of the presence of the speed limit sign in a sample speed limit sign image; determining a target time point according to the sequence of probability values; creating an initial variable window according to the target time point, wherein the initial variable window is used to determine a window of the sample speed limit sign image to be cleaned; generating at least one image sequence according to the initial variable window; performing cleaning processing on the at least one image sequence to obtain a sequence of processed speed limit sign images; determining a ratio of a number of probability values in the sequence of probability values that are greater than or equal to a preset probability threshold to a number of probability values in the sequence of probability values; in response to determining that the ratio is greater than or equal to a preset threshold, determining the initial variable window as a target variable window; in response to determining that the ratio is less than the preset threshold, adjusting the initial variable window and the initial target detection model, taking the sequence of processed speed limit sign images as the sample sequence of speed limit sign images, and respectively determining the adjusted initial variable window and the adjusted initial target detection model as the initial variable window and the initial target detection model, and performing the determination steps again.

2. The method of claim 1, wherein, The determination of the target time point according to the sequence of probability values comprises: selecting probability values greater than or equal to the preset probability threshold from the sequence of probability values to obtain a target sequence of probability values; determining a target sequence of sample speed limit sign images corresponding to the target sequence of probability values as a target sequence of sample speed limit sign images; determining a collection time point corresponding to a target sample speed limit sign image located at an initial position in the target sequence of sample speed limit sign images as the target time point.

3. The method of claim 2, wherein, The creation of the initial variable window according to the target time point comprises: determining a time point corresponding to a difference from the target time point within a first preset time range as a start time point; selecting a sample speed limit sign image corresponding to the start time point from the sample sequence of speed limit sign images as a start sample speed limit sign image; determining a time point corresponding to a difference from the target time point within a second preset time range as an end time point; selecting a sample speed limit sign image corresponding to the end time point from the sample sequence of speed limit sign images as an end sample speed limit sign image; determining a sequence of sample speed limit sign images between the start sample speed limit sign image and the end sample speed limit sign image in the sample sequence of speed limit sign images as a sequence of sample speed limit sign images included in the initial variable window.

4. The method of claim 3, wherein, The generation of the at least one image sequence according to the initial variable window comprises: based on the first probability value sequence, wherein the first probability value sequence is a probability value sequence between the target time point and a sequence termination time point in the probability value sequence, and the sequence termination time point is a time point corresponding to a probability value at a termination position of the probability value sequence, the following sequence determination step is performed: in response to determining that the initial variable window does not include a sample speed limit sign image sequence corresponding to the same acquisition time point as the sequence termination time point, expanding the initial variable window to obtain an expanded variable window and an expanded window, wherein the expanded window is a window corresponding to a part of the expanded variable window larger than the initial variable window; in response to determining that there is no probability value greater than or equal to the preset probability threshold in the probability value sequence corresponding to the sample speed limit sign image sequence included in the expanded window, adding the sample speed limit sign image sequence included in the expanded variable window to at least one preset image sequence; in response to determining that there is a second probability value greater than or equal to the preset probability threshold in the second probability value sequence, screening the second probability value greater than or equal to the preset probability value from the second probability value sequence to obtain a target screening probability value sequence, wherein the second probability value sequence is a probability value sequence between an expanded probability value and a probability value corresponding to the sequence termination time point in the probability value sequence, and the expanded probability value is a probability value corresponding to a sample speed limit sign image sequence at a termination position in the expanded variable window; creating a sample variable window for a sample speed limit sign image corresponding to a probability value at a starting position in the target screening probability value sequence; in response to determining that the sample variable window includes a sample speed limit sign image corresponding to the same acquisition time point as the sequence termination time point, determining the at least one preset image sequence as the at least one image sequence; in response to determining that the sample variable window does not include a sample speed limit sign image corresponding to the same acquisition time point as the sequence termination time point, determining a probability value sequence between a window termination time point and the sequence termination time point in the probability value sequence as a first probability value sequence, and performing the sequence determination step again, wherein the window termination time point is a time point corresponding to a sample speed limit sign image at a termination position of the sample variable window.

5. The method of claim 4, wherein, The expanding of the initial variable window to obtain an expanded variable window and an expanded window includes: based on the initial variable window, the following expansion step is performed: determining a probability value corresponding to a sample speed limit sign image between the target time point and the termination time point as a window probability value sequence; in response to determining that there is a window probability value greater than or equal to the preset probability threshold in the window probability value sequence, determining at least one window probability value greater than or equal to the preset probability threshold as a target window probability value sequence; determining a target window probability value at a termination position of the target window probability value sequence as a target termination window probability value; A sample speed limit sign image corresponding to the target termination window probability value is collected at the adjustment time point; According to the adjustment time point, the initial variable window is expanded to obtain an expanded variable window and the expanded window; In response to determining that there is no probability value greater than or equal to the preset probability threshold in the probability value sequence corresponding to the sample speed limit sign image sequence included in the expanded window, the expanding step is ended.

6. The method of claim 5, wherein, The method further comprises: In response to determining that there is a probability value greater than or equal to the preset probability threshold in the probability value sequence corresponding to the sample speed limit sign image sequence included in the expanded window, the collection time point of the sample speed limit sign image located at the termination position in the initial variable window is determined as a target time point, the expanded variable window is determined as the initial variable window, and the expanding step is executed again. 7.An image cleaning device, comprising: a control unit configured to control a collection vehicle to collect data for a speed limit sign to obtain a speed limit sign image sequence; an execution unit configured to select a sample speed limit sign image sequence from the speed limit sign image sequence, and perform the following determination step: inputting the sample speed limit sign image sequence into an initial target detection model in sequence to obtain a probability value sequence, wherein a probability value in the probability value sequence represents a probability value of the presence of the speed limit sign in the sample speed limit sign image; determining a target time point according to the probability value sequence; creating an initial variable window according to the target time point, wherein the initial variable window is used to determine a window of a sample speed limit sign to be cleaned; generating at least one image sequence according to the initial variable window; performing cleaning processing on the at least one image sequence to obtain a processed speed limit sign image sequence; determining a ratio of a number of probability values greater than or equal to a preset probability threshold in the probability value sequence to a number of probability values in the probability value sequence; and in response to determining that the ratio is greater than or equal to a preset threshold, determining the initial variable window as a target variable window; an adjustment unit configured to, in response to determining that the ratio is less than the preset threshold, adjust the initial variable window and the initial target detection model, take the processed speed limit sign image sequence as a sample speed limit sign image sequence, and respectively determine the adjusted initial variable window and the adjusted initial target detection model as the initial variable window and the initial target detection model, and execute the determination step again. 8.An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-6.

9. A computer readable medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-6. The computer program is executed by the processor to implement the method of any one of claims 1-6.

Images