Camera cleaning method, system, electronic device and storage medium

Abstract

The present invention discloses a camera cleaning method, system, electronic device and storage medium, and relates to the field of image processing technology. The method comprises: obtaining a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively training a target detection model through the labeled data set to obtain a stain type recognition model; obtaining a stain image to be recognized on a camera of a target device; performing stain recognition on the stain image to be recognized through the stain type recognition model to obtain stain information; and controlling a cleaning device of the target device to clean the camera according to the stain information. In this way, a more accurate intersection-over-union ratio can optimize the training direction of the target detection model, thereby obtaining a stain type recognition model with higher recognition accuracy, so as to perform more efficient camera cleaning.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a camera cleaning method, system, electronic device and storage medium. Background Art

[0002] With the development of visual technology, cameras have been used in a variety of different environments. Different application environments may cause contaminants to deposit on the camera surface, thus affecting the normal operation of the equipment where the camera is located. Especially in the application environment of firefighting robots, the speed and nature of camera contamination will change significantly with the development stage of the fire scale. At present, a single cleaning method often cannot flexibly adapt to the diverse and sudden pollution challenges in different application environments.

[0003] In summary, the current automatic cleaning method of the fire robot camera in a fire scene is not conducive to the operation of the fire robot. Summary of the invention

[0004] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and to provide a camera cleaning method, system, electronic device and storage medium, which are used to judge different stain information through image processing, so as to perform camera cleaning in a targeted manner according to the stain information.

[0005] The present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a camera cleaning method, the method comprising:

[0007] Obtain a labeled dataset, which is a camera stain image dataset with stain category labeled bounding boxes;

[0008] The target detection model is iteratively trained through the labeled data set until the target loss result meets the preset loss threshold to obtain a stain type recognition model; wherein the i-th round of training includes: the target detection model of the i-th round is trained according to the labeled data set to output the predicted bounding box corresponding to the i-th round, the predicted bounding box output by the target detection model of the i-th round and the real bounding box in the labeled data set are calculated using the intersection-over-union calculation formula to obtain the target loss result of the i-th round, and the model parameters of the target detection model are optimized according to the target loss result of the i-th round to obtain the target detection model of the i+1th round;

[0009] Obtaining a stain image to be identified from a camera of a target device;

[0010] The stain type recognition model is used to perform stain recognition on the stain image to be recognized, thereby obtaining stain information;

[0011] The cleaning device of the target device is controlled to clean the camera according to the stain information.

[0012] In one implementation, the intersection-over-union ratio calculation formula is: , , , ,in, is the true bounding box in the annotation dataset, is the predicted bounding box output by the target detection model, is the intersection area of ​​the true bounding box and the predicted bounding box, is the union area of ​​the true bounding box and the predicted bounding box, is the intersection and comparison result, is the width of the true bounding box, is the height of the true bounding box, is the width of the predicted bounding box, is the height of the predicted bounding box, The target pixel The distance to the line of the real bounding box, is the distance from the intersection of the vertical line and the midpoint of the real bounding box to the midpoint of the real bounding box. The vertical line passes through the target pixel , and are kernel functions, and Used to determine the target pixel Whether it is in the intersection of the true bounding box and the predicted bounding box.

[0013] In one embodiment, the predicted bounding box output by the target detection model of the i-th round and the real bounding box in the labeled data set are calculated using the intersection-over-union ratio calculation formula to obtain the target loss result of the i-th round, including:

[0014] Calculate the intersection-and-union ratio result of the i-th round according to the prediction bounding box of the i-th round and its corresponding true bounding box according to the intersection-and-union ratio calculation formula;

[0015] The target loss result of the i-th round is determined according to the intersection-combination ratio result of the i-th round; the calculation formula of the target loss result is: ,in is the target loss result, is a positive sample pair in the labeled dataset.

[0016] In one embodiment, the target detection model includes a lightweight backbone network and a global attention mechanism, and the target detection model of the i-th round is trained according to the labeled data set to output a predicted bounding box corresponding to the i-th round, including:

[0017] The feature extraction of the labeled data set is performed through a lightweight backbone network to obtain the target features;

[0018] Enhance the target features through the global attention mechanism;

[0019] The enhanced target features are learned through a lightweight backbone network to obtain the predicted bounding box corresponding to the i-th round.

[0020] In one embodiment, stain identification is performed on the stain image to be identified by using a stain type identification model to obtain stain information, including:

[0021] If the cleaning device is started, the stain type recognition model is used to perform stain recognition on the stain image to be recognized to obtain stain information;

[0022] If the cleaning device is not started, the stain image to be identified is processed based on the background anti-differential algorithm to obtain a static stain image; the static stain image is subjected to stain identification through the stain type identification model to obtain stain information.

[0023] In one embodiment, the stain information includes the type of stain, and controlling a cleaning device of the target device to clean the camera according to the stain information includes:

[0024] Determining a target cleaning component in a cleaning device according to the type of stain;

[0025] Control the target cleaning component to clean the camera.

[0026] In one embodiment, the stain information further includes the stain position and the stain area, and controlling the target cleaning component to clean the camera includes:

[0027] Determine the stain position and stain area according to the recognition bounding box pixels corresponding to the stain image to be recognized in the stain recognition process;

[0028] Determine the cleaning location based on the location of the stain;

[0029] Determine the cleaning method based on the area of ​​the stain;

[0030] Based on the cleaning mode, the target cleaning component is controlled to clean the cleaning position.

[0031] In a second aspect, the present invention provides a camera cleaning system, the system comprising:

[0032] The control end is used to obtain a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively train the target detection model through the labeled data set until the target loss result meets the preset loss threshold, so as to obtain a stain type recognition model; wherein the i-th round of training includes: the target detection model of the i-th round is trained according to the labeled data set to output the predicted boundary box corresponding to the i-th round, and the predicted boundary box output by the target detection model of the i-th round and the real boundary box in the labeled data set are calculated by using the intersection and union ratio calculation formula to obtain the target loss result of the i-th round, and the model parameters of the target detection model are optimized according to the target loss result of the i-th round to obtain the target detection model of the i+1th round; obtain the stain image to be identified on the camera of the target device; perform stain recognition on the stain image to be identified through the stain type recognition model to obtain stain information; and control the cleaning device of the target device to clean the camera according to the stain information;

[0033] A cleaning device is used to clean the camera according to the stain information; wherein the cleaning device includes an outer protective cover, a cleaning component, an inner protective cover and a base; the outer protective cover is used to protect the cleaning component and the inner protective cover; the cleaning component is used to protect the inner protective cover and avoid blocking the camera; the inner protective cover is used to rotate relative to the cleaning component to prevent the camera from directly contacting the stain and enable the cleaning component to remove the stain on the inner protective cover; the base is used to discharge the waste liquid generated by the cleaning component and prevent stains.

[0034] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the camera cleaning method of the first aspect is implemented.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which implements the camera cleaning method of the first aspect when the computer program is executed by a processor.

[0036] The camera cleaning method, system, electronic device and storage medium disclosed in the present invention obtain a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively train a target detection model through the labeled data set to obtain a stain type recognition model; obtain a stain image to be recognized on the camera of a target device; perform stain recognition on the stain image to be recognized through the stain type recognition model to obtain stain information; and control a cleaning device of the target device to clean the camera according to the stain information. In this way, by iteratively training a target detection model including an optimized intersection-and-union ratio calculation formula, a more accurate intersection-and-union ratio can optimize the training direction of the target detection model, thereby obtaining a stain type recognition model with higher recognition accuracy, so as to perform more efficient camera cleaning; at the same time, according to the recognition result of the stain type recognition model, that is, the predicted labeling box, the stain information of the current camera can be obtained, so that the camera cleaning can be carried out in a targeted manner according to the current stain information. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0038] Figure 1 A schematic diagram of a process of the camera cleaning method proposed in this embodiment is shown;

[0039] Figure 2 A schematic diagram of a real bounding box proposed in this embodiment is shown;

[0040] Figure 3 A schematic structural diagram of the cleaning device proposed in this embodiment is shown;

[0041] Figure 4 A structural schematic diagram of the camera cleaning system proposed in this embodiment is shown.

[0042] Figure ID:

[0043] 201-real boundary frame; 202-real boundary frame center line; 203-vertical line; 301-outer protective cover; 302-cleaning component; 303-inner protective cover; 304-base; 400-camera cleaning system; 401-control terminal; 402-cleaning device. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0045] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0047] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0048] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. Terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0049] Example 1

[0050] The disclosed embodiment provides a camera cleaning method for determining different stain information through visual recognition, thereby performing camera cleaning in a targeted manner according to the stain information.

[0051] See also Figure 1 The camera cleaning method includes steps S101 to S105, and each step is described in detail below.

[0052] Step S101, obtaining a labeled dataset, where the labeled dataset is a camera stain image dataset with a stain category labeled boundary box.

[0053] In this embodiment, before building the model, it is necessary to collect a large number of camera stain images of fire-fighting robots at the fire scene, and use the image annotation tool (labelimg) to label the stains on the camera stain images according to the different types of stains, thereby obtaining a labeled data set with stain category labeled bounding boxes, so that the model can learn the image features of different stain types.

[0054] It should be noted that there are two main sources for image collection. One is to search for existing data sets or scattered pictures on the Internet. The other is to build experiments to simulate the real environment of a fire scene through literature reading and data query, take camera pollution pictures by ourselves, and simulate different types of fires to make the data set as large as possible.

[0055] It should be noted that the camera cleaning method proposed in this embodiment is also applicable to the cleaning of cameras of other devices, so the corresponding camera images can be obtained according to needs for subsequent model training.

[0056] Step S102, iteratively training the target detection model by annotating the data set until the target loss result meets the preset loss threshold, so as to obtain a stain type recognition model.

[0057] Among them, the i-th round of training includes: the i-th round target detection model is trained according to the labeled data set to output the predicted bounding box corresponding to the i-th round, the predicted bounding box output by the i-th round target detection model and the real bounding box in the labeled data set are calculated using the intersection-over-union calculation formula to obtain the i-th round target loss result, and the model parameters of the target detection model are optimized according to the i-th round target loss result to obtain the i+1-th round target detection model.

[0058] In this embodiment, the target detection model may be a version of YOLO (You Only Look Once) YOLOv8. The target detection model is iteratively trained according to the labeled data set to obtain a stain type recognition model that can identify and label the stain types. The preset loss threshold may be set according to the model performance requirements.

[0059] In a specific embodiment, the target detection model includes an intersection-over-union calculation formula: , , , ,in, is the true bounding box in the annotation dataset, is the predicted bounding box output by the target detection model, is the intersection area of ​​the true bounding box and the predicted bounding box, is the union area of ​​the true bounding box and the predicted bounding box, is the intersection and comparison result, is the width of the true bounding box, is the height of the true bounding box, is the width of the predicted bounding box, is the height of the predicted bounding box, The target pixel The distance to the line of the real bounding box, is the distance from the intersection of the vertical line and the midpoint of the real bounding box to the midpoint of the real bounding box. The vertical line passes through the target pixel , and are kernel functions, and Used to determine the target pixel Whether it is in the intersection of the true bounding box and the predicted bounding box.

[0060] In this embodiment, based on the intersection-over-union ratio calculation formula, the target detection model is iteratively trained for multiple rounds according to the labeled data set. The intersection-over-union ratio calculation formula of the target detection model is: , , , . is the true bounding box in the annotation dataset, is the predicted bounding box output by the target detection model, is the intersection area of ​​the true bounding box and the predicted bounding box, is the union area of ​​the true bounding box and the predicted bounding box, is the intersection and comparison result, is the width of the predicted bounding box, is the height of the predicted bounding box, and are kernel functions respectively. Figure 2 , is the width of the ground-truth bounding box 201, is the height of the ground-truth bounding box 201, The target pixel The distance to the line 202 of the real bounding box, is the intersection of the vertical line 203 and the middle line 202 of the real bounding box The distance to point c in the ground truth bounding box, where the vertical line 203 passes through the target pixel .

[0061] and Used to determine the target pixel of the judgment point Is it in the intersection area of ​​the real bounding box and the predicted bounding box? Inside.

[0062] when and are close to 1, then Approaching 1, we know the target pixel Within the true bounding box; when and / or Approaching 0, Approaching 0, we know the target pixel Not within the true bounding box; when and are close to 1, then Approaching 1, we know the target pixel Within the predicted bounding box; when and / or Approaching 0, Approaching 0, we know the target pixel Not within the predicted bounding box.

[0063] like and tend to 1, then we know that the target pixel In the intersection area of ​​the true bounding box and the predicted bounding box inside, if and / or tends to 0, then we know that the target pixel Not in the intersection of the true bounding box and the predicted bounding box Inside.

[0064] Among them, the calculation formula of the kernel function can be , Used to control the target pixel sensitivity.

[0065] It should be noted that by calculating the nearest boundary distance of the target pixel in the height direction and the nearest boundary distance in the width direction of the real bounding box and the predicted bounding box, and using the kernel function to calculate the similarity of the nearest boundary distances of the real bounding box and the predicted bounding box in the two directions, and then multiplying the similarities in the two directions, the target pixel Evaluation values ​​under the true bounding box and predicted bounding box and . Further, by evaluating the value and Determine the target pixel Whether it is in the intersection area of ​​the real bounding box and the predicted bounding box, and for the target pixel in the intersection area By summing up the products of the evaluation values, a more accurate intersection area of ​​the true bounding box and the predicted bounding box can be obtained, thereby optimizing the training direction of the target detection model, prompting the target detection model to learn more accurate positioning capabilities, and then obtaining a stain type recognition model with stronger recognition capabilities.

[0066] It should be noted that the intersection-over-union calculation method of this embodiment further integrates support for extreme-angle rotation boxes, more sophisticated size adaptability, or more complex shape matching metrics to solve more diverse detection challenges; a more sophisticated gradient adjustment mechanism is designed to ensure that stable and effective gradient signals can be provided in various difficult-to-detect situations (such as extremely small targets, overlapping targets, etc.), accelerate training and improve the accuracy of the final model; and introduce a more advanced adaptive mechanism to dynamically adjust the weight of the loss function according to the specific attributes of the target (such as scale, shape, category) to enhance the model's generalization ability on unseen data; and further exploration is made on optimizing computing costs to ensure that while improving detection performance, the computing burden is maintained or reduced, which is particularly important for real-time applications.

[0067] It should be noted that the union area of ​​the true bounding box and the predicted bounding box is obtained by subtracting the intersection area of ​​the true bounding box and the predicted bounding box from the sum of the true bounding box area and the predicted bounding box area. A more accurate union area can be calculated based on a more accurate intersection area, thereby further optimizing the optimization direction of the target detection model. The calculation formula for the union area is .

[0068] In a specific embodiment, the intersection-and-union ratio result of the i-th round is calculated according to the prediction bounding box of the i-th round and its corresponding true bounding box according to the intersection-and-union ratio calculation formula; the target loss result is determined according to the intersection-and-union ratio result of the i-th round; the calculation formula of the target loss result is ,in is the target loss result, is a positive sample pair in the labeled dataset.

[0069] In this embodiment, in each training of the iterative training process of the model, the intersection-and-union ratio result is calculated according to the intersection-and-union ratio calculation formula in the previous text based on the predicted bounding box of the target detection model. The target loss result is determined based on the intersection-and-union ratio result, and the calculation formula is: ,in is the target loss result, is a positive sample pair in the labeled dataset. In this way, the target loss function constructed according to the optimized intersection-over-union ratio measures the difference between the true bounding box and the predicted bounding box, thereby guiding the training direction of the target detection model according to the obtained target loss result, so as to perform back propagation and update the model parameters, and thus obtain a stain type recognition model with higher recognition accuracy.

[0070] In a specific embodiment, the target detection model includes a lightweight backbone network and a global attention mechanism. The target detection model of the i-th round is trained according to the labeled data set to output the predicted bounding box corresponding to the i-th round, including: extracting features of the labeled data set through the lightweight backbone network to obtain target features; enhancing the target features through the global attention mechanism; and performing feature learning on the enhanced target features through the lightweight backbone network to obtain the predicted bounding box corresponding to the i-th round.

[0071] In this embodiment, forward propagation is required before calculating the loss value. Specifically, after the labeled data set is input into the target detection model, the lightweight backbone network of the target detection model extracts features from the labeled data set. At the same time, the global attention mechanism in the target detection model enhances the extracted target features, so that the lightweight backbone network learns features based on the enhanced target features, and then inputs the learned features into the output layer to generate a prediction result, i.e., a predicted bounding box.

[0072] It should be noted that the lightweight backbone network can be MobileNetV3, which greatly improves the computational efficiency while maintaining high performance, ensuring fast and accurate target detection in resource-constrained environments. A global attention mechanism is inserted into the lightweight backbone network to further enhance the target feature map so that each level can obtain more comprehensive contextual information. At the same time, the global attention mechanism is inserted before the feature pyramid network (FPN) in the neck area. The global attention mechanism can help fuse feature maps of different scales and promote contextual interaction between scales, which is particularly important for detecting targets of different sizes.

[0073] Step S103, obtaining the stain image to be identified on the camera of the target device.

[0074] In this embodiment, an image of stains to be identified is obtained from a camera when a firefighting robot is operating in a real fire scene.

[0075] Step S104, stain recognition is performed on the stain image to be recognized by using a stain type recognition model to obtain stain information.

[0076] In this embodiment, the stain type recognition model is used to predict and annotate the stain image to be identified, and the stain information on the fire robot camera at this time can be obtained according to the pixel information of the predicted annotation box and the stain category information. Since the stain type recognition model is a model trained under the guidance of the optimized intersection-union ratio, the stain information can be identified more quickly and accurately at this time.

[0077] In a specific embodiment, step S104 includes: if the cleaning device is started, stain identification is performed on the stain image to be identified through a stain type recognition model to obtain stain information; if the cleaning device is not started, the stain image to be identified is processed based on a background anti-differential algorithm to obtain a static stain image; stain identification is performed on the static stain image through a stain type recognition model to obtain stain information.

[0078] Please note that, see Figure 3 The cleaning device includes an outer protective cover 301, a cleaning component 302, an inner protective cover 303 and a base 304. The outer protective cover 301 is used to protect the cleaning component 302 and the inner protective cover 303 from damage, and the inner and outer protective covers are made of high-strength materials and seal the space where the camera is located to ensure that the camera is not physically or chemically damaged in a complex fire environment; the cleaning component 302 protects the inner protective cover 303 by scraping strips, spraying cleaning agents, etc., and can remain stationary relative to the camera to avoid blocking the camera; the inner protective cover 303 is used to rotate relative to the cleaning component through its own rotation, so that the cleaning component removes the stains on the inner protective cover and prevents the camera from directly contacting the stains, thereby fixing the relative positions of the outer protective cover, the cleaning device and the camera to ensure that the camera can always keep the field of view clean in the fire scene; the base 304 is provided with a drainage slit to ensure the discharge of waste liquid generated by the cleaning component, and the high-pressure gas is sprayed into the air to prevent stains.

[0079] Therefore, when the firefighting robot just enters the fire scene, the inner protective cover 303 is in a clean state, the inner protective cover 303 does not rotate, and the relationship with the camera is relatively static. After entering the fire scene, the firefighting robot is generally in a moving state. At this time, if some small stains or dust fall on the inner protective cover 303, it is easy to merge with the background. The background anti-differential algorithm can be used to remove the constantly changing background, obtain a static stain image on the inner protective cover 303, and input the static stain image into the stain type recognition model for recognition, which is conducive to quickly starting the cleaning device in the early stage and ensuring the cleanliness of the camera. After the cleaning device is started, the inner protective cover 303 rotates, and the stains on it also rotate accordingly. At this time, the background anti-differential algorithm is not very meaningful, so the use of the anti-differential algorithm is stopped, and the stain image to be identified is directly input into the stain type recognition model for identification.

[0080] Step S105, controlling a cleaning device of the target device to clean the camera according to the stain information.

[0081] In this embodiment, the cleaning device is controlled to clean the fire-fighting robot according to the stain information, so that the stain information on the camera at this time can be quickly determined according to the identification annotation box information of the stain type identification model, thereby ensuring in real time that the cleaning status of the camera does not affect the operating status of the fire-fighting robot.

[0082] In a specific embodiment, the stain information includes the type of stain, and step S105 includes: determining a target cleaning component in the cleaning device according to the type of stain; and controlling the target cleaning component to clean the camera.

[0083] In this embodiment, the target cleaning agent and target cleaning component required to clean the stains of this stain category on the camera at this time can be determined based on the more accurate stain category output by the stain category model, so that the stains on the camera can be quickly cleaned by using targeted cleaning agents and cleaning components.

[0084] It should be noted that the cleaning component 302 includes a microfiber cloth, a non-alkaline detergent solution, 75% medical alcohol, pure water, a brush, a scraper, etc. The base 304 is provided with a drainage slit to facilitate the discharge of the liquid cleaned by the scraper to prevent the accumulation of the liquid. According to the type of stain, different cleaning components 302 can be controlled to start, and the corresponding relationship of cleaning is as follows: dust-fiber cloth, smoke-non-alkaline detergent solution, oil-alcohol, water vapor-fiber cloth, chemical pollution-pure water, biological pollution-alcohol, spider webs or other biological attachments-brush. According to the above correspondence, when the corresponding stain is identified, the corresponding cleaning component is activated. However, for example, water vapor and dust do not require the cleaning component to be activated, and cleaning can be achieved by rotating the inner protective cover. Therefore, the specific response plan is: when dust, water vapor, spider webs or other biological attachments are identified, the rotation of the inner protective cover is activated; when smoke is identified, a non-alkaline detergent solution is released, and the inner protective cover starts to rotate at the same time; when oil is identified, alcohol is released, and the inner protective cover starts to rotate at the same time; when chemical pollution is identified, pure water is released, and the inner protective cover starts to rotate at the same time; when biological pollution is identified, alcohol is released, and the inner protective cover starts to rotate at the same time. In this way, by mixing and using a variety of cleaning agents, the cleaning of the camera can be completed more efficiently.

[0085] In a specific embodiment, the stain information also includes the stain location and the stain area, and controlling the target cleaning component to clean the camera includes: determining the stain location and the stain area according to the identification boundary box pixels corresponding to the stain image to be identified in the stain identification process; determining the cleaning position according to the stain location; determining the cleaning method according to the stain area; and based on the cleaning method, controlling the target cleaning component to clean the cleaning position.

[0086] In this embodiment, the stain type recognition model will mark the stain when identifying the stain image to be identified. Therefore, the stain position on the camera at this time can be determined according to the marked boundary box, and the total number of pixels in the marked boundary box can be divided by the resolution of the camera to obtain the percentage of the camera's polluted field of view at the current moment, so as to determine the current stain area according to the percentage.

[0087] Furthermore, according to the stain position information, the cleaning position of the cleaning component detergent is controlled according to the situation where detergent needs to be sprayed, ensuring that it can be sprayed more accurately to the height where the stain is located, thereby cleaning the stain more accurately and reducing the waste of detergent.

[0088] At the same time, based on the stain area information, the area size of different stains at each moment can be obtained, thereby determining a targeted cleaning method, such as the detergent injection rate.

[0089] Exemplarily, whether to change the amount of detergent sprayed depends on whether the current amount of spraying can effectively control the stain. Therefore, by dividing the difference in the stain area between two frames by the time difference between the two frames, the change gradient of a stain can be obtained, which is recorded as , the calculation formula is: .

[0090] Among them, ∆s is the difference in the stain area between the two frames, and ∆t is the time difference between the two frames. Because the value representing the size of the area is the percentage of the recognition frame to the total area of ​​the camera, the range of the stain area size is 0-1; and the time difference between the two frames needs to be calculated based on fps. Assuming 30 frames, the time difference between the two frames should be 1 / 30≈0.033, and the range of the gradient value should be -1 / (1 / 30)- 1 / (1 / 30), that is, -30-30. When When >=0, it means that the stain is not effectively controlled at present, and the area of ​​the stain is expanding or remains unchanged. Therefore, it can be seen that the cleaning speed is not as high as the contamination speed. It is necessary to increase the spray speed of the detergent or the rotation speed of the inner protective cover to speed up the cleaning speed of the stain. If the cleaning device has a good cleaning effect on the stain within 0.5-2 seconds, at this time a large stain accounts for about 20% of the video image, then the large stain is removed within one second, so, Need to satisfy 0.2 / 2<= <=0.2 / 0.5, that is, 0.1<= <=0.4. Therefore, when When it is less than 0.1, the cleaning rate needs to be slightly increased; when When it is greater than 0.4, the release of the detergent can be appropriately slowed down; when When >=1, the cleaning rate is too fast, the cleaning agent is consumed too much, and the rate needs to be greatly reduced. The cleaning rate adjustment scheme is shown in Table 1 below.

[0091] Table 1:

[0092]

[0093] The camera cleaning method proposed in this embodiment obtains a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively trains the target detection model through the labeled data set to obtain a stain type recognition model; obtains the stain image to be recognized on the camera of the target device; performs stain recognition on the stain image to be recognized through the stain type recognition model to obtain stain information; and controls the cleaning device of the target device to clean the camera according to the stain information. In this way, by iteratively training the target detection model including the optimized intersection-and-union ratio calculation formula, a more accurate intersection-and-union ratio can optimize the training direction of the target detection model, thereby obtaining a stain type recognition model with higher recognition accuracy, so as to perform more efficient camera cleaning; at the same time, according to the recognition result of the stain type recognition model, that is, the predicted labeling box, the stain information of the current camera can be obtained, so as to perform camera cleaning in a targeted manner according to the current stain information.

[0094] Example 2

[0095] In addition, the present disclosure provides a camera cleaning system 400, see Figure 4 , the system includes:

[0096] The control terminal 401 is used to obtain a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively train the target detection model through the labeled data set until the target loss result meets the preset loss threshold to obtain a stain type recognition model; wherein the i-th round of training includes: the target detection model of the i-th round is trained according to the labeled data set to output the predicted boundary box corresponding to the i-th round, and the predicted boundary box output by the target detection model of the i-th round and the real boundary box in the labeled data set are calculated using the intersection-and-union ratio calculation formula to obtain the target loss result of the i-th round, and the model parameters of the target detection model are optimized according to the target loss result of the i-th round to obtain the target detection model of the i+1th round; obtain the stain image to be identified on the camera of the target device; perform stain recognition on the stain image to be identified through the stain type recognition model to obtain stain information; control the cleaning device of the target device to clean the camera according to the stain information; the intersection-and-union ratio calculation formula is: , , , ,in, is the true bounding box in the annotation dataset, is the predicted bounding box output by the target detection model, is the intersection area of ​​the true bounding box and the predicted bounding box, is the union area of ​​the true bounding box and the predicted bounding box, is the intersection and comparison result, is the width of the true bounding box, is the height of the true bounding box, is the width of the predicted bounding box, is the height of the predicted bounding box, The target pixel The distance to the line of the real bounding box, is the distance from the intersection of the vertical line and the midpoint of the real bounding box to the midpoint of the real bounding box. The vertical line passes through the target pixel , and are kernel functions, and Used to determine the target pixel Whether it is within the intersection of the real bounding box and the predicted bounding box;

[0097] The cleaning device 402 is used to clean the camera according to the stain information; wherein the cleaning device includes an outer protective cover, a cleaning component, an inner protective cover and a base; the outer protective cover is used to protect the cleaning component and the inner protective cover; the cleaning component is used to protect the inner protective cover and avoid blocking the camera; the inner protective cover is used to rotate relative to the cleaning component to prevent the camera from directly contacting the stain and enable the cleaning component to remove the stain on the inner protective cover; the base is used to discharge the waste liquid generated by the cleaning component and prevent stains.

[0098] Optionally, the control terminal 401 is further used to calculate the intersection-and-union ratio result of the i-th round according to the prediction bounding box of the i-th round and its corresponding real bounding box according to the intersection-and-union ratio calculation formula; determine the target loss result according to the intersection-and-union ratio result; the calculation formula of the target loss result is: ,in is the target loss result, is a positive sample pair in the labeled dataset.

[0099] Optionally, the target detection model includes a lightweight backbone network and a global attention mechanism. The control end 401 is also used to extract features of the labeled data set through the lightweight backbone network to obtain target features; enhance the target features through the global attention mechanism; and perform feature learning on the enhanced target features through the lightweight backbone network to obtain a predicted bounding box corresponding to the i-th round.

[0100] Optionally, the control end 401 is also used to, if the cleaning device is started, perform stain recognition on the stain image to be recognized through a stain type recognition model to obtain stain information; if the cleaning device is not started, process the stain image to be recognized based on a background anti-differential algorithm to obtain a static stain image; and perform stain recognition on the static stain image through a stain type recognition model to obtain stain information.

[0101] Optionally, the stain information includes the type of stain, and the control end 401 is further used to determine a target cleaning component in the cleaning device according to the type of stain; and control the target cleaning component to clean the camera.

[0102] Optionally, the stain information also includes the stain position and the stain area. The control end 401 is also used to determine the stain position and the stain area according to the recognition boundary box pixels corresponding to the stain image to be identified in the stain identification process; determine the cleaning position according to the stain position; determine the cleaning method according to the stain area; and based on the cleaning method, control the target cleaning component to clean the cleaning position.

[0103] The system provided in the embodiment of the present disclosure can execute the steps of the camera cleaning method provided in Example 1, which will not be described again to avoid repetition.

[0104] The camera cleaning system proposed in this embodiment obtains a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively trains the target detection model through the labeled data set to obtain a stain type recognition model; obtains a stain image to be recognized on the camera of the target device; performs stain recognition on the stain image to be recognized through the stain type recognition model to obtain stain information; and controls the cleaning device of the target device to clean the camera according to the stain information. In this way, by iteratively training the target detection model including the optimized intersection-and-union ratio calculation formula, a more accurate intersection-and-union ratio can optimize the training direction of the target detection model, thereby obtaining a stain type recognition model with higher recognition accuracy, so as to perform more efficient camera cleaning; at the same time, according to the recognition result of the stain type recognition model, that is, the predicted labeling box, the stain information of the current camera can be obtained, so that the camera cleaning can be carried out in a targeted manner according to the current stain information.

[0105] Example 3

[0106] In addition, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the camera cleaning method described in Example 1 is implemented.

[0107] The device provided in the embodiment of the present disclosure can execute the steps of the camera cleaning method provided in Example 1, which will not be described again to avoid repetition.

[0108] Example 4

[0109] The embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, which implements the camera cleaning method described in Embodiment 1 when executed by a processor.

[0110] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0111] The computer-readable storage medium provided in this embodiment can implement the camera cleaning method provided in Example 1, and will not be described again here to avoid repetition.

[0112] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0113] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0114] The above embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it should not be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A camera cleaning method, characterized in that: The method comprises: Acquire a labeled data set, wherein the labeled data set is a camera stain image data set with a stain category labeled boundary box; The target detection model is iteratively trained using the annotated data set until the target loss result meets a preset loss threshold, so as to obtain a stain type recognition model; wherein the i-th round of training includes: the target detection model of the i-th round is trained according to the annotated data set to output a predicted bounding box corresponding to the i-th round, and the predicted bounding box output by the target detection model of the i-th round and the real bounding box in the annotated data set are calculated using an intersection-over-union calculation formula to obtain the target loss result of the i-th round, and the model parameters of the target detection model are optimized according to the target loss result of the i-th round to obtain the target detection model of the i+1th round; Obtaining a stain image to be identified from a camera of a target device; Performing stain recognition on the stain image to be recognized by using the stain type recognition model to obtain stain information; Controlling a cleaning device of the target device to clean the camera according to the stain information; The intersection-over-combination ratio calculation formula is: , , , ,in, is the true bounding box in the labeled dataset, is the predicted bounding box output by the target detection model, is the intersection area of ​​the true bounding box and the predicted bounding box, is the union area of ​​the true bounding box and the predicted bounding box, is the intersection and comparison result, is the width of the ground-truth bounding box, is the height of the ground-truth bounding box, is the width of the predicted bounding box, is the height of the predicted bounding box, The target pixel The distance to the middle line of the real bounding box, the middle line of the real bounding box is parallel to the width of the real bounding box, is the distance from the intersection of the vertical line and the midpoint of the real bounding box to the midpoint of the real bounding box, the vertical line is perpendicular to the midpoint of the real bounding box, and the vertical line passes through the target pixel , and are kernel functions, and Used to determine the target pixel Whether it is within the intersection of the true bounding box and the predicted bounding box.

2. The camera cleaning method according to claim 1, characterized in that: The prediction bounding box output by the target detection model of the i-th round and the real bounding box in the labeled data set are calculated by using the intersection-over-union ratio calculation formula to obtain the target loss result of the i-th round, including: Calculate the intersection-and-union ratio result of the i-th round according to the predicted bounding box of the i-th round and its corresponding true bounding box according to the intersection-and-union ratio calculation formula; The target loss result of the i-th round is determined according to the intersection-and-combination result of the i-th round; the calculation formula of the target loss result is: ,in is the target loss result, is a positive sample pair in the labeled dataset.

3. The camera cleaning method according to claim 1, characterized in that: The target detection model includes a lightweight backbone network and a global attention mechanism. The target detection model of the i-th round is trained according to the labeled data set to output a predicted bounding box corresponding to the i-th round, including: Performing feature extraction on the labeled data set through the lightweight backbone network to obtain target features; Performing feature enhancement on the target feature through the global attention mechanism; The enhanced target features are subjected to feature learning through the lightweight backbone network to obtain the prediction bounding box corresponding to the i-th round.

4. The camera cleaning method according to any one of claims 1 to 3, characterized in that: The step of performing stain recognition on the stain image to be recognized by using the stain type recognition model to obtain stain information includes: If the cleaning device has been started, stain recognition is performed on the stain image to be recognized by using the stain type recognition model to obtain the stain information; If the cleaning device is not started, the stain image to be identified is processed based on a background anti-differential algorithm to obtain a static stain image; stain identification is performed on the static stain image using the stain type identification model to obtain the stain information.

5. The camera cleaning method according to claim 1, characterized in that: The stain information includes the type of stain, and controlling the cleaning device of the target device to clean the camera according to the stain information includes: Determining a target cleaning component in the cleaning device according to the type of the stain; The target cleaning component is controlled to clean the camera.

6. The camera cleaning method according to claim 5, characterized in that: The stain information also includes a stain position and a stain area, and the controlling the target cleaning component to clean the camera includes: Determine the stain position and the stain area according to the recognition boundary box pixels corresponding to the stain image to be recognized in the stain recognition process; determining a cleaning position according to the stain position; Determining a cleaning method according to the stain area; Based on the cleaning mode, the target cleaning component is controlled to clean the cleaning position.

7. A camera cleaning system, characterized in that: The system comprises: A control end is used to obtain a labeled data set, which is a camera stain image data set with a stain category labeled boundary box; iteratively train the target detection model through the labeled data set until the target loss result meets the preset loss threshold to obtain a stain type recognition model; wherein the i-th round of training includes: the target detection model of the i-th round is trained according to the labeled data set to output the predicted boundary box corresponding to the i-th round, the predicted boundary box output by the target detection model of the i-th round and the real boundary box in the labeled data set are calculated using the intersection-and-union ratio calculation formula to obtain the target loss result of the i-th round, and the model parameters of the target detection model are optimized according to the target loss result of the i-th round to obtain the target detection model of the i+1th round; obtain the stain image to be identified on the camera of the target device; perform stain recognition on the stain image to be identified through the stain type recognition model to obtain stain information; control the cleaning device of the target device to clean the camera according to the stain information; the intersection-and-union ratio calculation formula is: , , , ,in, is the true bounding box in the labeled dataset, is the predicted bounding box output by the target detection model, is the intersection area of ​​the true bounding box and the predicted bounding box, is the union area of ​​the true bounding box and the predicted bounding box, is the intersection and comparison result, is the width of the ground-truth bounding box, is the height of the ground-truth bounding box, is the width of the predicted bounding box, is the height of the predicted bounding box, The target pixel The distance to the middle line of the real bounding box, the middle line of the real bounding box is parallel to the width of the real bounding box, is the distance from the intersection of the vertical line and the midpoint of the real bounding box to the midpoint of the real bounding box, the vertical line is perpendicular to the midpoint of the real bounding box, and the vertical line passes through the target pixel , and are kernel functions, and Used to determine the target pixel Whether it is within the intersection of the true bounding box and the predicted bounding box; A cleaning device, used for cleaning the camera according to the stain information; wherein the cleaning device comprises an outer protective cover, a cleaning component, an inner protective cover and a base; the outer protective cover is used to protect the cleaning component and the inner protective cover; the cleaning component is used to protect the inner protective cover and avoid blocking the camera; the inner protective cover is used to rotate relative to the cleaning component to prevent the camera from directly contacting the stain and enable the cleaning component to remove the stain on the inner protective cover; the base is used to discharge the waste liquid generated by the cleaning component and prevent stains.

8. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the camera cleaning method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the camera cleaning method as described in any one of claims 1 to 6.

Images