Liquid spraying control method and device, automatic spraying equipment and storage medium

Abstract

Embodiments of the present application are applicable to the field of artificial intelligence technology, and provide a liquid spraying control method and device, an automatic spraying device and a storage medium. The method comprises: collecting a target image of a to-be-sprayed region by an image sensor arranged on the automatic spraying device; the target image comprises a target object; performing target identification on the target object by using a target identification model to obtain a target category of the target object; the target identification model is obtained by training a preset initial model based on a preset training set; each piece of training data in the preset training set at least comprises a training image containing a training object, a center point of the training object in the training image and an occupied region; if the target category is a preset category, the automatic spraying device stops spraying the to-be-sprayed region; the preset category at least comprises living beings; otherwise, the automatic spraying device continues to spray the to-be-sprayed region. The above method can completely disinfect the to-be-sprayed region.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a liquid spraying control method, device, automatic spraying equipment and storage medium. Background Art

[0002] In recent years, highly contagious and widespread viruses have continued to wreak havoc around the world, severely impacting human safety, social order, and economic stability. Public places with dense crowds and high mobility are often key targets for disinfection. Failure to thoroughly disinfect these environments can cause significant social harm.

[0003] Currently, manual spraying and disinfection often consumes considerable time and labor costs, and can be detrimental to the health of workers. Robotic disinfection typically begins with a model of the area to be sprayed, followed by a planned path for the robot. If an obstacle is detected in the path, the robot stops and its sprayer is deactivated. Otherwise, the robot continues spraying the area.

[0004] However, non-pedestrian obstacles also need to be disinfected. Therefore, existing robots do not recognize and distinguish obstacles when spraying. When an obstacle is identified, spraying stops, resulting in incomplete disinfection of the sprayed area. Summary of the Invention

[0005] The embodiments of the present application provide a liquid spraying control method, device, automatic spraying equipment and storage medium, which can solve the problem of incomplete disinfection of the area to be sprayed due to failure to identify and distinguish obstacles when disinfecting the area to be sprayed.

[0006] In a first aspect, an embodiment of the present application provides a liquid spraying control method, the method comprising:

[0007] The target image of the area to be sprayed is collected by an image sensor provided on the automatic spraying device; the target image includes a target object;

[0008] A target recognition model is used to identify a target object and obtain a target category of the target object; the target recognition model is obtained by training a preset initial model based on a preset training set, the preset training set includes multiple training data, each training data includes at least a training image containing the training object, a center point of the training object in the training image, and an area occupied by the training object;

[0009] If the target category is a preset category, the automatic spraying device is controlled to stop spraying the area to be sprayed; the preset category includes at least living things;

[0010] If the target category is not a preset category, the automatic spraying equipment is controlled to continue spraying the area to be sprayed.

[0011] In a second aspect, an embodiment of the present application provides a liquid spray control device, the device comprising:

[0012] An acquisition module is used to acquire a target image of the area to be sprayed by using an image sensor provided on the automatic spraying device; the target image includes a target object;

[0013] The recognition module is used to identify the target object using a target recognition model to obtain the target category of the target object; the target recognition model is obtained by training a preset initial model based on a preset training set, and the preset training set includes multiple training data, each training data includes at least a training image containing the training object, the center point of the training object in the training image, and the area occupied by the training object;

[0014] A spraying module, configured to control the automatic spraying device to stop spraying the area to be sprayed if the target category is a preset category; the preset category includes at least living things;

[0015] The stop module is used to control the automatic spraying equipment to stop spraying the area to be sprayed if the target category is not a preset category.

[0016] In a third aspect, an embodiment of the present application provides an automatic spraying device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of the first aspect described above when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of the first aspect described above.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on an automatic spraying device, enables the automatic spraying device to execute the method of the first aspect.

[0019] Compared to the prior art, the embodiments of the present application have the following advantages: the image sensor in the automatic spraying device captures target images of the area to be sprayed, and then uses a pre-trained target recognition model to recognize the target images and determine the target category of the target object in the target image. The target recognition model is obtained by training a preset initial model based on a preset training set, and each training data in the preset training set includes a training image containing the training object, as well as the center point and area occupied by the training object in the training image. Thus, the trained target recognition model can accurately obtain the area occupied by the target object from the target image, and then classify the occupied area, thereby improving the accuracy of target object category prediction. Subsequently, if the target category is a pedestrian, which is one of the preset categories, the automatic spraying device is controlled to stop spraying the area to be sprayed, thereby avoiding a certain degree of harm to pedestrians during disinfection in the area to be sprayed. Otherwise, if the target category is not a preset category, the automatic spraying device continues spraying the area to be sprayed, thereby thoroughly disinfecting the area to be sprayed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flow chart of an implementation of a liquid spraying control method provided in one embodiment of the present application;

[0022] Figure 2 This is a schematic diagram of an implementation method for generating training data in a liquid spraying control method provided in one embodiment of the present application;

[0023] Figure 3 This is a structural diagram of a label feature map in a liquid spraying control method provided by an embodiment of the present application;

[0024] Figure 4 This is a schematic diagram of an implementation method for generating a target recognition model in a liquid spraying control method provided in one embodiment of the present application;

[0025] Figure 5 A network structure block diagram of an initial model in a liquid spraying control method provided in an embodiment of the present application;

[0026] Figure 6 It is a network structure diagram of a neural network model in the prior art;

[0027] Figure 7This is a structural diagram of a liquid spray control device provided in one embodiment of the present application;

[0028] Figure 8 This is a structural diagram of an automatic spraying device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0030] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0031] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0032] The liquid spraying control method provided in the embodiments of this application can be applied to automatic spraying equipment. The automatic spraying equipment includes, but is not limited to, unmanned vehicles equipped with spraying devices, spraying robots, and other equipment. The embodiments of this application do not impose any restrictions on the specific type of automatic spraying equipment.

[0033] Among them, the above-mentioned liquids include but are not limited to water, disinfectants and other liquids. Taking disinfectants as an example, in recent years, highly contagious and widely spread viruses are still raging around the world, seriously affecting human life safety, normal social order and stable economic development. Public places with dense crowds and high mobility are usually important targets for epidemic prevention and control. If the environment is not thoroughly disinfected, it will cause huge social harm. At present, the use of manual spray disinfection methods often consumes a lot of time and manpower costs, and may cause a certain degree of damage to the health of the staff performing the spraying. Therefore, a liquid spraying control method for spraying disinfectant is proposed.

[0034] See also Figure 1 , Figure 1 The following is a flow chart showing an implementation of a liquid spraying control method provided in an embodiment of the present application, wherein the method comprises the following steps:

[0035] S101. The automatic spraying device collects a target image of the area to be sprayed through an image sensor provided on the automatic spraying device; the target image includes a target object.

[0036] In one embodiment, the image sensor includes, but is not limited to, sensors capable of capturing images, such as lidar, millimeter-wave radar, or cameras. The image sensors may be multiple and distributed in different directions of the automatic spraying equipment to capture target images of the sprayed area from multiple directions. In this embodiment, there are no restrictions on the type, number, or placement of the image sensors on the automatic spraying equipment.

[0037] Wherein, the area to be sprayed is the area where the automatic spraying device needs to spray liquid. Before spraying, the automatic spraying device needs to perform the above method to determine whether the area to be sprayed needs to be sprayed.

[0038] The target image is an image generated after collecting images of the area to be sprayed, and the target objects include but are not limited to pedestrians, animals, plants, and other objects.

[0039] S102. The automatic spraying equipment uses a target recognition model to identify the target object and obtain a target category of the target object; the target recognition model is obtained by training a preset initial model based on a preset training set, and the preset training set includes multiple training data, each training data includes at least a training image containing the training object, the center point of the training object in the training image, and the area occupied by the training object.

[0040] In one embodiment, the object recognition model is obtained by training a preset initial model based on a preset training set, wherein the preset training set generally includes multiple training data, each of which includes at least a training image containing a training object, a center point of the training object in the training image, and an area occupied by the training object.

[0041] It should be noted that in this embodiment, the above-mentioned recognition model is specifically a model based on the CenterNet framework. CenterNet detects the position of the training object based on its center point in the training image (which can also be considered the center of the area occupied by the training object). This converts the target object recognition problem into a standard keypoint estimation (center point prediction) problem. Specifically, only the center point of the training object in the training image is used as the anchor point, and only the position, width, and height of this center point in the training image are regressed.

[0042] Therefore, during training, the training data must include at least a training image containing the training object, the center point of the training object in the training image, and the area occupied by the training object. The occupied area refers to the area of ​​the training object in the training image. Typically, the initial model processes the training object based on the width and height of the occupied area, and then performs model training based on the processed training object.

[0043] The training data typically includes the true category of the training object. During training, the initial model performs recognition processing on the training image to predict the area occupied by the training object. It then predicts the training object's category based on the occupied area. The prediction loss is then calculated based on the predicted category and the true category, and some model parameters in the initial model are iteratively updated to produce the object recognition model.

[0044] It should be noted that because the initial model is trained based on a preset training set, when processing training images, the initial model does not need to consider the intersection-over-union ratio between the predicted area occupied by the training object and the actual area occupied. In other words, there is no need to use an anchor-based algorithm to manually set thresholds to classify the training object into foreground and background.

[0045] Usually, since each type of training object has only one positive sample, it is no longer necessary to perform non-maximum suppression on the feature map generated during the training process. Instead, it is only necessary to extract the pixel with the highest peak value from the feature map for subsequent processing.

[0046] In addition, the initial model based on the CenterNet framework has a larger dimension of the feature map it outputs, typically 1 / 4 the size of the training image input to the initial model. However, the model used by the anchor-based method typically outputs a feature map with a dimension size of 1 / 16 the size of the training image input to the model. Therefore, the model used by the point-based method requires multiple anchors of different lengths and widths. However, the initial model based on the CenterNet framework does not need to use multiple anchors of different lengths and widths, greatly reducing the computational complexity of the anchor part.

[0047] S103: If the target category is a preset category, the automatic spraying device stops spraying the area to be sprayed; the preset category at least includes living things.

[0048] S104: If the target category is not a preset category, the automatic spraying device continues to spray the area to be sprayed.

[0049] In one embodiment, the aforementioned living entities include, but are not limited to, pedestrians, animals, and other living entities. The aforementioned preset categories primarily take pedestrians as an example. If the target object is determined to be a pedestrian, spraying the area to be sprayed is stopped. If the target object is not a pedestrian, spraying is continued in the area to be sprayed.

[0050] It should be noted that when an automatic spraying device captures a target image of the area to be sprayed, it typically captures a broader original image. Specifically, this original image will include the target image corresponding to the area to be sprayed, as well as other boundary images not within the area to be sprayed. Based on this, in actual processing, the automatic spraying device may process the entire original image to determine the category of the target object within the original image. Subsequently, if the target object is of a preset category, the automatic spraying device further determines the position of the target object relative to the automatic spraying device using image sensors positioned in multiple directions. These multiple positions are then combined to determine the exact position of the target object relative to the automatic spraying device. If the target object's exact position is determined to be within the area to be sprayed, spraying is stopped. Alternatively, the automatic spraying device may issue a warning to the target object to maintain a certain distance from the automatic spraying device or to leave the area to be sprayed, allowing the automatic spraying device to continue spraying. If the target object's exact position is determined to be outside the area to be sprayed, spraying is continued.

[0051] In another embodiment, when using an automatic spraying device to spray liquid, it is typically necessary to first create a model of the area to be sprayed and store the model in the automatic spraying device. Therefore, when acquiring a raw image, the automatic spraying device can also perform similarity calculation on the raw image based on the pre-stored model of the area to be sprayed, thereby determining a target image of the area to be sprayed from the raw image. Finally, the aforementioned steps S101-S104 are performed on the target image.

[0052] In one embodiment, the non-preset categories may include, but are not limited to, non-living objects such as trees and vehicles, which are also objects that need to be disinfected.

[0053] In this embodiment, the image sensor in the automatic spraying device captures target images of the area to be sprayed. The target images are then recognized using a pre-trained target recognition model to determine the target category of the target object in the target image. The target recognition model is trained using a preset initial model based on a preset training set, and each piece of training data in the preset training set includes a training image containing the training object, as well as the center point and area occupied by the training object in the training image. Thus, the trained target recognition model can accurately determine the area occupied by the target object from the target image and then classify the occupied area, thereby improving the accuracy of target object category prediction. Subsequently, if the target category is a pedestrian, which is one of the preset categories, the automatic spraying device is controlled to stop spraying in the spraying area to avoid causing a certain degree of harm to pedestrians during disinfection in the spraying area. Otherwise, if the target category is not a preset category, the automatic spraying device continues spraying in the spraying area to thoroughly disinfect the area.

[0054] In other embodiments, in the liquid spraying control method, the automatic spraying device can also control spraying based on external or self-recognition results. The recognition results include, but are not limited to, the disinfection requirements of the spraying area, external wind speed and direction, the movement speed of the automatic spraying device, and the remaining amount of liquid.

[0055] For example, when the vehicle speed exceeds a preset speed, the liquid spraying speed is increased; when the vehicle speed is less than or equal to the preset speed, the liquid spraying speed is reduced; when the wind is headwindy, the liquid spraying direction is adjusted; when the wind speed exceeds a preset speed, the liquid spraying speed is reduced; when the wind speed is less than or equal to the preset speed, the liquid spraying speed is increased; and when the remaining liquid amount is less than a preset amount, the liquid spraying speed is reduced and an early warning is issued. For example, the warning can be issued through a warning light on the automatic spraying device or a voice prompt.

[0056] In one embodiment, the target object is identified by a target recognition model to obtain the target category of the target object. However, when training a target recognition model, the area occupied by the training object in the training image and the true category are usually manually marked, and then the training is input into the initial model for training. However, when iterating the network parameters of the initial model, this method only calculates the loss value based on the true category and the predicted category, and then iterates based on the loss value. However, when the initial model actually makes a category prediction, the initial model must first detect the center point and the area occupied by the training object in the training image, and then make a category prediction. Therefore, the predicted category will also be affected by the center point and the area occupied by the initial model. If the detected center point and the area occupied are different from the actual center point and the area occupied by the training object in the training image, the accuracy of the predicted category will be low when the initial model makes a prediction based on the different center point and the area occupied.

[0057] Based on this, refer to Figure 2 In order to improve the accuracy of the predicted category, when training the target recognition model, the automatic spraying equipment also needs to generate training data through the following S201-S204. The details are as follows,

[0058] S201: The automatic spraying device obtains a training image containing a training object.

[0059] In one embodiment, the training images can be pre-stored in a designated storage path of the automatic spraying device, or retrieved by the automatic spraying device from a network crawler, without limitation. There are typically multiple training images, each of which includes at least one training object of a preset category. In other words, a single training image can include training objects corresponding to multiple preset categories, or it can also include multiple different training objects corresponding to a single preset category, without limitation.

[0060] S202. The automatic spraying device calculates the feature map position of the center point in the training feature map and the Gaussian kernel radius of the center point in the training feature map according to the image size of the training image and the dimensional size of the training feature map output by the initial model.

[0061] In one embodiment, the image size of the training image is the size at the time of input to the initial model. Typically, the image size received by the initial model is fixed. Therefore, if the original image size of the training image is inconsistent with the image size received by the initial model, the original image size of the training image needs to be processed.

[0062] Specifically, the automatic spraying device needs to perform affine transformation processing on the original image size according to the image size received by the initial model. For example, if the image size received by the initial model is 512*512 size, and the original image size is 1280*720 size, the automatic spraying device can scale the original image size to 512*288 size at a constant ratio. After that, because the 288 size after scaling at a constant ratio is insufficient for 512 size, the automatic spraying device also needs to perform pixel point filling on the scaled image, and fill each pixel point of the filling with a pixel value of 0 to form a training image of 512*512 size.

[0063] In an embodiment, the training feature map described above is a feature map output by the initial model after model processing on the training image. Generally, the initial model needs to process the training feature map to obtain the predicted category of the training object.

[0064] In an embodiment, when calculating the feature map position of the center point in the training feature map, the automatic spraying device can find the center point of the training object in advance according to the occupied area of the training object in the training image; then, the position of the center point in the training image is determined. Then, according to the image size of the training image and the dimension size of the training feature map, the scaling multiple when the training image is processed by the initial model is calculated. Then, the position of the center point in the training image is scaled by the same scaling multiple to obtain the feature map position.

[0065] For example, the scaling multiple is S, and the position of the center point in the training image is (x, y), then the position of the center point in the training feature map after scaling is (x / S, y / S). If the values of x / S and y / S are not integers, they can be rounded down for subsequent processing.

[0066] In an embodiment, the Gaussian kernel radius is used to fill the Gaussian function calculation value of each pixel point in the training feature map with the center point as the center and the Gaussian kernel radius as the training feature map. Specifically, the center point is the maximum value, and decreases along the radius outward according to the Gaussian function.

[0067] Among them, for the calculation of the Gaussian kernel radius of each training object, it can be calculated according to the occupied area of the training object in the training image. Specifically, the automatic spraying device can load the width and length of the occupied area of each training object in the respective training image; then, the average value of the width and length of all occupied areas is calculated; then, for the Gaussian kernel radius of any training object, the standard deviation between the width and length corresponding to the training object and the average value of the width and length is calculated respectively. The standard deviation corresponding to the width and the standard deviation corresponding to the height are both used as the Gaussian kernel radius of the training object.

[0068] S203: The automatic spraying device processes the training feature map based on the Gaussian kernel radius to obtain a label feature map of the training image.

[0069] Conventional technology typically calculates the area occupied by the training object based on the area predicted by the initial model. For example, the actual area and the predicted area often overlap. Therefore, the radius of the inscribed circle or circumscribed circle between the two diagonal points of the predicted area and the actual area can be determined as the Gaussian kernel radius.

[0070] However, after calculating the Gaussian kernel radius using this method, a circular Gaussian distribution with the same radius as the kernel is typically drawn on the training feature map, with the center point being the circle, to generate the label feature map. However, when training on pedestrians, the area occupied by pedestrians in the training image is typically rectangular. Therefore, the circular label feature map cannot fully cover the pedestrian information. Alternatively, even if the pedestrian information is fully covered, the circular label feature map will still contain a lot of noise.

[0071] Based on this, in this embodiment, after calculating the standard deviation corresponding to the width and the standard deviation corresponding to the height, the Gaussian kernel radius for width and height can be obtained. The automatic spraying device can then draw an elliptical Gaussian distribution on the training feature map based on the Gaussian kernel radius for width and height, reducing the included noise information and improving the algorithm accuracy.

[0072] It should be noted that the above S202 has explained that the Gaussian kernel radius is used to fill the Gaussian function calculation value for each pixel in the training feature map with the center point as the center of the circle and the Gaussian kernel radius. Therefore, it can be considered that the Gaussian function calculation value filled with each pixel can be used to represent the probability value of the pixel belonging to the center point. The pixel at the center of the circle is the center point, so its Gaussian function calculation value is 1, and the Gaussian kernel radius on the width and the Gaussian kernel radius on the height are the size of the training object on the label feature map. In this way, the initial model can perform model training for center point position prediction based on the label feature map.

[0073] It should be added that, since there are two Gaussian kernel radii at this time, when determining the probability value of each pixel in the training feature map based on the Gaussian kernel radius, the calculation formula can be:

[0074]

[0075] Where x and y are the coordinates of the pixel relative to the center point; f is the probability value at the coordinate (x, y).

[0076] Usually, it is difficult to accurately define the center point by the position of a certain pixel point, and the image information represented by the pixels near the center point is usually similar to the image information represented by the center point. If the pixels near the center point are directly marked as negative samples (that is, the probability value is 0), it may interfere with the training of the initial model. Based on this, in this embodiment, the training feature map is processed by the Gaussian kernel radius, and the probability values ​​of the pixels near the center point are reasonably assigned so that the initial model can converge better. In this way, the label feature map generated by the Gaussian kernel radius can add a directional guidance when training the initial model. That is, the closer to the center point, the greater the probability value, so that the initial model network can quickly determine the center point in a direction, thereby improving the convergence efficiency of the initial model.

[0077] Among them, the probability value of each pixel in the generated label feature map can be as follows Figure 3 As shown. Among them, Figure 3 The horizontal coordinate axis is used to represent the coordinates of the pixel points, and the numerical axis represents the probability value of the pixel point. Among them, the coordinate (0, 0) is the center point, and its corresponding probability value is the largest, that is, 1. In addition, Figure 3 The label feature diagram shown is only an example in this embodiment.

[0078] S204: The automatic spraying device generates training data based on the training image and the label feature map.

[0079] In one embodiment, since the target recognition model is used to classify the target object, the training data should generally also include the label category corresponding to the label feature map. In other words, the label feature map must also include the true category of the training object to generate the training data.

[0080] Afterwards, during the training process, the initial model processes the training image and outputs the predicted center point position, predicted occupied area size and predicted category of the training object in the training image; then, the model is iterated with the actual center point position, actual occupied area size and true category represented by the label feature map to obtain the target recognition model.

[0081] Since the actual center point position, occupied area size, and true category are mapped into the output format of the initial model, which is the same as the output format after the initial model processes the training image, the initial model can quickly calculate the loss function based on the label feature map, improving the iteration efficiency of the model.

[0082] It should be noted that if a training image contains multiple training objects of the same category, a single label feature map can be used to represent them. That is, in the training feature map, the center point position corresponding to each training object is determined. Then, the training feature map is processed using the center point position and the Gaussian kernel radius corresponding to each training object to obtain a label feature map.

[0083] However, it should be added that, if there is only one training object in a training image, the above-mentioned S202-S204 method can be used to generate a label feature map.

[0084] However, if a training image contains multiple training objects, and at least two training objects overlap, for each training object, it is still necessary to first generate a label feature map corresponding to each training object based on the method of S203-S204. However, in this case, the center points calculated based on the above S202 may overlap in the feature map positions of the training feature map. In other words, the feature map positions corresponding to multiple training objects are the same. Therefore, in the label feature map of each overlapping training object generated using the above S203-S204 method, there may be multiple overlapping pixels, and each overlapping pixel generally has multiple probability values.

[0085] However, each pixel in the label feature map only needs one probability value. Based on this, in this embodiment, for any overlapping pixel, the maximum probability value among the overlapping pixels is determined as the final probability value of the overlapping pixel. In this way, a label feature map is generated based on the final probability values ​​of each pixel in the label feature maps corresponding to at least two training objects.

[0086] It should be noted that at this time, the label feature map can simultaneously represent the image information of two training objects. That is, the area occupied by the training object in the training image is changed to include the area occupied by both training objects. Alternatively, the automatic spraying device can determine the unobstructed training object in the training image corresponding to the overlapping pixel points. Afterwards, the label feature map at this time is used as the label feature map of the unobstructed training object. At the same time, the occupied area corresponding to the obscured training object is copied to the remaining positions in the training image, and the remaining positions must not belong to the occupied areas of other training objects. Afterwards, the label feature map of the obscured training object is regenerated. In this way, the problem that when the center points of multiple training objects of the same category coincide, only one label feature map of the training object can be generated for model processing is solved.

[0087] In other embodiments, if a training image contains multiple training objects of different categories, multiple label feature maps are required to represent them. That is, each label feature map is only used to represent image information of training objects of the same category.

[0088] In one embodiment, in order to accurately classify the target object, it is necessary to accurately determine the area occupied by the target object from the target image. Therefore, in order for the target recognition model to accurately determine the area occupied by the target object from the target image, when training the initial model, the initial model may include three output branches for outputting three feature maps. Specifically, they are: a predicted label feature map of the center point of the training object, a predicted offset feature map representing the predicted offset of the center point, and a predicted occupied area size feature map representing the training object. Among them, the above three feature maps are generated after the initial model processes the input training image, and the loss function calculation is performed corresponding to the label feature map of the training object in the training image, the offset feature map of the center point of the training object, and the occupied area size feature map of the training object, respectively, to iteratively update the model parameters of the three output branches in the initial model.

[0089] Specifically, refer to Figure 4 , the automatic spraying equipment can iterate the initial model through the following steps S401-S403. The details are as follows:

[0090] S401. After the automatic spraying equipment obtains the initial model and processes the training image, it outputs a predicted label feature map representing the center point of the training object, a predicted offset feature map representing the predicted offset of the center point, and a predicted occupied area size feature map representing the training object.

[0091] The label feature map has been explained above and will not be described again. The predicted offset feature map and the predicted occupied area size feature map can be generated in the following manner.

[0092] For example, in combination with the example in S202 above: the scaling factor is S, and the position of the center point in the training image is (x, y), then the position of the scaled center point in the training feature map is (x / S, y / S). Among them, if the values ​​of x / S and y / S are not integers, they can be rounded down and used in subsequent processing. Based on this, for the offset of the center point, it can be the value of x / S minus the value of x / S rounded down as the offset of the center point on the horizontal axis, and the value of y / S minus the value of y / S rounded down as the offset of the center point on the vertical axis. Finally, the above two offsets are used as the offset feature map of the center point. Similarly, if the offset feature map of other pixel points is needed, it can also be obtained in the above manner.

[0093] Similar to the offset feature map, when obtaining the occupied area size feature map of the training object, the automatic spraying device can first determine the area occupied by the training object in the training image. Then, the length and width of the occupied area are scaled by S times, and the scaled values ​​are used as the occupied area size feature map.

[0094] The label feature map can be represented by N*Label*128*128, the offset feature map can be represented by N*2*128*128, and the occupied area feature map can be represented by N*2*128*128. N is the number of training object categories in the training image. As mentioned above, a label feature map is generated for each category. Label is the actual category of the training object. 128*128 represents the dimension of the input feature map. 2 indicates that the feature map includes both length and width feature information.

[0095] S402. The automatic spraying equipment uses a first loss function to process the predicted label feature map and the label feature map to obtain the position prediction loss of the center point; uses a second loss function to process the offset feature map and the predicted offset feature map to obtain the offset prediction loss; and uses a second loss function to process the occupied area size feature map and the predicted occupied area size feature map to obtain the occupied area size prediction loss.

[0096] S403: The automatic spraying device iteratively updates the initial model according to the position prediction loss, the offset prediction loss, and the occupied area size prediction loss to obtain a target recognition model.

[0097] In one embodiment, the above-mentioned first loss function can be specifically a balanced cross entropy loss function (FocalLoss); the second loss function can be specifically a smoothed mean absolute error loss function (smooth L1 Loss). Among them, for the second loss function, in the prior art, L1 Loss is usually used to process the above-mentioned offset feature map, predicted offset feature map, occupied area size feature map and predicted occupied area size feature map. However, the derivative of the L1 Loss at point 0 is not unique, which will affect the convergence of the initial model. However, in this embodiment, when using Smooth L1 to calculate the loss, Smooth L1 can be processed using a square function near point 0 to make it smoother at point 0, so as to improve the convergence efficiency of the initial model.

[0098] Among them, the specific calculation formula of Focal Loss can be:

[0099] f central point (p t )=-α t (1-p t ) γlog(p t );

[0100] Among them, f central point (p t ) is the position prediction loss of the t-th center point; p t is the predicted probability value of the pixel where the t-th center point is located, where p t is∈[0,1];α t is the preset balance parameter; γ is the adjustment factor. The adjustment factor can adjust the imbalance between positive samples and negative samples.

[0101] The specific calculation formula for Smooth L1 can be:

[0102]

[0103] in,

[0104] k=f(k i )-l i ;

[0105] Among them, f(k i ) is the predicted value of the i-th pixel, and l i is the actual value of the i-th pixel. Among them, L1(k) is the calculated prediction loss. Specifically, when calculating the offset prediction loss, its k i is the predicted offset of the i-th pixel in the predicted offset feature map, l i is the actual offset of the i-th pixel in the offset feature map; when calculating the prediction loss of the occupied area size, its k i is the size of the predicted area occupied by the i-th pixel in the feature map, l i is the actual area occupied by the i-th pixel in the occupied area size feature map. Since each feature map typically contains multiple pixels, the L1(k) value for each pixel can be calculated separately. Finally, the L1(k) values ​​for each pixel are summed to obtain the final offset prediction loss and occupied area size prediction loss.

[0106] It is understood that when the generated target recognition model processes the target image, it can output the predicted position of the center point in the target image, as well as the predicted offset of the center point and the predicted size of the area occupied. The predicted position is then processed based on the predicted offset to accurately output the position of the center point in the target image. Based on this position and the size of the area occupied, the target object is identified in the target image. This improves the accuracy of the target recognition model in locating the center point.

[0107] Specifically, refer to Figure 5 and Figure 6 , Figure 5 is a network structure block diagram of an initial model in the embodiment of the present application; Figure 6 is a network structure block diagram of a neural network model in the prior art. Referring to Figure 6 , the initial model specifically includes a lightweight network, a pixel upsampling network, a first output network, a second output network, a third output network, and a fourth output network.

[0108] The lightweight network specifically refers to a Mobilenetv2 network in Figure 5 , which includes a 1*32*256*256 convolution layer (Conv) and a plurality of bottleneck layers (bottleneck) in sequence. Each bottleneck outputs an image or a feature map of different dimensions. Each layer of the network includes 1*a*b*c information. In the information, 1 represents a feature map; a represents the number of channels, for example, the initial input training image is a 3-channel image; and a*b represents the dimension size of the input image or feature map.

[0109] Specifically, a bottleneck is usually composed of two 1*1 convolution layers and a 3*3 convolution layer, with the 3*3 convolution layer between the two 1*1 convolution layers. The two 1*1 convolution layers are respectively responsible for reducing and then increasing the dimension of the feature map, so that the 3*3 convolution layer becomes a bottleneck with smaller input / output dimension. Specifically, before being processed by the 3*3 convolution layer, the 1*1 convolution layer is used to reduce the number of channels of the feature map, thereby reducing the parameter amount. After being processed by the 3*3 convolution layer, the 1*1 convolution layer is used to restore the number of channels of the feature map. As can be seen from Figure 6 , the prior art uses a Resnet18 network as a lightweight network, which includes a 1*64*256*256 convolution layer (Conv) and a plurality of decomposition layers (Resblock) in sequence. In each decomposition layer, the number of channels corresponding to the decomposition layer is much larger than the number of channels corresponding to the bottleneck layer. That is, compared with the Resnet18 network, the bottleneck used in the embodiment reduces the parameter amount and optimizes the calculation amount.

[0110] In addition, in the embodiment, the pixel upsampling network includes a plurality of upsampling layers, each of which includes a pixel adjustment layer (PixelShuffle) + convolution (Conv) layer in Figure 5 . In the prior art, the upsampling network is usually composed of a plurality of deconvolution layers, specifically Figure 6However, when multiple deconvolution layers are used to process feature maps, the feature information of the training object will be lost to a certain extent. However, when pixel upsampling networks are used to process feature maps, the feature information can be retained while increasing the resolution, and no grid effect will occur.

[0111] In one embodiment, the feature map output by the training image after being processed by the lightweight network and the pixel upsampling network is the reused feature map, wherein the reused feature map needs to be processed by the first output network, the third output network, and the fourth output network respectively.

[0112] in, Figure 5 The third and fourth output networks in the initial model that process the reused feature maps are Figure 6 The network structure for processing the reused feature map is similar to that in

[15] , and will not be described in detail.

[0113] It should be noted that when outputting the predicted label feature map, the initial model first uses the first output network to process the reused feature map to obtain the initial predicted label feature map. Afterwards, the reused feature map is processed again using the second output network to obtain the final predicted label feature map. Figure 6 It can be seen that in the prior art, when the model outputs the predicted label feature map, it only uses a first output network for processing.

[0114] Among them, the second output network specifically includes a normalization layer and a maximum pooling layer, which are used to normalize and pool the initial predicted label feature map. Specifically, after the probability value of each pixel in the initial predicted label feature map is normalized, a 3*3 maximum pooling layer is used to determine the maximum value of the probability value of each pixel in the domain pixel. That is, there is only one maximum value in every nine pixels. Afterwards, for all the maximum values ​​on the initial predicted label feature map, k maximum values ​​are selected in descending order according to the numerical value to be retained, and the predicted label feature map is obtained. At this time, the initial model can directly determine the pixel points corresponding to the k maximum values ​​as the center points. Among them, the purpose of normalization and maximum pooling is to better retain the texture features of the training object in the training image.

[0115] It should be noted that in S403 above, when iteratively updating the initial model based on the position prediction loss, offset prediction loss, and occupied area size prediction loss, only the model parameters of the lightweight network, pixel upsampling network, first output network, second output network, third output network, and fourth output network are iterated. The above initial model should also include a classification layer for performing category prediction based on the above feature maps. The model parameters of the classification layer should be iterated based on the training loss calculated based on the actual category of the training object and the predicted category.

[0116] See also Figure 7 , Figure 7 This is a structural block diagram of a liquid spraying control device provided in an embodiment of the present application. The liquid spraying control device in this embodiment includes modules for executing Figure 1 、 Figure 2 and Figure 4 Each step in the corresponding embodiment. Please refer to Figure 1 、 Figure 2 and Figure 4 as well as Figure 1 、 Figure 2 and Figure 4 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 7 The liquid spraying control device 700 may include: a collection module 710, a recognition module 720, a spraying module 730 and a stop module 740, wherein:

[0117] The acquisition module 710 is used to acquire a target image of the area to be sprayed by using an image sensor provided on the automatic spraying device; the target image includes a target object.

[0118] The recognition module 720 is used to use a target recognition model to identify the target object and obtain the target category of the target object; the target recognition model is obtained by training a preset initial model based on a preset training set, and the preset training set includes multiple training data, each training data includes at least a training image containing the training object, the center point of the training object in the training image, and the area occupied by the training object.

[0119] The spraying module 730 is used to control the automatic spraying device to stop spraying the area to be sprayed if the target category is a preset category; the preset category at least includes living things.

[0120] The stop module 740 is used to control the automatic spraying equipment to stop spraying the area to be sprayed if the target category is not a preset category.

[0121] In one embodiment, the liquid spray control device 700 further includes the following modules to generate training data:

[0122] A training image acquisition module is used to acquire a training image containing a training object;

[0123] A calculation module, configured to calculate a feature map position of a center point in the training feature map and a Gaussian kernel radius of the center point in the training feature map according to an image size of the training image and a dimension of a training feature map output by the initial model;

[0124] A label feature map generation module is used to process the training feature map based on the Gaussian kernel radius to obtain the label feature map of the training image;

[0125] The training data generation module is used to generate training data based on training images and label feature maps.

[0126] In one embodiment, the calculation module is further configured to:

[0127] Based on the image size and dimension size, the scaling factor of the training image when processed by the initial model is calculated; based on the scaling factor and the position of the center point in the training image, the feature map position of the center point in the training feature map is calculated.

[0128] In one embodiment, the training image includes a training object, and the label feature map generation module is further configured to:

[0129] Taking the feature map position as the center, the probability value of each pixel in the training feature map is determined based on the Gaussian kernel radius, and the label feature map is obtained according to the probability value of each pixel in the training feature map; the probability value is used to indicate the probability that each pixel in the label feature map belongs to the center point.

[0130] In one embodiment, the training image includes multiple training objects; the label feature map generation module is further configured to:

[0131] The feature map position of the center point of each training object in the training feature map is obtained respectively; with the feature map position as the center, the probability value of each pixel point in the training feature map is determined based on the Gaussian kernel radius, and the label feature map is obtained according to the probability value of each pixel point in the training feature map; the probability value is used to represent the probability that each pixel point in the label feature map belongs to the center point; if the feature map positions corresponding to at least two training objects overlap, then for any overlapping pixel points in the label feature maps corresponding to at least two training objects, the maximum value of the probability value in the overlapping pixel points is determined as the final probability value of the overlapping pixel point; according to the final probability value of each pixel point in the label feature maps corresponding to at least two training objects, the final label feature map is obtained.

[0132] In one embodiment, the training data further includes a feature map of the offset of the center point and a feature map of the size of the area occupied by the training object; the liquid spray control device 700 further includes the following modules for training the initial model:

[0133] The feature map output module is used to obtain the predicted label feature map representing the center point of the training object, the predicted offset feature map representing the predicted offset of the center point, and the predicted area size feature map representing the training object after the initial model processes the training image.

[0134] The loss calculation module is used to use the first loss function to process the predicted label feature map and the label feature map to obtain the position prediction loss of the center point; use the second loss function to process the offset feature map and the predicted offset feature map to obtain the offset prediction loss; and use the second loss function to process the occupied area size feature map and the predicted occupied area size feature map to obtain the occupied area size prediction loss.

[0135] The iteration module is used to iteratively update the initial model according to the position prediction loss, offset prediction loss and occupied area size prediction loss to obtain the target recognition model.

[0136] In one embodiment, the initial model includes a lightweight network, a pixel upsampling network, a first output network, a second output network, a third output network, and a fourth output network;

[0137] The lightweight network and the pixel upsampling network are used in turn to process the training image to obtain a reused feature map; the first output network is used to process the reused feature map to obtain an initial predicted label feature map; the second output network includes a normalization layer and a maximum pooling layer in turn, which are used to normalize and pool the initial predicted label feature map to obtain a final predicted label feature map; the third output network is used to process the reused feature map to obtain a predicted offset feature map; the fourth output network is used to process the reused feature map to obtain a predicted occupied area size feature map.

[0138] When it is understood that Figure 7 In the structural block diagram of the liquid spray control device shown, each module is used to execute Figure 1 、 Figure 2 and Figure 4 The steps in the corresponding embodiment, and for Figure 1 、 Figure 2 and Figure 4 Each step in the corresponding embodiment has been explained in detail in the above embodiment. Figure 1 、 Figure 2 and Figure 4 as well as Figure 1 、 Figure 2 and Figure 4 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0139] Figure 8This is a structural diagram of an automatic spraying device provided by an embodiment of the present application. Figure 8 As shown, the automatic spraying device 800 of this embodiment includes: a processor 810, a memory 820, and a computer program 830 stored in the memory 820 and executable by the processor 810, such as a program for a liquid spraying control method. When the processor 810 executes the computer program 830, the steps in each embodiment of the above-mentioned liquid spraying control method are implemented, such as Figure 1 Alternatively, the processor 810 executes the computer program 830 to implement the above Figure 7 The functions of each module in the corresponding embodiment are, for example, Figure 7 For details on the functions of modules 710 to 740, please refer to Figure 7 Related description in the corresponding embodiment.

[0140] Exemplarily, the computer program 830 can be divided into one or more modules, one or more of which are stored in the memory 820 and executed by the processor 810 to implement the liquid spraying control method provided in the embodiments of the present application. One or more modules can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 830 in the automatic spraying device 800. For example, the computer program 830 can implement the liquid spraying control method provided in the embodiments of the present application.

[0141] The automatic spraying device 800 may include, but is not limited to, a processor 810 and a memory 820. It will be understood by those skilled in the art that Figure 8 This is merely an example of the automatic spraying device 800 and does not constitute a limitation of the automatic spraying device 800. The automatic spraying device 800 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the automatic spraying device may also include input and output devices, network access devices, buses, etc.

[0142] The processor 810 may be a central processing unit, or other general-purpose processor, a digital signal processor, an application-specific integrated circuit, an off-the-shelf programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] The memory 820 may be an internal storage unit of the automatic spraying device 800, such as a hard disk or memory of the automatic spraying device 800. The memory 820 may also be an external storage device of the automatic spraying device 800, such as a plug-in hard disk, smart memory card, flash memory card, etc. equipped on the automatic spraying device 800. Furthermore, the memory 820 may include both an internal storage unit of the automatic spraying device 800 and an external storage device.

[0144] An embodiment of the present application provides a computer-readable storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the liquid spraying control method described in the above-mentioned embodiments is implemented.

[0145] An embodiment of the present application provides a computer program product. When the computer program product is run on an automatic spraying device, the automatic spraying device executes the liquid spraying control method in each of the above embodiments.

[0146] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A liquid spraying control method, characterized in that: The method comprises: Capturing a target image of the area to be sprayed by an image sensor provided on the automatic spraying device; the target image includes a target object; The target object is identified by a target recognition model to obtain a target category of the target object; the target recognition model is obtained by training a preset initial model based on a preset training set, the preset training set includes a plurality of training data, each of the training data includes at least a training image containing a training object, a center point of the training object in the training image, and an area occupied by the training object; If the target category is a preset category, the automatic spraying device is controlled to stop spraying the area to be sprayed; the preset category includes at least living things; If the target category is not a preset category, controlling the automatic spraying equipment to continue spraying the area to be sprayed; The training data is obtained by: Acquiring a training image containing the training object; Calculating the feature map position of the center point in the training feature map and the Gaussian kernel radius of the center point in the training feature map according to the image size of the training image and the dimensional size of the training feature map output by the initial model; Processing the training feature map based on the Gaussian kernel radius to obtain a label feature map of the training image; Generating the training data according to the training image and the label feature map; Calculating the Gaussian kernel radius of the center point in the training feature map includes: Determine the width and length of the area occupied by each training object in the respective training image; calculate the average values ​​of the widths and lengths of all the occupied areas; for any training object, calculate the standard deviation between the width and the average values ​​of the length and the length based on the width and length of the area occupied by the training object; and use the standard deviation corresponding to the width and the standard deviation corresponding to the length as the Gaussian kernel radius of the training object.

2. The method according to claim 1, characterized in that Calculating the feature map position of the center point in the training feature map according to the image size and the dimensional size of the training feature map output by the initial model includes: Calculating a scaling factor for the training image when it is processed by the initial model according to the image size and the dimension size; The feature map position of the center point in the training feature map is calculated according to the scaling factor and the position of the center point in the training image.

3. The method according to claim 1, characterized in that The training image includes a training object; the processing of the feature map based on the Gaussian kernel radius to obtain a label feature map includes: Taking the feature map position as the center, the probability value of each pixel point in the training feature map is determined based on the Gaussian kernel radius, and a label feature map is obtained according to the probability value of each pixel point in the training feature map; the probability value is used to represent the probability that each pixel point in the label feature map belongs to the center point.

4. The method according to claim 1, wherein The training image includes a plurality of the training objects; and the processing of the feature map based on the Gaussian kernel radius to obtain a label feature map includes: Respectively obtain the feature map position of the center point of each training object in the training feature map; Taking the feature map position as the center, determining the probability value of each pixel in the training feature map based on the Gaussian kernel radius, and obtaining a label feature map according to the probability value of each pixel in the training feature map; the probability value is used to represent the probability that each pixel in the label feature map belongs to the center point; If the feature map positions corresponding to at least two training objects overlap, then for any overlapping pixel points in the label feature maps corresponding to the at least two training objects, the maximum probability value of the overlapping pixel points is determined as the final probability value of the overlapping pixel point; The final label feature map is obtained according to the final probability value of each pixel point in the label feature map corresponding to the at least two training objects.

5. The method according to claim 1, wherein The training data also includes a feature map of the offset of the center point and a feature map of the size of the area occupied by the training object; The target recognition model is trained by the following steps: After obtaining the initial model to process the training image, the outputs include a predicted label feature map representing the center point of the training object, a predicted offset feature map representing the predicted offset of the center point, and a predicted occupied area size feature map representing the training object; Using a first loss function to process the predicted label feature map and the label feature map to obtain a position prediction loss of the center point; using a second loss function to process the offset feature map and the predicted offset feature map to obtain an offset prediction loss; and using the second loss function to process the occupied area size feature map and the predicted occupied area size feature map to obtain an occupied area size prediction loss; The target recognition model is obtained by iteratively updating the initial model according to the position prediction loss, the offset prediction loss, and the occupied area size prediction loss.

6. The method according to claim 5, characterized in that The initial model includes a lightweight network, a pixel upsampling network, a first output network, a second output network, a third output network and a fourth output network; The lightweight network and the pixel upsampling network are used to process the training image in sequence to obtain a reused feature map; The first output network is used to process the reused feature map to obtain an initial predicted label feature map; the second output network includes a normalization layer and a maximum pooling layer in sequence, which are used to perform normalization and pooling processing on the initial predicted label feature map to obtain the final predicted label feature map; The third output network is used to process the multiplexed feature map to obtain the predicted offset feature map; The fourth output network is used to process the reused feature map to obtain the predicted occupied area size feature map.

7. A liquid spray control device, characterized in that: The device comprises: An acquisition module is configured to acquire a target image of the area to be sprayed by using an image sensor provided on the automatic spraying device; the target image includes a target object; an identification module, configured to identify the target object using a target recognition model to obtain a target category of the target object; the target recognition model is obtained by training a preset initial model based on a preset training set, the preset training set including a plurality of training data, each of the training data including at least a training image containing the training object, a center point of the training object in the training image, and an area occupied by the training object; a stop module, configured to control the automatic spraying device to stop spraying the area to be sprayed if the target category is a preset category; the preset category includes at least living things; a spraying module, configured to control the automatic spraying device to continue spraying the area to be sprayed if the target category is not a preset category; The training data is obtained by: A training image acquisition module, used to acquire a training image containing the training object; a calculation module, configured to calculate a feature map position of the center point in the training feature map and a Gaussian kernel radius of the center point in the training feature map according to an image size of the training image and a dimensional size of the training feature map output by the initial model; a label feature map generating module, configured to process the training feature map based on the Gaussian kernel radius to obtain a label feature map of the training image; A training data generation module, configured to generate the training data based on the training image and the label feature map; The computing module is also used to: Determine the width and length of the area occupied by each training object in the respective training image; calculate the average values ​​of the widths and lengths of all the occupied areas; for any training object, calculate the standard deviation between the width and the average values ​​of the length and the length based on the width and length of the area occupied by the training object; and use the standard deviation corresponding to the width and the standard deviation corresponding to the length as the Gaussian kernel radius of the training object.

8. An automatic spraying device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images