Target Object Aggregation Method, Device, and Storage Medium

Abstract

A method for aggregating target objects includes: performing target object density analysis on an environmental image to obtain a density map of the environmental image and a density level of the density map; when the density level meets a preset level threshold, determining a target area of the target object in the density map according to the density map and the density level; generating a control instruction according to the target area, where the control instruction is used to control the target object aggregation device to run to the target area and then aggregate the target objects.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to a method, device, and storage medium for aggregating target objects. Background Art

[0002] The statements herein only provide background information related to the present application and do not necessarily constitute exemplary techniques.

[0003] With the continuous progress of computer technology and artificial intelligence technology, automated working devices similar to intelligent target object aggregation devices have gradually entered people's lives. For example, a small and low-power target object aggregation device is used to achieve the aggregation of target objects such as paper scraps, fallen leaves, and sundries, so as to improve the cleaning efficiency. However, for currently common target object aggregation devices with cleaning functions, they will execute the cleaning function once regardless of whether there are target objects to be cleaned in the area where the target object aggregation device is located, which greatly increases the workload of the target object aggregation device and affects the aggregation efficiency of the target objects. Summary of the Invention

[0004] According to various embodiments of the present application, a method, device, and storage medium for aggregating target objects are provided.

[0005] In a first aspect, an embodiment of the present application provides a method for aggregating target objects, the method including:

[0006] Performing target object density analysis on an environmental image to obtain a density map of the environmental image and a density level of the density map;

[0007] When the density level meets a preset level threshold, determining a target area of the target object in the density map according to the density map and the density level;

[0008] Generating a control instruction according to the target area, where the control instruction is used to control the target object aggregation device to run to the target area and then aggregate the target objects.

[0009] In a second aspect, an embodiment of the present application provides a target object aggregation device, the device including:

[0010] An acquisition module configured to perform target object density analysis on an environmental image to obtain a density map of the environmental image and a density level of the density map;

[0011] A determination module configured to, when the density level meets a preset level threshold, determine a target area of the target object in the density map according to the density map and the density level;

[0012] A generation module, configured to generate a control instruction according to the target area, where the control instruction is used to control the target object collection device to run to the target area and then collect the target object.

[0013] In a third aspect, an embodiment of the present application provides a target object collection device, including a memory and a processor;

[0014] The memory is configured to store a computer program;

[0015] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the target object collection method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to implement the steps of the target collection method described in the first aspect above.

[0017] Details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present application will become apparent from the specification, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a scenario architecture diagram of the target object collection method in an embodiment of the present application.

[0020] Figure 2 It is a flowchart of the target object collection method in an embodiment of the present application.

[0021] Figure 3 It is a schematic diagram of the network structure of the density and density level collaborative estimation model in an embodiment of the present application.

[0022] Figure 4 It is a schematic diagram of the density map of the environmental image in an embodiment of the present application.

[0023] Figure 5 It is a schematic diagram of the density level of the density map in an embodiment of the present application.

[0024] Figure 6 It is a schematic diagram of the regional density map in an embodiment of the present application.

[0025] Figure 7 Schematic block diagram of the target object collection device provided by the embodiment of the present application. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0027] The flowchart shown in the accompanying drawings is only an example illustration, and does not necessarily include all the content and operations / steps, nor does it necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged, so the actual execution order may be changed according to the actual situation.

[0028] It should be noted that the target object collection method, device, equipment, and storage medium provided by the embodiments of the present application can reduce the workload of the target object collection device and improve the target object collection efficiency.

[0029] It can be understood that the target object collection method provided by the embodiments of the present application can be applied to any target object collection device with image processing capabilities. The target object collection device can be a device with a Central Processing Unit (CPU) and / or a Graphics Processing Unit (GPU). Among them, the target object collection device includes but is not limited to self-mobile devices such as lawn mowing devices, sweeping robots, or garbage cleaning devices. The target object collection device can also have both a CPU and a GPU. In some cases, the target object collection device can cooperate with a terminal and / or a server to implement the above target object collection method. Among them, the terminal includes but is not limited to personal computers, workstations, etc. The server can be independent or a server cluster.

[0030] In practical applications, the target object collection method provided by the present application includes but is not limited to being applied in, for example, Figure 1 the application environment shown as follows.

[0031] Such as Figure 1As shown, the target object collection device 120 is connected to the camera 140 and the terminal 160 through a network. The camera 140 can capture an environmental image of a geographical area, such as a square, a park, etc., and the environmental image includes target objects such as paper scraps, fallen leaves, and / or other sundries. A target object collection device 1200 is deployed in the target object collection device 120. The function of the target object collection device 1200 can be logically divided into multiple modules, each module can have different functions, and the function of each module is implemented by the processor in the target object collection device 120 reading and executing instructions in the memory.

[0032] Exemplarily, the target object collection device 1200 may include an acquisition module 1201, a determination module 1202, and a generation module 1203. In a specific implementation manner, the target object collection device 1200 may execute the content described in steps S201 - S203 described below. It should be noted that the embodiments of the present application only exemplify the structure and functional modules of the target object collection device 1200.

[0033] Among them, the acquisition module 1201 is configured to obtain a density map of the distribution of target objects in the environmental image and the density level corresponding to the density map by performing target object density analysis on the environmental image obtained from the camera 140; the determination module 1202 is configured to determine, according to the density level obtained by the acquisition module 1201 and a preset level threshold, that when the density level meets the preset level threshold, determine the target area of the target object in the density map according to the density map and the density level obtained by the acquisition module 1201; the generation module 1203 is configured to generate a control instruction according to the target area of the target object in the density map, where the control instruction is used to control the target object collection device to run to the target area and then collect the target objects. Further, in this embodiment, the target object collection device 120 may also send the control instruction to the terminal 160, and the terminal controls the target object collection device 120 to run to the target area and then collect the target objects according to the control instruction.

[0034] In addition, in some possible cases, some of the multiple modules included in the above-mentioned target object collection device 1200 may also be combined into one module. For example, the above-mentioned acquisition module 1201 and determination module 1202 may be combined into an analysis module, that is, the analysis module combines the functions of the acquisition module 1201 and the determination module 1202.

[0035] In the embodiments of the present application, the above-described target object collection device 1200 can be flexibly deployed. For example, the target object collection device 1200 can be deployed on the terminal 160. The terminal 160 reads and executes instructions in the memory through the processor to generate control instructions for controlling the target object collection device, and controls the target object collection device to run to the target area according to the generated control instructions to collect the target objects.

[0036] Please refer to Figure 2 as shown in Figure 2 which is a flowchart of the target object collection method in the embodiments of the present application. It should be noted that Figure 2 this is a detailed description of each step of the target object collection method provided in the embodiments of the present application from the perspective of the target object collection device. Among them, the target object collection device can be a garbage cleaning device, a lawn mowing device, or a floor sweeping robot, etc.

[0037] It can be seen from Figure 2 that the target object collection method provided in the embodiments of the present application includes steps S201 to S203. Details are as follows:

[0038] S201, perform target object density analysis on the environmental image to obtain the density map of the environmental image and the density level of the density map.

[0039] Among them, the environmental image includes the target objects to be collected, and the target objects can be any objects to be collected. In different geographical regions, the target objects can be different objects to be collected. For example, in the square area, the objects to be collected can be garbage such as paper scraps and fruit peels. By performing garbage density analysis on the environmental image of the square area and obtaining the density map of the corresponding environmental image and the density level corresponding to the density map, it is possible to determine the areas where there is a certain degree of garbage accumulation from the environmental image based on the density map and density level of the environmental image, effectively preventing the garbage collection device from collecting garbage in areas where there is no garbage and improving the efficiency of garbage collection.

[0040] For another example, in the lawn mowing area, the objects to be collected can be fallen leaves, etc. By performing fallen leaf density analysis on the environmental image of the lawn mowing area and obtaining the density map of the environmental image of the lawn mowing area and the density level corresponding to the density map, it is possible to determine the areas where there is a certain degree of fallen leaf accumulation from the environmental image based on the density map of the environmental image and the density level corresponding to the density map, effectively preventing the lawn mowing device from collecting fallen leaves in areas where there are no fallen leaves and improving the efficiency of fallen leaf collection.

[0041] It can be understood that cameras are generally deployed in geographical areas where object aggregation is required, such as squares, roads, or lawns. The camera at least includes a camera lens. The camera can capture the above geographical area through the camera lens to obtain an environmental image, and the object aggregation device can obtain the corresponding environmental image from the camera. It should be noted that the object aggregation device can receive the environmental image sent by the camera, or actively obtain the corresponding environmental image from the camera, or directly install the camera on the object aggregation device. The object aggregation device can obtain the corresponding environmental image in real time, or obtain the corresponding environmental image according to a preset period. This embodiment does not make a limitation in this regard.

[0042] In addition, the camera installed in a geographical area where object aggregation is required, such as a square, road, or lawn, can also send the captured environmental image to a terminal or a server. After the terminal or the server analyzes the environmental image and obtains the target area of the object in the density map of the environmental image, the terminal or the server generates a control instruction, and the terminal or the server sends the generated control instruction to the object aggregation device to control the object aggregation device to operate to the target area to aggregate the object. This embodiment does not make a limitation in this regard.

[0043] In one implementation manner, when an object is detected, an environmental image is obtained.

[0044] Specifically, when it is detected by a deep learning detection algorithm that there is an object in the image captured by the camera, the image including the object is determined as the environmental image, so that the image not including the object does not undergo object density analysis processing, reducing the computational amount of the machine, thereby improving the efficiency of object density analysis.

[0045] In specific implementation, performing object density analysis on the environmental image can be a process of evaluating the density degree of the object. Specifically, it is necessary to evaluate the area in the environmental image where the density degree of the object meets the preset collection condition and use it as the target area for object aggregation, so as to prevent the object aggregation device from aggregating the object in an area that does not meet the preset collection condition, that is, preventing the object aggregation device from aggregating the object in an area where the object is not dense enough. The object aggregation device aggregating the object in an area that meets the preset collection condition can effectively reduce the workload of the object aggregation device and improve the object aggregation efficiency.

[0046] Exemplarily, performing object density analysis on the environmental image and obtaining the density map of the environmental image and the density level of the density map may include: obtaining the feature map of the environmental image; segmenting the extracted feature map to obtain the density map of the environmental image; classifying the feature map of the environmental image according to a preset density value to determine the density level corresponding to the density map of the environmental image, and the density value corresponds to the density level one by one.

[0047] Specifically, based on a pre-trained density and density level co-estimation model, a density map of the environmental image and the density level corresponding to the density map can be obtained. Among them, the pre-trained density and density level co-estimation model can detect the objects in the environmental image, determine the density of the objects in the environmental image, and obtain the density map; further classify the density map of the environmental image to evaluate the density level corresponding to the density map.

[0048] Exemplarily, as Figure 3 shown, Figure 3 is a schematic diagram of the network structure of the density and density level co-estimation model provided by the embodiment of the present application.

[0049] As can be seen from Figure 3 in this embodiment, the density and density level co-estimation model 300 includes a feature map extraction network layer 301, a feature map segmentation network layer 302, and a feature map classification network layer 303. Among them, the density and density level co-estimation model 300 is a network for estimating the density and density level of the objects in the image, which takes the environmental image as the input and outputs the density map corresponding to the environmental image and the density level of the density map. Specifically, the environmental image is input into the feature map extraction network layer 301, and the feature map extraction network layer 301 can extract the feature map in the environmental image. The feature map segmentation network layer 302 segments the feature map extracted by the feature map extraction network layer 301 to obtain the density map of the environmental image. The feature map classification network layer 303 classifies the feature map extracted by the feature map extraction network layer 301 based on a preset density value to obtain the density level corresponding to the density map of the environmental image.

[0050] Specifically, the feature map extraction network layer 301 can be a shared feature extraction layer of the density and density level co-estimation model 300, and this shared feature extraction layer is used to extract the feature map in the environmental image. The feature map segmentation network layer 302 can be a segmentation task layer of the density and density level co-estimation model 300, and this segmentation task layer can be used to segment the feature map extracted by the feature map extraction network layer 301 to obtain the density map of the environmental image. The feature map classification network layer 303 can be a classification task layer of the density and density level co-estimation model 300, and this classification task layer can classify the feature map extracted by the feature map extraction network layer 301 based on a preset density value to obtain the density level corresponding to the density map of the environmental image.

[0051] Specifically, reference can be made to Figure 4 and Figure 5 shown, Figure 4 shows the pixel values of each pixel point in the density map of the environmental image. Figure 5 shows the density level of the density map.

[0052] Among them, Figure 4 For each pixel point in the density map 304 shown, the pixel value of each pixel point is used to characterize the density of the corresponding pixel point in the input environmental image. Specifically, such as pixel point 1, pixel point 2, pixel point 3, …, pixel point n, etc. This density map 304 can display the density distribution of the environmental image, and further display the distribution of the target object. Specifically, the pixel value of the corresponding pixel point in the density map 304 can characterize the aggregation status of the target object. The pixel point with a larger pixel value represents that the target object is more aggregated, and the pixel point with a smaller pixel value represents that the target object is more dispersed. For example, the pixel values of the pixel points in the area 3041 are 0.2 or greater than 0.2, and the pixel values of the pixel points in the area 3042 are 0. Then, based on the pixel values of the pixel points in the area 3041 and the pixel values of the pixel points in the area 3042, it can be determined that the density distribution in the area 3041 is greater than the density distribution in the area 3042. Furthermore, it can be determined that the target object is concentrated in the area 3041, and there is no target object in the area 3042, or there are target objects with a lower density. And in the area where the density of the target object is low, the condition for cleaning the target object in this area cannot be triggered. That is to say, in the area corresponding to the pixel point with a larger pixel value, the target object is more concentrated, and in the area corresponding to the pixel point with a smaller pixel value, the target object is more dispersed. Through the density map of the environmental image, the distribution of the target object can be effectively reflected, which is beneficial to determining the area where the target object exists based on the distribution of the target object, avoiding collecting the target object in the area where there is no target object, and improving the efficiency of collecting the target object.

[0053] Figure 5 Each value in the density level schematic diagram 305 shown represents a different density value. In the embodiment of the present application, the density probability value is used to represent the density value, that is, the density probability value corresponds one-to-one with the density level. Specifically, such as the density probability values are 0.1, 0.2, 0.3, …, 1 respectively, and they represent the corresponding density levels as the 1st level, the 2nd level, the 3rd level, …, the 10th level, etc. That is to say, the density map and the density level collaborative estimation model 300 will segment the feature map of the environmental image to obtain the density map corresponding to each feature map, and classify the corresponding density map. After obtaining the density level of each density map according to the density probability value of each density map, the density map of the environmental image and the density level of the corresponding density map are output simultaneously. When the target object collection device determines that the density level corresponding to the density map meets the preset level threshold, the target area where the target object exists in the environmental image is determined according to the density map and the corresponding density level, so as to control the target object collection device to collect the target object in the target area, and improve the efficiency of collecting the target object. Specifically, the density probability value is in a proportional relationship with the distribution of the target object. That is to say, the greater the density probability, the higher the corresponding density level, and the more concentrated the distribution of the target object. The smaller the density probability value, the lower the corresponding density level, and the more discrete the distribution of the target object.

[0054] Specifically, when the density level is greater than or equal to a preset level threshold, it is determined that the density level meets the preset level threshold. Then, based on the density map and the corresponding density level, the target area where the target object exists in the environmental image is determined to control the target object collection device to collect the target object in the target area, thereby improving the target object collection efficiency.

[0055] Among them, the preset level threshold can be predefined by professionals, such as 0.7 or 0.8, etc.

[0056] It should be noted that the multi-task collaborative deep learning model provided in the embodiments of the present application can be obtained by training a deep learning model. Among them, the deep learning model can be a convolutional neural network model, such as a multicolumn convolutional neural network (MCNN), a scale-adaptive CNN (SaCNN), a congested scene recognition net (CSRNet), a multi-task collaborative neural network (Perspective Crowd Counting via Spatial Convolutional Network, PCC-NET), etc.

[0057] Taking PCC-NET as an example, compared with the common multi-task collaborative neural network, this network removes the data transfer process between the transmission layer and the segmentation layer. Specifically, the common multi-task collaborative neural network needs to transmit the feature map extracted by the shared feature extraction layer to the segmentation layer through the transmission layer. After the segmentation layer performs semantic segmentation and instance segmentation on the feature map, the image parts with different semantics are then transmitted back to the corresponding task detection layer through the transmission layer. In this embodiment, however, this network is directly connected to the segmentation task layer and the classification task layer through the shared feature extraction layer, effectively avoiding the data transfer process between the transmission layer and the segmentation layer, and realizing the output of two types of information. Among them, the output of the two types of information is the density map and the density map level respectively. This embodiment realizes the combination of the density map and the density map level to determine the distribution of the target object in the environmental image, and further realizes controlling the target object collection device to collect the target object in the area where the target object exists, avoiding the target object collection device from performing the target object collection operation in the area where the target object does not exist. Especially when the processing capacity of the target object collection device is limited, or the power of the target object collection device is insufficient, this embodiment can effectively avoid the waste of resources of the target object collection device and improve the work efficiency.

[0058] In specific implementation, the training process of the above density and density level collaborative estimation model can be implemented by a server. For example, the training of the density and density level collaborative estimation model can be completed by a cloud server or a local server, and then sent by the cloud server or the local server to the target object collection device. The training process of the density and density level collaborative estimation model is not specifically limited in this embodiment.

[0059] S202. When the density level meets a preset level threshold, determine a target area of the target object in the density map according to the density map and the density level.

[0060] If the density level of the density map of the environmental image is greater than or equal to the preset level threshold, it is determined that the density level of the density map of the environmental image meets the preset level threshold. The density level threshold can be predefined according to the actual scenario.

[0061] When determining the target area of the target object in the density map according to the density map and the density level, the density map can be first preprocessed according to the pixel values of each pixel point in the density map to remove the area in the density map that does not contain the target object, and then the preprocessed density map is fused with the density level to determine the target area containing the target object. In this way, the target object collection device is controlled to collect the target object in the target area, effectively avoiding the waste of power and CPU caused by the target object collection device performing the target object collection task in the area that does not contain the target object, improving the target object collection efficiency, and achieving the purpose of saving the resource consumption of the target object collection device. The first preprocessing process is a process of filtering pixel points based on pixel values, filtering out the points with pixel values lower than the preset pixel threshold in the density map, and further determining the area where the target object is dense in the density map based on the pixel values of the filtered pixel points.

[0062] In one embodiment, determining the target area of the target object in the density map according to the density map and the density level may include: performing a first preprocessing on the density map according to a preset pixel threshold. Fusing the preprocessed density map with the density level to obtain the distribution areas of each target object in the preprocessed density map. Screening each distribution area to determine the target area.

[0063] In one embodiment, performing a first preprocessing on the density map according to a preset pixel threshold may include: performing a second preprocessing on the density map. Taking the pixel points with pixel values greater than the preset pixel threshold in the preprocessed density map as first pixel points, and taking the pixel points other than the first pixel points as second pixel points. Performing binarization processing on the first pixel points and the second pixel points to obtain a binarized image. Fusing the preprocessed density map and the binarized image to obtain the preprocessed density map.

[0064] In specific implementation, a first preprocessing is performed on the density map according to a preset pixel value, and some pixel points in the density map can be filtered out. For example, 10% of the pixel points are filtered out to effectively reduce the power consumption of the target object aggregation device for processing the density map and improve the efficiency of processing the density map. Exemplarily, the first preprocessing can filter a preset number of pixel points in the density map by means of a preset pixel threshold. Specifically, the preset pixel threshold is related to the number of pixel points to be filtered. For example, if 10% of the pixel points need to be filtered out, the preset pixel threshold is taken as the integer value obtained by dividing the maximum pixel value by 10. Another example, if 15% of the pixel points need to be filtered out, the preset pixel threshold is taken as the integer value obtained by dividing the maximum pixel value by 15. This embodiment can filter some pixel points in the density image, which not only improves the analysis efficiency of the target object distribution, but also can effectively reduce the power consumption of the target object aggregation device.

[0065] In specific implementation, the process of performing a second processing on the density map may include: performing grayscale processing on the density map; normalizing the grayscale-processed density map, and using the normalized density map as the density map after the second preprocessing.

[0066] Among them, the density map can be grayscaled based on any one of the algorithms of the component method, the maximum value method, the average value method, or the weighted average method, and the multi-channel density map is converted into a single-channel grayscale map.

[0067] Specifically, the component method is to use the brightness of the three components in the multi-channel color image as the grayscale values of three grayscale images, and in a specific embodiment, the grayscale values of the grayscale images can be selected according to application needs. The maximum value method is to use the maximum value of the brightness of the three components in the multi-channel color image as the grayscale value of the grayscale map for grayscale processing. The average value method is to obtain the grayscale value of the grayscale map by averaging the brightness of the three components in the multi-channel color image. The weighted average method is to perform weighted averaging on the three components in the multi-channel color image with different weights according to importance and other indicators to obtain the grayscale value of the grayscale image for grayscale processing.

[0068] In specific implementation, any one of the above grayscale processing methods can be flexibly selected for grayscaling the density map, and no limitation is made here.

[0069] It should be understood that in this embodiment, after the single-channel density map after grayscale processing is further normalized, the pixel values of each pixel point in the density map can be normalized between 0 and 255. This embodiment can further compare the pixel value of each pixel point with the preset pixel threshold, use the pixel points greater than the preset pixel threshold as the first pixel points, and use the other pixel points except the first pixel points as the second pixel points.

[0070] In specific implementation, the pixel values of each first pixel point can be set to 1 respectively, and the pixel values of each second pixel point can be set to 0, so as to implement the binarization processing of the first pixel points and the second pixel points and obtain a binary image; further, the pixel values of each pixel point in the density map after the second preprocessing are multiplied by the pixel values of the corresponding pixel points in the binary image respectively, so as to implement the fusion of the density map after the second preprocessing and the binary image. When the density map after the second preprocessing and the binary image are fused, it is possible to effectively remove each pixel point with a pixel value of 0 in the binary image, and remove the part of the density map that does not contain the target object area. Furthermore, by analyzing the distribution of the target object in the density map that only contains the target object, it is possible to efficiently and accurately screen the distribution area of the target object, so as to determine the target area that needs to perform the target object collection, and improve the collection efficiency of the target object.

[0071] In one embodiment, fusing the density map after the first preprocessing with the density level to obtain the distribution area of each target object in the density map after the first preprocessing may include: multiplying each pixel value in the density map after the first preprocessing by the density level to obtain a regional density map; performing binarization processing on the regional density map; and dividing the distribution area where each target object is located according to the regional density map after the binarization processing.

[0072] Among them, the density map after the first preprocessing is a density map from which some pixel points are filtered out. Specifically, the pixel values of the filtered-out part of the pixel points do not meet the preset pixel threshold, for example, are less than or equal to the preset pixel threshold.

[0073] In a specific embodiment, each pixel value in the density map from which some pixel points are filtered out can be multiplied by the density level, effectively reducing the workload of the target object collection device, reducing the power consumption of the target object collection device, and improving the work efficiency at the same time.

[0074] Specifically, the pixel value of each pixel point in the regional density map can be used to represent the density of the corresponding pixel point in the environmental image. Therefore, the regional density map can display the density distribution of the environmental image, and further display the distribution of the target object. Specifically, the area where the pixel points with a pixel value of 0 in the regional density map gather can represent that there is no target object in the corresponding area of the environmental image, and the area where the pixel points with a pixel value not equal to 0 in the regional density map gather can represent that there is a target object in the corresponding area of the environmental image. The target object in the environmental image can be represented by multiple pixel points with different pixel value distributions in the regional density map. Therefore, the distribution areas where the target object is located in the regional density map can be divided according to the pixel values of multiple pixel points.

[0075] In one embodiment, the region density map can be binarized. For example, the pixel values of multiple pixel points in the region representing the aggregation of the target objects are binarized respectively with a preset threshold. The pixel values of each pixel point in the region representing the aggregation of the target objects are set to 1, and the values of each pixel point in the region representing no aggregation of the target objects are set to 0. According to the binarized region density map, the respective distribution regions where the target objects are located can be divided more efficiently.

[0076] Specifically, please refer to Figure 6 shown in Figure 6 which is the binarized region density map corresponding to the region density map in the embodiment of the present application. As can be seen from Figure 6 it, in the binarized region density map 610, the pixel values in regions 611 and 612 are 1, and the pixel values in the remaining regions are 0. A pixel value of 1 indicates that there are target objects in the corresponding regions 611 and 612, and a pixel value of 0 indicates that there are no target objects in other regions except regions 611 and 612. In specific implementation, the region label categories can be set in advance according to the pixel values in each region, or also referred to as setting the pixel value categories according to the pixel values in each region first. Specifically, there is an associated mapping relationship between the region label categories and the pixel values in the region. The region label category with a pixel value of 1 represents that there is an aggregation of target objects in the region, and the region label category with a pixel value of 0 represents that there is no aggregation of target objects in the region. Among them, the region label category can be composed of numbers, symbols, characters, or any combination of numbers, symbols, and characters, and no specific limitation is made.

[0077] In one embodiment, screening the respective distribution regions to determine the target region may include: obtaining the categories corresponding to the respective distribution regions. Connecting the distribution regions of the same category to obtain multiple connected regions. Obtaining the areas of the multiple connected regions; determining the connected regions with areas greater than or equal to a preset area threshold as the target regions.

[0078] Among them, the categories corresponding to the respective distribution regions include the pre-set region label categories, and the region label category represents the presence or absence of target objects in the corresponding distribution region. Connecting the respective distribution regions where the target objects are present can obtain the connected regions of the respective distribution regions where the target objects are present. Further, the size of the preset area threshold can be set in advance according to the need for target object aggregation, and no specific limitation is made.

[0079] S203. Generate a control instruction according to the target region, where the control instruction is used to control the target object aggregation device to run to the target region and then aggregate the target objects.

[0080] The target area is the distribution area containing the target objects, and the area of the target area is greater than a preset area threshold. A control instruction is generated according to the target area, and the control instruction is used to control the target object collection device to run to the target area and collect the target objects in the target area.

[0081] As can be seen from the above analysis, for the target object collection method, device, equipment and storage medium provided by the embodiments of the present application, first, it is necessary to perform target object density analysis on the environmental image to obtain the density map of the environmental image and the density level of the density map; then, when the density level meets the preset level threshold, determine the target area of the target object in the density map according to the density map and the density level; and then generate a control instruction according to the target area, where the control instruction is used to control the target object collection device to collect the target objects after running to the target area. By determining the target area where the target objects are located according to the density map of the target objects in the environmental image and the density level corresponding to the density map, it is possible to effectively control the target object collection device to collect the target objects in the target area where the target objects are relatively concentrated, without the need to perform full-area target object collection, effectively reducing the workload of the target object collection device and improving the efficiency of target object collection.

[0082] In the above text, in combination with Figures 1 to 6 ..., the target object collection method provided by the present application is described in detail. Next, the target object collection device and the target object collection equipment provided by the present application will be described with reference to the accompanying drawings.

[0083] See Figure 1 In the schematic structural diagram of the target object collection device in the system architecture diagram shown, the target object collection device 1200 includes:

[0084] An acquisition module 1201, configured to perform target object density analysis on the environmental image to obtain the density map of the environmental image and the density level of the density map;

[0085] A determination module 1202, configured to determine the target area of the target object in the density map according to the density map and the density level when the density level meets the preset level threshold;

[0086] A generation module 1203, configured to generate a control instruction according to the target area, and the control instruction is used to control the target object collection device to collect the target objects after running to the target area.

[0087] In one embodiment, the acquisition module 1201 includes:

[0088] An acquisition unit for acquiring a feature map of the environmental image; a generation unit for segmenting the feature map to obtain the density map; a classification unit for classifying the feature map according to a preset density value to determine the density level, where the density value corresponds to the density level one by one.

[0089] In one embodiment, the determination module 1202 includes: a first processing unit for performing a first preprocessing on the density map according to a preset pixel threshold; a second processing unit for fusing the density map after the first preprocessing with the density level to obtain the distribution regions of each target object in the density map after the first preprocessing; and a screening unit for screening each distribution region to determine the target region.

[0090] In one embodiment, the first processing unit includes: a first processing subunit for performing a second preprocessing on the density map; taking the pixel points in the density map after the second preprocessing whose pixel values are greater than the preset pixel threshold as first pixel points, and taking the pixel points other than the first pixel points as second pixel points; a second processing subunit for performing a binarization process on the first pixel points and the second pixel points to obtain a binarized image; and a fusion subunit for fusing the density map after the second preprocessing and the binarized image to obtain the density map after the first preprocessing.

[0091] In one embodiment, the first processing subunit is specifically configured to: perform a grayscale process on the density map; perform a normalization process on the density map after the grayscale process, and use the density map obtained by the normalization process as the density map after the second preprocessing.

[0092] In one embodiment, the second processing unit includes: a first generation subunit for multiplying each pixel value in the density map after the first preprocessing by the density level to obtain a regional density map; a third processing subunit for performing a binarization process on the regional density map; and a division subunit for dividing the distribution regions where each target object is located according to the regional density map after the binarization process.

[0093] In one embodiment, the screening unit includes: a first acquisition subunit for acquiring the categories corresponding to each distribution region; a second generation subunit for connecting the distribution regions of the same category to obtain a plurality of connected regions; a second acquisition subunit for acquiring the areas of the plurality of connected regions; and a determination subunit for determining the connected regions whose areas are greater than or equal to a preset area threshold as the target regions.

[0094] In one embodiment, the acquisition module 1201 includes an environmental image acquisition unit for acquiring an environmental image when a target object is detected.

[0095] The object counting device according to the embodiment of the present application may correspond to executing the methods described in the embodiments of the present application, and the above and other operations and / or functions of each module in the object counting device are respectively for implementing Figure 2 the corresponding processes of the respective methods in

[0096] In addition, it should be noted that the above-described embodiments are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0097] Please refer to Figure 7 as shown in Figure 7 the schematic block diagram of the target object collection device provided by the embodiment of the present application.

[0098] As Figure 7 shown, the target object collection device 100 includes a processor 101, a memory 102, a communication interface 103, and a bus 104. Among them, the processor 101, the memory 102, and the communication interface 103 communicate through the bus 104, and can also achieve communication through other means such as wireless transmission. The memory 102 stores executable program codes, and the processor 101 can call the program codes stored in the memory 102 to execute the target object collection method in the foregoing method embodiments.

[0099] It should be understood that in the embodiment of the present application, the processor 101 may be a central processing unit CPU, and the processor 101 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0100] The memory 102 may include a read-only memory and a random access memory, and provide instructions and data to the processor 101. The memory 102 may also include a non-volatile random access memory. For example, the memory 102 may also store a data set.

[0101] The memory 102 may be a volatile memory, a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0102] In addition to including a data bus, the bus 104 may further include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, all kinds of buses are labeled as the bus 104 in the figure.

[0103] It should be understood that the target object collection device 120 according to the embodiment of the present application may correspond to the target object collection device in the embodiment of the present application, and may correspond to the corresponding main body that executes the method shown in the embodiment of the present application Figure 2 and the above and other operations and / or functions of each device in the target object collection device 120 are respectively for implementing the corresponding processes of each method in Figure 2 For the sake of brevity, they will not be described in detail here.

[0104] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits or dedicated circuits, etc.

[0105] However, for this application, in more cases, software program implementation is a better implementation method. Based on such an understanding, the technical solution of this application, in essence, or the part that makes contributions to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disc of a computer, etc., and includes several instructions to enable a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of this application.

[0106] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0107] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices.

[0108] The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device or data center to another website, computer, training device or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A method for aggregating target objects, the method comprising: Perform target object density analysis on the environmental image to obtain the density map of the environmental image and the density level of the density map; The environmental image is obtained by a camera; When the density level meets the preset level threshold, determine the target area of the target object in the density map according to the density map and the density level; Generate a control instruction according to the target area. The control instruction is used to control the target object collection device to run to the target area and collect the target object in the target area; Wherein, the target object collection device is connected to the camera through a network; The determining the target area of the target object in the density map according to the density map and the density level includes: Obtain the category corresponding to each distribution area; Connect the distribution areas of the same category to obtain multiple connected areas; Obtain the areas of multiple connected areas; Determine the connected areas with an area greater than or equal to the preset area threshold as the target area.

2. The method according to claim 1, wherein, The performing target object density analysis on the environmental image to obtain the density map of the environmental image and the density level of the density map includes: Obtain the feature map of the environmental image; Segment the feature map to obtain the density map; Classify the feature map according to a preset density value to determine the density level, and the density value corresponds to the density level one by one.

3. The method according to claim 1, wherein, The determining the target area of the target object in the density map according to the density map and the density level includes: Perform a first preprocessing on the density map according to a preset pixel threshold; Fuse the density map after the first preprocessing with the density level to obtain the distribution areas of each target object in the density map after the first preprocessing; Screen each distribution area to determine the target area.

4. The method according to claim 3, wherein, The performing a first preprocessing on the density map according to a preset pixel threshold includes: Perform a second preprocessing on the density map; Take the pixel points in the density map after the second preprocessing with pixel values greater than the preset pixel threshold as the first pixel points, and take the pixel points other than the first pixel points as the second pixel points; Perform binarization processing on the first pixel points and the second pixel points to obtain a binarized image; Fuse the density map after the second preprocessing and the binarized image to obtain the density map after the first preprocessing.

5. The method according to claim 4, wherein, The performing a second preprocessing on the density map includes: Perform grayscale processing on the density map; Normalize the density map after the grayscale processing and use the normalized density map as the density map after the second preprocessing.

6. The method according to claim 4, wherein, The fusing the density map after the first preprocessing with the density level to obtain the distribution areas of each target object in the density map after the first preprocessing includes: Multiply each pixel value in the density map after the first preprocessing by the density level to obtain a regional density map; Perform binarization processing on the regional density map; Divide the distribution areas where each target object is located according to the binarized regional density map.

7. The method according to claim 1, wherein, The method further includes: When a target object is detected, obtain an environmental image.

8. A target object aggregation device, comprising a memory and a processor; The memory is configured to store a computer program; The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the target object collection method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, where when the computer program is executed by a processor, the processor is caused to implement the steps of the target object collection method according to any one of claims 1 to 7.

Images