Cockroach Bait Placement Method, Device, Equipment and Storage Medium

Abstract

The present invention relates to the field of artificial intelligence, and discloses a cockroach bait dispensing method, device, equipment and storage medium. The method includes: collecting video data of a region to be recognized within a preset time interval, and performing frame division processing on the video data to obtain multiple video screenshots; performing cockroach recognition on the multiple video screenshots to obtain a first recognition quantity of a preset region within the preset time interval; calculating a current second recognition quantity of the region to be recognized according to the first recognition quantity; calculating a total bait dispensing quantity of the current region to be recognized according to the second recognition quantity; formulating a bait dispensing plan according to the total bait dispensing quantity, and after implementing the bait dispensing plan, collecting disinfection and extermination data in real time; and adjusting the bait dispensing plan in real time based on the disinfection and extermination data. This method recognizes cockroaches through image recognition technology and records the corresponding cockroach trajectories, collects big data of cockroach activity trajectories based on the records of the system, establishes an information model, and can calculate the optimal dispensing plan.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and particularly to a method, device, equipment and storage medium for placing cockroach bait. Background Art

[0002] Cockroaches like to appear in places such as warm, humid, food-rich and multi-crack areas, especially in Guangdong which is located in a low-latitude area. As one of the four pests, cockroaches carry various bacteria, worms, etc., making people extremely hate them. Moreover, cockroaches have characteristics such as strong reproductive ability and environmental adaptability, which increases the difficulty of killing and eliminating them. Currently, the main method of killing and eliminating cockroaches on the market is to place bait, but it is all done based on experience. When placing bait according to experience, the house owner cannot know the specific number of cockroaches in the house and the growth situation of the cockroaches in the house, and just places cockroach bait based on experience, resulting in poor effects. The cockroaches in the house are not completely killed, and the number of cockroaches returns to the number before cockroach control after a period of time, making the cockroach control ineffective. Summary of the Invention

[0003] The main purpose of the present invention is to solve the technical problem of poor killing and eliminating effect of the existing cockroach bait placement.

[0004] The first aspect of the present invention provides a method for placing cockroach bait, including: collecting video data of a to-be-identified area within a preset time interval, and performing frame division processing on the video data to obtain multiple video screenshots; performing cockroach recognition on the multiple video screenshots to obtain a first recognition quantity of a preset area within the preset time interval; calculating a second recognition quantity of the to-be-identified area currently according to the first recognition quantity; calculating a total bait placement quantity of the to-be-identified area currently according to the second recognition quantity; formulating a bait placement plan according to the total bait placement quantity, and after implementing the bait placement plan, collecting killing and eliminating data in real time; and adjusting the bait placement plan in real time based on the killing and eliminating data.

[0005] In this embodiment, in the first implementation manner of the first aspect of the present invention, the performing cockroach recognition on the multiple video screenshots to obtain a first recognition quantity of a preset area within the preset time interval: performing image preprocessing on the multiple video screenshots to determine a foreground area in the multiple video screenshots; based on a preset cockroach recognition algorithm, performing cockroach recognition on the foreground area in the multiple video screenshots to determine and mark the cockroach contours in each video screenshot; and calculating a first recognition quantity of the preset area within the preset time interval based on the cockroach contours in each video screenshot.

[0006] In this embodiment, in the second implementation manner of the first aspect of the present invention, the image preprocessing of the multi-frame video screenshots to determine the foreground regions in the multi-frame video screenshots includes: performing grayscale processing on the multi-frame video screenshots to obtain grayscale images corresponding to each video screenshot; performing binary segmentation on the grayscale images by the maximum inter-class variance method to obtain corresponding image segmentation results; performing opening operation processing and closing operation processing on the segmentation results to obtain the foreground regions corresponding to each frame of video screenshot.

[0007] In this embodiment, in the third implementation manner of the first aspect of the present invention, the cockroach recognition of the foreground regions in the multi-frame video screenshots based on a preset cockroach recognition algorithm, and determining and marking the cockroach contours in each frame of video screenshot includes: extracting the HOG features of the foreground regions in the multi-frame video screenshots; inputting the HOG features into a preset SVM classifier, and classifying the foreground regions through the SVM classifier to determine the foreground regions in the multi-frame video screenshots classified as cockroaches; marking the foreground regions classified as cockroaches.

[0008] In this embodiment, in the fourth implementation manner of the first aspect of the present invention, the extraction of the HOG features of the multi-frame video screenshots includes: performing color space standardization on the grayscale images by the Gamma correction method; calculating the gradient information of each pixel in the standardized grayscale images, and dividing the grayscale images into multiple cells according to the image size; calculating the gradient histogram of each cell based on the gradient information; forming each n cells into a block according to the positional relationship of the cells in the grayscale images; concatenating the gradient histograms of all blocks to obtain the HOG features corresponding to the video screenshots.

[0009] In this embodiment, in the fifth implementation manner of the first aspect of the present invention, the calculation of the first recognition quantity of the set region within a preset time interval based on the cockroach contours in each frame of video screenshot includes: tracking the marked cockroach contours in each frame of video screenshot through a preset tracking algorithm to obtain the cockroach trajectories in the video data; counting the number of times of passing through a preset virtual dividing line in the to-be-recognized region by the cockroach trajectories; dividing the number of times by the time length of the time interval to obtain the first recognition quantity.

[0010] In this embodiment, in the sixth implementation manner of the first aspect of the present invention, the real-time adjustment of the bait placement plan based on the disinfection and killing data includes: determining a third recognition quantity in the area to be recognized after implementing the bait placement plan based on the disinfection and killing data; calculating the total bait placement quantity in the area to be recognized after the bait placement plan based on the third recognition quantity; and adjusting the bait placement plan based on the total bait placement quantity in the area to be recognized after the bait placement plan.

[0011] The second aspect of the present invention provides a cockroach bait placement device, including: a video processing module, configured to collect video data of an area to be recognized within a preset time interval, and perform frame division processing on the video data to obtain multiple video screenshots; an identification module, configured to perform cockroach identification on the multiple video screenshots to obtain a first recognition quantity in a preset area within a preset time interval; a first calculation module, configured to calculate a current second recognition quantity in the area to be recognized according to the first recognition quantity; a second calculation module, configured to calculate the total bait placement quantity in the current area to be recognized according to the second recognition quantity; a plan formulation module, configured to formulate a bait placement plan according to the total bait placement quantity, and collect disinfection and killing data in real time after implementing the bait placement plan; and an adjustment module, configured to perform real-time adjustment of the bait placement plan based on the disinfection and killing data.

[0012] In this embodiment, in the first implementation manner of the second aspect of the present invention, the preprocessing unit is specifically configured to: a preprocessing unit, configured to perform image preprocessing on the multiple video screenshots to determine a foreground area in the multiple video screenshots; a contour marking unit, configured to perform cockroach identification on the foreground area in the multiple video screenshots based on a preset cockroach identification algorithm, and determine and mark the cockroach contours in each video screenshot; and a quantity calculation unit, configured to calculate a first recognition quantity in a preset area within a preset time interval based on the cockroach contours in each video screenshot.

[0013] In this embodiment, in the second implementation manner of the second aspect of the present invention, the preprocessing unit is specifically configured to: perform grayscale processing on the multiple video screenshots to obtain grayscale images corresponding to the respective video screenshots; perform binary segmentation on the grayscale images by using the maximum inter-class variance method to obtain corresponding image segmentation results; and perform opening operation processing and closing operation processing on the segmentation results to obtain foreground areas corresponding to the respective video screenshots.

[0014] In this embodiment, in the third implementation manner of the second aspect of the present invention, the contour marking unit specifically includes: a feature extraction subunit extracts the HOG features of the foreground regions in multiple frames of the video screenshots; a classification subunit, which inputs the HOG features into a preset SVM classifier, classifies the foreground regions through the SVM classifier, and determines the foreground regions in multiple frames of the video screenshots that are classified as cockroaches; and a marking subunit, which is used to mark the foreground regions classified as cockroaches.

[0015] In this embodiment, in the fourth implementation manner of the second aspect of the present invention, the feature extraction subunit is specifically configured to: standardize the color space of the grayscale image through the Gamma correction method; calculate the gradient information of each pixel in the standardized grayscale image, and divide the grayscale image into multiple cells according to the image size; calculate the gradient histogram of each cell based on the gradient information; form each n cells into a block according to the positional relationship of the cells in the grayscale image; and concatenate the gradient histograms of all the blocks to obtain the HOG features corresponding to the video screenshots.

[0016] In this embodiment, in the fifth implementation manner of the second aspect of the present invention, the quantity calculation unit is specifically configured to: track the marked cockroach contours in each frame of the video screenshot through a preset tracking algorithm to obtain the cockroach trajectories in the video data; count the number of times the cockroach trajectories cross a preset virtual dividing line in the to-be-identified area; and divide the number of times by the time length of the time interval to obtain the first identification quantity.

[0017] In this embodiment, in the sixth implementation manner of the second aspect of the present invention, the adjustment module is specifically configured to: determine the third identification quantity in the to-be-identified area after implementing the bait placement plan based on the pest control data; calculate the total bait placement amount in the to-be-identified area after the bait placement plan based on the third identification quantity; and adjust the bait placement plan based on the total bait placement amount in the to-be-identified area after the bait placement plan.

[0018] The third aspect of the present invention provides a cockroach bait placement device, including: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected through a line; the at least one processor calls the instructions in the memory so that the cockroach bait placement device executes the steps of the above-mentioned cockroach bait placement method.

[0019] The fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when the instructions are run on a computer, the computer is made to execute the steps of the above-mentioned cockroach bait placement method.

[0020] In the technical solution of the present invention, video data of the area to be recognized within a preset time interval is collected, and the video data is frame-divided to obtain multiple video screenshots; cockroach recognition is performed on the multiple video screenshots to obtain the first recognition quantity of the preset area within the preset time interval; the current second recognition quantity of the area to be recognized is calculated according to the first recognition quantity; the total bait delivery quantity of the area to be recognized currently is calculated according to the second recognition quantity; a bait delivery plan is formulated according to the total bait delivery quantity, and after implementing the bait delivery plan, disinfection data is collected in real time; the bait delivery plan is adjusted in real time based on the disinfection data. This method recognizes cockroaches through image recognition technology and records the corresponding cockroach trajectories, collects big data of cockroach activity trajectories based on the records of the system, establishes an information model, and can calculate the optimal delivery plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the first embodiment of the cockroach bait delivery method in the embodiment of the present invention;

[0022] Figure 2 Schematic diagram of the second embodiment of the cockroach bait delivery method in the embodiment of the present invention;

[0023] Figure 3 Schematic diagram of the third embodiment of the cockroach bait delivery method in the embodiment of the present invention;

[0024] Figure 4 Schematic diagram of an embodiment of the cockroach bait delivery device in the embodiment of the present invention;

[0025] Figure 5 Schematic diagram of another embodiment of the cockroach bait delivery device in the embodiment of the present invention;

[0026] Figure 6 Schematic diagram of an embodiment of the cockroach bait delivery equipment in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In the technical solution of the present invention, video data of a region to be recognized within a preset time interval is collected, and the video data is frame-divided to obtain multiple video screenshots; cockroach recognition is performed on the multiple video screenshots to obtain a first recognition quantity of a preset region within the preset time interval; a second recognition quantity of the region to be recognized at present is calculated according to the first recognition quantity; the total bait delivery quantity of the region to be recognized at present is calculated according to the second recognition quantity; a bait delivery plan is formulated according to the total bait delivery quantity, and after the bait delivery plan is implemented, disinfection data is collected in real time; and the bait delivery plan is adjusted in real time based on the disinfection data. This method uses image recognition technology to recognize cockroaches and record the corresponding cockroach trajectories, collects trajectory big data of cockroach activities based on the records of the system, establishes an information model, and can calculate the optimal delivery plan.

[0028] In the description of the present invention, the terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that comprises a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these process, method, product or device.

[0029] For ease of understanding, the specific process of the embodiments of the present invention will be described below. Please refer to Figure 1 , the first embodiment of the cockroach bait delivery method in the embodiments of the present invention includes:

[0030] 101. Collect video data of a region to be recognized within a preset time interval, and perform frame division on the video data to obtain multiple video screenshots;

[0031] It can be understood that the execution subject of the present invention can be a cockroach bait delivery device, or a terminal or a server, and specific limitations are not made here. In the embodiments of the present invention, the server is used as the execution subject for illustration. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0032] In practical applications, the video data may be for a specific area where cockroaches are likely to breed, such as the back kitchen of a catering enterprise or the kitchen area in a normally inhabited building. The present invention is not limited in this regard. Taking the back kitchen of a catering enterprise as an application scenario, the preset area is an area covering links such as raw material storage, cleaning, cutting, cooking, sample retention, and tableware washing and disinfection. The video data is collected by placing cameras in the above areas and through the interfaces provided by the cameras to achieve video monitoring of the entire process of the application scenario to be recognized.

[0033] In this embodiment, the cameras may include a black-and-white CCD (Charge-coupled Device) camera and a color CCD camera for visible light monitoring. Among them, the black-and-white camera can sense the characteristics of infrared light and cooperate with an infrared lamp to achieve night vision video live broadcast; the color CCD camera is used to achieve video live broadcast in an illuminated environment. Therefore, in a preferred embodiment of the present invention, the combination of the black-and-white CCD camera and the color CCD camera forms a set of all-weather video monitoring system.

[0034] In this embodiment, various processing tools can be used to disassemble the frames of the video according to different video frame rates.

[0035] Specifically, one implementation method can use the OpenCV package in Python to perform frame splitting on the video data to be frame-split. It should be noted that the frame splitting operation cannot be performed in units of time such as seconds, but the frame rate of the video should be obtained to achieve the splitting process of each frame of the picture.

[0036] 102. Identify cockroaches in multiple video screenshots to obtain the first identification quantity in the preset area within the preset time interval;

[0037] In this embodiment, it is necessary to perform image preprocessing on multiple video screenshots. After loading the video, draw in the video window based on OpenCV and Python to distinguish the movement of the background and the object (foreground) to separate the object. Generally, a black image is used as the background, and a white image is the object to be detected. Morphological operations in the image, including erosion, dilation, and opening and closing operations, etc., are used to refine the video for image preprocessing and post-processing. This is convenient for subsequent model training and image contour separation.

[0038] In this embodiment, the image contour of the preprocessed image is separated, and the separated contour is recognized, so as to recognize the contour of the cockroach therein. The contour of the cockroach will be marked, and through a preset tracking algorithm, the marked contour of the cockroach is calculated, and the corresponding first recognition quantity of the cockroach can be obtained. The first recognition quantity is the average number of cockroaches within the statistical time window, denoted as M2.

[0039] 103. Calculate the current second recognition quantity of the area to be recognized according to the first recognition quantity;

[0040] In this embodiment, taking the back kitchen of a catering enterprise as an application scenario, the preset area is the area covering links such as raw material storage, cleaning, cutting, cooking, sample retention, tableware washing and disinfection, which is the area captured by the camera, and the area to be recognized is the entire back kitchen of the catering enterprise. Taking the kitchen in the house as the preset area, the area to be recognized is the entire house.

[0041] In practical applications, due to the strong reproductive ability of cockroaches, a single female can produce at least more than 10 and at most more than 90 oothecae; in one ootheca, at least 10 and at most more than 50 small cockroaches can hatch. Therefore, a cockroach reproduction model needs to be added to the base number statistically obtained from the video trajectory to obtain the final total number of cockroaches. The vitality of cockroaches is very strong. A mature female cockroach can produce 14 - 40 eggs in about a week. At the beginning of the statistics, 1 / 4 of the existing cockroaches are taken as the number of female adult cockroaches, which is 0.25M1. For each adult female cockroach, starting from the 7th day, every 7 days, it can produce an ootheca containing 14 - 40 eggs. The female cockroaches will not die within the statistical window, and the larvae will mature and produce new oothecae. Through the above process, combined with the recursive simulation algorithm, the final total number of cockroaches (including oothecae) is calculated, which is M2.

[0042] 104. Calculate the total bait delivery amount of the current area to be recognized according to the second recognition quantity;

[0043] In this embodiment, it is assumed that a 30 - gram Jie Bing cockroach gel bait can be used for an area of 50 - 100 square meters. For an area of 100 square meters, 2 baits are used to kill 300 cockroaches and cockroach eggs. The bait base number for each cockroach is 2 / 100 * 300 g / m2 * cockroach. Then the total bait delivery amount is M2 / 100 * 300 * 7 baits.

[0044] 105. Formulate a bait delivery plan according to the total bait delivery amount, and after implementing the bait delivery plan, collect the disinfection and killing data in real time;

[0045] 106. Adjust the bait delivery plan in real time based on the disinfection and killing data.

[0046] In this embodiment, the real-time adjustment of the bait delivery plan based on the disinfection and killing data includes determining the third recognition quantity in the area to be recognized after implementing the bait delivery plan based on the disinfection and killing data; calculating the total amount of bait to be delivered in the area to be recognized after the bait delivery plan based on the third recognition quantity; and adjusting the bait delivery plan based on the total amount of bait to be delivered in the area to be recognized after the bait delivery plan.

[0047] In this embodiment, by collecting video data of the area to be recognized within a preset time interval and performing frame-by-frame processing on the video data, multiple video screenshots are obtained; cockroaches are recognized from the multiple video screenshots to obtain the first recognition quantity in the preset area within the preset time interval; the current second recognition quantity of the area to be recognized is calculated according to the first recognition quantity; the total amount of bait to be delivered in the current area to be recognized is calculated according to the second recognition quantity; a bait delivery plan is formulated according to the total amount of bait to be delivered, and after implementing the bait delivery plan, disinfection and killing data are collected in real time; the bait delivery plan is adjusted in real time based on the disinfection and killing data. This method uses image recognition technology to recognize cockroaches and record the corresponding cockroach trajectories, collects big data on cockroach activity trajectories based on the system's records, establishes an information model, and can calculate the optimal delivery plan.

[0048] Please refer to Figure 2 , the second embodiment of the cockroach bait delivery method in the embodiment of the present invention includes:

[0049] 201. Collect video data of the area to be recognized within a preset time interval and perform frame-by-frame processing on the video data to obtain multiple video screenshots;

[0050] 202. Perform grayscale processing on the multiple video screenshots to obtain grayscale images corresponding to the respective video screenshots;

[0051] In this embodiment, in order to adjust the contrast of the image color, the average of the red, green, and blue color values of each pixel point in the sample test result picture is taken as the grayscale value of the image, and normalization processing is performed on each pixel value in the grayscale image to obtain a normalized image matrix G.

[0052] 203. Perform binary segmentation on the grayscale image by the maximum inter-class variance method to obtain the corresponding image segmentation result;

[0053] In practical applications, the Otsu method divides an image into a background part and an object part according to the gray-scale characteristics of the image. The greater the between-class variance between the background and the object, the greater the difference between the two parts that make up the image. When some objects are misclassified as the background or some background is misclassified as an object, the difference between the two parts will become smaller. Therefore, the segmentation that maximizes the between-class variance means the minimum misclassification probability. The Otsu method divides the image into a background part and an object part, and then performs segmentation on the background and object parts to obtain the segmentation result.

[0054] 204. Perform opening operation and closing operation on the segmentation result to obtain the foreground region corresponding to each frame of the video screenshot;

[0055] In this embodiment, the opening operation is mainly an erosion operation first and then a dilation operation, and the closing operation is a dilation operation first and then an erosion operation. The opening operation can eliminate isolated points higher than their neighboring points, eliminate thin points between object boundaries without significantly changing their area, and at the same time play a smoothing role; the closing operation can eliminate isolated points lower than their neighboring points, fill and smooth the boundaries of neighboring objects without significantly changing their area.

[0056] 205. Standardize the color space of the grayscale image through the Gamma correction method;

[0057] To reduce the influence caused by local shadows and illumination changes in the image, and at the same time suppress the interference of noise, define the correction value as u, perform Gamma correction on each pixel value G(x, y) in G, and calculate each pixel value G'(x, y) in the corrected image matrix G' = G(x, y)u; where G(x, y) represents the pixel value corresponding to the element in the x-th row and y-th column of the image matrix G.

[0058] 206. Calculate the gradient information of each pixel in the standardized grayscale image, and divide the grayscale image into multiple cells according to the image size;

[0059] In this embodiment, to provide a coding for a local image region and at the same time be weakly sensitive to the pose and appearance of the human object in the image. Divide the image matrix G' into several cells, each cell contains m*m (such as 8*8) pixels, and statistically calculate the gradient information of each cell.

[0060] 207. Calculate the gradient histogram of each cell based on the gradient information;

[0061] In this embodiment, in order to capture contour information and further weaken the interference of illumination, the gradients of each pixel value G'(x, y) in the image matrix G' after Gama correction are calculated in the horizontal and vertical directions, and the gradient direction value and gradient direction of G'(x, y) are calculated accordingly, where G'(x, y) represents the pixel value corresponding to the element in the x-th row and y-th column of G'. The specific steps are as follows:

[0062] Calculate the horizontal gradient dx(x, y) of G'(x, y) = G'(x, y + 1) - G'(x, y).

[0063] Calculate the vertical gradient dy(x, y) of G'(x, y) = G'(x + 1, y) - G'(x, y)

[0064] Calculate the gradient direction value and gradient direction of G'(x, y).

[0065] In this context, the 360-degree gradient direction is divided into 18 direction blocks f1, f2,..., f18. That is, the gradient direction range of the first direction block f1 is [0, 1 / 18), and the gradient direction calculation formula is [((i - 1) * 360) / k, (i * 360) / k). i represents the number of the direction block.

[0066] Judge each pixel in the cell. If the gradient direction α(x, y) of the pixel value G(x, y) belongs to fi, then αi = αi + d(x, y), where αi represents the variable of the number of gradients in the i-th direction. Such an operation can obtain the gradient information of each cell.

[0067] 208. According to the positional relationship of the cells in the grayscale image, every n cells are grouped into a block;

[0068] 209. Concatenate the gradient histograms of all blocks to obtain the HOG features corresponding to the video screenshot;

[0069] In this embodiment, every several cells are grouped into a block (for example, 3 * 3 cells / block). The feature descriptors of all cells within a block are concatenated to obtain the HOG feature of that block.

[0070] In this embodiment, the HOG features of all blocks within the object to be detected in the image are concatenated to obtain the HOG feature of the object to be detected. And they are combined into a final feature vector for classification use.

[0071] 210. Input the HOG features into a preset SVM classifier, classify the foreground regions through the SVM classifier, and determine the foreground regions in multiple video screenshots classified as cockroaches.

[0072] In this embodiment, before cockroach recognition, an SVM classifier for classification needs to be trained, mainly including:

[0073] 1) Sample feature calculation: Extract the feature values of the test result sample pictures through the HOG feature algorithm, and synthesize a set of training samples (A1, B1), (A2, B2),..., (Ai, Bi) from the feature values and the corresponding expected error types. Ai represents the feature value of the i-th sample image, and Bi represents the type of error reason corresponding to the feature value of this image.

[0074] 2) Train the SVM classifier: Define an objective function

[0075] At the same time, calculate β = {β1, β2,..., βi}

[0076] Maximize the Q(β) function while satisfying the condition

[0077] Calculate the value of β, calculate the weight vector and the bias value b = 1 - W*X, and finally obtain the SVM class discrimination function f(x) = sgn(W*X) + b. Extract the feature vector values according to the HOG feature values of the foreground regions, input the feature vector values into all trained SVM classifiers, count the results obtained by each classifier, and the category with the highest number of votes is the classification result of the foreground region, and determine the foreground region classified as a cockroach.

[0078] 211. Mark the foreground regions classified as cockroaches.

[0079] 212. Track the marked cockroach contours in each frame of the video screenshot through a preset tracking algorithm to obtain the cockroach trajectories in the video data.

[0080] In this embodiment, in the application based on video structuring, after passing through the tracking algorithm, the target will obtain a unique identifier and its corresponding motion trajectory. Based on the above red dot marking definition, track the moving direction of the object. In the first frame, save the initial position of the detected ID object. Then, in the next frame, to continue tracking the object, the contour of the object in the frame must be matched with the ID at the first appearance, and the coordinates of the object are saved. Then, after the object crosses the boundary (or a certain amount of limit) of the video, use the stored position to evaluate the moving direction and draw the relevant trajectory.

[0081] 213. Count the number of times the cockroach trajectories cross the preset virtual dividing line in the area to be recognized.

[0082] In the coordinates of the relevant trajectory, within the range where cockroaches appear, the principle of automatic counting is used to estimate the total number of cockroaches. The counting algorithm should also assume a virtual dividing line. When the target crosses this virtual dividing line, the total count is incremented by one. As mentioned before, after the target is processed by the tracking algorithm, a unique target identifier and a corresponding set of moving trajectory coordinate points will be obtained. With these two pieces of data, it is possible to quickly determine whether a certain target crosses a certain straight line.

[0083] 214. Divide the number of times by the time length of the time interval to obtain the first recognition quantity.

[0084] 215. Calculate the current second recognition quantity of the area to be recognized according to the first recognition quantity.

[0085] 216. Calculate the total amount of bait to be put in the current area to be recognized according to the second recognition quantity.

[0086] 217. Formulate a bait placement plan according to the total amount of bait to be put, and after implementing the bait placement plan, collect disinfection and extermination data in real time.

[0087] 218. Adjust the bait placement plan in real time based on the disinfection and extermination data.

[0088] Based on the previous embodiment, this embodiment details the process of cockroach recognition of the multi-frame video screenshots to obtain the first recognition quantity in the preset area within the preset time interval. By performing image preprocessing on the multi-frame video screenshots, the foreground area in the multi-frame video screenshots is determined; based on the preset cockroach recognition algorithm, cockroaches in the foreground area of the multi-frame video screenshots are recognized, and the cockroach contours in each frame of the video screenshot are determined and marked; based on the cockroach contours in each frame of the video screenshot, the first recognition quantity in the preset area within the preset time interval is calculated. The present invention first recognizes the foreground area of the video screenshot through the preset cockroach recognition algorithm, and then classifies the foreground area to achieve cockroach recognition. Through two recognitions, the accuracy of cockroach recognition is improved. In addition, this method recognizes cockroaches through image recognition technology and records the corresponding cockroach trajectories, collects the trajectory big data of cockroach activities based on the records of the system, establishes an information model, and can calculate the optimal placement plan.

[0089] Please refer to Figure 3 , the third embodiment of the cockroach bait placement method in the embodiment of the present invention includes:

[0090] 301. Collect video data of the area to be recognized within the preset time interval, and perform frame splitting on the video data to obtain multi-frame video screenshots.

[0091] 302. Perform cockroach recognition on the multi-frame video screenshots to obtain the first recognition quantity in the preset area within the preset time interval.

[0092] 303. Calculate the current second recognition quantity of the area to be recognized according to the first recognition quantity;

[0093] 304. Calculate the total amount of bait to be put in the area to be recognized currently according to the second recognition quantity;

[0094] 305. Formulate a bait placement plan according to the total amount of bait to be put, and after implementing the bait placement plan, collect disinfection and killing data in real time;

[0095] 306. Determine the third recognition quantity in the area to be recognized after implementing the bait placement plan based on the disinfection and killing data;

[0096] 307. Calculate the total amount of bait to be put in the area to be recognized after the bait placement plan based on the third recognition quantity;

[0097] 308. Adjust the bait placement plan based on the total amount of bait to be put in the area to be recognized after the bait placement plan.

[0098] In this embodiment, after implementing the bait placement plan, video data of the area to be recognized is still collected, and the number of cockroaches in the area to be recognized after implementing the bait placement plan is calculated in the same way as the above process, denoted as M3. The result obtained by M2 - M3 is the number of cockroaches eliminated by implementing the bait placement plan. The effect of implementing the bait placement plan is judged based on the number of eliminated cockroaches, and the total amount of bait to be put in the area to be recognized in the future is judged based on the effect of implementing the bait placement plan, and then the bait placement plan is adjusted.

[0099] Based on the previous embodiment, this embodiment describes in detail the process of real-time adjustment of the bait placement plan based on the disinfection and killing data. By determining the third recognition quantity in the area to be recognized after implementing the bait placement plan based on the disinfection and killing data; calculating the total amount of bait to be put in the area to be recognized after the bait placement plan based on the third recognition quantity; adjusting the bait placement plan based on the total amount of bait to be put in the area to be recognized after the bait placement plan. This method uses image recognition technology to identify cockroaches and record the corresponding cockroach trajectories, collects the trajectory big data of cockroach activities based on the records of the system, establishes an information model, and can calculate the optimal placement plan.

[0100] The cockroach bait placement method in the embodiment of the present invention is described above. Next, the cockroach bait placement device in the embodiment of the present invention will be described. Please refer to Figure 4 , an embodiment of the cockroach bait placement device in the embodiment of the present invention includes:

[0101] A video processing module 401, configured to collect video data of the area to be recognized within a preset time interval, and perform frame splitting on the video data to obtain multiple video screenshots;

[0102] An identification module 402 for identifying cockroaches from the multi-frame video screenshots to obtain a first identification quantity of a preset area within a preset time interval;

[0103] A first calculation module 403 for calculating a current second identification quantity of the area to be identified according to the first identification quantity;

[0104] A second calculation module 404 for calculating a total bait delivery quantity of the area to be identified currently according to the second identification quantity;

[0105] A plan formulation module 405 for formulating a bait delivery plan according to the total bait delivery quantity, and collecting disinfection and extermination data in real time after implementing the bait delivery plan;

[0106] An adjustment module 406 for adjusting the bait delivery plan in real time based on the disinfection and extermination data.

[0107] In an embodiment of the present invention, the cockroach bait delivery device runs the above cockroach bait delivery method. The cockroach bait delivery device collects video data of an area to be identified within a preset time interval, performs frame division processing on the video data to obtain multi-frame video screenshots, identifies cockroaches from the multi-frame video screenshots to obtain a first identification quantity of a preset area within a preset time interval, calculates a current second identification quantity of the area to be identified according to the first identification quantity, calculates a total bait delivery quantity of the area to be identified currently according to the second identification quantity, formulates a bait delivery plan according to the total bait delivery quantity, and collects disinfection and extermination data in real time after implementing the bait delivery plan, and adjusts the bait delivery plan in real time based on the disinfection and extermination data. This method identifies cockroaches through image recognition technology and records the corresponding cockroach trajectories, collects big data of cockroach activity trajectories based on the records of the system, establishes an information model, and can calculate the optimal delivery plan.

[0108] Please refer to Figure 5 , the second embodiment of the cockroach bait delivery device in the embodiment of the present invention includes:

[0109] A video processing module 401 for collecting video data of an area to be identified within a preset time interval and performing frame division processing on the video data to obtain multi-frame video screenshots;

[0110] An identification module 402 for identifying cockroaches from the multi-frame video screenshots to obtain a first identification quantity of a preset area within a preset time interval;

[0111] A first calculation module 403 for calculating a current second identification quantity of the area to be identified according to the first identification quantity;

[0112] A second calculation module 404, configured to calculate the total amount of bait to be put in the area to be recognized currently according to the second recognition quantity;

[0113] A scheme formulation module 405, configured to formulate a bait placement scheme according to the total amount of bait to be put, and collect disinfection and killing data in real time after implementing the bait placement scheme;

[0114] An adjustment module 406, configured to adjust the bait placement scheme in real time based on the disinfection and killing data.

[0115] In this embodiment, the recognition module 402 is specifically configured as follows: A preprocessing unit 4021, configured to perform image preprocessing on the multi-frame video screenshots to determine the foreground areas in the multi-frame video screenshots; A contour marking unit 4022, configured to perform cockroach recognition on the foreground areas in the multi-frame video screenshots based on a preset cockroach recognition algorithm, and determine and mark the cockroach contours in each frame of video screenshot; A quantity calculation unit 4023, configured to calculate the first recognition quantity of the set area within a preset time interval based on the cockroach contours in each frame of video screenshot.

[0116] In this embodiment, the preprocessing unit 4021 is specifically configured as follows: Perform grayscale processing on the multi-frame video screenshots to obtain grayscale images corresponding to the respective video screenshots; Perform binary segmentation on the grayscale images by the maximum inter-class variance method to obtain corresponding image segmentation results; Perform opening operation processing and closing operation processing on the segmentation results to obtain the foreground areas corresponding to each frame of video screenshot.

[0117] In this embodiment, the contour marking unit 4022 specifically includes: A feature extraction subunit 40221, configured to extract the HOG features of the foreground areas in the multi-frame video screenshots; A classification subunit 40222, configured to input the HOG features into a preset SVM classifier, classify the foreground areas through the SVM classifier, and determine the foreground areas in the multi-frame video screenshots classified as cockroaches; A marking subunit 40223, configured to mark the foreground areas classified as cockroaches.

[0118] In this embodiment, the feature extraction subunit 40221 is specifically configured as follows: Perform color space standardization on the grayscale images by the Gamma correction method; Calculate the gradient information of each pixel in the standardized grayscale images, and divide the grayscale images into multiple cells according to the image size; Calculate the gradient histogram of each cell based on the gradient information; According to the positional relationship of the cells in the grayscale images, form each n cells into a block; Concatenate the gradient histograms of all blocks to obtain the HOG features corresponding to the video screenshots.

[0119] In this embodiment, the quantity calculation unit 4023 is specifically configured to: track the marked cockroach contours in each frame of video screenshot through a preset tracking algorithm to obtain the cockroach trajectories in the video data; count the number of times of passing through the preset virtual dividing line in the to-be-identified area collected for the cockroach trajectories; divide the number of times by the time length of the time interval to obtain the first identification quantity.

[0120] In this embodiment, the adjustment module 406 is specifically configured to: determine a third identification quantity in the to-be-identified area after implementing the bait delivery plan based on the pest control data; calculate the total bait delivery amount in the to-be-identified area after the bait delivery plan based on the third identification quantity; and adjust the bait delivery plan based on the total bait delivery amount in the to-be-identified area after the bait delivery plan.

[0121] On the basis of the previous embodiment, this embodiment details the specific functions of each module. Each module on the cockroach bait delivery device first identifies the foreground area of the video screenshot through a preset cockroach recognition algorithm, and then classifies the foreground area to achieve cockroach recognition. Through two identifications, the accuracy of cockroach recognition is improved. In addition, cockroaches are identified through image recognition technology and the corresponding cockroach trajectories are recorded. Based on the records of the system, big data on the cockroach activity trajectories are collected, an information model is established, and the optimal delivery plan can be calculated.

[0122] Above Figure 4 And Figure 5 The cockroach bait delivery device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. Next, the cockroach bait delivery device in the embodiment of the present invention will be described in detail from the perspective of hardware processing.

[0123] Figure 6FIG. 0 is a schematic structural diagram of a cockroach bait dispensing device provided by an embodiment of the present invention. The cockroach bait dispensing device 600 may vary significantly due to different configurations or performances, and may include one or more central processing units (CPUs) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. Among them, the memory 620 and the storage media 630 may be transient storage or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the cockroach bait dispensing device 600. Further, the processor 610 may be configured to communicate with the storage media 630 and execute a series of instruction operations in the storage media 630 on the cockroach bait dispensing device 600 to implement the steps of the above-mentioned cockroach bait dispensing method.

[0124] The cockroach bait dispensing device 600 may further include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating systems 631, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, and so on. Those skilled in the art can understand that Figure 6 the shown structural diagram of the cockroach bait dispensing device does not limit the cockroach bait dispensing device provided by the present application, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0125] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions run on a computer, the computer is caused to execute the steps of the cockroach bait dispensing method.

[0126] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, or unit can refer to the corresponding processes in the foregoing method embodiments and will not be described herein again.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0128] As described above, the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for placing cockroach bait, characterized in that, The cockroach bait placement method includes: Collecting video data of the area to be identified within a preset time interval, and performing frame division processing on the video data to obtain multiple video screenshots; Performing cockroach identification on the multiple video screenshots to obtain a first identification quantity of a preset area within a preset time interval. The performing cockroach identification on the multiple video screenshots to obtain a first identification quantity of a preset area within a preset time interval includes: performing image preprocessing on the multiple video screenshots to determine the foreground area in the multiple video screenshots; based on a preset cockroach identification algorithm, performing cockroach identification on the foreground area in the multiple video screenshots, determining and marking the cockroach contours in each video screenshot; tracking the marked cockroach contours in each video screenshot through a preset tracking algorithm to obtain the cockroach trajectories in the video data; counting the number of times the cockroach trajectories cross a preset virtual dividing line in the area to be identified; dividing the number of times by the time length of the time interval to obtain the first identification quantity; Calculating a current second identification quantity of the area to be identified according to the first identification quantity. The second identification quantity is determined based on the first identification quantity and a cockroach reproduction model, and the cockroach reproduction model is used to simulate the reproduction process of the cockroaches; Calculating the total bait placement quantity of the current area to be identified according to the second identification quantity; Formulating a bait placement plan according to the total bait placement quantity, and after implementing the bait placement plan, collecting disinfection data in real time; Performing real-time adjustment on the bait placement plan based on the disinfection data.

2. The cockroach bait dispensing method according to claim 1, characterized in that The performing image preprocessing on the multiple video screenshots to determine the foreground area in the multiple video screenshots includes: Performing grayscale processing on the multiple video screenshots to obtain grayscale images corresponding to the video screenshots; Performing binary segmentation on the grayscale images through the maximum inter-class variance method to obtain corresponding image segmentation results; Performing opening operation processing and closing operation processing on the segmentation results to obtain the foreground area corresponding to each video screenshot.

3. The cockroach bait dispensing method according to claim 2, characterized in that, The performing cockroach identification on the foreground area in the multiple video screenshots based on a preset cockroach identification algorithm, determining and marking the cockroach contours in each video screenshot includes: Extracting the HOG features of the foreground area in the multiple video screenshots; Inputting the HOG features into a preset SVM classifier, and classifying the foreground area through the SVM classifier to determine the foreground areas in the multiple video screenshots classified as cockroaches; Marking the foreground areas classified as cockroaches.

4. The cockroach bait dispensing method according to claim 3, wherein, The extracting the HOG features of the multiple video screenshots includes: Performing color space standardization on the grayscale images through the Gamma correction method; Calculating the gradient information of each pixel in the standardized grayscale images, and dividing the grayscale images into multiple cells according to the image size; Calculating the gradient histogram of each cell based on the gradient information; According to the positional relationship of the cells in the grayscale images, forming a block by every n cells; Connecting the gradient histograms of all blocks in series to obtain the HOG features corresponding to the video screenshots.

5. The cockroach bait dispensing method according to any one of claims 1-4, characterized in that, The real-time adjustment of the bait delivery scheme based on the disinfecting data includes: Determine a third identification quantity in the area to be identified after implementing the bait delivery scheme based on the disinfection data; The total amount of bait placed in the area to be identified after calculating the bait placement plan based on the third identification quantity; The bait delivery plan is adjusted based on the total amount of bait delivered to the to-be-identified area after the bait delivery plan.

6. A cockroach bait dispensing device, characterized in that, The cockroach bait delivery device comprises: A video processing module is used to collect video data of the area to be identified within a preset time interval, and perform frame processing on the video data to obtain multi-frame video screenshots; An identification module is used to identify cockroaches on the multiple frames of video screenshots to obtain a first identification number of a preset area within a preset time interval, wherein the identification of cockroaches on the multiple frames of video screenshots to obtain the first identification number of the preset area within the preset time interval includes: performing image preprocessing on the multiple frames of video screenshots to determine the foreground area in the multiple frames of the video screenshots; based on a preset cockroach identification algorithm, identifying cockroaches in the foreground area in the multiple frames of the video screenshots, determining and marking the outline of the cockroaches in each frame of the video screenshot; tracking the outline of the cockroaches marked in each frame of the video screenshot by a preset tracking algorithm to obtain the cockroach track in the video data; counting the number of times the cockroach track passes through a preset virtual dividing line in the acquisition area to be identified; and dividing the number by the time length of the time interval to obtain the first identification number; A first calculation module, used for calculating a current second identification number of the to-be-identified area according to the first identification number, wherein the second identification number is determined based on the first identification number and a cockroach reproduction model, wherein the cockroach reproduction model is used for simulating the reproduction process of the cockroaches; A second calculation module, used for calculating the total amount of bait put into the current area to be identified according to the second identification quantity; A plan formulation module is used to formulate a bait delivery plan according to the total amount of bait delivered, and to collect disinfection data in real time after the bait delivery plan is implemented; An adjustment module is used to make real-time adjustments to the bait delivery plan based on the pest control data.

7. A cockroach bait dispensing device, characterized in that, The cockroach bait delivery device comprises: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instructions in the memory to enable the cockroach bait delivery device to perform the steps of the cockroach bait delivery method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the cockroach bait delivery method according to any one of claims 1 to 5 are implemented.

Images