An object dropping method, device, terminal and computer readable storage medium

Abstract

The application provides an object feeding method, device, terminal and computer readable storage medium, and the object feeding method comprises: performing segmentation processing on a plurality of continuous video frames contained in a target video stream respectively to obtain mask images corresponding to the video frames; counting motion pixel point information corresponding to a target object in the mask images of the video frames; determining a motion pixel point change feature of the target video stream according to the motion pixel point information corresponding to the video frames; and determining an object feeding strategy of feeding a target object to a target area based on the motion pixel point change feature of the target video stream. According to the motion pixel point information of the video frames in the target video stream, the application determines the change trend of the motion pixel points corresponding to the video stream, and according to the change trend of the motion pixel points, the state parameters of fish eating food can be determined, and then the object feeding strategy of feeding the target object to the target area is determined according to the state parameters of the fish eating food, so that accurate feeding is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an object feeding method and device, a terminal and a computer readable storage medium. BACKGROUND

[0002] China is a large country of aquaculture, and the total amount of aquatic product cultivation ranks first in the world. Feed is the most important variable cost in aquaculture, generally accounting for more than 60% of the total cost of cultivation. Determining the amount of feed per meal, reducing feed waste, and improving feed utilization have an important role in increasing the efficiency of cultivation. On the basis of ensuring the rapid growth of aquatic organisms, feed management directly determines the growth environment and growth rate of organisms. Insufficient feeding will lead to slow growth or even stagnation of the cultivated objects, affecting yield and reducing economic benefits. Excessive feeding will cause the cultivated objects to be unable to consume all the feed, and the feed will be oxidized and decomposed in the water, causing a significant increase in ammonia nitrogen, nitrite and other substances in the water, reducing the feeding amount of fish, and thus slowing down the growth rate, and in severe cases, even leading to death and causing huge economic losses. SUMMARY

[0003] The technical problem solved by the present application is to provide an object feeding method and device, a terminal and a computer readable storage medium, which solves the problem of being unable to achieve precise feeding in the prior art.

[0004] To solve the above technical problems, the first technical solution adopted by the present application is to provide an object feeding method, which comprises: performing segmentation processing on a plurality of continuous video frames contained in a target video stream respectively to obtain mask images corresponding to each video frame; the target video stream is collected for a target area; statistical information of motion pixels corresponding to a target object in the mask images of each video frame; determining the motion pixel change characteristics of the target video stream according to the motion pixel information corresponding to each video frame; and determining an object feeding strategy for feeding a target object to the target area based on the motion pixel change characteristics of the target video stream.

[0005] After the step of determining the object feeding strategy for feeding the target object to the target area based on the motion pixel change characteristics of the target video stream, the method further comprises: feeding the target object to the target area by an object feeding device associated with the target area according to the object feeding strategy.

[0006] The step of determining the object feeding strategy for feeding the target object to the target area based on the motion pixel change characteristics of the target video stream comprises: determining a state parameter of the target object contained in the target video stream based on the motion pixel change characteristics of the target video stream; the state parameter represents the activity level of the target object using the target object; and determining the object feeding strategy for feeding the target object to the target area according to the state parameter.

[0007] The method further comprises: constructing a motion pixel point information change curve of the target object in the whole process before and after the target object; the motion pixel point information change curve comprises a plurality of preset change characteristics; matching a preset state parameter corresponding to each preset change characteristic in the motion pixel point information change curve; and matching an object dropping strategy corresponding to each preset state parameter.

[0008] The preset state parameter comprises at least one of a first state parameter, a second state parameter, a third state parameter and a fourth state parameter; and the matching of the object dropping strategy corresponding to each preset state parameter comprises at least one of the following: matching a first dropping strategy to the first state parameter in response to the first state parameter corresponding to the preset change characteristic; matching a second dropping strategy to the second state parameter in response to the second state parameter corresponding to the preset change characteristic; matching a third dropping strategy to the third state parameter in response to the third state parameter corresponding to the preset change characteristic; and matching a fourth dropping strategy to the fourth state parameter in response to the fourth state parameter corresponding to the preset change characteristic.

[0009] The motion pixel point information change curve comprises a preset change characteristic corresponding to the first state parameter; and the method further comprises: collecting a video of the target region before the target object as a reference video stream; determining a motion pixel point change characteristic of the reference video stream based on motion pixel point information corresponding to each video frame in the reference video stream; and correcting the motion pixel point information change curve based on the motion pixel point change characteristic of the reference video stream and the preset change characteristic corresponding to the first state parameter.

[0010] The motion pixel point information comprises a number of motion pixel points; and the determination of the motion pixel point change characteristic of the target video stream based on the motion pixel point information corresponding to each video frame comprises: generating a number of motion pixel points change trend corresponding to the target video stream based on the number of motion pixel points corresponding to each video frame.

[0011] The preset change characteristic comprises a preset change trend; and the determination of the object dropping strategy of dropping the target object to the target region based on the motion pixel point change characteristic of the target video stream comprises: comparing the number of motion pixel points change trend of the target video stream with each preset change trend in the motion pixel point information change curve respectively; and determining the object dropping strategy corresponding to the preset change trend as the object dropping strategy of dropping the target object to the target region in response to the number of motion pixel points change trend being consistent with the preset change trend.

[0012] The object feeding strategy includes at least one of feeding amount specification, feeding rate specification, feeding time length specification, feeding location specification, and feeding amount specification corresponding to each feeding location.

[0013] To solve the above technical problems, a second technical solution adopted by the present application is to provide an object feeding method, characterized in that the object feeding method comprises: performing segmentation processing on each of a plurality of continuous video frames contained in an acquired target video stream to obtain a mask image corresponding to each video frame; the target video stream is collected for a target region; and a recurrent neural network is used to determine an object feeding strategy for feeding a target object to the target region based on the mask images of the video frames in the target video stream.

[0014] The training method of the recurrent neural network comprises: acquiring a plurality of training video streams, wherein each training video stream comprises a plurality of continuous sample images, each sample image is associated with a corresponding mask image, and each training video stream is associated with a corresponding labeled object feeding strategy; inputting the mask images of all sample images contained in the training video stream to the recurrent neural network to obtain a predicted object feeding strategy corresponding to the training video stream; and iteratively training the recurrent neural network based on an error value between the labeled object feeding strategy and the predicted object feeding strategy corresponding to the same training video stream.

[0015] To solve the above technical problems, a third technical solution adopted by the present application is to provide an object feeding device, comprising: a preprocessing module configured to perform segmentation processing on each of a plurality of continuous video frames contained in a target video stream to obtain a mask image corresponding to each video frame; the target video stream is collected for a target region; a statistical module configured to count motion pixel point information corresponding to a target object in the mask image of each video frame; an analysis module configured to determine a motion pixel point change feature of the target video stream according to the motion pixel point information corresponding to each video frame; and a determination module configured to determine an object feeding strategy for feeding a target object to the target region based on the motion pixel point change feature of the target video stream.

[0016] To solve the above technical problems, a fourth technical solution adopted by the present application is to provide an object feeding device, comprising: a preprocessing module configured to perform segmentation processing on each of a plurality of continuous video frames contained in an acquired target video stream to obtain a mask image corresponding to each video frame; the target video stream is collected for a target region; and a processing module configured to determine an object feeding strategy for feeding a target object to the target region based on the mask images of the video frames in the target video stream using a recurrent neural network.

[0017] To solve the above technical problems, a fifth technical solution adopted by the present application is to provide a terminal, the terminal comprising a memory, a processor, and a computer program stored in the memory and running on the processor, the processor being configured to execute the program data to implement the steps of the object dropping method described above.

[0018] To solve the above technical problems, a sixth technical solution adopted by the present application is to provide a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the object dropping method described above.

[0019] The present application has the following advantages: Unlike the prior art, the present application provides an object dropping method, device, terminal and computer readable storage medium, the object dropping method comprising: performing segmentation processing on each of a plurality of continuous video frames included in a target video stream to obtain a mask image corresponding to each video frame; the target video stream is collected for a target region; counting motion pixel point information corresponding to a target object in the mask image of each video frame; determining a motion pixel point change feature of the target video stream according to the motion pixel point information corresponding to each video frame; and determining an object dropping strategy for dropping a target object into the target region based on the motion pixel point change feature of the target video stream. The present application determines the change trend of the motion pixel points corresponding to the video stream according to the motion pixel point information of each video frame in the target video stream, determines the state parameters of the fish eating food according to the change trend of the motion pixel points, and then determines the object dropping strategy for dropping the target object into the target region according to the state parameters of the fish eating food, thereby realizing accurate feeding. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0021] Figure 1 is a flowchart of the object dropping method provided by the present application;

[0022] Figure 2 is a flowchart of a specific embodiment of the object dropping method provided by the present application;

[0023] Figure 3 is a motion pixel point information change curve in a specific embodiment of the object dropping method provided by the present application;

[0024] Figure 4 is Figure 2 is a flowchart of a specific embodiment of step S204 in the object dropping method provided by the present application;

[0025] Figure 5 is a frame diagram of an embodiment of the object throwing device provided by the present application;

[0026] Figure 6 is a frame diagram of another embodiment of the object throwing device provided by the present application;

[0027] Figure 7 is a frame diagram of an embodiment of the terminal provided by the present application;

[0028] Figure 8 is a frame diagram of an embodiment of the computer readable storage medium provided by the present application. DETAILED DESCRIPTION

[0029] The scheme of the embodiments of the present application will be described in detail below in combination with the accompanying drawings.

[0030] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. The present application may, however, be practiced without some or all of these details. Moreover, the present application can also be practiced with only other system structures, interfaces, techniques, etc.

[0031] The term "and / or" in the present description is merely used to describe an association relationship of associated objects, and indicates that there can be three relationships, for example, A and / or B can represent three cases of existence of A alone, existence of A and B simultaneously, and existence of B alone. In addition, the character " / " in the present description generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" in the present description represents two or more than two.

[0032] In order for those skilled in the art to better understand the technical solutions of the present application, a kind of object throwing method provided by the present application will be further described in detail below in combination with the accompanying drawings and specific embodiments.

[0033] The research and application of the precise feeding method aims to realize the maximum growth rate of breeding by using scientific and reasonable feeding amount, and to reduce the waste of feed to the maximum extent.

[0034] Domestic research on feeding methods has been conducted for many years, and most of them are still in a state of serious dependence on manual participation, that is, manual feeding or semi-automatic feeding. The significant feature of this mode is to feed a fixed amount at a fixed time and in a fixed place. Manual feeding is suitable for small production systems and requires full-time manual participation. Semi-automatic feeding is suitable for large and medium-sized production systems and requires regular management of the feeding system. It can generally be timed or quantified, does not need to be operated on site, and liberates part of the workforce, but both require experienced managers to observe whether the breeding objects are full, so as to quickly correct the feeding amount. Since the manual and semi-automatic feeding methods are a kind of unquantifiable feeding method relying solely on breeding experience and are uncontrollable, they have significant limitations and are extremely prone to unreasonable feeding, which not only cannot promote growth, but also may weaken their resistance to diseases, increase the burden of water purification, and make it difficult to ensure biological health and rapid growth. Therefore, an automatic and quantifiable method is needed to control feeding.

[0035] Please refer to Figure 1 , Figure 1 is a flowchart of the object feeding method provided by the present application. In this embodiment, an object feeding method is provided, which is suitable for feeding fish ponds indoors and outdoors, and can also be suitable for feeding animals such as cows and sheep on a pasture. The object feeding method comprises the following steps.

[0036] S11: The plurality of continuous video frames contained in the target video stream are respectively segmented to obtain the mask images corresponding to each video frame.

[0037] Specifically, a motion pixel point information change curve corresponding to the target object in the entire process before and after the use of the target object is constructed in advance; the motion pixel point information change curve comprises a plurality of preset change characteristics; each preset change characteristic in the motion pixel point information change curve is matched with a corresponding preset state parameter; and each preset state parameter is matched with a corresponding object feeding strategy. The preset state parameters include at least one of a first state parameter, a second state parameter, a third state parameter, and a fourth state parameter.

[0038] In an embodiment, in response to the preset change characteristic corresponding to the first state parameter, the first state parameter is matched with a first feeding strategy; in response to the preset change characteristic corresponding to the second state parameter, the second state parameter is matched with a second feeding strategy; in response to the preset change characteristic corresponding to the third state parameter, the third state parameter is matched with a third feeding strategy; and in response to the preset change characteristic corresponding to the fourth state parameter, the fourth state parameter is matched with a fourth feeding strategy.

[0039] In an embodiment, a video of a target region before use of a target object is collected as a reference video stream; based on motion pixel point information corresponding to each video frame in the reference video stream, a motion pixel point change feature of the reference video stream is determined; and based on the motion pixel point change feature of the reference video stream and a preset change feature corresponding to the first state parameter, a motion pixel point information change curve is corrected.

[0040] In an embodiment, a real-time collected video stream is acquired; a frame difference method is used to perform motion detection on each video frame in the video stream, and a corresponding mask image of each video frame is determined.

[0041] S12: Motion pixel point information corresponding to a target object in each video frame of the mask image is counted.

[0042] Specifically, the motion pixel point information includes a number of motion pixel points.

[0043] S13: Based on the motion pixel point information corresponding to each video frame, a motion pixel point change feature of the target video stream is determined.

[0044] Specifically, the motion pixel point information includes a number of motion pixel points. Based on the number of motion pixel points corresponding to each video frame, a motion pixel point number change trend corresponding to the target video stream is generated.

[0045] In an embodiment, the preset change feature includes a preset change trend. The motion pixel point number change trend of the target video stream is compared with each preset change trend in the motion pixel point information change curve respectively; and in response to the motion pixel point number change trend being consistent with a preset change trend, an object dropping strategy corresponding to the preset change trend is determined to drop the target object into the target region.

[0046] In an embodiment, the object dropping strategy includes at least one of a feeding amount specification, a feeding rate specification, a feeding time length specification, a feeding location specification, and a feeding amount specification corresponding to each feeding location.

[0047] S14: Based on the motion pixel point change feature of the target video stream, an object dropping strategy for dropping the target object into the target region is determined.

[0048] Specifically, based on the motion pixel point change feature of the target video stream, a state parameter of a target object included in the target video stream is determined; the state parameter represents an activity level of the target object using the target object; and based on the state parameter, an object dropping strategy for dropping the target object into the target region is determined.

[0049] In an embodiment, a recurrent neural network can also be directly used to determine an object dropping strategy for dropping the target object into the target region based on the mask image of each video frame in the target video stream.

[0050] In an embodiment, the object delivery device associated with the target area delivers the target object to the target area according to an object delivery strategy.

[0051] The object delivery method provided by the embodiment comprises: performing segmentation processing on each of a plurality of continuous video frames included in a target video stream to obtain a mask image corresponding to each video frame; the target video stream is collected for a target area; statistical information of motion pixel points corresponding to a target object in the mask image of each video frame is obtained; a motion pixel point change feature of the target video stream is determined according to the motion pixel point information corresponding to each video frame; and an object delivery strategy for delivering a target object to the target area is determined based on the motion pixel point change feature of the target video stream. According to the motion pixel point information of each video frame in the target video stream, the change trend of the motion pixel points corresponding to the video stream is determined, the state parameters of the fish eating food are determined according to the change trend of the motion pixel points, the object delivery strategy for delivering the target object to the target area is determined according to the state parameters of the fish eating food, and accurate feeding is realized.

[0052] Please refer to Figure 2 , Figure 2 is a flowchart of an embodiment of the object delivery method provided by the embodiment. The object delivery method provided by the embodiment is suitable for feeding fish in an indoor or outdoor fish pond, and can also be suitable for feeding animals such as cows and sheep in a pasture. The object delivery method comprises the following steps. The embodiment is described by taking feeding fish in a fish pond as an example.

[0053] S201: A motion pixel point information change curve corresponding to the whole process before and after eating is constructed in advance.

[0054] Specifically, an image acquisition device is installed at a preset position of the fish pond, so that the image acquisition device can acquire images of the water surface of the fish pond.

[0055] Please refer to Figure 3 , Figure 3 is a motion pixel point information change curve in a specific embodiment of the object delivery method provided by the embodiment.

[0056] The fish pond images collected before and after the fish in the fish pond eat are sequentially formed into a first state video stream, a second state video stream, a third state video stream, and a fourth state video stream. The motion pixel point information change curve is generated according to the change curves respectively corresponding to the first state video stream, the second state video stream, the third state video stream, and the fourth state video stream.

[0057] The first state video stream corresponds to a first state parameter, and the first state parameter is a parameter of the fish in an unfed state. Since the fish in the fishpond is in the unfed state, the fish in the fishpond is in a free activity state. That is, the water surface of the fishpond is relatively calm in the unfed state of the fish. Motion pixel points of each fishpond image collected in the first state video stream are detected to obtain the number of motion pixel points corresponding to each fishpond image. According to the number of motion pixel points of each fishpond image in the first state video stream, a motion pixel point change curve corresponding to the unfed state of the fish is determined. In the unfed state of the fish, the number of motion pixel points corresponding to each fishpond image in the first state video stream is basically in a constant range. That is, the activity level of the fish in the unfed state does not change greatly, and the motion pixel point change curve corresponding to the unfed state of the fish is approximately a straight line.

[0058] The second state video stream corresponds to a second state parameter, and the second state parameter is a parameter of the fish in an initial feeding state. The fish in the fishpond is initially fed, and the fish school will appear group gathering, food fighting, jumping and other situations. Motion pixel points of each fishpond image collected in the second state video stream are detected to obtain the number of motion pixel points corresponding to each fishpond image. According to the number of motion pixel points of each fishpond image in the second state video stream, a motion pixel point change curve corresponding to the initial feeding state of the fish is determined. In the initial feeding state of the fish, the number of motion pixel points corresponding to each fishpond image arranged in time in the second state video stream will rapidly increase. That is, the activity level of the fish in the initial feeding state is significantly improved, and the motion pixel point change curve corresponding to the initial feeding state of the fish is a growth trend.

[0059] The third state video stream corresponds to a third state parameter, and the third state parameter is a parameter of the fish in a re-feeding state. When the fish continues to fight for food, part of the fish has not yet eaten and needs to continue to feed the fish in the fishpond. The fish school will appear active and other situations. Since part of the fish has finished eating, only part of the fish is still active and eating. Motion pixel points of each fishpond image collected in the third state video stream are detected to obtain the number of motion pixel points corresponding to each fishpond image. According to the number of motion pixel points of each fishpond image in the third state video stream, a motion pixel point change curve corresponding to the re-feeding state of the fish is determined. In the re-feeding state of the fish, the fish in the fishpond gradually becomes full, and the number of motion pixel points corresponding to each fishpond image arranged in time in the third state video stream will gradually decrease. That is, the activity level of the fish in the re-feeding state is reduced, and the motion pixel point change curve corresponding to the re-feeding state of the fish is a decreasing trend.

[0060] The fourth state video stream corresponds to a fourth state parameter, and the fourth state parameter is a parameter of the fish in an end-of-feeding state. After the fish finishes feeding, the activity of the fish in the fishpond gradually decreases until the fish returns to a free activity state. That is, after the fish finishes feeding, the fish group gradually becomes less active, and the activity of the fish group gradually decreases until it is stable. In the fourth state video stream collected in this state, the motion pixel points of each fishpond image are detected to obtain the number of motion pixel points corresponding to each fishpond image. According to the number of motion pixel points of each fishpond image in the fourth state video stream, a motion pixel point change curve corresponding to the fish in the end-of-feeding state is determined. In the end-of-feeding state, the number of motion pixel points corresponding to each fishpond image in the fourth state video stream gradually decreases until it is stable. That is, the activity of the fish in the end-of-feeding state is basically unchanged, and the motion pixel point change curve corresponding to the fish in the end-of-feeding state is gradually decreasing until it is a straight line. Since the activity of the fish group after finishing feeding is higher than that of the fish group in the unfed state, the straight line in the change curve corresponding to the fish in the end-of-feeding state is higher than the change curve corresponding to the fish in the unfed state.

[0061] S202: Matching each preset state parameter corresponding to a preset change feature in the motion pixel point information change curve.

[0062] Specifically, the preset state parameter is a preset feeding stage. Based on the change trend of the motion pixel point information change curve, the motion pixel point information change curve is divided into a preset number of preset change features, and then the preset change features are matched with the corresponding preset feeding stages. The preset change features include preset change trends.

[0063] In this embodiment, the motion pixel point information change curve is divided into four preset change features according to the change trend of the motion pixel point information change curve. Among them, the four preset change features are constant trend, upward trend, downward trend and gradually decreasing to constant trend. Among them, the preset change feature with constant trend in the motion pixel point information change curve corresponds to the pre-feeding stage. The preset change feature with upward trend in the motion pixel point information change curve corresponds to the initial feeding stage. The preset change feature with downward trend in the motion pixel point information change curve corresponds to the re-feeding stage. The preset change feature with decreasing to constant trend in the motion pixel point information change curve corresponds to the post-feeding stage. And the straight line corresponding to the constant trend in the pre-feeding stage is lower than the straight line corresponding to the constant trend in the post-feeding stage.

[0064] S203: Matching each preset state parameter with a corresponding object release strategy.

[0065] Specifically, the object feeding strategy includes at least one of feeding amount specification, feeding rate specification, feeding time length specification, feeding location specification, and feeding amount specification corresponding to each feeding location.

[0066] In the embodiment, the first state parameter is matched with the first feeding strategy in response to the preset change feature corresponding to the first state parameter, the second state parameter is matched with the second feeding strategy in response to the preset change feature corresponding to the second state parameter, the third state parameter is matched with the third feeding strategy in response to the preset change feature corresponding to the third state parameter, and the fourth state parameter is matched with the fourth feeding strategy in response to the preset change feature corresponding to the fourth state parameter. The first object feeding strategy, the second object feeding strategy, the third object feeding strategy, and the fourth object feeding strategy correspond to at least one fish feeding scheme, respectively.

[0067] S204: training the recurrent neural network.

[0068] In the embodiment, the recurrent neural network can be RNN (Recurrent Neural Networks) and improved version of LSTM (Long Short Term Mermory network). The recurrent neural network can also be other network models trained by deep learning.

[0069] Specifically, the training method of the recurrent neural network includes the following steps.

[0070] Please refer to Figure 4 , Figure 4 is Figure 2 a flowchart of a specific embodiment of step S204 in the object feeding method provided by the application.

[0071] S2041: obtaining a plurality of training video streams.

[0072] Specifically, the training video stream includes a plurality of continuous sample images, each sample image is associated with a corresponding mask image, and each training video stream is associated with a corresponding labeled object feeding strategy. The labeled object feeding strategy corresponding to each training video stream is determined according to experience, the type and size of fish in the fish pond, etc.

[0073] S2042: inputting the mask images of all sample images included in the training video stream to the recurrent neural network to obtain a predicted object feeding strategy corresponding to the training video stream.

[0074] Specifically, the mask images of the sample images in the training video stream are grouped into a mask image sequence according to time, and the mask image sequence corresponding to the training video stream is input to the recurrent neural network. The recurrent neural network determines the predicted object feeding strategy corresponding to the training video stream according to the mask image sequence.

[0075] S2043: iteratively train the recurrent neural network based on the error value between the labeled object dropping strategy and the predicted object dropping strategy corresponding to the same training video stream.

[0076] Specifically, the error value between the labeled object dropping strategy and the predicted object dropping strategy corresponding to the same training video stream is calculated, and the recurrent neural network is iteratively trained based on the error value.

[0077] In an optional embodiment, the result of the recurrent neural network is back propagated, and the weight of the recurrent neural network is corrected according to the error value between the labeled object dropping strategy and the predicted object dropping strategy corresponding to the same training video stream, so as to realize the training of the recurrent neural network.

[0078] The mask image sequence corresponding to the training video stream is input into the recurrent neural network, and the recurrent neural network detects the object dropping strategy corresponding to the training video stream. When the error value between the labeled object dropping strategy and the predicted object dropping strategy corresponding to the training video stream is less than a preset threshold, the training of the recurrent neural network is stopped.

[0079] In an embodiment, in order to avoid the influence of special weather such as rain and strong wind on the feeding accuracy, it is necessary to correct the motion pixel point information change curve before feeding, so that the motion pixel point information change curve is more suitable for the current feeding.

[0080] S205: collect the video of the target area before the target object as a reference video stream.

[0081] Specifically, the image of the fishpond water surface before feeding is collected to obtain a video of a preset time length as a reference video stream.

[0082] In an embodiment, the feeding machine needs to feed for the first time at 6 o'clock in the morning, and the video data from 5 o'clock to 6 o'clock in the morning is selected as the reference video stream.

[0083] S206: determine the motion pixel point change feature of the reference video stream based on the motion pixel point information corresponding to each video frame in the reference video stream.

[0084] Specifically, the frame difference method can be used to detect the segmentation of each video frame in the reference video stream to obtain the mask image corresponding to each video frame. In the mask image, the foreground image and the background image are distinguished. In an embodiment, the first frame image of the reference video stream can be taken as the background image, and the second frame image and the video frames after the second frame image are subtracted from the first frame image to determine the motion area of the second frame image and the video frames after the second frame image relative to the first frame image. The motion area of each video frame relative to the first frame image is taken as the foreground image of each video frame. The number of pixel points corresponding to the foreground image of each video frame is counted as the number of motion pixel points corresponding to each video frame. The motion pixel point change curve corresponding to the video stream is generated according to the number of motion pixel points corresponding to each video frame. The motion pixel point change curve is taken as the motion pixel point change feature.

[0085] In another embodiment, the difference between the number of motion pixel points of the next frame video frame and the previous frame video frame can also be calculated, and the difference between the number of motion pixel points corresponding to adjacent video frames in the reference video stream is calculated. The motion pixel point change curve of the reference video stream is determined based on the difference between the number of motion pixel points corresponding to adjacent video frames.

[0086] The motion pixel point change curve takes time as the horizontal axis and the number of motion pixel points as the vertical axis.

[0087] S207: Correct the motion pixel point information change curve based on the motion pixel point change feature of the reference video stream and the preset change feature corresponding to the first state parameter.

[0088] Specifically, in special environments such as strong wind and heavy rain, the water surface of the fishpond will fluctuate, and a large number of motion pixel points will appear in the image of the fishpond water surface without feeding, resulting in a large number of motion pixel points in the whole process before and after feeding the fish. In order to improve the feeding accuracy, the motion pixel point information change curve needs to be corrected according to the motion pixel point change curve corresponding to the pre-feeding video currently collected.

[0089] In an embodiment, the difference between the motion pixel point change curve corresponding to the pre-feeding video and the preset change curve corresponding to the pre-feeding stage in the motion pixel point information change curve is calculated. That is, the difference between the average number of motion pixel points corresponding to the reference video stream and the preset average number of motion pixel points corresponding to the preset change curve corresponding to the pre-feeding stage is calculated.

[0090] In response to the difference being a positive value, the running pixel point change curve is shifted upward on the vertical axis by an absolute value of the difference, to obtain a corrected running pixel point change curve. In response to the difference being a negative value, the running pixel point change curve is shifted downward on the vertical axis by an absolute value of the difference, to obtain a corrected running pixel point change curve.

[0091] The running pixel point change curve needs to be corrected before each round of feeding, to improve the feeding accuracy.

[0092] S208: Obtain a target video stream collected in real time.

[0093] Specifically, the target video stream of a preset time length is collected by the image collection device. The target video stream of a preset number of frames can also be collected by the image collection device. The target video stream is collected for a target region. Each video frame in the video stream contains a target object, which can be an animal swimming in water, such as fish, shrimp, crab, etc.

[0094] S209: Perform motion detection on each video frame in the target video stream, to determine a corresponding mask image of each video frame.

[0095] Specifically, each video frame in the video stream is segmented by using a frame difference method, to obtain a mask image of each video frame relative to a first frame image in the video stream, to determine a foreground region of each video frame relative to the first frame image, to distinguish the foreground region and the background region in each video frame.

[0096] The next frame image can also be compared with the previous video frame, to determine a motion region of the next frame image relative to the previous frame image, and the motion region is taken as the foreground image. All adjacent video frames in the video stream are traversed, to determine the mask image of each video frame.

[0097] S210: Statistically determine motion pixel point information corresponding to the target object in the mask image of each video frame.

[0098] Specifically, the motion pixel point information includes the number of motion pixel points. The number of motion pixel points corresponding to the foreground region in the mask image of each video frame is statistically determined. In a specific embodiment, the number of motion pixel points corresponding to the position of the target object as the foreground image in each video frame is statistically determined.

[0099] S211: Generate a motion pixel point number change trend corresponding to the target video stream according to the number of motion pixel points corresponding to each video frame.

[0100] Specifically, a motion pixel point change curve corresponding to the target video stream is generated according to the number of motion pixel points corresponding to each video frame. The motion pixel point change trend can be determined according to the motion pixel point change curve.

[0101] In another embodiment, the maximum number of motion pixels, the minimum number of motion pixels, the average number of motion pixels, etc. in the target video stream are determined according to the number of motion pixels corresponding to each video frame.

[0102] S212: Determine the state parameter of the target object contained in the target video stream based on the motion pixel change trend of the target video stream.

[0103] The state parameter represents the activity level of the target object using the target object. The motion pixel change trend of the target video stream is compared with each preset change trend in the motion pixel information change curve.

[0104] Specifically, the motion pixel change trend corresponding to the target video stream is compared with each preset change trend in the motion pixel information change curve, and the preset change trend consistent with the motion pixel change trend of the target video stream is determined. The state parameter of the target object corresponding to the preset change trend is determined as the state parameter of the motion pixel change trend of the target video stream.

[0105] The maximum number of motion pixels corresponding to the target video stream is compared with the maximum number of motion pixels corresponding to each preset change trend in the motion pixel information change curve; the minimum number of motion pixels corresponding to the video stream is compared with the minimum number of motion pixels corresponding to each preset change trend in the motion pixel information change curve; the average number of motion pixels corresponding to the video stream is compared with the average number of motion pixels corresponding to each preset change trend in the motion pixel information change curve, to determine the state parameter of the target object contained in the target video stream.

[0106] S213: Determine the object feeding strategy for feeding the target object to the target area according to the state parameter.

[0107] Specifically, in response to the motion pixel change trend being consistent with a preset change trend, the object feeding strategy corresponding to the state parameter of the preset change trend is used for feeding.

[0108] Specifically, in response to the motion pixel change trend being consistent with a preset change trend in the corrected motion pixel information change curve, the object feeding strategy corresponding to the state parameter of the preset change trend can be used for feeding the fish in the fishpond.

[0109] In a specific embodiment, in response to the motion pixel point quantity change trend being consistent with a preset change trend, and a difference between the maximum motion pixel point quantity corresponding to the video stream and the maximum motion pixel point quantity corresponding to the trend-consistent preset change trend being less than a first threshold, a difference between the minimum motion pixel point quantity corresponding to the video stream and the minimum motion pixel point quantity corresponding to the trend-consistent preset change trend being less than a second threshold, and a difference between the average motion pixel point quantity corresponding to the video stream and the average motion pixel point quantity corresponding to the trend-consistent preset change trend being less than a third threshold, the fish in the fishpond can be fed according to the object feeding strategy corresponding to the state parameter of the trend-consistent preset change trend.

[0110] In another embodiment, a recurrent neural network is used to determine the object feeding strategy corresponding to the video stream based on the mask images respectively corresponding to the video frames of the video stream, and the object feeding strategy is used for feeding.

[0111] In an embodiment, after the object feeding strategy corresponding to the motion pixel point quantity change trend is determined, the object feeding strategy is transmitted to the operation terminal of the staff to inform the staff of the current fish feeding state or to allow the staff to select one from the recommended multiple fish feeding schemes, and then the automatic fish feeder arranged around the fishpond feeds the fish in the fishpond according to the fish feeding scheme selected by the staff, etc. In a specific embodiment, the target object is fed into the target area by the object feeding device associated with the target area according to the object feeding strategy. The target object can be food for feeding the target object in the target area.

[0112] After the current feeding, video data is continuously collected, and the video data is processed according to the above steps S208 to S213, and the feeding specification corresponding to the video data is determined according to the motion pixel point quantity change trend corresponding to the video data and the motion pixel point quantity change trend corresponding to the adjacent previous video stream, so as to improve the feeding accuracy.

[0113] The above steps S208 to S213 are repeated until feeding is no longer needed, and the current round of fish feeding is completed.

[0114] The object feeding method provided in the embodiment can replace direct identification of fish by using water movement caused by fish movement in the state that the information of fish in the image is not very clear, and then determine the feeding stage of the fish, and then determine the corresponding object feeding strategy according to the feeding stage of the fish, so as to improve the feeding accuracy.

[0115] In the embodiment, the motion pixel point information change curve is fine-tuned before feeding to solve the problem that special weather such as rain and strong wind disturbs the motion pixel point quantity in the image of the fishpond water surface, and further improve the accuracy of determining the feeding stage, and then improve the feeding accuracy.

[0116] The object feeding method provided by the embodiment comprises: performing segmentation processing on a plurality of continuous video frames contained in a target video stream respectively to obtain mask images corresponding to the video frames; the target video stream is collected for a target region; motion pixel point information corresponding to a target object in the mask images of the video frames is counted; motion pixel point change characteristics of the target video stream are determined according to the motion pixel point information corresponding to the video frames; and an object feeding strategy of feeding a target object to the target region is determined based on the motion pixel point change characteristics of the target video stream. According to the motion pixel point information of the video frames in the target video stream, the change trend of the motion pixel points corresponding to the video stream is determined, the state parameters of fish eating food are determined according to the change trend of the motion pixel points, the object feeding strategy of feeding the target object to the target region is determined according to the state parameters of the fish eating food, and then accurate feeding is realized.

[0117] Referring to Figure 5 , Figure 5 is a frame schematic diagram of an embodiment of the object feeding device provided by the application. The embodiment provides an object feeding device 60, which comprises a preprocessing module 61, a counting module 62, an analysis module 63 and a determining module 64.

[0118] The preprocessing module 61 is used for performing segmentation processing on a plurality of continuous video frames contained in a target video stream respectively to obtain mask images corresponding to the video frames; the target video stream is collected for a target region.

[0119] The preprocessing module 61 is also used for pre-constructing a motion pixel point information change curve of a target object in the whole process before and after the target object is used; the motion pixel point information change curve comprises a plurality of preset change characteristics; a preset state parameter corresponding to each preset change characteristic in the motion pixel point information change curve is matched; and a corresponding object feeding strategy is matched for each preset state parameter. The preset state parameter comprises at least one of a first state parameter, a second state parameter, a third state parameter and a fourth state parameter.

[0120] The preprocessing module 61 is also used for matching a first object feeding strategy for the preset change characteristic corresponding to a pre-feeding stage; the preprocessing module 61 is also used for matching a first feeding strategy for the first state parameter corresponding to the preset change characteristic; matching a second feeding strategy for the second state parameter corresponding to the preset change characteristic; matching a third feeding strategy for the third state parameter corresponding to the preset change characteristic; and matching a fourth feeding strategy for the fourth state parameter corresponding to the preset change characteristic.

[0121] The preprocessing module 61 is also configured to collect a video of the target area before the target object is used as a reference video stream, determine a motion pixel point change feature of the reference video stream based on motion pixel point information corresponding to each video frame in the reference video stream, and correct a motion pixel point information change curve based on the motion pixel point change feature of the reference video stream and a preset change feature corresponding to the first state parameter. The motion pixel point information change curve at least includes the preset change feature corresponding to the first state parameter.

[0122] The statistical module 62 is configured to count motion pixel point information corresponding to the target area in a mask image of each video frame. The motion pixel point information includes a number of motion pixel points.

[0123] The analysis module 63 is configured to determine a motion pixel point change feature of the video stream according to the motion pixel point information corresponding to each video frame. The analysis module 63 is configured to generate a motion pixel point number change trend corresponding to the video stream according to the number of motion pixel points corresponding to each video frame.

[0124] The determination module 64 is configured to determine to feed the target object according to an object feeding strategy corresponding to the motion pixel point change feature based on the motion pixel point change feature of the video stream. The object feeding strategy includes at least one of a feeding amount specification, a feeding rate specification, a feeding time length specification, a feeding location specification, and a feeding amount specification corresponding to each feeding location.

[0125] The determination module 64 is configured to determine a state parameter of the target object included in the target video stream based on the motion pixel point change feature of the target video stream, the state parameter representing an activity level of the target object using the target object, and determine an object feeding strategy for feeding the target object to the target area according to the state parameter.

[0126] The determination module 64 is also configured to feed the target object to the target area by an object feeding device associated with the target area according to the object feeding strategy.

[0127] The determination module 64 is configured to compare the motion pixel point number change trend of the target video stream with each preset change trend in the motion pixel point information change curve respectively, and determine to feed the target object to the target area according to an object feeding strategy corresponding to a preset change trend in response to the motion pixel point number change trend being consistent with the preset change trend.

[0128] According to the motion pixel point information of each video frame in the video stream, the application determines a change trend of the motion pixel points corresponding to the video stream, determines a fish feeding stage according to the change trend of the motion pixel points, determines a quantity of fish food to be fed to a fish pond according to the fish feeding stage, and thus realizes accurate feeding.

[0129] Referring to Figure 6 , Figure 6is a frame diagram of another embodiment of the object dropping device provided by the present application. The embodiment provides an object dropping device 70, which comprises a preprocessing module 71 and a processing module 72.

[0130] The preprocessing module 71 is configured to perform segmentation processing on each of a plurality of continuous video frames included in an obtained target video stream to obtain a mask image corresponding to each video frame. The target video stream is collected for a target region.

[0131] The processing module 72 is configured to determine an object dropping strategy for dropping a target object to the target region based on the mask images of the video frames in the target video stream by using a recurrent neural network.

[0132] The present application determines the amount of fish food to be fed to the fish pond according to the mask images corresponding to the video frames in the video stream, and thus realizes accurate feeding.

[0133] Please refer to Figure 7 , Figure 7 is a frame diagram of an embodiment of the terminal provided by the present application. The terminal 80 comprises a memory 81 and a processor 82 coupled with each other. The processor 82 is configured to execute program instructions stored in the memory 81 to implement the steps of any of the object dropping method embodiments described above. In a specific implementation scenario, the terminal 80 can include but is not limited to a microcomputer, a server, and in addition, the terminal 80 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited herein.

[0134] Specifically, the processor 82 is configured to control itself and the memory 81 to implement the steps of any of the object dropping method embodiments described above. The processor 82 can also be referred to as a CPU (Central Processing Unit). The processor 82 can be an integrated circuit chip with a processing capability of signals. The processor 82 can also be a general processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 82 can be realized by integrated circuit chips together.

[0135] Please refer to Figure 8 , Figure 8A framework schematic diagram of an embodiment of the computer readable storage medium provided in the present application is shown. The computer readable storage medium 90 stores program instructions 901 capable of being executed by a processor, the program instructions 901 being used to implement the steps of any of the object dropping method embodiments described above.

[0136] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can be referred to the description of the above method embodiments. For brevity, details are not repeated here.

[0137] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be mutually referred to, and for brevity, details are not repeated here.

[0138] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus implementation described above is only schematic, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a unit or component can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0139] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0140] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that makes contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0141] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that the personal information collection range has been entered, and the personal information will be collected. If the individual voluntarily enters the collection range, it is considered to agree to collect the personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the individual to upload his / her personal information, etc. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0142] The above is only an embodiment of the present application, and does not limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An object launching method, characterized by, The object delivery method comprises: respectively performing segmentation processing on a plurality of continuous video frames contained in a target video stream to obtain mask images corresponding to the video frames; the target video stream is collected for a target region; statistically obtaining motion pixel point information corresponding to a target object in the mask images of the video frames; determining motion pixel point change characteristics of the target video stream according to the motion pixel point information corresponding to the video frames; determining an object delivery strategy for delivering a target object to the target region based on the motion pixel point change characteristics of the target video stream; The object delivery method comprises: previously constructing a motion pixel point information change curve of the target object in the entire process before and after the target object is used; the motion pixel point information change curve comprises a plurality of preset change characteristics; matching preset state parameters corresponding to each of the preset change characteristics in the motion pixel point information change curve; the motion pixel point information change curve at least comprises the preset change characteristics corresponding to a first state parameter; collecting a video of the target region before the target object is used as a reference video stream; determining motion pixel point change characteristics of the reference video stream based on the motion pixel point information corresponding to the video frames in the reference video stream; correcting the motion pixel point information change curve based on the motion pixel point change characteristics of the reference video stream and the preset change characteristics corresponding to the first state parameter.

2. The object delivery method according to claim 1, wherein after the step of determining the object delivery strategy for delivering the target object to the target region based on the motion pixel point change characteristics of the target video stream, the method further comprises: delivering the target object to the target region by an object delivery device associated with the target region according to the object delivery strategy.

3. The object delivery method according to claim 1 or 2, wherein the step of determining the object delivery strategy for delivering the target object to the target region based on the motion pixel point change characteristics of the target video stream comprises: determining a state parameter of a target object contained in the target video stream based on the motion pixel point change characteristics of the target video stream; the state parameter represents an activity level of the target object using the target object; determining the object delivery strategy for delivering the target object to the target region according to the state parameter.

4. The object delivery method according to claim 3, wherein before the step of respectively performing segmentation processing on a plurality of continuous video frames contained in a target video stream to obtain mask images corresponding to the video frames, the method further comprises: matching an object delivery strategy corresponding to each of the preset state parameters.

5. The object dropping method according to claim 4, wherein The preset state parameters comprise at least one of a first state parameter, a second state parameter, a third state parameter and a fourth state parameter; The step of matching an object delivery strategy corresponding to each of the preset state parameters comprises at least one of the following: In response to the preset change feature corresponding to the first state parameter, the first state parameter is matched with a first delivery strategy. In response to the preset change feature corresponding to the second state parameter, the second state parameter is matched with a second delivery strategy. In response to the preset change feature corresponding to the third state parameter, the third state parameter is matched with a third delivery strategy. In response to the preset change feature corresponding to the fourth state parameter, the fourth state parameter is matched with a fourth delivery strategy.

6. The object dropping method according to claim 3, wherein The motion pixel point information includes a number of motion pixel points. The motion pixel point information corresponding to each of the video frames is used to determine a motion pixel point change feature of the target video stream, including: The number of motion pixel points corresponding to each of the video frames is used to generate a motion pixel point number change trend of the target video stream.

7. The object dropping method according to claim 6, wherein The preset change feature includes a preset change trend. The motion pixel point change feature of the target video stream is used to determine an object delivery strategy for delivering target objects to the target area, including: The motion pixel point number change trend of the target video stream is compared with each of the preset change trends in the motion pixel point information change curve. In response to the motion pixel point number change trend being consistent with a preset change trend, the object delivery strategy corresponding to the preset change trend is determined for delivering target objects to the target area.

8. The object dropping method according to claim 1, wherein The object delivery strategy includes at least one of a feeding amount specification, a feeding rate specification, a feeding time length specification, a feeding location specification, and a feeding amount specification corresponding to each of the feeding locations.

9. An object launching device, characterized by The object delivery device includes: A preprocessing module is configured to pre-construct a motion pixel point information change curve of a target object in a whole process before and after using target objects, the motion pixel point information change curve includes a plurality of preset change features, each of the preset change features in the motion pixel point information change curve is matched with a corresponding preset state parameter, the motion pixel point information change curve includes at least a preset change feature corresponding to a first state parameter, a video of a target area before using the target objects is collected as a reference video stream, a motion pixel point change feature of the reference video stream is determined based on motion pixel point information corresponding to each video frame in the reference video stream, the motion pixel point information change curve is corrected based on the motion pixel point change feature of the reference video stream and the preset change feature corresponding to the first state parameter, a plurality of continuous video frames included in a target video stream are segmented to obtain a mask image corresponding to each of the video frames, and the target video stream is collected for a target area; A statistics module is configured to count motion pixel point information of a target object in a mask image of each of the video frames. An analysis module is configured to determine a motion pixel point change feature of the target video stream based on the motion pixel point information corresponding to each of the video frames. A determination module is configured to determine an object delivery strategy for delivering target objects to the target area based on the motion pixel point change feature of the target video stream.

10. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is configured to execute program data to implement the steps in the object dropping method according to any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps in the object dropping method according to any one of claims 1-8.

Images