Data detection method and device and electronic equipment

Abstract

The invention provides a data detection method and device and electronic equipment, and relates to the technical field of monitoring. The method comprises the following steps: firstly, acquiring a grain detection image, and performing color space conversion on the grain detection image to obtain a region conversion result; then judging whether the region conversion result meets a preset threshold value requirement or not; if yes, it is determined that grain filling is completed, the grain type is recognized based on the continuous image frames after grain filling is completed, and a grain detection model corresponding to the grain type is selected for grain impurity-containing breakage rate detection; if not, the grain filling state of the next frame of grain detection image is detected, and by adopting the mode, the efficiency and accuracy of impurity-containing crushing detection of various grains can be improved.

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to a data detection method, device and electronic equipment. Background Art

[0002] With the development of intelligent combine harvesters, grain impurity and breakage monitoring technology based on machine vision and deep learning has been widely used. In the related technology, there is a method of setting up an impurity and breakage detection device to detect the impurity and breakage rate of grains. Specifically, the impurity and breakage detection device includes a box body and a breakage detector. The top and bottom of the box body are respectively provided with a feed port and a discharge port that penetrate inside and outside. The discharge port is equipped with a switch conveyor that can be opened and closed. A material level sensor is provided on the top of the box body to detect whether the grain in the box is full; a detection hole is provided on the box body, and a breakage detector is installed at the detection hole to detect the impurity and breakage rate of grains in the box body; in addition, there is a method of sending the grain image collected by the image acquisition device to the ARM processor based on the image segmentation network Unet, and the ARM processor processes to obtain the pixel a of the broken grain, the pixel b of the intact grain, and the pixel c of the stem and branch, and combines the moisture content sent by the host computer to calculate the impurity and breakage rate of the grain.

[0003] The first method is to detect the grain fullness through the material level sensor. When there are a large number of fallen grains, the fallen grains will form an obstruction, resulting in false detection and affecting the detection accuracy of the crushing detector.

[0004] Method 2 can detect the impurity and breakage rate of a specific type of grain, but it cannot meet the accuracy requirements for detecting the impurity and breakage rate of multiple types of grains. In addition, Method 2 usually feeds back the impurity and breakage rate on the vehicle side based on a single-frame sampling graph. The impurity and breakage rate obtained based on a single-frame sampling graph is not conducive to vehicle-side regulation. Summary of the invention

[0005] The present invention application provides a data detection method, device and electronic device to improve the efficiency and accuracy of grain impurity and breakage detection. The specific technical solution is as follows:

[0006] In a first aspect, the present application provides a data detection method, comprising:

[0007] Acquire grain detection images;

[0008] Performing color space conversion on the grain detection image to obtain a region conversion result;

[0009] Determining whether the area conversion result meets a preset threshold requirement;

[0010] If yes, it is determined that the grain filling is completed, the type of grain is identified based on the continuous image frames of the grain filling completion, and a grain detection model corresponding to the grain type is selected to perform grain impurity and breakage rate detection;

[0011] If not, the grain filling status of the next grain detection image is detected.

[0012] Grain detection images are collected through image acquisition equipment, and multiple grain detection models are trained based on deep learning technology. Automatic switching of grain detection models is achieved according to the grain type recognition results, which can improve the efficiency of detecting the impurities and breakage of various grains. The grain detection images are converted into color space to obtain the regional conversion results, and then whether the grain is full is determined based on whether the regional conversion results meet the preset threshold requirements, which is convenient for subsequent detection of the impurities and breakage rate of the grains, thereby improving the detection efficiency of the impurities and breakage rate of the grains.

[0013] In a possible implementation, performing color space conversion on the grain detection image to obtain a region conversion result includes:

[0014] Dividing the grain detection image into a first region image and a second region image according to regions;

[0015] Performing grayscale binarization processing on the first region image to obtain a first region conversion result, and performing color space conversion on the second region image according to a set sliding window to obtain a second region conversion result corresponding to each sliding window;

[0016] The preset threshold requirements include:

[0017] The number of target identifiers included in the first area conversion result is greater than or equal to a first threshold, and the second area conversion results corresponding to each of the sliding windows are greater than a second threshold.

[0018] By dividing the grain detection image into a first area image and a second area image, performing grayscale binarization processing on the first area image to obtain the first area conversion result, and performing color space conversion on the second area image according to a set sliding window to obtain the second area conversion result corresponding to each sliding window, it is possible to facilitate the subsequent identification of the grain fullness of the detection box based on the first area conversion result and the second area conversion result, thereby improving the accuracy of grain fullness identification.

[0019] In a possible implementation, the identifying the type of grain based on the continuous image frames after the grain filling is completed includes:

[0020] Acquire sample data of multiple grains, and expand the sample data to obtain amplified sample data;

[0021] Based on the sample data and the amplified sample data, a preset grain recognition model is trained and optimized to obtain a target grain recognition model;

[0022] The continuous image frames are input into the target grain recognition model to obtain a grain type recognition result.

[0023] By expanding the sample data, the accuracy of the grain recognition model can be improved.

[0024] In a possible implementation, the step of selecting a grain detection model corresponding to the grain type to perform grain impurity and breakage rate detection includes:

[0025] Based on the improved image segmentation network U-Net, the corresponding weights of multiple grain detection models are trained for different grain types.

[0026] When it is determined that a grain type indication message sent by the target object is received, based on the grain type in the grain type indication message, a grain detection model corresponding to the grain type is selected from the multiple grain detection models to perform grain impurity and breakage rate detection;

[0027] When the grain type indication message is not received, based on the grain type identification result output by the target grain identification model, a grain detection model corresponding to the grain type identification result is selected from the multiple grain detection models to perform grain impurity and breakage rate detection.

[0028] Based on the above method, it is possible to automatically select a grain detection model corresponding to the grain type for grain impurity and breakage rate detection for different grain types, thereby improving the efficiency and accuracy of the detection.

[0029] In a possible implementation, the grain impurity and breakage rate is detected by:

[0030] Performing mixed and fragmented identification on the continuous image frames, and calculating the mixed and fragmented rate corresponding to each frame of the continuous image frames;

[0031] Using a moving average algorithm, filtering the impurity and breakage rate corresponding to each frame of the continuous image frames to obtain a filtered target impurity and breakage rate, wherein the target impurity and breakage rate is used to characterize the harvest quality of the grain of the target object within a preset operation time;

[0032] The target impurity crushing rate is pushed to the target object.

[0033] Based on the above method, the filtered target debris and crushing rate is continuously pushed to the data analysis interface of the display device, so that the operator can make timely adjustments according to the change of the debris and crushing rate to improve the harvesting level.

[0034] In a second aspect, the present application provides a data detection device, comprising:

[0035] A data acquisition module, used for acquiring grain detection images;

[0036] A data conversion module, used for performing color space conversion on the grain detection image to obtain a region conversion result;

[0037] A data detection module, used to determine whether the area conversion value meets a preset threshold requirement;

[0038] If yes, it is determined that the grain filling is completed, the type of grain is identified based on the continuous image frames of the grain filling completion, and a grain detection model corresponding to the grain type is selected to perform grain impurity and breakage rate detection;

[0039] If not, the grain filling status of the next grain detection image is detected.

[0040] In a possible implementation, the data conversion module is specifically used to:

[0041] Dividing the grain detection image into a first region image and a second region image according to regions;

[0042] Performing grayscale binarization processing on the first region image to obtain a first region conversion result, and performing color space conversion on the second region image according to a set sliding window to obtain a second region conversion result corresponding to each sliding window;

[0043] The preset threshold requirements include:

[0044] The number of target identifiers included in the first area conversion result is greater than or equal to a first threshold, and the second area conversion results corresponding to each of the sliding windows are greater than a second threshold.

[0045] In a possible implementation, the data detection module is specifically used to:

[0046] Acquire sample data of multiple grains, and expand the sample data to obtain amplified sample data;

[0047] Based on the sample data and the amplified sample data, a preset grain recognition model is trained and optimized to obtain a target grain recognition model;

[0048] The continuous image frames are input into the target grain recognition model to obtain a grain type recognition result.

[0049] In a possible implementation, the data detection module is specifically used to:

[0050] Based on the improved image segmentation network U-Net, the corresponding weights of multiple grain detection models are trained for different grain types.

[0051] When it is determined that a grain type indication message sent by the target object is received, based on the grain type in the grain type indication message, a grain detection model corresponding to the grain type is selected from the multiple grain detection models to perform grain impurity and breakage rate detection;

[0052] When the grain type indication message is not received, based on the grain type identification result output by the target grain identification model, a grain detection model corresponding to the grain type identification result is selected from the multiple grain detection models to perform grain impurity and breakage rate detection.

[0053] In a possible implementation, the data detection module is specifically used to:

[0054] Performing mixed and fragmented identification on the continuous image frames, and calculating the mixed and fragmented rate corresponding to each frame of the continuous image frames;

[0055] Using a moving average algorithm, filtering the impurity and breakage rate corresponding to each frame of the continuous image frames to obtain a filtered target impurity and breakage rate, wherein the target impurity and breakage rate is used to characterize the harvest quality of the grain of the target object within a preset operation time;

[0056] The target impurity crushing rate is pushed to the target object.

[0057] In a third aspect, the present application provides an electronic device, including:

[0058] Memory, used to store computer programs;

[0059] The processor is used to implement the steps of the above-mentioned data detection method when executing the computer program stored in the memory.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data detection method are implemented.

[0061] For each aspect from the second to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flow chart of a data detection method provided in an embodiment of the present application;

[0063] Figure 2 A schematic diagram of the data detection system architecture provided in an embodiment of the present application;

[0064] Figure 3 An overall flow chart of impurity and crushing detection provided in an embodiment of the present application;

[0065] Figure 4 A flow chart of grain fullness identification provided in an embodiment of the present application;

[0066] Figure 5 A flow chart of impurity and crushing detection provided in an embodiment of the present application;

[0067] Figure 6 A schematic diagram of the structure of a data detection device provided in an embodiment of the present application;

[0068] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0070] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0071] With the development of intelligent combine harvesters, grain impurity and breakage monitoring technology based on machine vision and deep learning has been widely used. In the related technology, there is a method of setting up an impurity and breakage detection device to detect the impurity and breakage rate of grains. Specifically, the impurity and breakage detection device includes a box body and a breakage detector. The top and bottom of the box body are respectively provided with a feed port and a discharge port that penetrate inside and outside. The discharge port is equipped with a switch conveyor that can be opened and closed. A material level sensor is provided on the top of the box body to detect whether the grain in the box is full; a detection hole is provided on the box body, and a breakage detector is installed at the detection hole to detect the impurity and breakage rate of grains in the box body; in addition, there is a method of sending the grain image collected by the image acquisition device to the ARM processor based on the image segmentation network Unet, and the ARM processor processes to obtain the pixel a of the broken grain, the pixel b of the intact grain, and the pixel c of the stem and branch, and combines the moisture content sent by the host computer to calculate the impurity and breakage rate of the grain.

[0072] In the above-mentioned method 1, the fullness of grain is detected by a material level sensor. When a large amount of fallen grains occur, the fallen grains will form an obstruction, resulting in false detection, which affects the detection accuracy of the crushing detector.

[0073] Method 2 can detect the impurity and breakage rate of a specific type of grain, but it cannot meet the accuracy requirements for detecting the impurity and breakage rate of multiple types of grains. In addition, Method 2 usually feeds back the impurity and breakage rate on the vehicle side based on a single-frame sampling graph. The impurity and breakage rate obtained based on a single-frame sampling graph is not conducive to vehicle-side regulation.

[0074] In view of this, in order to improve the detection accuracy of the impurity and breakage rate of various types of grains, the present application provides a data detection method, which specifically includes: first acquiring a grain detection image, performing color space conversion on the grain detection image, and obtaining a region conversion result; then determining whether the region conversion result meets a preset threshold requirement; if so, determining that the grain filling is completed, identifying the type of grain based on continuous image frames of completed grain filling, and selecting a grain detection model corresponding to the grain type to perform grain impurity and breakage rate detection; if not, detecting the grain filling status of the next frame of grain detection image.

[0075] The present application collects grain detection images through image acquisition equipment, trains multiple grain detection models based on deep learning technology, and realizes automatic switching of grain detection models according to grain type recognition results, which can improve the efficiency of detecting the impurities and breakage of various grains; by performing color space conversion on the grain detection image, the regional conversion result is obtained, and then whether the grain is full is determined based on whether the regional conversion result meets the preset threshold requirements, which is convenient for subsequent detection of the impurities and breakage rate of the grain, thereby improving the detection efficiency of the impurities and breakage rate of the grain.

[0076] Reference Figure 1As shown, it is a flow chart of a data detection method provided in an embodiment of the present application, the method comprising:

[0077] S1, obtain grain detection image.

[0078] First of all, the method provided in this application can be applied to Figure 2 In the system architecture shown, the system architecture includes: image acquisition equipment, image processing controller, harvester core electronic control unit VCU, grain unloading motor, display device, and the method provided in the present application can be run in the image processing controller.

[0079] The image acquisition device is used to collect grain detection images in the grain detection box at the grain drum elevator of the harvester, and send the grain detection images to the image processing controller for corresponding processing. The image acquisition device can be various types of detection devices, such as cameras, etc.

[0080] The image processing controller is deployed on the harvester to perform grain filling recognition (full grain recognition), grain type recognition, and grain impurity and breakage rate detection on the grain detection image; in addition, the image processing controller can also transmit the collected grain detection image to the cloud through the intelligent vehicle terminal (T-Box) data, and the cloud will train and iterate the model, and return the trained target recognition model and target detection model to the image processing controller.

[0081] The harvester's core electronic control unit VCU determines the driver's driving intention by collecting signals from the accelerator pedal, gear position, brake pedal, etc.; in the present application, the communication between the image processing controller and the above-mentioned components can be forwarded by the VCU. For example, when the grain full signal output by the image processing controller is received, a control signal is sent to the grain unloading motor to perform the grain unloading operation or the instruction of the target object (driver, operator) is received to switch the instructions.

[0082] The display device is used to display the grain impurity and breakage rate detection index value pushed by the image processing controller, so that the driver can adjust the operation based on the current grain impurity and breakage rate, thereby improving the intelligence level and harvesting quality of the harvester.

[0083] In the embodiments of the present application, as shown in the attached Figure 3 As shown, before the image processing controller detects the impurities and breakage of grains, it first needs to obtain the grain detection image, then perform data sampling, then perform grain type recognition, and finally switch the grain detection model.

[0084] Data sampling can be divided into two parts: image cropping and grain fullness recognition. When performing image cropping, since the area photographed by the image acquisition device in the grain detection box usually contains unnecessary image information, when obtaining the sampled image taken by the image acquisition device, the origin of the cropping area can be determined according to the corner detection method, and the sampled image can be cropped to obtain a valid grain detection image. Preferably, the size of the grain detection image can be 480*480, and this application does not impose any specific restrictions on this.

[0085] By cropping the sampled image acquired by the image acquisition device using the corner detection method, a grain detection image can be obtained, which is convenient for subsequent detection of the grain impurity and breakage rate based on the grain detection image.

[0086] S2, performing color space conversion on the grain detection image to obtain a region conversion result.

[0087] When a traditional harvester is operating, a feeding-detection-discharging cycle is implemented in the detection box at the grain transport drum, and a material level sensor is usually installed in the detection box to determine the material level of the grain in the detection box. When it is determined that the grain in the detection box is full, the camera collects the detection image, identifies the impurities and crushing, and controls the motor to unload the grain.

[0088] In order to reduce the misjudgment of the grain fullness in the detection box, refer to the attached Figure 4 As shown, the image processing controller can first perform color space conversion on the grain detection image to obtain a region conversion result. The specific process is as follows:

[0089] The image processing controller can divide the grain detection image according to the area, into a first area image and a second area image, wherein the first area image may be an image collected in the area near the top of the detection box. In a specific implementation, the upper one-tenth area of ​​the grain detection image can be used as the first area image; the second area image may be an image collected in other areas of the detection box. In a specific implementation, the lower nine-tenths area of ​​the grain detection image can be used as the second area image.

[0090] In an embodiment of the present application, the reflectivity of the detection box background and the grains is different. In order to improve the rate of identifying full grains, the image processing controller can first perform grayscale binarization processing on the first area image to obtain a first area conversion result, and the first area conversion result can be a binary image. Although it is possible to determine whether the detection box is full based on the number of values ​​representing the grains in the binary image, the number of values ​​representing the grains in the binary image will increase when a large number of grains fall, creating the illusion that the detection box is full. Therefore, the image processing controller can also perform color space conversion on the second area image according to the set sliding window to obtain the second area conversion results corresponding to each sliding window, and then determine whether the detection box is full based on the second area conversion result.

[0091] By dividing the grain detection image into a first area image and a second area image, performing grayscale binarization processing on the first area image to obtain the first area conversion result, and performing color space conversion on the second area image according to a set sliding window to obtain the second area conversion result corresponding to each sliding window, it is possible to facilitate the subsequent identification of the grain fullness of the detection box based on the first area conversion result and the second area conversion result, thereby improving the accuracy of grain fullness identification.

[0092] S3, determining whether the area conversion result meets a preset threshold requirement.

[0093] If the region conversion result meets the preset threshold requirement, step S41 is executed; if the region conversion result does not meet the preset threshold requirement, step S42 is executed.

[0094] In the embodiment of the present application, after obtaining the area conversion result, the image processing controller can determine whether the inspection box is full of grain according to whether the area conversion result meets the preset threshold requirement.

[0095] Specifically, the image processor can obtain the area conversion result, which includes the first area conversion result and the second area conversion result; first, it is determined whether the number of target identifiers contained in the first area conversion result is greater than or equal to the first threshold, and the target identifier can be an identifier that characterizes the degree of grain reflectivity; then it is determined whether the second area conversion result corresponding to each sliding window is greater than the second threshold; when it is determined that the number of target identifiers is greater than or equal to the first threshold, and the second area conversion result corresponding to each sliding window is greater than the second threshold, it is determined that the area conversion result meets the preset threshold requirement and the grain in the inspection box is filled, otherwise, it is determined that the grain in the inspection box has not yet been filled.

[0096] It should be pointed out that determining whether the second area conversion result corresponding to each sliding window is greater than the second threshold can be used to further determine whether there is a background area of ​​the detection box. If there is a background area of ​​the detection box in any sliding window, it means that the grain in the detection box is not full.

[0097] S41, determining that the grain filling is completed, identifying the type of grain based on the continuous image frames of the grain filling completion, and selecting a grain detection model corresponding to the grain type to perform grain impurity and breakage rate detection.

[0098] In an embodiment of the present application, when the image processing controller determines that the grain filling is complete, it can unload the grain according to the set unloading speed. During the unloading process, the image sampling of the unloading process is continuously performed to obtain continuous image frames, and the similarity of the sampled continuous image frames is judged to avoid sampling the same image frames. Finally, the continuous image frames determined to be full of grain are input into the crop type recognition module to identify the type of grain.

[0099] In the embodiment of the present application, the process of identifying the type of grain is as follows:

[0100] Before each operation of the harvester, it is necessary to identify the type of grain. The image processing controller can establish a cloud database to obtain sample data of various grains, and then expand the sample data to obtain amplified sample data. Then, based on the sample data and the amplified sample data, the preset grain recognition model is trained and optimized to obtain the target grain recognition model with the best recognition effect. Finally, the grain type is identified based on the target grain recognition model to obtain the grain type recognition result. In the specific implementation, the grain recognition model can adopt a deep neural network structure GoogLeNet, which introduces a core sub-network structure to enable the network to extract features at multiple scales, thereby enhancing the prediction ability of the network model.

[0101] In an embodiment of the present application, in view of the large difference in characteristics of different grains (such as corn, wheat, soybeans, etc.), in order to detect the impurity and breakage rate of grains for different grain types, the accuracy of the detection of impurity and breakage rate of grains is improved. The image processing controller can select the grain detection model corresponding to the grain type to detect the impurity and breakage rate of grains. Specifically, the image processing controller can train the weights corresponding to each of the multiple grain detection models for different grain types based on the improved image segmentation network U-Net network; when it is determined that the grain type indication message sent by the target object is received, for example, the grain type manually selected by the operator (driver), according to the grain type in the grain type indication message, the grain detection model corresponding to the grain type is selected from multiple grain detection models to detect the impurity and breakage rate of grains. When it is determined that the grain type indication message is not received (such as the operator does not manually select the grain type), the image processing controller can select the grain detection model corresponding to the grain type from multiple grain detection models based on the grain type recognition result output by the target grain recognition model to detect the impurity and breakage rate of grains.

[0102] This application trains the corresponding weights of multiple grain detection models for different grain types, and automatically switches the grain detection model corresponding to the grain type, which can improve the speed and accuracy of subsequent detection of grain impurities and breakage rates.

[0103] In the embodiment of the present application, after confirming the grain detection model corresponding to the grain type, the image processing controller can perform grain impurity and breakage rate detection, and the specific process is as follows:

[0104] See attached Figure 5 As shown, when the image processing controller samples the image in the detection box, it still performs grain fullness recognition on the sampled image to ensure that the grain detection image size is also 480*480 and does not contain invalid information areas, and then samples the grain full and non-repetitive data to obtain continuous image frames, and inputs the continuous image frames into the impurity and broken recognition module; the impurity and broken recognition module performs impurity and broken recognition on the continuous image frames, and calculates the impurity and broken rate corresponding to each frame in the continuous image frames. At this stage, the image processing controller can interact with the operator through the window provided by the display device, and the operator can feedback whether the recognition accuracy of the impurity and broken recognition model meets the accuracy requirements. If the recognition effect of the impurity and broken recognition model fed back does not meet the accuracy requirements, the sampling module of the image processing controller can save the sampled image, transmit the sampled image back to the cloud, expand the cloud data, iteratively train the impurity and broken recognition model, and improve the recognition accuracy of the impurity and broken recognition model.

[0105] In an embodiment of the present application, in order to be able to provide feedback to the operator on the harvest quality of grain within a preset operating time, the image processing controller can use a moving average algorithm to filter the impurity and breakage rate corresponding to each frame of the continuous image frames to obtain a filtered target impurity and breakage rate, wherein the target impurity and breakage rate is used to characterize the harvest quality of grain of the target object within the preset operating time.

[0106] After obtaining the target impurity crushing rate, the image processing controller can push the target impurity crushing rate to the operator based on the following strategies:

[0107] When the sampling image is in a state where the grain is not full, for example, the harvester is suspended, stopped or turning, the trash and breakage rate and related data will not be pushed at this time; when normal sampling is performed for trash and breakage identification, the filtered target trash and breakage rate will be continuously pushed to the data analysis interface of the display device, so that the operator can make timely operation adjustments according to the changes in the trash and breakage rate to improve the grain harvesting level.

[0108] S42, detecting the grain filling status of the next frame of grain detection image.

[0109] In an embodiment of the present application, when the image processing controller determines that the area conversion result does not meet the preset threshold requirement, it can detect the grain filling status of the next frame of grain detection image. The method for detecting the grain filling status can refer to the embodiment of the grain full identification method, which will not be repeated here.

[0110] In summary, the method provided in the present application utilizes an image acquisition device to acquire grain detection images, and identifies whether the grain is full based on the grain detection images, without relying on a material level sensor for full grain identification; designs a data sampling method, and trains multiple grain detection models based on deep learning technology; determines the crop type based on continuous image frames, thereby realizing the switching of automated grain detection models, and can improve the efficiency of detecting impurities and breakage of various grains; by designing a cloud data expansion strategy, users can provide feedback on the performance of the impurity and breakage identification model through the human-computer interaction interface, thereby achieving the purpose of iterative optimization of the grain detection model on the cloud.

[0111] Based on the method provided in the above embodiment, the embodiment of the present application also provides a data detection device, such as Figure 6 The figure is a schematic diagram of the structure of a data detection device in an embodiment of the present application, the device comprising:

[0112] The data acquisition module 601 is used to acquire grain detection images;

[0113] A data conversion module 602 is used to perform color space conversion on the grain detection image to obtain a region conversion result;

[0114] A data detection module 603 is used to determine whether the region conversion value meets a preset threshold requirement;

[0115] If yes, it is determined that the grain filling is completed, the type of grain is identified based on the continuous image frames of the grain filling completion, and a grain detection model corresponding to the grain type is selected to perform grain impurity and breakage rate detection;

[0116] If not, the grain filling status of the next grain detection image is detected.

[0117] In a possible implementation, the data conversion module 602 is specifically used for:

[0118] Dividing the grain detection image into a first region image and a second region image according to regions;

[0119] Performing grayscale binarization processing on the first region image to obtain a first region conversion result, and performing color space conversion on the second region image according to a set sliding window to obtain a second region conversion result corresponding to each sliding window;

[0120] The preset threshold requirements include:

[0121] The number of target identifiers included in the first area conversion result is greater than or equal to a first threshold, and the second area conversion results corresponding to each of the sliding windows are greater than a second threshold.

[0122] In a possible implementation, the data detection module 603 is specifically used to:

[0123] Acquire sample data of multiple grains, and expand the sample data to obtain amplified sample data;

[0124] Based on the sample data and the amplified sample data, a preset grain recognition model is trained and optimized to obtain a target grain recognition model;

[0125] The continuous image frames are input into the target grain recognition model to obtain a grain type recognition result.

[0126] In a possible implementation, the data detection module 603 is specifically used to:

[0127] Based on the improved image segmentation network U-Net, the corresponding weights of multiple grain detection models are trained for different grain types.

[0128] When it is determined that a grain type indication message sent by the target object is received, based on the grain type in the grain type indication message, a grain detection model corresponding to the grain type is selected from the multiple grain detection models to perform grain impurity and breakage rate detection;

[0129] When the grain type indication message is not received, based on the grain type identification result output by the target grain identification model, a grain detection model corresponding to the grain type identification result is selected from the multiple grain detection models to perform grain impurity and breakage rate detection.

[0130] In a possible implementation, the data detection module 603 is specifically used to:

[0131] Performing mixed and fragmented identification on the continuous image frames, and calculating the mixed and fragmented rate corresponding to each frame of the continuous image frames;

[0132] Using a moving average algorithm, filtering the impurity and breakage rate corresponding to each frame of the continuous image frames to obtain a filtered target impurity and breakage rate, wherein the target impurity and breakage rate is used to characterize the harvest quality of the grain of the target object within a preset operation time;

[0133] The target impurity crushing rate is pushed to the target object.

[0134] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application, and the electronic device can realize the function of the aforementioned data detection device, referring to Figure 7 , the electronic device comprises:

[0135] At least one processor 701, and a memory 702 connected to the at least one processor 701. The specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application. Figure 7 In the example, the processor 701 and the memory 702 are connected via a bus 700. Figure 7 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 700 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 7 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 701 can also be called a controller, and there is no limitation on the name.

[0136] In the embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701. The at least one processor 701 can execute the data detection method discussed above by executing the instructions stored in the memory 702. The processor 701 can implement Figure 6 The functions of each module in the device shown.

[0137] Among them, the processor 701 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 702 and calling data stored in the memory 702, the various functions of the device and process data, the device can be monitored as a whole.

[0138] In one possible design, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0139] Processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the data detection method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0140] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 702 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0141] By designing and programming the processor 701, the code corresponding to the data detection method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 The steps of the data detection method of the embodiment shown are as follows: How to design and program the processor 701 is a technique well known to those skilled in the art and will not be described in detail here.

[0142] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the data detection method discussed above.

[0143] In some possible implementations, various aspects of the data detection method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the data detection method according to various exemplary implementations of the present application described above in this specification.

[0144] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0148] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data detection method, characterized in that: include: Acquire grain detection images; Performing color space conversion on the grain detection image to obtain a region conversion result; Determining whether the area conversion result meets a preset threshold requirement; If yes, it is determined that the grain filling is completed, the type of grain is identified based on the continuous image frames of the grain filling completion, and a grain detection model corresponding to the grain type is selected to perform grain impurity and breakage rate detection; If not, the grain filling status of the next grain detection image is detected.

2. The method according to claim 1, characterized in that The step of performing color space conversion on the grain detection image to obtain a region conversion result includes: Dividing the grain detection image into a first region image and a second region image according to regions; Performing grayscale binarization processing on the first region image to obtain a first region conversion result, and performing color space conversion on the second region image according to a set sliding window to obtain a second region conversion result corresponding to each sliding window; The preset threshold requirements include: The number of target identifiers included in the first area conversion result is greater than or equal to a first threshold, and the second area conversion results corresponding to each of the sliding windows are greater than a second threshold.

3. The method according to claim 1, characterized in that The method of identifying the type of grain based on the continuous image frames after the grain filling is completed includes: Acquire sample data of multiple grains, and expand the sample data to obtain amplified sample data; Based on the sample data and the amplified sample data, a preset grain recognition model is trained and optimized to obtain a target grain recognition model; The continuous image frames are input into the target grain recognition model to obtain a grain type recognition result.

4. The method according to claim 1, characterized in that The step of selecting a grain detection model corresponding to the grain type to detect the grain impurity and breakage rate comprises: Based on the improved image segmentation network U-Net, the corresponding weights of multiple grain detection models are trained for different grain types. When it is determined that a grain type indication message sent by the target object is received, based on the grain type in the grain type indication message, a grain detection model corresponding to the grain type is selected from the multiple grain detection models to perform grain impurity and breakage rate detection; When the grain type indication message is not received, based on the grain type identification result output by the target grain identification model, a grain detection model corresponding to the grain type identification result is selected from the multiple grain detection models to perform grain impurity and breakage rate detection.

5. The method according to claim 1, characterized in that The grain impurities and breakage rate are tested by: Performing mixed and fragmented identification on the continuous image frames, and calculating the mixed and fragmented rate corresponding to each frame of the continuous image frames; Using a moving average algorithm, filtering the impurity and breakage rate corresponding to each frame of the continuous image frames to obtain a filtered target impurity and breakage rate, wherein the target impurity and breakage rate is used to characterize the harvest quality of the grain of the target object within a preset operation time; The target impurity crushing rate is pushed to the target object.

6. A data detection device, characterized in that: include: A data acquisition module, used for acquiring grain detection images; A data conversion module, used for performing color space conversion on the grain detection image to obtain a region conversion result; A data detection module, used to determine whether the area conversion value meets a preset threshold requirement; If yes, it is determined that the grain filling is completed, the type of grain is identified based on the continuous image frames of the grain filling completion, and a grain detection model corresponding to the grain type is selected to perform grain impurity and breakage rate detection; If not, the grain filling status of the next grain detection image is detected.

7. The device according to claim 6, characterized in that The data conversion module is specifically used for: Dividing the grain detection image into a first region image and a second region image according to regions; Performing grayscale binarization processing on the first region image to obtain a first region conversion result, and performing color space conversion on the second region image according to a set sliding window to obtain a second region conversion result corresponding to each sliding window; The preset threshold requirements include: The number of target identifiers included in the first area conversion result is greater than or equal to a first threshold, and the second area conversion results corresponding to each of the sliding windows are greater than a second threshold.

8. The device according to claim 6, characterized in that The data detection module is specifically used for: Based on the improved image segmentation network U-Net, the corresponding weights of multiple grain detection models are trained for different grain types. When it is determined that a grain type indication message sent by the target object is received, based on the grain type in the grain type indication message, a grain detection model corresponding to the grain type is selected from the multiple grain detection models to perform grain impurity and breakage rate detection; When the grain type indication message is not received, based on the grain type identification result output by the target grain identification model, a grain detection model corresponding to the grain type identification result is selected from the multiple grain detection models to perform grain impurity and breakage rate detection.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method steps of any one of claims 1 to 5 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images