Image data processing method and device, storage medium and electronic device

Abstract

The invention discloses an image data processing method and device, a storage medium and an electronic device, and relates to the technical field of image data processing, and the image data processing method comprises the steps: carrying out the feature extraction of image data obtained by carrying out the image collection of a target object, and obtaining a current image feature vector corresponding to the image data, the current image feature vector has multiple dimensions; inputting the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model for the current image feature vector, the recognition result being used for indicating an object type of the target object; and generating a target execution instruction based on the object type, and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction, and by adopting the technical scheme, the technical problem of low bad fruit screening efficiency caused by inaccurate image recognition results of bad fruits during bad fruit screening is solved.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to an image data processing method, device, storage medium and electronic device. Background Art

[0002] At present, with the continuous advancement of artificial intelligence technology, its application in some traditional industries is becoming more and more extensive. For example, fruit and vegetable screening is generally done manually, but manual screening is time-consuming, labor-intensive and prone to errors.

[0003] Although related technologies have begun to use factors such as weight for machine screening, this approach still results in bad fruit after screening and requires manual review, making it difficult to implement a streamlined process and resulting in low screening efficiency. This machine screening method also often mistakenly screens out intact fruits and vegetables as bad fruit, causing losses. Therefore, related technologies face the technical problem of inaccurate image recognition results for bad fruit, leading to low bad fruit screening efficiency.

[0004] In the related art, an effective solution has not yet been proposed for the technical problem that when screening bad fruits, the image recognition results of bad fruits are inaccurate, resulting in low efficiency in screening bad fruits. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, storage medium, and electronic device for processing image data to at least solve the technical problem in the related art of inaccurate image recognition results of bad fruits, resulting in low bad fruit screening efficiency, when screening bad fruits.

[0006] According to one embodiment of the embodiments of the present application, a method for processing image data is provided, including: performing feature extraction on image data obtained by image acquisition of a target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions; inputting the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model for the current image feature vector, wherein the recognition result is used to indicate the object type of the target object; generating a target execution instruction based on the object type, and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate on the target object according to the target execution instruction.

[0007] In an exemplary embodiment, before performing feature extraction on image data obtained by capturing an image of a target object, the method further includes: obtaining initial image data obtained by an image capture device capturing an image of a target object located in an image capture area at a preset frame rate; determining image clarity of an initial image corresponding to the initial image data, and determining a first clear image from the initial image whose image clarity is greater than a first preset clarity; and determining the image data based on the first clear image.

[0008] In an exemplary embodiment, the method further includes: performing image sharpening processing on other images in the initial image except the first clear image to obtain a second clear image; and when it is determined that the image clarity of the second clear image is equal to the first preset clarity, determining the image data based on the first clear image and the second clear image.

[0009] In an exemplary embodiment, feature extraction is performed on image data obtained by image capture of a target object to obtain a current image feature vector corresponding to the image data, including: grouping the image data according to different capture angles to obtain multiple groups of data, wherein each group of data corresponds to an capture angle; analyzing each group of data in turn using a hierarchical analysis method to obtain target object features indicated by each group of data; feature extraction is performed on each group of data based on the target object features to obtain a first image feature vector for each group of data; and determining the current image feature vector based on multiple first feature vectors.

[0010] In an exemplary embodiment, feature extraction is performed on image data obtained by image capture of a target object to obtain a current image feature vector corresponding to the image data, including: classifying the image data according to different capture angles to obtain multiple first data sets, wherein each first data set in the first data sets corresponds to an capture angle; performing feature extraction on the data of each first data set to obtain a second image feature vector; grouping the image data according to different capture positions to obtain multiple second data sets, wherein each first data set in the first data set corresponds to an capture position; performing feature extraction on the data of each second data set to obtain a third image feature vector; and determining the current image feature vector based on the multiple second image feature vectors and the multiple third image feature vectors.

[0011] In an exemplary embodiment, a target execution instruction is generated based on the object type, and the target execution instruction is sent to an execution mechanism to control the execution mechanism to operate on the target object according to the target execution instruction, including: when it is determined that the object type is a damaged object, generating a first target execution instruction for performing a culling operation; sending the first target execution instruction to the execution mechanism to control the execution mechanism to perform a culling operation on the target object according to the first target execution instruction; when it is determined that the object type is an intact object, generating a second target execution instruction for performing a packing operation; sending the second target execution instruction to the execution mechanism to control the execution mechanism to perform a packing operation on the target object according to the second target execution instruction.

[0012] In an exemplary embodiment, after generating a target execution instruction based on the object type and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction, the method further includes: obtaining a first recognition number of multiple recognitions of different current image feature vectors using the object recognition model; determining a second recognition number that the object type is a damaged object from the first recognition number, and determining a first probability that the object is damaged based on the second recognition number and the first recognition number; determining a first number of times that the execution mechanism operates on the target object according to the target execution instruction, and determining a second probability based on the first number and the first recognition number; and generating an alarm message when it is determined that the difference between the first probability and the second probability is greater than a preset difference, and sending the alarm message to the first object, so that the first object optimizes the object recognition model based on the alarm information.

[0013] According to another aspect of an embodiment of the present application, a device for processing image data is also provided, including: a first obtaining module, used to extract features from image data obtained by image acquisition of a target object, and obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions; a second obtaining module, used to input the current image feature vector into an object recognition model, and obtain a recognition result of the object recognition model for the current image feature vector, wherein the recognition result is used to indicate the object type of the target object; a sending module, used to generate a target execution instruction based on the object type, and send the target execution instruction to an execution mechanism to control the execution mechanism to operate on the target object according to the target execution instruction.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the above-mentioned image data processing method when running.

[0015] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the image data processing method through the computer program.

[0016] In an embodiment of the present application, feature extraction is performed on image data obtained by image acquisition of a target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions; the current image feature vector is input into an object recognition model to obtain a recognition result of the object recognition model for the current image feature vector, wherein the recognition result is used to indicate the object type of the target object; a target execution instruction is generated based on the object type, and the target execution instruction is sent to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction. The present application can intelligently screen and filter fruits and vegetables that do not meet the standards, realize a fully automated and efficient fruit and vegetable screening process, solve the technical problem in the related art that, when screening bad fruits, the image recognition results of bad fruits are inaccurate, resulting in low bad fruit screening efficiency, improve the accuracy of the image recognition results of bad fruits, and thus improve the bad fruit screening efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a schematic diagram of the hardware environment of an image data processing method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for processing image data according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of a method for processing image data according to an embodiment of the present application;

[0022] Figure 4 This is a structural block diagram of an image data processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to one aspect of the embodiment of the present application, a method for processing image data is provided. The method for processing image data is widely used in smart factory, smart home, smart home, smart home device ecology, smart house ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned method for processing image data can be applied to Figure 1 In the hardware environment shown in FIG. 1 , which is composed of a terminal device 102 and a server 104. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0026] The aforementioned network may include, but is not limited to, at least one of the following: a wired network and a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth. The terminal device 102 may be, but is not limited to, a smart factory handheld device, a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing machine, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, smart curtains, smart audio and video, a smart socket, a smart speaker, a smart fresh air device, smart kitchen and bathroom equipment, smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purifier, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, and the like.

[0027] In this embodiment, a method for processing image data is provided, which is applied to the above-mentioned terminal device. Figure 2 : is a flowchart of a method for processing image data according to an embodiment of the present application, the process comprising the following steps:

[0028] Step S202: performing feature extraction on image data obtained by performing image acquisition on the target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions;

[0029] It is understandable that image feature extraction technology can be used to extract useful information from the original image data for further analysis, identification or classification. The current image feature vector contains the key information of the image of the target object, and the dimension of the current image feature vector depends on the feature vector extraction method used. The present application also provides some feature vector extraction methods, including but not limited to edge detection method, corner detection method, scale-invariant feature transformation method, speed robust feature method, deep learning model extraction method, and Hough transform method. The selection and design of feature vectors are crucial to the performance of image processing tasks. The current image feature vector extracted by the present application can capture the key information of the image and is robust to noise and changes.

[0030] The target group can be understood as fruits and vegetables. Fruits and vegetables refer to a general term for fruits and vegetables. They are an indispensable part of people's daily diet and provide the human body with a rich supply of vitamins, minerals, dietary fiber, and other nutrients. Fruits can include apples, bananas, oranges, grapes, strawberries, blueberries, etc. Vegetables can include tomatoes, carrots, spinach, broccoli, bell peppers, cucumbers, etc.

[0031] Step S204: inputting the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model on the current image feature vector, wherein the recognition result is used to indicate the object type of the target object;

[0032] The object recognition model is obtained by training using the historical image feature vectors of the historical images as input samples and the object types of the objects corresponding to the historical image feature vectors as output samples.

[0033] Step S206 : generating a target execution instruction based on the object type, and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction.

[0034] Through the above steps, feature extraction is performed on the image data obtained by image acquisition of the target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions; the current image feature vector is input into the object recognition model to obtain a recognition result of the object recognition model for the current image feature vector, wherein the recognition result is used to indicate the object type of the target object; a target execution instruction is generated based on the object type, and the target execution instruction is sent to the execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction. The present application can intelligently screen and filter fruits and vegetables that do not meet the standards, realize a fully automated and efficient fruit and vegetable screening process, solve the technical problem in the related art that, when screening bad fruits, the image recognition results of bad fruits are inaccurate, resulting in low bad fruit screening efficiency, improve the accuracy of the image recognition results of bad fruits, and thus improve the bad fruit screening efficiency.

[0035] Optionally, in one embodiment, for the bad fruit screening process, the present application may include an initial preparation stage, an image acquisition and preprocessing stage, an intelligent analysis and classification stage, a rejection operation stage, and a system optimization stage. In the initial preparation stage, the device is started to ensure that all components of all hardware devices (such as cameras, sensors, actuators, etc.) are in a standby state under normal operation. Then, the preset rejection criteria and parameter settings and the trained deep learning model (i.e., the above-mentioned object recognition model) are loaded, and the model is used to identify the characteristics of bad fruits. For the other stages, they are respectively described in the embodiments below.

[0036] In an exemplary embodiment, before executing the process of extracting features from the image data obtained by capturing an image of the target object in step S202, further, initial image data obtained by the image capture device capturing an image of the target object located in the image capture area at a preset frame rate can be obtained; the image clarity of the initial image corresponding to the initial image data is determined, and a first clear image having an image clarity greater than a first preset clarity is determined from the initial image; and the image data is determined based on the first clear image.

[0037] The image clarity can be quantified, for example, by the contrast of the image, but is not limited thereto.

[0038] In an exemplary embodiment, further, for other images in the initial image except the first clear image, image sharpening processing is performed on the other images to obtain a second clear image; when it is determined that the image clarity of the second clear image is equal to the first preset clarity, the image data is determined based on the first clear image and the second clear image.

[0039] Alternatively, in the case where it is determined that the image clarity of the second clear image is less than the first preset clarity, if it is determined that the image clarity of the second clear image is greater than the second preset clarity, the image data is determined based on the first clear image and the second clear image, wherein the second preset clarity is less than the first preset clarity.

[0040] The image sharpening process performed on the other images can be understood as improving the image quality through various technical means, making the image clearer and more detailed. This process is widely used in photography, video surveillance, medical imaging, satellite remote sensing and other fields. The image sharpening process technologies and methods are as follows:

[0041] 1. Denoising: Image denoising is the process of improving image quality by reducing random noise in the image. Common denoising algorithms include Gaussian filtering, median filtering, and bilateral filtering.

[0042] 2. Sharpening: This method enhances the edges and details of an image to make it appear clearer. Common sharpening methods include the Laplacian operator and Unsharp Masking (USM).

[0043] 3. Contrast enhancement: By adjusting the contrast of the image, the light and dark areas of the image are more distinct, thereby improving image clarity. Common contrast enhancement methods include histogram equalization and contrast-limited adaptive histogram equalization.

[0044] 4. Super-resolution: This algorithm generates higher-resolution images to enhance the clarity of image details. Super-resolution techniques include interpolation-based methods and learning-based methods.

[0045] 5. Image deblurring: For blurred images, restore image clarity through the inverse process. Common deblurring techniques include Wiener filtering and blind deconvolution.

[0046] 6. Image fusion: This technology combines useful information from multiple images to produce a clearer, more informative image. This technology has wide applications in fields such as multi-view stereo vision and multispectral imaging.

[0047] 7. Image Segmentation: By segmenting an image into different regions or objects, specific elements in the image can be identified and processed more clearly.

[0048] 8. Image registration: Align multiple images to the same coordinate system for comparison or fusion processing.

[0049] 9. Optical correction: Algorithms are used to correct aberrations in optical imaging systems to improve image clarity.

[0050] 10. Machine Learning and Deep Learning: In recent years, methods based on machine learning and deep learning have achieved remarkable results in image sharpening, such as using convolutional neural networks for image denoising and super-resolution.

[0051] Image sharpening can significantly improve the visual effect and application value of images and improve the accuracy of subsequent object recognition models.

[0052] The above embodiment corresponds to the image acquisition and preprocessing stage. Specifically, the image acquisition process includes: using a high-definition camera to capture the fruit and vegetable images transmitted on the production line (corresponding to the initial images described above) in real time, and adjusting the camera position and angle in real time to ensure that multiple sides or the entire image of the fruit and vegetable are captured.

[0053] Image preprocessing specifically includes: denoising, contrast enhancement, color correction, and other preprocessing of the collected images to improve image quality. At the same time, the image size is standardized to facilitate subsequent model processing.

[0054] In an exemplary embodiment, the following steps are used to extract features from the image data obtained by capturing the target object in step S202 to obtain a current image feature vector corresponding to the image data. The specific steps include: grouping the image data according to different acquisition angles to obtain multiple groups of data, wherein each group of data corresponds to an acquisition angle; analyzing each group of data in turn using the hierarchical analysis method to obtain target object features indicated by each group of data; extracting features from each group of data based on the target object features to obtain a first image feature vector for each group of data; and determining the current image feature vector based on multiple first feature vectors.

[0055] The target object features mentioned above may be understood as features that can be collected visually, including color features, shape features, texture features, size features, etc., but are not limited thereto.

[0056] Among them, the above-mentioned collection angles include but are not limited to the top, bottom, front, etc. of fruits and vegetables, and can also be set to collect from 360°.

[0057] In an exemplary embodiment, other technical solutions are further proposed for performing feature extraction on image data obtained by image capture of a target object to obtain a current image feature vector corresponding to the image data, specifically including: classifying the image data according to different capture angles to obtain multiple first data sets, wherein each first data set in the first data sets corresponds to an capture angle; performing feature extraction on the data of each first data set to obtain a second image feature vector; grouping the image data according to different capture positions to obtain multiple second data sets, wherein each first data set in the first data set corresponds to an capture position; performing feature extraction on the data of each second data set to obtain a third image feature vector; and determining the current image feature vector based on the multiple second image feature vectors and the multiple third image feature vectors.

[0058] The above embodiments correspond to the intelligent analysis and classification stage. Specifically, a deep learning model is used to extract features from the pre-processed image. These features include appearance features such as color, shape, texture, and size. In this way, the types of bad fruits that can be identified can be continuously expanded through continuous training and optimization of the model. The introduction of a deep learning model can achieve accurate identification of bad fruits, reduce the probability of incorrect screening, realize automatic operation of the assembly line, improve recognition accuracy and efficiency, and also improve the quality and market competitiveness of fruits and vegetables. At the same time, by adopting a standardized processing method with unified detection standards and algorithms, the subjectivity and inconsistency in manual detection are avoided, the accuracy and fairness of fruit and vegetable grading and elimination are ensured, and the screening needs of fruits and vegetables under different varieties and different growth conditions can be adapted. For the automated elimination of bad fruits, the manual participation links are greatly reduced, the labor intensity is reduced, and human resources are saved. At the same time, errors and losses caused by human factors are also reduced.

[0059] Optionally, in one embodiment, in addition to using high-resolution industrial camera lenses and customized light source systems to capture 360° images to obtain the appearance characteristics of fruits and vegetables, thereby achieving accurate detection of appearance defects such as scratches and rot, weight sensors and volume measurement equipment can also be used to collect weight and volume characteristics of fruits and vegetables, thereby identifying whether there is hollow rot inside the fruits and vegetables, and then filtering out bad fruits. Deep learning models and hardware equipment can be used to complete the assembly line operation of bad fruit screening. By comprehensively considering the appearance, weight, and volume characteristics of fruits and vegetables, bad fruits can be accurately identified, and then the operation of automatically screening out bad fruits and retaining good fruits can be performed by using robotic arms and conveyor belts, which greatly improves the bad fruit recognition rate and effectively solves the problem of manual screening being time-consuming, labor-intensive, and low in accuracy.

[0060] Furthermore, the process of inputting the current image feature vector into the object recognition model in the above-mentioned step S204 to obtain the recognition result of the current image feature vector by the object recognition model corresponds to the intelligent analysis and classification stage, including using the deep learning model to classify and recognize the extracted features to determine whether each fruit or vegetable is good or bad.

[0061] Among them, the deep learning model can identify the subtle differences between good and bad fruits by learning a large amount of labeled historical image data (corresponding to the above historical images).

[0062] The camera collects image information of fruits and vegetables on the conveyor belt in real time and inputs this information into the deep learning model as the image to be identified (the image corresponding to the above-mentioned current image feature vector). The deep learning model analyzes the collected fruit and vegetable photos based on big data and image recognition algorithms to identify characteristics such as color, size, and shape of the fruits and vegetables. As the model parameters of the deep learning model are continuously optimized, the image recognition accuracy of the deep learning model can be greatly improved. In this way, through machine vision and deep learning algorithms, real-time and continuous classification and detection can be performed on a large number of fruits and vegetables, and bad fruits can be automatically identified and removed quickly and efficiently, greatly improving work efficiency, reducing the time and cost of manual operations, significantly improving the efficiency of fruit and vegetable screening, and improving the efficiency of subsequent fruit and vegetable processing.

[0063] In an exemplary embodiment, the implementation process of generating a target execution instruction based on the object type in step S206 and sending the target execution instruction to the execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction specifically includes: when it is determined that the object type is a damaged object, generating a first target execution instruction for performing a culling operation; sending the first target execution instruction to the execution mechanism to control the execution mechanism to perform a culling operation on the target object according to the first target execution instruction; when it is determined that the object type is an intact object, generating a second target execution instruction for performing a packing operation; sending the second target execution instruction to the execution mechanism to control the execution mechanism to perform a packing operation on the target object according to the second target execution instruction.

[0064] This embodiment corresponds to the rejection operation stage. Based on the feature data extracted by the image recognition algorithm, image recognition is performed using a trained deep learning model. Fruits and vegetables can be classified to determine whether they meet quality standards. The recognition result is then output to determine whether the fruits and vegetables are defective. If the recognition result is bad fruit (corresponding to the object type of the damaged object mentioned above), then after the execution mechanism receives the target execution instruction, the execution mechanism performs a rejection operation on the identified bad fruit according to the target execution instruction. If the recognition result is good fruit (corresponding to the object type of the intact object mentioned above), then after the execution mechanism receives the target execution instruction, the execution mechanism performs a boxing operation on the identified bad fruit according to the target execution instruction. At the same time, information such as the number, type, and time of rejection of bad fruit each time is recorded. This can be used for subsequent data analysis and quality control. This embodiment packs good fruit and rejects bad fruit based on the recognition results of the fruit and vegetables by the model, thereby improving the accuracy of the rejection operation.

[0065] In an exemplary embodiment, after generating a target execution instruction based on the object type and sending the target execution instruction to the execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction, further, a first recognition number of multiple recognitions of different current image feature vectors using the object recognition model can be obtained; a second recognition number that the object type is a damaged object is determined from the first recognition number, and a first probability of the object being damaged is determined based on the second recognition number and the first recognition number; a first number of times the execution mechanism operates on the target object according to the target execution instruction is determined, and a second probability is determined based on the first number and the first recognition number; when it is determined that the difference between the first probability and the second probability is greater than a preset difference, an alarm message is generated, and the alarm message is sent to the first object, so that the first object optimizes the object recognition model based on the alarm information.

[0066] Optionally, if the difference between the first probability and the second probability is greater than a preset difference, it can be determined that the recognition result of the target object is inaccurate and needs further review, so an alarm message is generated.

[0067] Optionally, when it is determined that the difference between the first probability and the second probability is less than a preset difference, it is determined that the elimination is accurate, and no further review is required, so no alarm information is generated.

[0068] This embodiment corresponds to the system optimization phase. By regularly analyzing the culling data recorded by the system, the model parameters are adjusted or the model structure is optimized based on the analysis results to improve the culling efficiency and accuracy. At the same time, the stability and reliability of the system are ensured by regularly inspecting and maintaining the hardware equipment.

[0069] Optionally, during the system optimization phase, continuous monitoring and looping can be implemented: the image acquisition module continues to capture new images of fruits and vegetables, and then repeats the bad fruit removal process. By continuously collecting new fruit and vegetable image data, the deep learning model is retrained to adapt to the fruit and vegetable screening needs of different varieties and growing conditions. Continuous model optimization improves screening efficiency and accuracy, ensuring the sustainable development of the fruit and vegetable industry. While accurately removing bad fruit, it also reduces resource waste and environmental pollution. Furthermore, by improving the quality and standardization of fruits and vegetables, it helps enhance the competitiveness and profitability of the fruit and vegetable industry.

[0070] Optionally, as technology continues to develop, this application can also expand and optimize functions by upgrading hardware and software.

[0071] In order to better understand the process of the above-mentioned image data processing method, the implementation method flow of the above-mentioned image data processing is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of this application.

[0072] In this embodiment, a method for processing image data is provided. Figure 3 is a flow chart of a method for processing image data according to an embodiment of the present application, such as Figure 3 As shown, the specific steps are as follows:

[0073] Step S301: The pipeline is started.

[0074] Step S302: The fruit is put on the production line.

[0075] Step S303: Image acquisition and uploading: 360-degree images of fruits and vegetables are acquired in the image acquisition area.

[0076] Step S304: Matching intelligent learning algorithm rules. Using a deep learning model, identify whether the fruits and vegetables are bad.

[0077] Step S305: Identify whether the fruit is good. If so, proceed to step S307; if not, proceed to step S306.

[0078] Step S306: Control the robotic arm to put the rotten fruits into a waste box for disposal.

[0079] Step S307: Control the robotic arm to pack the good fruits into boxes for the next process.

[0080] Corresponding to the above steps, the present application further proposes a system for intelligently removing bad fruit, comprising:

[0081] Power supply module: Used to provide stable power supply for the entire system. It includes components such as transformers, rectifiers, filters, etc. to ensure the stability and reliability of the power supply.

[0082] Image acquisition device: This includes a high-definition camera and signal interface circuitry. This camera captures images of fruits and vegetables flowing through the production line in real time at a preset frame rate and transmits them to the main control board in the image processing and analysis module for processing.

[0083] Among them, the high-definition camera can be equipped with a specific light source and lighting system to ensure the quality and clarity of the image. The high-definition camera is connected to the main control board through a dedicated interface (such as USB, HDMI, etc.) to transmit image data.

[0084] Image acquisition and processing module: This module includes image sensors, analog-to-digital converters, digital signal processors, and programmable logic components, and is used to preprocess image data and extract features. Specifically, the main control board performs preprocessing on the captured images, such as denoising and contrast enhancement, to improve the accuracy of subsequent analysis. Deep learning models (such as convolutional neural networks) are used to extract features and perform classification and recognition on the preprocessed images. These deep learning models are trained using large amounts of labeled image data and are capable of identifying characteristics such as color, shape, size, and texture of fruits and vegetables, and determining whether they are rotten or not.

[0085] The main control board (MCU) is the core processing unit, typically based on a high-performance processor or microcontroller. After receiving image data, the MCU runs an AI-powered image processing algorithm to pre-process the image, extract features, and classify it, identifying bad fruit. This process can take from a few seconds to tens of seconds, depending on the algorithm's complexity and computing resources.

[0086] The main control board also connects to other modules, such as the actuators and the human-machine interface, for communication and control. Based on the identification results, the main control board also generates instructions for rejecting bad fruit. These instructions include the location, quantity, and rejection method of the bad fruit.

[0087] After the main control board receives the feedback signal from the actuator and confirms that the rejection operation has been completed, it records the rejected bad fruit information (such as quantity, location, etc.) and updates the inventory data.

[0088] The actuators, which include motor drive circuits, relay control circuits, and electromagnet control, enable precise operation of the robotic arms and air jets. The main control board generates instructions based on the image processing and analysis module's judgments, controlling the actuators' actions. For example, they can control the robotic arms and air jets to remove bad fruit, or control the robotic arms to pack good fruit. After the removal operation is complete, the actuators send a feedback signal to the main control board. The actuators can precisely locate and remove bad fruit while minimizing the risk of accidental damage to good fruit.

[0089] Among them, the rejection operation can be clamping by a robotic arm, blowing out by a jet device, etc., depending on the system design and application scenario.

[0090] Human-machine interface (HMI): This includes input devices such as a display screen, touch screen, and buttons, as well as corresponding interface circuits and driver circuits. It provides an interface for system operation, monitoring, and management, facilitating operator interaction with the system. It also offers options for parameter settings and function selection. The HMI displays real-time information such as system operating status, operation prompts, rejection results, and rejection quantity. It also provides corresponding alarm and notification functions, ensuring that operators are kept informed of the system's operating status and rejection effectiveness.

[0091] Communication module: used for communication and data transmission with external devices or host computers. It includes Ethernet interface, Wi-Fi module, Bluetooth module, etc. to meet different communication needs.

[0092] Auxiliary circuits: including reset circuit, clock circuit, power monitoring circuit, etc., to ensure the stability of the system.

[0093] In summary, the key points of the intelligent bad fruit removal system implemented in this application are its advantages in visual recognition technology based on deep learning models, real-time detection and removal, grid conveyor belt design, and high accuracy and removal efficiency. These key points together constitute the core competitiveness of the system, enabling it to exert significant advantages and effects in practical applications. It can be seen that the intelligent bad fruit removal system has multiple advantages such as high efficiency, accuracy, standardization, manpower saving and sustainability, which makes the system widely used in the fruit and vegetable industry and provides strong support for the development and upgrading of the fruit and vegetable industry. The intelligent bad fruit removal system realizes the automatic identification and removal of bad fruits in the fruit and vegetable processing process by integrating technologies such as image acquisition, image processing and analysis, control execution and human-computer interaction, providing an effective quality control means for the fruit and vegetable processing industry.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0095] Figure 4 is a structural block diagram of an image data processing device according to an embodiment of the present application; Figure 4 Shown, including:

[0096] A first obtaining module 42 is configured to perform feature extraction on image data obtained by image acquisition of the target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions;

[0097] a second obtaining module 44, configured to input the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model on the current image feature vector, wherein the recognition result is used to indicate an object type of the target object;

[0098] The sending module 46 is configured to generate a target execution instruction based on the object type, and send the target execution instruction to the execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction.

[0099] Through the above-mentioned device, feature extraction is performed on image data obtained by image acquisition of the target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions; the current image feature vector is input into an object recognition model to obtain a recognition result of the object recognition model for the current image feature vector, wherein the recognition result is used to indicate the object type of the target object; a target execution instruction is generated based on the object type, and the target execution instruction is sent to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction. The present application can intelligently screen and filter fruits and vegetables that do not meet the standards, realize a fully automated and efficient fruit and vegetable screening process, solve the technical problem in the related art that, when screening bad fruits, the image recognition results of bad fruits are inaccurate, resulting in low bad fruit screening efficiency, improve the accuracy of the image recognition results of bad fruits, and thus improve the bad fruit screening efficiency.

[0100] In an exemplary embodiment, the image data processing device further includes an image data determination module, which is used to further obtain initial image data obtained by the image acquisition device performing image acquisition on the target object located in the image acquisition area at a preset frame rate before executing the process of performing feature extraction on the image data obtained by performing image acquisition on the target object; determine the image clarity of the initial image corresponding to the initial image data, and determine a first clear image from the initial image whose image clarity is greater than a first preset clarity; and determine the image data based on the first clear image.

[0101] In an exemplary embodiment, further, the image data determination module is also used to perform image sharpening processing on other images in the initial image except the first clear image to obtain a second clear image; when it is determined that the image clarity of the second clear image is equal to the first preset clarity, the image data is determined based on the first clear image and the second clear image.

[0102] In an exemplary embodiment, the above-mentioned first obtaining module is also used to: group the image data according to different acquisition angles to obtain multiple groups of data, wherein each group of data in the multiple groups of data corresponds to an acquisition angle; for each group of data, use the hierarchical analysis method to analyze each group of data in turn to obtain the target object features indicated by each group of data; perform feature extraction on each group of data based on the target object features to obtain the first image feature vector of each group of data; determine the current image feature vector based on multiple first feature vectors.

[0103] In an exemplary embodiment, the above-mentioned first obtaining module is also used to: classify the image data according to different acquisition angles to obtain multiple first data sets, wherein each first data set in the first data sets corresponds to an acquisition angle; perform feature extraction on the data of each first data set to obtain a second image feature vector; group the image data according to different acquisition positions to obtain multiple second data sets, wherein each first data set in the first data set corresponds to an acquisition position; perform feature extraction on the data of each second data set to obtain a third image feature vector; and determine the current image feature vector based on multiple second image feature vectors and multiple third image feature vectors.

[0104] In an exemplary embodiment, the above-mentioned sending module is also used to: when it is determined that the object type is a damaged object, generate a first target execution instruction for performing a culling operation; send the first target execution instruction to the execution mechanism to control the execution mechanism to perform a culling operation on the target object according to the first target execution instruction; when it is determined that the object type is an intact object, generate a second target execution instruction for performing a packing operation; send the second target execution instruction to the execution mechanism to control the execution mechanism to perform a packing operation on the target object according to the second target execution instruction.

[0105] In an exemplary embodiment, the image data processing device further includes an optimization module for generating a target execution instruction based on the object type and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction, and then further obtaining a first recognition number of multiple recognitions of different current image feature vectors using the object recognition model; determining a second recognition number that the object type is a damaged object from the first recognition number, and determining a first probability of the object being damaged based on the second recognition number and the first recognition number; determining a first number of times that the execution mechanism operates on the target object according to the target execution instruction, and determining a second probability based on the first number and the first recognition number; and generating an alarm message when it is determined that the difference between the first probability and the second probability is greater than a preset difference, and sending the alarm message to the first object, so that the first object optimizes the object recognition model based on the alarm information.

[0106] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0107] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0108] S1, performing feature extraction on image data obtained by performing image acquisition on a target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions;

[0109] S2, inputting the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model on the current image feature vector, wherein the recognition result is used to indicate the object type of the target object;

[0110] S3: Generate a target execution instruction based on the object type, and send the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction.

[0111] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0112] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0113] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0114] S1, performing feature extraction on image data obtained by performing image acquisition on a target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions;

[0115] S2, inputting the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model on the current image feature vector, wherein the recognition result is used to indicate the object type of the target object;

[0116] S3: Generate a target execution instruction based on the object type, and send the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction.

[0117] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0118] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0119] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0120] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing image data, characterized in that: include: Performing feature extraction on image data obtained by performing image acquisition on the target object to obtain a current image feature vector corresponding to the image data, wherein the current image feature vector has multiple dimensions; inputting the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model on the current image feature vector, wherein the recognition result is used to indicate the object type of the target object; A target execution instruction is generated based on the object type, and the target execution instruction is sent to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction.

2. The method for processing image data according to claim 1, characterized in that: Before extracting features from the image data obtained by collecting images of the target object, the method further includes: Acquiring initial image data obtained by an image acquisition device acquiring images of a target object located in an image acquisition area at a preset frame rate; Determining the image definition of an initial image corresponding to the initial image data, and determining a first clear image whose image definition is greater than a first preset definition from the initial image; The image data is determined according to the first clear image.

3. The method for processing image data according to claim 2, characterized in that: The method further includes: for other images in the initial image except the first clear image, performing image clearing processing on the other images to obtain a second clear image; In a case where it is determined that the image definition of the second clear image is equal to the first preset definition, the image data is determined according to the first clear image and the second clear image.

4. The method for processing image data according to claim 1, characterized in that: Performing feature extraction on the image data obtained by performing image acquisition on the target object to obtain a current image feature vector corresponding to the image data includes: Grouping the image data according to different acquisition angles to obtain multiple groups of data, wherein each group of data in the multiple groups of data corresponds to one acquisition angle; For each set of data, each set of data is analyzed in turn using the hierarchical analysis method to obtain the target object characteristics indicated by each set of data; Performing feature extraction on each set of data according to the target object feature to obtain a first image feature vector of each set of data; The current image feature vector is determined according to a plurality of first feature vectors.

5. The method for processing image data according to claim 1, characterized in that: Performing feature extraction on the image data obtained by performing image acquisition on the target object to obtain a current image feature vector corresponding to the image data includes: Classifying the image data according to different acquisition angles to obtain a plurality of first data sets, wherein each of the first data sets corresponds to one acquisition angle; Extracting features from the data of each of the first data sets to obtain a second image feature vector; grouping the image data according to different acquisition positions to obtain a plurality of second data sets, wherein each of the first data sets in the first data sets corresponds to one acquisition position; Perform feature extraction on the data of each second data set to obtain a third image feature vector; and determine the current image feature vector based on the multiple second image feature vectors and the multiple third image feature vectors.

6. The method for processing image data according to claim 1, characterized in that: Generating a target execution instruction based on the object type, and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction, including: In the case where it is determined that the object type is a damaged object, generating a first target execution instruction for performing a culling operation; Sending the first target execution instruction to an execution mechanism to control the execution mechanism to perform a culling operation on the target object according to the first target execution instruction; In the case where it is determined that the object type is an intact object, generating a second target execution instruction for performing a boxing operation; The second target execution instruction is sent to the execution mechanism to control the execution mechanism to perform a boxing operation on the target object according to the second target execution instruction.

7. The method for processing image data according to claim 1, characterized in that: After generating a target execution instruction based on the object type and sending the target execution instruction to an execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction, the method further includes: obtaining a first recognition number of multiple recognitions of different current image feature vectors using the object recognition model; Determine a second recognition number in which the object type is a damaged object from the first recognition number, and determine a first probability that the object is damaged based on the second recognition number and the first recognition number; Determining a first number of times that the execution mechanism operates the target object according to the target execution instruction, and determining a second probability based on the first number of times and the first recognition number of times; When it is determined that the difference between the first probability and the second probability is greater than a preset difference, an alarm message is generated and the alarm message is sent to the first object, so that the first object optimizes the object recognition model based on the alarm message.

8. An image data processing device, characterized in that: include: A first obtaining module is used to extract features from image data obtained by collecting images of the target object, and obtain a current image feature vector corresponding to the image data; a second obtaining module, configured to input the current image feature vector into an object recognition model to obtain a recognition result of the object recognition model on the current image feature vector, wherein the recognition result is used to indicate an object type of the target object; The sending module is used to generate a target execution instruction based on the object type, and send the target execution instruction to the execution mechanism to control the execution mechanism to operate the target object according to the target execution instruction.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images