Machine vision-based intelligent sorting method and system for bird's nest raw materials

Abstract

This invention discloses a machine vision-based intelligent sorting method and system for bird's nest raw materials, relating to the field of food processing technology. The invention involves conveying the bird's nest raw materials to be sorted on a conveying device, setting several sensing positioning points on the device. At each sensing positioning point, multi-device image acquisition of the bird's nest raw materials is triggered. After obtaining the corresponding bird's nest calibration image set, image processing operations are performed to obtain the corresponding modal image feature set. Feature dimension vectors characterizing bird's nest quality are then extracted, constructing a bird's nest quality knowledge graph. A mapping dictionary between the feature dimension vectors and the bird's nest quality knowledge graph is established. Feature attribute fields recorded in the mapping dictionary are selected from the feature dimension vectors, and the feature attribute fields are summarized to generate a quality dataset of bird's nest raw materials for grade classification. Sorting control instructions are generated for each batch of bird's nest raw materials, and intelligent sorting is performed on bird's nest raw materials of different quality grades based on the corresponding sorting control instructions.

Description

Technical Field

[0001] This invention relates to the field of food processing technology, specifically to a machine vision-based intelligent sorting method and system for bird's nest raw materials. Background Technology

[0002] The quality grade of bird's nest determines its product value. Precise sorting of raw bird's nest is key to ensuring consistent quality and improving production efficiency. Currently, the sorting of raw bird's nest mainly relies on manual labor, with operators judging the quality characteristics of bird's nest such as appearance, color, and impurities based on experience. This has the following obvious drawbacks: First, manual sorting is highly subjective, and differences in the experience of operators lead to inconsistent judgment standards, which can easily result in misjudgment of grade and sorting deviation, affecting product quality and reducing sorting accuracy. Secondly, the detection methods of some existing single image acquisition devices have limited information collection, making it impossible to fully capture the multi-dimensional quality characteristics of bird's nest, and difficult to generate standardized quality datasets and subsequent automated intelligent sorting. Third, existing sorting technologies have not established a complete quality knowledge system, and the mapping relationship between feature dimensions and quality grades lacks unified standards, resulting in a disconnect between feature extraction and grade classification, making it difficult to meet actual production needs in terms of sorting efficiency and accuracy.

[0003] Therefore, given the shortcomings of existing bird's nest sorting methods, such as high subjectivity, low efficiency, poor accuracy, and lack of systematic data collection, mapping, and intelligent sorting mechanisms, there is an urgent need for an automated and intelligent sorting method for bird's nest raw materials. Summary of the Invention

[0004] The purpose of this invention is to provide a machine vision-based intelligent sorting method and system for bird's nest raw materials to address the shortcomings in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a machine vision-based intelligent sorting method for bird's nest raw materials, comprising the following steps: Step S1: The bird's nest raw materials to be sorted are transported on the conveyor device, and several sensor positioning points are set on the conveyor device. At the location of each sensor positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration map. Step S2: Perform image processing operations based on the bird's nest calibration image set to obtain the corresponding modal image feature set, and extract the feature dimension vector representing the quality of bird's nest based on the modal image feature set; Step S3: Construct a knowledge graph of bird's nest quality, establish a mapping dictionary between feature dimension vectors and bird's nest quality knowledge graph, filter feature attribute fields recorded in the mapping dictionary for feature dimension vectors, and summarize all feature attribute fields to generate a quality dataset of corresponding bird's nest raw materials. Step S4: Based on the quality dataset, classify the corresponding bird's nest raw materials by grade, generate sorting control instructions for each batch of bird's nest raw materials, and perform intelligent sorting for bird's nest raw materials of different quality grades based on the corresponding sorting control instructions.

[0006] In a preferred embodiment, the raw bird's nest to be sorted is conveyed on a conveyor device, and several sensing positioning points are set on the conveyor device. At the location of each sensing positioning point, a multi-device image acquisition of the raw bird's nest is triggered to obtain the corresponding bird's nest calibration image set. The process includes: Identify the bird's nest raw materials to be sorted; Several sensing and positioning points are set on the conveying device. These points are used to deploy various types of sensor devices to obtain three-dimensional modeling information, infrared sensing information, and internal quality information of the bird's nest raw materials. The information is stored in the form of image files and associated with their respective file authentication identifiers. Command execution devices are deployed at each sensor positioning point. The set acquisition commands are entered into the command execution devices at each sensor positioning point. The command execution devices select and enable the acquisition commands to perform work on the sensor devices, obtain image files of bird's nest raw materials, integrate all image files at each sensor positioning point, and obtain the bird's nest calibration map set for the corresponding sensor positioning point.

[0007] In a preferred embodiment, image processing is performed based on a bird's nest calibration atlas to obtain a corresponding modal image feature set. The process of extracting feature dimension vectors characterizing bird's nest quality based on the modal image feature set includes: A first processing layer, a second processing layer, and a third processing layer are set up to perform image processing operations on the bird's nest calibration image set, thereby obtaining the corresponding first modal features, second modal features, and third modal features; The first modal feature, the second modal feature, and the third modal feature each correspond to their respective image regions. By summing up their respective image regions, a modal image feature set corresponding to the bird's nest calibration image set at each sensing positioning point is constructed. The modal image feature set includes a number of feature pixel blocks. Each feature pixel block is normalized and converted into a grayscale pixel block. A pre-trained convolutional neural network is used as the feature extraction model to obtain one-dimensional vectors of grayscale pixel blocks at different dimensional levels. Feature aggregation, pooling, and dimensionality reduction are performed on all one-dimensional vectors to obtain feature dimension vectors used to characterize the quality of bird's nest.

[0008] In a preferred embodiment, the process of constructing a knowledge graph of bird's nest quality includes: Construct a subclass knowledge graph for each quality level, construct a blank graph container, arrange the subclass knowledge graphs in different positions of the blank graph container, establish a corresponding index sequence based on the subclass knowledge graph at each position, and create a corresponding interaction layer for each subclass knowledge graph. Using the index sequence as a retrieval credential, a corresponding mapping path is constructed between all interaction layers belonging to the same retrieval credential. A control domain and an identity domain are created in each interaction layer. When each subclass knowledge graph has completed the processing of the mapping path, control domain, and identity domain, the subclass knowledge graphs are associated at adjacent positions in the blank graph container. After the association is completed, a bird's nest quality knowledge graph representing the corresponding quality grade of bird's nest is constructed.

[0009] In a preferred embodiment, the process of establishing a mapping dictionary between feature dimension vectors and the bird's nest quality knowledge graph includes: For each subclass knowledge graph of a mapping path, a corresponding graph term set is created. The graph term set includes several feature words for the corresponding feature dimension vector. All feature words corresponding to the graph term set are taken as a dictionary item. The corresponding first-level dictionary index is set based on the dictionary item. All dictionary items are integrated to generate a mapping dictionary. A second-level dictionary index is set for the mapping dictionary. The index path of each feature dimension vector in the mapping dictionary is determined based on the first-level dictionary index and the second-level dictionary index. The index path is used to determine the position of each feature dimension vector in the mapping dictionary.

[0010] In a preferred embodiment, the process of filtering the feature attribute fields recorded in the mapping dictionary for the feature dimension vector and summarizing all feature attribute fields to generate the corresponding bird's nest raw material quality dataset includes: Determine all feature dimension vectors for each bird's nest raw material, obtain the index path for each feature dimension vector, determine all feature words in the mapping dictionary based on their respective index paths, and process each feature word into the corresponding feature attribute field based on the preset word segmentation symbols. Set up a blank data file, set up a number of fill queues on the blank data file, each fill queue consists of a number of fill cells, set up migration points and operation points, the migration points migrate feature attribute fields to the fill queues and perform filling at the fill cells in the appropriate positions in the fill queues, and the operation points determine the appropriate positions of each feature attribute field. Arrange each feature attribute field in the appropriate position to construct several quality description statements. Set connection qualifiers to connect different quality description statements to generate the corresponding quality dataset of bird's nest raw materials.

[0011] In a preferred embodiment, the process of classifying the corresponding bird's nest raw materials according to the quality dataset, generating sorting control instructions for each batch of bird's nest raw materials, and performing intelligent sorting of bird's nest raw materials of different quality grades based on the corresponding sorting control instructions includes: Historical datasets are obtained to construct a quality grade determination model for bird's nest raw materials. The quality dataset is input into the quality grade determination model, which classifies the quality grades based on the feature attribute fields, quality description statements, and preset grade division rules in the quality dataset, thus classifying bird's nest raw materials into different quality grades. Based on the quality grade of each batch of bird's nest raw materials, a preset grade-sorting strategy mapping table is invoked to generate sorting control instructions for different quality grades. The sorting execution mechanism is deployed and debugged, and the sorting execution mechanism performs intelligent sorting of bird's nest raw materials of different quality grades based on the sorting control instructions. The quality grade, sorting time, sorting location, and flow information of each batch of bird's nest raw materials are recorded simultaneously as sorting records for each batch of bird's nest raw materials.

[0012] In a preferred embodiment, the process by which the sorting actuator performs intelligent sorting of bird's nest raw materials of different quality grades based on sorting control commands includes: The sorting execution mechanism deploys and runs the sorting model, which consists of several categories of sub-sorting models. Each sub-sorting model is responsible for sorting bird's nest raw materials of a certain quality grade. The sorting models are stored in the model library. When the bird's nest raw material reaches the working range of the sorting execution mechanism on the conveying device, the sub-sorting model that meets the corresponding quality grade is taken out from the model library. The sub-sorting model loads the corresponding sorting execution mechanism to package the bird's nest raw material of the current quality grade and transport it to the storage warehouse along the conveying direction of the conveying device. When the sorting execution mechanism performs sorting based on the obtained sub-sorting model, it records the working parameters of the sub-sorting model, iterates on the corresponding sub-sorting model, optimizes the corresponding sub-sorting model, and adapts it to the sorting process of the sorting execution mechanism for sorting bird's nest raw materials. The sub-sorting model is then stored in the model library.

[0013] This invention also provides a machine vision-based intelligent sorting system for bird's nest raw materials, the system comprising: The image acquisition module is used to transport the bird's nest raw materials to be sorted on the conveyor device, and set several sensing positioning points on the conveyor device. At the location of each sensing positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration image set. The image processing module is used to perform image processing operations based on the bird's nest calibration image set, obtain the corresponding modal image feature set, and extract the feature dimension vector representing the quality of bird's nest based on the modal image feature set. The knowledge graph processing module is used to construct a knowledge graph of bird's nest quality, establish a mapping dictionary between feature dimension vectors and bird's nest quality knowledge graph, filter feature attribute fields recorded in the mapping dictionary for feature dimension vectors, and summarize all feature attribute fields to generate a quality dataset of corresponding bird's nest raw materials. The bird's nest intelligent sorting module is used to classify the corresponding bird's nest raw materials according to the quality dataset, generate sorting control instructions for each batch of bird's nest raw materials, and perform intelligent sorting of bird's nest raw materials of different quality grades according to the corresponding sorting control instructions.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention sets sensor positioning points on the transmission device to trigger multi-device acquisition of bird's nest calibration image sets. Compared with single-device acquisition, it can comprehensively capture multi-dimensional visual information of bird's nest, accurately locate and collect data to avoid feature loss, and provide reliable support for subsequent feature extraction. The calibration image sets are processed and feature dimension vectors are extracted to realize the quantification of quality features, construct a quality knowledge graph and mapping dictionary, associate feature dimension vectors with standardized feature attribute fields, and generate a standardized quality dataset, which solves the problems of chaotic quality judgment standards and strong subjectivity.

[0015] 2. This invention forms a complete sorting closed loop by classifying grades based on a quality dataset and generating sorting control instructions, replacing manual operation, avoiding sorting deviations, and achieving large-scale continuous sorting. Through a quality knowledge graph and mapping dictionary, the feature mapping relationship can be flexibly adjusted to adapt to the sorting needs of bird's nests of different specifications and qualities. In addition, the quality dataset can achieve data retention and traceability, providing data support for the iteration of subsequent sorting rules. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1 This is a flowchart of the intelligent sorting method for bird's nest raw materials based on machine vision according to the present invention.

[0018] Figure 2 This is a schematic diagram of the intelligent sorting process of the intelligent sorting method for bird's nest raw materials based on machine vision according to the present invention.

[0019] Figure 3 This is a system block diagram of the intelligent sorting system for bird's nest raw materials based on machine vision according to the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1, please refer to Figure 1 As shown in this embodiment, a machine vision-based intelligent sorting method for bird's nest raw materials includes the following steps: Step S1: The bird's nest raw materials to be sorted are transported on the conveyor device, and several sensor positioning points are set on the conveyor device. At the location of each sensor positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration map. Step S2: Perform image processing operations based on the bird's nest calibration image set to obtain the corresponding modal image feature set, and extract the feature dimension vector representing the quality of bird's nest based on the modal image feature set; Step S3: Construct a knowledge graph of bird's nest quality, establish a mapping dictionary between feature dimension vectors and bird's nest quality knowledge graph, filter feature attribute fields recorded in the mapping dictionary for feature dimension vectors, and summarize all feature attribute fields to generate a quality dataset of corresponding bird's nest raw materials. Step S4: Based on the quality dataset, classify the corresponding bird's nest raw materials by grade, generate sorting control instructions for each batch of bird's nest raw materials, and perform intelligent sorting for bird's nest raw materials of different quality grades based on the corresponding sorting control instructions.

[0022] It should be further explained that, in the specific implementation process, the bird's nest raw materials to be sorted are transported on a conveyor device, and several sensor positioning points are set on the conveyor device. At the location of each sensor positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration image set. The process includes: Determine the bird's nest raw materials to be sorted, including their weight, shape, and storage time; The production line used for transporting bird's nest raw materials is debugged, and the corresponding conveying device is deployed and debugged for the production line. The bird's nest raw materials to be sorted on the production line are transported through the conveying device. Several sensing positioning points are set on the transmission device. Each sensing positioning point is used to arrange several types of sensor devices, including 3D structured light sensors, infrared sensors and thermal imaging sensors. The 3D structured light sensor is used to obtain three-dimensional modeling information corresponding to the bird's nest raw materials; The infrared sensor is used to obtain infrared sensing information corresponding to the bird's nest raw materials; The thermal imaging sensor is used to obtain the internal quality information of the bird's nest raw materials; Among them, the 3D modeling information, infrared sensing information and internal quality information are all stored in the form of image files. The 3D modeling information, infrared sensing information and internal quality information are each associated with a corresponding file authentication identifier. The file authentication identifiers of the 3D modeling information, infrared sensing information and internal quality information are respectively denoted as Tag1, Tag2 and Tag3. Set the acquisition command, deploy the corresponding command execution device at each sensing location point, and input the acquisition command into the command execution device at the location of each sensing location point. The command execution device decides whether to enable the acquisition command to perform work on the sensor device at the corresponding location based on its own device working environment. Then, image files of bird's nest raw materials from different types of sensor devices are obtained, and all image files at each sensor positioning point are integrated to obtain the bird's nest calibration map set corresponding to the sensor positioning point.

[0023] It should be further explained that, in the specific implementation process, image processing operations are performed based on the bird's nest calibration image set to obtain the corresponding modal image feature set. The process of extracting feature dimension vectors representing the quality of bird's nest based on the modal image feature set includes: Set up a first processing layer, a second processing layer, and a third processing layer. Perform image processing operations on the bird's nest calibration atlas using these three layers. The specific process is as follows: The first processing layer is used to process the image file with the file authentication identifier Tag1, thereby obtaining the first modal feature corresponding to the three-dimensional modeling information. The first modal feature is a number of first feature pixel blocks in the three-dimensional modeling information that conform to the preset modeling feature set, and the image region composed of these first feature pixel blocks. The second processing layer is used to process the image file with the file authentication identifier Tag2, thereby obtaining the second modal feature corresponding to the infrared sensing information. The second modal feature is a number of second feature pixel blocks in the infrared sensing information that satisfy the preset infrared reference information, and the image area is composed of these second feature pixel blocks. The third processing layer is used to process image files with the file authentication identifier Tag3, thereby obtaining the third modal features corresponding to the internal quality information. The third modal features are several third feature pixel blocks in the internal quality information that meet the preset quality reference standard, and the image region is composed of these third feature pixel blocks.

[0024] The image regions corresponding to the first modal features, the second modal features, and the third modal features are summarized, and then the modal image feature set corresponding to the bird's nest calibration map set at each sensing positioning point is constructed. Normalization is performed on each feature pixel block corresponding to the modal image feature set, mapping each pixel value included in the feature pixel block from [0, 255] to [0, 1], thereby converting the feature pixel block into a grayscale pixel block; Based on a pre-trained convolutional neural network as the feature extraction model, the feature extraction model obtains one-dimensional vectors corresponding to grayscale pixel blocks at different dimensional levels. Feature aggregation, pooling, and dimensionality reduction are performed on all one-dimensional vectors to extract the final feature dimension vectors used to characterize the quality of bird's nest.

[0025] The feature extraction model includes shallow convolutional kernels and deep convolutional kernels of convolutional neural networks. The former is used to extract low-dimensional features corresponding to bird's nest raw materials, such as edge information, color and texture, while the latter is used to extract high-dimensional features corresponding to bird's nest raw materials, such as shape, object parts and overall category.

[0026] It should be further explained that, in the specific implementation process, the process of constructing a knowledge graph of bird's nest quality and establishing a mapping dictionary between feature dimension vectors and the knowledge graph of bird's nest quality includes: The data information corresponding to different quality grades of bird's nest raw materials is determined, and a subclass knowledge graph corresponding to each quality grade is constructed based on the determined data information. A blank graph container is constructed, and the subclass knowledge graphs of several quality grades are arranged in different positions in the blank graph container. At each location, a corresponding index sequence is established based on the graph identifier of the subclass knowledge graph. Each index sequence is used as a retrieval credential for the corresponding subclass knowledge graph in the blank graph container. A corresponding interaction layer is created for each subclass knowledge graph. Construct corresponding mapping paths between all interaction layers belonging to the same retrieval credential, thereby obtaining all mapping paths of interaction layers corresponding to different retrieval credentials. Among them, there are several subclass knowledge graphs under a mapping path. A control domain and an identity domain are created for each interaction layer. The control domain is set with a corresponding control period and control object, and the identity domain is set with a corresponding set of identity interaction objects. The set of identity interaction objects consists of a number of identity targets. Each identity target is used to perform a type of interaction operation on the interaction layer within the control domain. When each subclass knowledge graph has completed the processing of the corresponding mapping path, control domain, and identity domain, the processed subclass knowledge graphs are associated at adjacent positions in the blank graph container. The association method is as follows: establish a layer interaction path between two subclass knowledge graphs. After the association is completed, construct a bird's nest quality knowledge graph that represents the corresponding quality grade of bird's nest.

[0027] For each subclass knowledge graph corresponding to a mapping path, a corresponding graph term set is created. The graph term set includes several feature words for corresponding feature dimension vectors. A matching threshold is set for the feature words. The matching threshold is used to determine whether the graph term set is matched by a certain feature dimension vector, and then to determine the position of the feature dimension vector in the bird's nest quality knowledge graph. Each graph word set is used as an encapsulation unit. All the feature words included in the graph word set are set as a dictionary item. A first-level dictionary index is set based on the dictionary item. The first-level dictionary index is used to retrieve each feature word in the dictionary item. All dictionary items are integrated to generate a mapping dictionary. A second-level dictionary index is set for the mapping dictionary. The second-level dictionary index is used to retrieve each dictionary item in the mapping dictionary. The index path in the mapping dictionary corresponding to each feature dimension vector is determined based on the first-level dictionary index and the second-level dictionary index, where the first-level dictionary index is denoted as... The second-level dictionary index is denoted as The index path is denoted as The index path determines the position of the feature dimension vector in the mapping dictionary, as shown below: = -> ; Here, index1 and index2 are both natural numbers greater than 0. index2 represents the sequential indexing of several second-level dictionary indices, incrementing by 1 from 1. The selected feature dimension vector corresponds to the dictionary entry at the position of the sub-mapping dictionary. index1 represents the sequential indexing of several first-level dictionary indices, incrementing by 1 from 1. The selected feature dimension vector corresponds to the position of the feature word in the dictionary entry. Therefore, the index path... This is used to represent the position of all feature words corresponding to each feature dimension vector in the mapping dictionary. Under the same index path, index2 has only one number, while index1 has several.

[0028] It should be further explained that, in the specific implementation process, the process of selecting the feature attribute fields recorded in the mapping dictionary for the feature dimension vector and summarizing all feature attribute fields to generate the corresponding bird's nest raw material quality dataset includes: Determine all feature dimension vectors corresponding to each bird's nest raw material, and then obtain the index path corresponding to each feature dimension vector. Based on their respective index paths, determine all feature words corresponding to each in the mapping dictionary. Process each feature word based on preset word segmentation symbols, and then process the feature words into corresponding feature attribute fields. Set up a blank data file, and set up a number of fill queues on the blank data file. Each fill queue consists of a number of fill cells. Set up migration points and operation points. The migration points are used to migrate the feature attribute fields from the current position to the fill queue and perform filling at the fill cell of the appropriate position in the fill queue. The operation points are used to determine the appropriate position corresponding to each feature attribute field. Arrange each feature attribute field in its corresponding adaptation position in the filling queue to construct several quality description statements. Set connection qualifiers to connect different quality description statements, and then generate the corresponding quality dataset of bird's nest raw materials.

[0029] It should be noted that the acquisition of feature attribute fields is as follows: For each batch of bird's nest raw materials, the corresponding feature dimension vectors in all dimensions are extracted. Based on the dimension identifier of the feature dimension vector, the corresponding index path is determined. The index path is the positioning path of the feature dimension vector in the mapping dictionary. The mapping dictionary pre-stores a set of feature words corresponding to each feature dimension vector. According to the index path, all feature words corresponding to the bird's nest raw material are retrieved from the mapping dictionary. The feature words are segmented and cleaned using preset word segmentation symbols (such as commas, semicolons, spaces, etc.). After removing redundant characters, the feature words are transformed into structured feature attribute fields. The feature attribute fields include, but are not limited to, fields corresponding to the core quality dimensions of bird's nest such as origin, variety, moisture content, protein content, color, taste, and impurity content. For operations related to blank data files, configure N filling queues in the blank data file (N is a positive integer ≥1, which can be adjusted according to the number of bird's nest feature dimensions). Each filling queue contains M filling cells (M is a positive integer ≥1, and the number matches the number of feature dimensions corresponding to the queue). The filling queues are divided according to bird's nest feature categories, such as basic attribute queue (corresponding to origin and variety), nutritional attribute queue (corresponding to moisture content and protein content), and quality attribute queue (corresponding to color, taste, and impurity content).

[0030] The operation point determines the appropriate location through a preset feature type-location mapping rule. Specifically, the preset mapping rule establishes a one-to-one mapping relationship table between the type of feature attribute field and the fill queue / fill cell, specifying the fill queue index and fill cell index corresponding to each feature type. An example is shown below: Feature type: Origin → First Fill Queue, First Fill Cell; Feature type: Variety → First fill queue, second fill cell; Feature type: Moisture content → 2nd fill queue, 1st fill cell; Feature type: Protein content → 2nd fill queue, 2nd fill cell; Feature type: Color → 3rd fill queue, 1st fill cell; Feature type: Texture → 3rd filler queue, 2nd filler slot; Feature type: Impurity content → 4th filling queue, 1st filling cell; The operation point sequentially reads the type identifier corresponding to the feature attribute field to be filled, matches it with the above mapping table, and outputs the fill queue index and fill cell index corresponding to the feature attribute field, i.e., the matching position; if there is no corresponding mapping record for the feature type, the operation point outputs an invalid position indicator, and the feature attribute field will not participate in the filling for the time being.

[0031] The migration point receives the adaptation position information output by the operation point, migrates the feature attribute fields to the adaptation filling cells of the corresponding filling queue, and completes the orderly filling of all feature attribute fields. During the filling process, the filling cells only store non-empty feature attribute fields, and the filling cells without corresponding feature attribute fields remain blank. It traverses each filling queue, extracts the non-empty feature attribute fields in the filling queue, concatenates the field content using field connection qualifiers (such as commas), and then applies the preset statement template corresponding to the filling queue to generate the quality description statement. The preset statement templates are as follows: Basic attribute queue template: The {queue name} of this bird's nest raw material is: {concatenated field content}; Nutritional attribute queue template: Its {queue name} is: {concatenated field content}; Quality attribute queue template: Regarding the {queue name}: {concatenated field content}, all quality description statements are arranged according to the preset order of filling the queue. The quality description statements are concatenated into a complete statement using inter-module connection qualifiers (such as "and"), and a statement ending qualifier (such as a period) is added at the end to form a single quality description statement for bird's nest raw material; If there are multiple bird's nest raw materials, repeat the above steps to generate multiple quality description statements, integrate the quality description statements corresponding to all bird's nest raw materials, and write them into a blank data file according to a preset format (such as text format, table format) to form a complete quality dataset of bird's nest raw materials.

[0032] It should be further explained that, in the specific implementation process, the corresponding bird's nest raw materials are classified into grades based on the quality dataset, and sorting control instructions are generated for each batch of bird's nest raw materials. The process of intelligent sorting of bird's nest raw materials of different quality grades based on the corresponding sorting control instructions includes: Obtain historical datasets of bird's nest raw materials classified by manual inspection, construct a quality grade determination model for bird's nest raw materials based on the historical datasets, and input the currently obtained quality datasets into the quality grade determination model. The quality grade determination model classifies bird's nest raw materials into premium, first-grade, second-grade, third-grade, and unqualified grades based on the feature attribute fields, quality description statements, and preset grading rules in the quality dataset. Based on the quality grade of each batch of bird's nest raw materials, a preset grade-sorting strategy mapping table is called to generate sorting control instructions corresponding to different quality grades. The sorting control instructions include sorting channel number, sorting execution mechanism action parameters, sorting timing information and sorting location information. Deploy and debug the sorting execution mechanism, and use the sorting execution mechanism to perform intelligent sorting of bird's nest raw materials of different quality grades based on sorting control instructions. After the intelligent sorting is completed, the bird's nest raw materials are transported to the sorting channel, sorting warehouse and sorting area that match their respective quality grades. At the same time, the quality grade, sorting time, sorting location and flow information of each batch of bird's nest raw materials are recorded as sorting records for each batch of bird's nest raw materials.

[0033] For a flowchart illustrating the intelligent sorting process, please refer to [link / reference needed]. Figure 2 As shown, the specific process of intelligent sorting is as follows: the sorting execution mechanism corresponds to a robotic arm, and the sorting execution mechanism deploys and runs a sorting model constructed by machine vision technology. The sorting model consists of several sub-sorting models of different categories, and each sub-sorting model is used to sort bird's nest raw materials corresponding to a certain quality grade. The sorting models corresponding to several categories of sub-sorting models are stored in the model library. When the bird's nest raw material reaches the working range of the sorting execution mechanism on the conveying device, the sub-sorting model that meets the quality grade of the corresponding bird's nest raw material is taken out from the model library. The sub-sorting model loads the corresponding sorting execution mechanism, which then completes the packaging of the bird's nest raw materials of the current quality grade and transports them to the storage warehouse along the corresponding conveying direction of the conveying device. This process continues until all bird's nest raw materials are stored in the corresponding warehouse, thus completing the sorting of all batches of bird's nest raw materials. Each bird's nest raw material is stored in a designated storage area within the warehouse that meets the corresponding quality grade. When the sorting execution mechanism performs sorting on bird's nest raw materials based on the corresponding obtained sub-sorting model, it simultaneously records the working parameters corresponding to the sub-sorting model, performs iteration on the corresponding sub-sorting model through the working parameters, and then optimizes the corresponding sub-sorting model to adapt to the sorting process of the sorting execution mechanism in performing sorting on bird's nest raw materials, and puts the sub-sorting model into the model library.

[0034] Example 2, please refer to Figure 3 As shown in this embodiment, a machine vision-based intelligent sorting system for bird's nest raw materials includes: The image acquisition module is used to transport the bird's nest raw materials to be sorted on the conveyor device, and set several sensing positioning points on the conveyor device. At the location of each sensing positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration image set. The image processing module is used to perform image processing operations based on the bird's nest calibration image set, obtain the corresponding modal image feature set, and extract the feature dimension vector representing the quality of bird's nest based on the modal image feature set. The knowledge graph processing module is used to construct a knowledge graph of bird's nest quality, establish a mapping dictionary between feature dimension vectors and bird's nest quality knowledge graph, filter feature attribute fields recorded in the mapping dictionary for feature dimension vectors, and summarize all feature attribute fields to generate a quality dataset of corresponding bird's nest raw materials. The bird's nest intelligent sorting module is used to classify the corresponding bird's nest raw materials according to the quality dataset, generate sorting control instructions for each batch of bird's nest raw materials, and perform intelligent sorting of bird's nest raw materials of different quality grades according to the corresponding sorting control instructions.

[0035] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A machine vision-based intelligent sorting method for bird's nest raw materials, characterized in that, Includes the following steps: Step S1: The bird's nest raw materials to be sorted are transported on the conveyor device, and several sensor positioning points are set on the conveyor device. At the location of each sensor positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration map. Step S2: Perform image processing operations based on the bird's nest calibration image set to obtain the corresponding modal image feature set, and extract the feature dimension vector representing the quality of bird's nest based on the modal image feature set; Step S3: Construct a knowledge graph of bird's nest quality, establish a mapping dictionary between feature dimension vectors and bird's nest quality knowledge graph, filter feature attribute fields recorded in the mapping dictionary for feature dimension vectors, and summarize all feature attribute fields to generate a quality dataset of corresponding bird's nest raw materials. Step S4: Based on the quality dataset, classify the corresponding bird's nest raw materials by grade, generate sorting control instructions for each batch of bird's nest raw materials, and perform intelligent sorting for bird's nest raw materials of different quality grades based on the corresponding sorting control instructions.

2. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 1 is characterized in that, The process of conveying the raw bird's nest to be sorted on a conveyor device, setting several sensor positioning points on the conveyor device, and triggering a multi-device image acquisition of the raw bird's nest at the location of each sensor positioning point to obtain the corresponding bird's nest calibration image set includes: Identify the bird's nest raw materials to be sorted; Several sensing and positioning points are set on the conveying device. These points are used to deploy various types of sensor devices to obtain three-dimensional modeling information, infrared sensing information, and internal quality information of the bird's nest raw materials. The information is stored in the form of image files and associated with their respective file authentication identifiers. Command execution devices are deployed at each sensor positioning point. The set acquisition commands are entered into the command execution devices at each sensor positioning point. The command execution devices select and enable the acquisition commands to perform work on the sensor devices, obtain image files of bird's nest raw materials, integrate all image files at each sensor positioning point, and obtain the bird's nest calibration map set for the corresponding sensor positioning point.

3. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 2, characterized in that, Image processing is performed based on the bird's nest calibration image set to obtain the corresponding modal image feature set. The process of extracting feature dimension vectors representing the quality of bird's nest based on the modal image feature set includes: A first processing layer, a second processing layer, and a third processing layer are set up to perform image processing operations on the bird's nest calibration image set, thereby obtaining the corresponding first modal features, second modal features, and third modal features; The first modal feature, the second modal feature, and the third modal feature each correspond to their respective image regions. By summing up their respective image regions, a modal image feature set corresponding to the bird's nest calibration image set at each sensing positioning point is constructed. The modal image feature set includes a number of feature pixel blocks. Each feature pixel block is normalized and converted into a grayscale pixel block. A pre-trained convolutional neural network is used as the feature extraction model to obtain one-dimensional vectors of grayscale pixel blocks at different dimensional levels. Feature aggregation, pooling, and dimensionality reduction are performed on all one-dimensional vectors to obtain feature dimension vectors used to characterize the quality of bird's nest.

4. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 3, characterized in that, The process of constructing a knowledge graph of bird's nest quality includes: Construct a subclass knowledge graph for each quality level, construct a blank graph container, arrange the subclass knowledge graphs in different positions of the blank graph container, establish a corresponding index sequence based on the subclass knowledge graph at each position, and create a corresponding interaction layer for each subclass knowledge graph. Using the index sequence as a retrieval credential, a corresponding mapping path is constructed between all interaction layers belonging to the same retrieval credential. A control domain and an identity domain are created in each interaction layer. When each subclass knowledge graph has completed the processing of the mapping path, control domain, and identity domain, the subclass knowledge graphs are associated at adjacent positions in the blank graph container. After the association is completed, a bird's nest quality knowledge graph representing the corresponding quality grade of bird's nest is constructed.

5. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 4, characterized in that, The process of establishing a mapping dictionary between feature dimension vectors and the bird's nest quality knowledge graph includes: For each subclass knowledge graph of a mapping path, a corresponding graph term set is created. The graph term set includes several feature words for the corresponding feature dimension vector. All feature words corresponding to the graph term set are taken as a dictionary item. The corresponding first-level dictionary index is set based on the dictionary item. All dictionary items are integrated to generate a mapping dictionary. A second-level dictionary index is set for the mapping dictionary. The index path of each feature dimension vector in the mapping dictionary is determined based on the first-level dictionary index and the second-level dictionary index. The index path is used to determine the position of each feature dimension vector in the mapping dictionary.

6. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 5, characterized in that, The process of filtering the feature attribute fields recorded in the mapping dictionary for feature dimension vectors and summarizing all feature attribute fields to generate the corresponding quality dataset for bird's nest raw materials includes: Determine all feature dimension vectors for each bird's nest raw material, obtain the index path for each feature dimension vector, determine all feature words in the mapping dictionary based on their respective index paths, and process each feature word into the corresponding feature attribute field based on the preset word segmentation symbols. Set up a blank data file, set up a number of fill queues on the blank data file, each fill queue consists of a number of fill cells, set up migration points and operation points, the migration points migrate feature attribute fields to the fill queues and perform filling at the fill cells in the appropriate positions in the fill queues, and the operation points determine the appropriate positions of each feature attribute field. Arrange each feature attribute field in the appropriate position to construct several quality description statements. Set connection qualifiers to connect different quality description statements to generate the corresponding quality dataset of bird's nest raw materials.

7. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 6, characterized in that, Based on the quality dataset, the corresponding bird's nest raw materials are classified into grades, and sorting control instructions are generated for each batch of bird's nest raw materials. The process of intelligent sorting of bird's nest raw materials of different quality grades based on the corresponding sorting control instructions includes: Historical datasets are obtained to construct a quality grade determination model for bird's nest raw materials. The quality dataset is input into the quality grade determination model, which classifies the quality grades based on the feature attribute fields, quality description statements, and preset grade division rules in the quality dataset, thus classifying bird's nest raw materials into different quality grades. Based on the quality grade of each batch of bird's nest raw materials, a preset grade-sorting strategy mapping table is invoked to generate sorting control instructions for different quality grades. The sorting execution mechanism is deployed and debugged, and the sorting execution mechanism performs intelligent sorting of bird's nest raw materials of different quality grades based on the sorting control instructions. The quality grade, sorting time, sorting location, and flow information of each batch of bird's nest raw materials are recorded simultaneously as sorting records for each batch of bird's nest raw materials.

8. The intelligent sorting method for bird's nest raw materials based on machine vision according to claim 7, characterized in that, The sorting process, based on sorting control commands, involves the following steps for the sorting execution mechanism to intelligently sort bird's nest raw materials of different quality grades: The sorting execution mechanism deploys and runs the sorting model, which consists of several categories of sub-sorting models. Each sub-sorting model is responsible for sorting bird's nest raw materials of a certain quality grade. The sorting models are stored in the model library. When the bird's nest raw material reaches the working range of the sorting execution mechanism on the conveying device, the sub-sorting model that meets the corresponding quality grade is taken out from the model library. The sub-sorting model loads the corresponding sorting execution mechanism to package the bird's nest raw material of the current quality grade and transport it to the storage warehouse along the conveying direction of the conveying device. When the sorting execution mechanism performs sorting based on the obtained sub-sorting model, it records the working parameters of the sub-sorting model, iterates on the corresponding sub-sorting model, optimizes the corresponding sub-sorting model, and adapts it to the sorting process of the sorting execution mechanism for sorting bird's nest raw materials. The sub-sorting model is then stored in the model library.

9. A machine vision-based intelligent sorting system for bird's nest raw materials, used to implement the intelligent sorting method for bird's nest raw materials according to any one of claims 1 to 8, characterized in that, The system includes: The image acquisition module is used to transport the bird's nest raw materials to be sorted on the conveyor device, and set several sensing positioning points on the conveyor device. At the location of each sensing positioning point, a multi-device image acquisition of the bird's nest raw materials is triggered to obtain the corresponding bird's nest calibration image set. The image processing module is used to perform image processing operations based on the bird's nest calibration image set, obtain the corresponding modal image feature set, and extract the feature dimension vector representing the quality of bird's nest based on the modal image feature set. The knowledge graph processing module is used to construct a knowledge graph of bird's nest quality, establish a mapping dictionary between feature dimension vectors and bird's nest quality knowledge graph, filter feature attribute fields recorded in the mapping dictionary for feature dimension vectors, and summarize all feature attribute fields to generate a quality dataset of corresponding bird's nest raw materials. The bird's nest intelligent sorting module is used to classify the corresponding bird's nest raw materials according to the quality dataset, generate sorting control instructions for each batch of bird's nest raw materials, and perform intelligent sorting of bird's nest raw materials of different quality grades according to the corresponding sorting control instructions.

Images