Training methods for tray status recognition models and tray status recognition methods

Abstract

This invention provides a training method and a tray status recognition method for a tray status recognition model, belonging to the field of image recognition. The training method includes: acquiring a desired image captured by an image acquisition device; segmenting the desired image based on the tray temporary storage region and labeling the tray status category to obtain a training set; inputting the training set into a classification model to be trained for iterative training to obtain a classification model; inputting the training set into a retrieval model to be trained for iterative training to obtain a retrieval model; and combining the classification model and the retrieval model to obtain a tray status recognition model.

Description

Technical Field

[0001] This invention relates to the field of image recognition, specifically to a training method for a tray status recognition model, a tray status recognition method, and a tray storage and retrieval control system. Background Technology

[0002] Pallet detection at workstations refers to detecting the material information of pallets temporarily stored in complex industrial production scenarios and measuring their position and orientation. It is a key link in enabling fully automated and flexible operation of laser-guided forklift AGVs (Automated Guided Vehicles). Currently, in assembly stations of large engineering machinery, workers discover missing materials during operation, notify forklift operators to transport empty pallets to the warehouse, or transport the corresponding materials to the temporary storage area, and then manually input material scheduling information to complete the pallet storage and retrieval process. Relying on real-time human monitoring undoubtedly leads to labor waste, and untimely feedback from periodic inspections can easily miss faults, resulting in material shortages and impacting work efficiency, production planning, and logistics and warehousing information management, which does not comply with industrial safety standards. With the continuous expansion of production scale, how to quickly and conveniently identify pallet status to meet the pallet status recognition requirements of high-speed and high-precision manufacturing is an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this application is to overcome the problem that existing pallet detection methods cannot quickly identify pallet status under complex lighting conditions, and to provide a training method for a pallet status recognition model, a pallet status recognition method, and a pallet storage and retrieval control system.

[0004] The first aspect of this application provides a method for training a tray state recognition model, including:

[0005] Acquire the desired image captured by the image acquisition device;

[0006] The desired image is segmented based on the temporary storage region of the tray and labeled with the tray state category to obtain a training set;

[0007] The training set is input into the classification model to be trained for iterative training to obtain the classification model, wherein the classification model is used to output at least one state category for each temporary region based on the image to be classified.

[0008] The training set is input into the retrieval model to be trained for iterative training to obtain the retrieval model. The retrieval model is used to perform similarity retrieval between the image to be classified and the image features in the database, and outputs the similarity ranking of the state category of each temporary region according to the similarity between each temporary region of the image to be classified and each state category.

[0009] The classification model and the retrieval model are combined to obtain the tray status recognition model.

[0010] In one embodiment of this application, before acquiring the desired image captured by the image acquisition device, the method further includes:

[0011] Acquire images captured by the image acquisition device;

[0012] Based on the image's average grayscale value and the current time point, determine whether to turn on the supplementary lighting source;

[0013] The exposure time of the image acquisition device is adjusted based on the grayscale mean and the image entropy of the image.

[0014] In one embodiment of this application, determining whether to turn on the supplementary lighting source based on the average grayscale value of the image and the current time point includes:

[0015] Determine whether the current time point is within the bright period based on the sunrise and sunset time database, where the bright period is from sunrise to sunset;

[0016] If the current time is not within a bright period, turn on the supplementary light source and provide supplementary light according to the piecewise supplementary light function;

[0017] If the current time point is within a bright period, determine whether the average grayscale value is less than the first threshold.

[0018] When the average grayscale value is less than the first threshold, the supplementary light source is turned on, and supplementary light is applied according to the piecewise supplementary light function.

[0019] In one embodiment of this application, adjusting the exposure time of the image acquisition device based on the grayscale mean and the image entropy of the image includes:

[0020] The exposure time of the image acquisition device is adjusted based on the difference between the mean gray level and the expected mean gray level, and the difference between the image entropy and the expected image entropy.

[0021] A second aspect of this application provides a tray status recognition method, including:

[0022] Acquire the image to be detected by the image acquisition device;

[0023] The image to be detected is segmented based on the temporary storage region of the tray and the region of interest is labeled to obtain the image to be classified;

[0024] The classification model in the tray state recognition model outputs at least one state category for each temporary region based on the image to be classified.

[0025] The retrieval model in the tray state recognition model is used to perform similarity retrieval between the image to be classified and the image features in the database. The similarity ranking of the state categories of each temporary region of the image to be classified is output according to the similarity between each temporary region of the image to be classified and each state category.

[0026] The state category of the tray in each temporary storage area is determined based on at least one state category output by the similarity ranking and classification model, wherein the tray state recognition model is obtained by the training method of the tray state recognition model in the above embodiment.

[0027] A third aspect of this application provides a training apparatus for a tray state recognition model, which includes a classification model and a retrieval model. The apparatus includes:

[0028] The dataset acquisition unit is used to acquire the desired image captured by the image acquisition device;

[0029] The dataset processing unit is used to segment the desired image based on the pallet storage region and label the pallet state category to obtain a training set.

[0030] The model training unit is used to input the training set into the classification model to be trained in the model to be trained for iterative training to obtain the classification model. The classification model is used to output at least one state category for each temporary region based on the image to be classified.

[0031] The training set is input into the retrieval model to be trained for iterative training to obtain the retrieval model. The retrieval model is used to perform similarity retrieval between the image to be classified and the image features in the database, and outputs the similarity ranking of the state category of each temporary region according to the similarity between each temporary region of the image to be classified and each state category.

[0032] The model generation unit is used to combine the classification model and the retrieval model to obtain the tray status recognition model.

[0033] A fourth aspect of this application discloses a tray status recognition device, comprising:

[0034] The image acquisition unit is used to acquire the image to be detected acquired by the image acquisition device;

[0035] The image processing unit is used to perform segmentation and region of interest labeling based on the tray storage area of ​​the image to be detected, so as to obtain the image to be classified.

[0036] The classification unit is used to output at least one state category for each temporary region based on the image to be classified using the classification model in the tray state recognition model.

[0037] The retrieval model in the tray state recognition model is used to perform similarity retrieval between the image to be classified and the image features in the database. The similarity ranking of the state categories of each temporary region of the image to be classified is output according to the similarity between each temporary region of the image to be classified and each state category.

[0038] The state category of the tray in each temporary storage area is determined based on at least one state category output by the similarity ranking and classification model, wherein the tray state recognition model is obtained by the training method of the tray state recognition model in the above embodiment.

[0039] The fifth aspect of this application provides a tray access control system, including:

[0040] The tray status recognition device in the above embodiment is used to determine the status category of the tray in each temporary storage area based on the image to be detected acquired by the image acquisition device.

[0041] Signal processing unit for generating tray access tasks based on state category;

[0042] The actuator includes multiple transport units for storing and retrieving pallets according to pallet storage and retrieval tasks.

[0043] The sixth aspect of this application provides an electronic device, including a memory and a processor. The memory stores a computer program, which, when run on the processor, executes the training method for the tray state recognition model in the above embodiments, or the tray state recognition method in the above embodiments.

[0044] The seventh aspect of this application provides a computer-readable storage medium storing a computer program, which, when run on a processor, executes the training method for the tray state recognition model in the above embodiments, or the tray state recognition method in the above embodiments.

[0045] Through the above technical solution, the pallet status recognition model combines the advantages of both classification and retrieval models, resulting in more accurate classification results. The processor, after training the pallet status recognition model, can quickly and accurately identify the status of pallets in complex industrial environments, solving problems such as untimely and time-consuming material placement on pallets. This enables timely material replenishment of pallets and achieves efficient transportation for dynamic warehousing allocation under complex conditions.

[0046] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0047] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0048] Figure 1 The illustration shows a flowchart of a training method for a tray state recognition model according to an embodiment of this application;

[0049] Figure 2 The illustration shows a schematic flowchart of a tray status recognition method according to an embodiment of this application. Detailed Implementation

[0050] The specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0051] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and reversal of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0052] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0053] Figure 1 This schematically illustrates a flowchart of a training method for a tray state recognition model according to an embodiment of this application, as shown below. Figure 1 As shown, in one embodiment of this application, a training method for a pallet state recognition model is provided. The pallet state recognition model includes a classification model and a retrieval model. The method may include steps S100-S500.

[0054] Step S100: Acquire the desired image captured by the image acquisition device.

[0055] Images of the temporary storage area for industrial pallets serve as the data source for training the pallet status recognition model. In one embodiment of this application, the image acquisition device is a large-area CMOS industrial camera used to acquire images of the temporary storage area for industrial pallets. To ensure that the images acquired by the image acquisition device under complex lighting conditions are high-quality desired images, i.e., to ensure the accuracy of the output results of the trained pallet status recognition model, it is necessary to adaptively adjust the exposure time and supplementary lighting source of the image acquisition device based on the lighting conditions of the industrial site.

[0056] In one embodiment of this application, before acquiring the desired image captured by the image acquisition device, the method further includes:

[0057] Acquire images captured by the image acquisition device;

[0058] Based on the image's average grayscale value and the current time point, determine whether to turn on the supplementary lighting source;

[0059] The exposure time of the image acquisition device is adjusted based on the grayscale mean and the image entropy of the image.

[0060] For example, if there is a focused skylight in the industrial site, natural light can illuminate the temporary storage area of ​​the tray. When the natural light is sufficient, the supplementary lighting source inside the industrial site does not need to be turned on to meet the lighting conditions required for the image acquisition device to acquire the desired image. When the natural light is insufficient, the supplementary lighting source inside the industrial site needs to be turned on to meet the lighting conditions required for the image acquisition device to acquire the desired image. The processor determines whether supplementary lighting is needed in the industrial site based on the grayscale average of the image acquired by the image acquisition device.

[0061] In one embodiment of this application, determining whether to turn on the supplementary lighting source based on the average grayscale value of the image and the current time point includes:

[0062] Determine whether the current time point is within the bright period based on the sunrise and sunset time database, where the bright period is from sunrise to sunset;

[0063] If the current time is not within a bright period, turn on the supplementary light source and provide supplementary light according to the piecewise supplementary light function;

[0064] If the current time point is within a bright period, determine whether the average grayscale value is less than the first threshold.

[0065] When the average grayscale value is less than the first threshold, the supplementary light source is turned on, and supplementary light is applied according to the piecewise supplementary light function.

[0066] It is understandable that sunrise and sunset times differ between cities. Therefore, in this embodiment, the sunrise and sunset time database can be a database established based on the year-round sunrise and sunset times of the city where the industrial site is located. The processor first determines whether the current time point is within a bright period with sufficient natural light. After obtaining the current time point, the processor can determine whether the current time point is within a bright period based on the built-in sunrise and sunset time database. If the current time point is between sunrise and sunset, i.e., during the day, then the current time point is within a bright period. If the current time point is not between sunrise and sunset, i.e., at night, then the current time point is not within a bright period. If the current time point is not within a bright period, the processor directly issues an instruction to turn on the supplementary lighting source to provide supplementary lighting for the industrial site. If the current time point is within a bright period, the processor also needs to determine whether to turn on the supplementary lighting source based on the average grayscale value of the image, because even if the time point is within a bright period, different weather conditions can affect natural light, resulting in insufficient natural light, such as cloudy or rainy days.

[0067] The processor determines whether the average grayscale value of the image is less than a first threshold. For example, the first threshold can be set to 100. The specific value of the first threshold can be adjusted and set according to the image acquisition requirements and the actual environment of the industrial site. This application does not limit this. If the average grayscale value of the image is less than the first threshold, it indicates that the natural lighting in the industrial site is insufficient. The processor issues an instruction to turn on the supplementary lighting source and performs supplementary lighting according to a preset supplementary lighting segmentation function.

[0068] To ensure the quality and effect of the acquired images, it is also necessary to make adaptive adjustments to the exposure time of the image acquisition device. The adjustment of the exposure time is based on the grayscale mean and image entropy of the image.

[0069] In one embodiment of this application, adjusting the exposure time of the image acquisition device based on the grayscale mean and the image entropy of the image includes:

[0070] The exposure time of the image acquisition device is adjusted based on the difference between the mean gray level and the expected mean gray level, and the difference between the image entropy and the expected image entropy.

[0071] The desired grayscale mean and desired image entropy can be determined in advance using the image mean and image entropy of a superior image. For example, the desired grayscale mean can be 150, and the desired image entropy can be 8. Based on the difference between the grayscale mean and the desired grayscale mean, and the difference between the image entropy and the desired image entropy, the exposure time of the image acquisition device is adjusted to improve the image acquisition effect. Specifically, the exposure time can be adjusted using the following formula:

[0072]

[0073] Among them, H i E is the grayscale mean of the obtained image. i The image entropy of the acquired image is given by H, the expected grayscale mean is given by E, the expected image entropy is given by f(i), and the exposure time adjustment is given by a. i The current exposure time is denoted by k, which is the preset gain coefficient.

[0074] Specifically, the processor can periodically extract the grayscale mean and image entropy of the images acquired by the image acquisition device, and adjust the exposure time of the image acquisition device in a timely manner, or periodically detect whether the acquired image produces a bright background or high contrast. When the acquired image has a bright background or high contrast, the processor can extract the grayscale mean and image entropy of the image, and adjust the exposure time of the image acquisition device in a timely manner.

[0075] Step S200: Segment the desired image based on the tray temporary storage region and label the tray state category to obtain a training set.

[0076] In one embodiment of this application, the desired image can first be preprocessed with random scaling and cropping, image enhancement, and PCA (Principal Component Analysis) noise reduction, and then the desired image can be segmented based on the temporary storage area of ​​the pallet. For example, a desired image can be segmented into four temporary storage areas, and each temporary storage area can be labeled with a state category. For example, the state category can be: no pallet, empty pallet without material, temporary storage area with people, and pallet with material.

[0077] Step S300: Input the training set into the classification model to be trained in the model to be trained for iterative training to obtain the classification model, wherein the classification model is used to output at least one state category for each temporary region based on the image to be classified.

[0078] The training model includes a classification model and a retrieval model. The training set is input into the classification model for iterative training to obtain the classification model. When applying the classification model to classify an image, the model extracts image features from the image and directly outputs the image category through an embedded classifier. Therefore, it is suitable for image scenarios with significant inter-class differences. For example, the specific category of the classification model is VGG16 (Visual Geometry Group). Because the image to be classified input to the classification model may have features of multiple state categories, the image classification model may output more than one state category for each temporary region of the image. Determining the final state category requires combining it with the retrieval model.

[0079] Step S400: Input the training set into the retrieval model to be trained in the training model for iterative training to obtain the retrieval model. The retrieval model is used to perform similarity retrieval between the image to be classified and the image features in the database, and output the similarity ranking of the state category of each temporary region according to the similarity between each temporary region of the image to be classified and each state category.

[0080] Step S500: Combine the classification model and the retrieval model to obtain the tray status recognition model.

[0081] The training set is input into the retrieval model to be trained iteratively to obtain the retrieval model. The retrieval model does not have a built-in classifier. When applying the classification model to classify the image to be classified, the retrieval model extracts image features from the image but does not perform direct classification. Instead, it performs a similarity search with the features of all images in a database (such as a gallery), and outputs a ranking of the state category similarity for each temporary region of the image to be classified based on the similarity between each temporary region and each state category. The retrieval model is suitable for image scenarios with small intra-class differences and can expand its categories by registering new categories in the database. After the classification model outputs multiple categories for the temporary region, it selects the category with the highest similarity as the state category for that temporary region based on the similarity ranking of the state categories output by the retrieval model. In other words, the tray state recognition model provided in this application combines the characteristics and advantages of both classification and retrieval models, resulting in a more accurate final classification result.

[0082] Figure 2 This illustration schematically shows a flowchart of a tray status recognition method according to an embodiment of this application, such as... Figure 2 As shown, in one embodiment of this application, a tray status recognition method is provided, including:

[0083] Step S600: Acquire the image to be detected by the image acquisition device;

[0084] Step S700: Perform segmentation and region of interest labeling on the image to be detected based on the tray temporary storage area to obtain the image to be classified;

[0085] Step S800: Using the classification model in the tray state recognition model, output at least one state category for each temporary area based on the image to be classified;

[0086] The retrieval model in the tray state recognition model is used to perform similarity retrieval between the image to be classified and the image features in the database. The similarity ranking of the state categories of each temporary region of the image to be classified is output according to the similarity between each temporary region of the image to be classified and each state category.

[0087] Step S900: Determine the state category of the tray for each temporary storage area based on at least one state category output by the similarity ranking and classification model, wherein the tray state recognition model is obtained by the training method of the tray state recognition model provided in the above embodiment.

[0088] The trained pallet state recognition model can be directly applied to pallet state recognition in industrial settings. After the image acquisition device acquires the image to be detected, it can first perform preprocessing such as random scaling and cropping, image enhancement, and PCA noise reduction. Then, the desired image is segmented based on the pallet's temporary storage area, and the temporary storage area is labeled with a region of interest (ROI) to obtain the image to be classified. This image is then input into the pallet state recognition model to determine the pallet state category for each temporary storage area. Notably, before the image acquisition device acquires the image to be detected, the processor also adjusts the exposure time of the supplementary lighting source and the image acquisition device according to the lighting conditions in the industrial setting, ensuring that the pallet state recognition model can perform state classification in complex lighting environments.

[0089] The classification model classifies the image to be classified. It extracts image features from the image and directly outputs the image category through an embedded classifier, including at least one state category. The retrieval model extracts image features but does not perform direct classification. Instead, it performs a similarity search with the features of all images in a database (such as a gallery), outputting a similarity ranking of the state categories for each temporary region of the image to be classified, based on the similarity between each temporary region and each state category. The processor, based on the similarity ranking of the multiple categories output by the classification model for each temporary region and the state categories output by the retrieval model, selects the category with the highest similarity as the state category for that temporary region.

[0090] Through the technical solutions described in the above embodiments, the pallet status recognition model combines the advantages of both classification and retrieval models, resulting in more accurate classification results. The processor, through the trained pallet status recognition model, can quickly and accurately identify the pallet status in complex industrial environments, solving problems such as untimely and time-consuming material placement on pallets. This enables timely material replenishment of pallets and achieves efficient transportation for dynamic warehousing allocation under complex conditions.

[0091] In one embodiment of this application, a training apparatus for a tray state recognition model is provided. The tray state recognition model includes a classification model and a retrieval model. The apparatus includes:

[0092] The dataset acquisition unit is used to acquire the desired image captured by the image acquisition device;

[0093] The dataset processing unit is used to segment the desired image based on the pallet storage region and label the pallet state category to obtain a training set.

[0094] The model training unit is used to input the training set into the classification model to be trained in the model to be trained for iterative training to obtain the classification model. The classification model is used to output at least one state category for each temporary region based on the image to be classified.

[0095] The training set is input into the retrieval model to be trained for iterative training to obtain the retrieval model. The retrieval model is used to perform similarity retrieval between the image to be classified and the image features in the database, and outputs the similarity ranking of the state category of each temporary region according to the similarity between each temporary region of the image to be classified and each state category.

[0096] The model generation unit is used to combine the classification model and the retrieval model to obtain the tray status recognition model.

[0097] In one embodiment of this application, a tray status recognition device is provided, comprising:

[0098] The image acquisition unit is used to acquire the image to be detected acquired by the image acquisition device;

[0099] The image processing unit is used to perform segmentation and region of interest labeling based on the tray storage area of ​​the image to be detected, so as to obtain the image to be classified.

[0100] The classification unit is used to output at least one state category for each temporary region based on the image to be classified using the classification model in the tray state recognition model.

[0101] The retrieval model in the tray state recognition model is used to perform similarity retrieval between the image to be classified and the image features in the database. The similarity ranking of the state categories for each temporary region is output according to the similarity between each temporary region of the image to be classified and each state category.

[0102] The state category of the tray in each temporary storage area is determined based on at least one state category output by the similarity ranking and classification model, wherein the tray state recognition model is obtained by the training method of the tray state recognition model as described in the above embodiments.

[0103] In one embodiment of this application, a tray access control system is provided, comprising:

[0104] The tray status recognition device in the above embodiment is used to determine the status category of the tray in each temporary storage area based on the image to be detected acquired by the image acquisition device.

[0105] Signal processing unit for generating tray access tasks based on state category;

[0106] The actuator includes multiple transport units for storing and retrieving pallets according to pallet storage and retrieval tasks.

[0107] In one embodiment of this application, the pallet storage and retrieval control system further includes a control execution device for generating an execution control signal based on the pallet storage and retrieval task and sending it to multiple transport units in the execution device to realize an automatic storage and retrieval process based on the pallet status category.

[0108] In one embodiment of this application, the transport unit is a laser forklift AGV.

[0109] In one embodiment of this application, an electronic device is provided, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor can execute the machine-executable instructions to implement the training method of the tray state recognition model in the above embodiment, or the tray state recognition method in the above embodiment.

[0110] In one embodiment of this application, a machine-readable storage medium is provided, on which instructions are stored. When executed by a processor, the instructions cause the processor to implement the training method of the tray state recognition model in the above embodiments, or the tray state recognition method in the above embodiments.

[0111] In one embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the training method for the tray state recognition model in the above embodiments, or the tray state recognition method in the above embodiments.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0115] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0116] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0117] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0118] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A training method for a tray state recognition model, characterized in that, The method includes: Acquire images captured by the image acquisition device; Based on the average grayscale value of the image and the current time point, determine whether to turn on the supplementary light source; The exposure time of the image acquisition device is adjusted based on the mean gray level and the image entropy of the image. Acquire the desired image captured by the image acquisition device; The desired image is segmented based on the temporary storage region of the tray and labeled with the tray state category to obtain a training set; The training set is input into the classification model to be trained in the model to be trained for iterative training to obtain the classification model, wherein the classification model is used to output at least one state category for each of the temporary regions based on the image to be classified. The training set is input into the retrieval model to be trained for iterative training to obtain the retrieval model. The retrieval model is used to perform similarity retrieval between the image to be classified and the image features in the database, and output the state category similarity ranking of each temporary region of the image to be classified according to the similarity between each temporary region of the image to be classified and each state category. The classification model and the retrieval model are combined to obtain the tray status recognition model; The step of determining whether to turn on the supplementary lighting source based on the average grayscale value of the image and the current time point includes: Determine whether the current time point is within a bright period based on the sunrise and sunset time database, wherein the bright period is from sunrise time to sunset time; If the current time point is not within the bright time period, the supplementary light source is turned on, and supplementary light is provided according to the supplementary light segmentation function; If the current time point is within the bright time period, determine whether the average gray value is less than a first threshold. When the average grayscale value is less than the first threshold, the supplementary light source is turned on, and supplementary light is applied according to the supplementary light piecewise function.

2. The method according to claim 1, characterized in that, Adjusting the exposure time of the image acquisition device based on the grayscale mean and the image entropy of the image includes: The exposure time of the image acquisition device is adjusted based on the difference between the mean gray level and the desired mean gray level, and the difference between the image entropy and the desired image entropy.

3. A method for identifying the status of a tray, characterized in that, include: Acquire images captured by the image acquisition device; Based on the average grayscale value of the image and the current time point, determine whether to turn on the supplementary light source; The exposure time of the image acquisition device is adjusted based on the mean gray level and the image entropy of the image. Acquire the image to be detected by the image acquisition device; The image to be detected is segmented based on the temporary storage area of ​​the tray and the region of interest is labeled to obtain the image to be classified; Using the classification model in the tray state recognition model, at least one state category is output for each of the temporary storage areas based on the image to be classified. The retrieval model in the tray state recognition model is used to perform similarity retrieval between the image to be classified and the image features in the database, and the similarity ranking of the state categories of each temporary storage area of ​​the image to be classified is output according to the similarity between each temporary storage area of ​​the image to be classified and each state category; The state category of each of the temporary storage areas is determined based on the similarity ranking and at least one state category output by the classification model, wherein the tray state recognition model is obtained by the training method of the tray state recognition model as described in any one of claims 1 to 2; The step of determining whether to turn on the supplementary lighting source based on the average grayscale value of the image and the current time point includes: Determine whether the current time point is within a bright period based on the sunrise and sunset time database, wherein the bright period is from sunrise time to sunset time; If the current time point is not within the bright time period, the supplementary light source is turned on, and supplementary light is provided according to the supplementary light segmentation function; If the current time point is within the bright time period, determine whether the average gray value is less than a first threshold. When the average grayscale value is less than the first threshold, the supplementary light source is turned on, and supplementary light is applied according to the supplementary light piecewise function.

4. A training device for a pallet state recognition model, characterized in that, The tray status recognition model includes a classification model and a retrieval model, and the device includes: The image acquisition unit is used to acquire images captured by the image acquisition device; The supplementary light source activation unit is used to determine whether to activate the supplementary light source based on the average grayscale value of the image and the current time point; An exposure time adjustment unit is used to adjust the exposure time of the image acquisition device according to the grayscale mean and the image entropy of the image. The dataset acquisition unit is used to acquire the desired image captured by the image acquisition device; The dataset processing unit is used to segment the desired image based on the tray temporary storage area and label the tray state category to obtain a training set. The model training unit is used to input the training set into the classification model to be trained in the model to be trained for iterative training to obtain the classification model, wherein the classification model is used to output at least one state category for each of the temporary regions based on the image to be classified. The training set is input into the retrieval model to be trained for iterative training to obtain the retrieval model. The retrieval model is used to perform similarity retrieval between the image to be classified and the image features in the database, and output the state category similarity ranking of each temporary region of the image to be classified according to the similarity between each temporary region of the image to be classified and each state category. A model generation unit is used to combine the classification model and the retrieval model to obtain the tray status recognition model; The step of determining whether to turn on the supplementary lighting source based on the average grayscale value of the image and the current time point includes: Determine whether the current time point is within a bright period based on the sunrise and sunset time database, wherein the bright period is from sunrise time to sunset time; If the current time point is not within the bright time period, the supplementary light source is turned on, and supplementary light is provided according to the supplementary light segmentation function; If the current time point is within the bright time period, determine whether the average gray value is less than a first threshold. When the average grayscale value is less than the first threshold, the supplementary light source is turned on, and supplementary light is applied according to the supplementary light piecewise function.

5. A pallet status recognition device, characterized in that, include: The image acquisition unit is used to acquire images captured by the image acquisition device; The supplementary light source activation unit is used to determine whether to activate the supplementary light source based on the average grayscale value of the image and the current time point; An exposure time adjustment unit is used to adjust the exposure time of the image acquisition device according to the grayscale mean and the image entropy of the image. The image acquisition unit is used to acquire the image to be detected acquired by the image acquisition device; The image processing unit is used to segment and label the image to be detected based on the tray temporary storage area to obtain the image to be classified; A classification unit is used to output at least one state category for each temporary region based on the image to be classified using the classification model in the tray state recognition model. The retrieval model in the tray state recognition model is used to perform similarity retrieval between the image to be classified and the image features in the database, and the similarity ranking of the state categories of each temporary storage area of ​​the image to be classified is output according to the similarity between each temporary storage area of ​​the image to be classified and each state category; The state category of each temporary storage area is determined based on the similarity ranking and at least one state category output by the classification model, wherein the tray state recognition model is obtained by the training method of the tray state recognition model as described in any one of claims 1 to 2; The step of determining whether to turn on the supplementary lighting source based on the average grayscale value of the image and the current time point includes: Determine whether the current time point is within a bright period based on the sunrise and sunset time database, wherein the bright period is from sunrise time to sunset time; If the current time point is not within the bright time period, the supplementary light source is turned on, and supplementary light is provided according to the supplementary light segmentation function; If the current time point is within the bright time period, determine whether the average gray value is less than a first threshold. When the average grayscale value is less than the first threshold, the supplementary light source is turned on, and supplementary light is applied according to the supplementary light piecewise function.

6. A tray access control system, characterized in that, include: The pallet status recognition device according to claim 5 is used to determine the status category of the pallet in each temporary storage area based on the image to be detected acquired by the image acquisition device. A signal processing device is used to generate a tray access task based on the state category; The execution device includes multiple transport units, which are used to store and retrieve pallets according to the pallet storage and retrieval task.

7. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, which, when run on the processor, executes the training method of the tray state recognition model as described in any one of claims 1 to 2, or the tray state recognition method as described in claim 3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, executes the training method for the tray status recognition model as described in any one of claims 1 to 2, or the tray status recognition method as described in claim 3.

Images