Methods, apparatus, equipment, and media for acquiring images of items used in weighing equipment

Abstract

This disclosure provides an embodiment of a method, apparatus, device, and medium for acquiring images of objects on a weighing device. One specific implementation of the method includes: capturing an image of an object on the weighing device to obtain an initial object image; inputting the initial object image into a pre-trained image classification model to obtain a classification result for the initial object image; determining that the initial object image meets a temporary retention condition based on the classification result indicating that a plastic bag obstructs the image; performing plastic bag feature recognition on the initial object image to determine whether the initial object image meets a retention condition; and identifying the initial object image as the acquired object image based on the retention condition. This implementation ensures the quality of the acquired object images while increasing the number of effectively acquired object images, thus improving the efficiency of acquiring usable object images.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically to methods, apparatus, devices, and media for acquiring images of items for weighing equipment. Background Technology

[0002] During the establishment of the product database, the products are diverse, with varying image angles, necessitating the collection of a large number of real product images. Currently, most food scales use cameras attached to the scale for image acquisition.

[0003] However, when using the above method to acquire images of objects, the following technical problems often arise:

[0004] Items on food scales are often obscured by plastic bags. If these items are directly added to the database, the image quality will be reduced, affecting the training of the corresponding model. If all items are removed, a large amount of effective data will be lost, resulting in a lack of diversity in the item image data.

[0005] The information disclosed in this background section is only intended to enhance the understanding of the background of the present disclosure concept, and therefore may contain information that does not form prior art known to those skilled in the art. Summary of the Invention

[0006] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of this disclosure provide methods, apparatus, electronic devices, and computer-readable media for acquiring images of items using weighing equipment to address one or more of the technical problems mentioned in the background section above.

[0008] In a first aspect, some embodiments of this disclosure provide a method for acquiring images of objects on a weighing device. The method includes: capturing an image of an object on the weighing device to obtain an initial image of the object; inputting the initial image of the object into a pre-trained image classification model to obtain a classification result for the initial image of the object; in response to determining that the classification result of the initial image of the object indicates that a plastic bag is obstructing the image, determining that the initial image of the object satisfies a temporary retention condition; in response to determining that the initial image of the object satisfies the temporary retention condition, performing plastic bag feature recognition on the initial image of the object to obtain plastic bag feature information corresponding to the initial image of the object, wherein the plastic bag feature information includes plastic bag obstruction ratio information; determining whether the initial image of the object satisfies the retention condition based on the plastic bag obstruction ratio information included in the plastic bag feature information; and in response to determining that the initial image of the object satisfies the retention condition, identifying the initial image of the object as the acquired image of the object.

[0009] Secondly, some embodiments of this disclosure provide an image acquisition device for a weighing device, the device comprising: a shooting unit configured to shoot an object on the weighing device to obtain an initial image of the object; a classification unit configured to input the initial image of the object into a pre-trained image classification model to obtain a classification result of the initial image of the object; a first determining unit configured to determine that the initial image of the object meets a temporary retention condition in response to determining that the classification result of the initial image of the object indicates that a plastic bag occlusion exists; a second determining unit configured to perform plastic bag feature recognition on the initial image of the object in response to determining that the initial image of the object meets the temporary retention condition, to obtain plastic bag feature information corresponding to the initial image of the object, wherein the plastic bag feature information includes plastic bag occlusion ratio information; a third determining unit configured to determine whether the initial image of the object meets the retention condition based on the plastic bag occlusion ratio information included in the plastic bag feature information; and a fourth determining unit configured to determine the initial image of the object as the acquired image of the object in response to determining that the initial image of the object meets the retention condition.

[0010] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0011] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0012] The above embodiments of this disclosure have the following beneficial effects: Through the image acquisition method for weighing equipment disclosed herein, the retention conditions of the acquired image can be automatically set according to a downstream model (such as item recognition) by predicting the plastic bag information in an image containing a plastic bag. This allows for flexible selection of images containing plastic bags but with high usability. Specifically, the reason why most image acquisition methods cannot flexibly select images with high usability is that most methods do not further filter images of items obscured by plastic bags, resulting in a small number and poor quality of acquired images. Based on this, some embodiments of the image acquisition method for weighing equipment disclosed herein include: First, taking a picture of an object on the weighing equipment to obtain an initial image. This yields an image of the item to be processed. Then, the initial image is input into a pre-trained image classification model to obtain an initial image classification result. This allows for an initial coarse classification of the item using the first model. Then, in response to determining that the initial image classification result indicates plastic bag obscuration, the initial image is determined to meet the temporary retention conditions. This lays the foundation for subsequent further classification. Then, in response to determining that the initial item image meets the aforementioned temporary retention conditions, plastic bag feature recognition is performed on the initial item image to obtain plastic bag feature information corresponding to the initial item image, wherein the plastic bag feature information includes plastic bag occlusion ratio information. Then, based on the plastic bag occlusion ratio information included in the plastic bag feature information, it is determined whether the initial item image meets the retention conditions. Finally, in response to determining that the initial item image meets the aforementioned retention conditions, the initial item image is determined as the acquired item image. Thus, data cleaning of the acquired item images is completed. The above-described item image acquisition method for weighing equipment uses two deep learning models to perform data cleaning on the acquired item images and uses preset conditions to filter out usable item images. Therefore, while ensuring the quality of the acquired item images, the number of acquired item images is increased as much as possible, improving the efficiency of acquiring usable item images. Attached Figure Description

[0013] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0014] Figure 1 This is a flowchart of some embodiments of the article image acquisition method for weighing equipment according to the present disclosure;

[0015] Figure 2This is a schematic diagram of the structure of some embodiments of the article image acquisition device for weighing equipment according to the present disclosure;

[0016] Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0019] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0020] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0021] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0022] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Figure 1 This is a flowchart illustrating some embodiments of the article image acquisition method for weighing equipment according to the present disclosure. The article image acquisition method for weighing equipment includes the following steps:

[0024] Step 101: Take a picture of the object on the weighing device to obtain an initial image of the object.

[0025] In some embodiments, the execution entity of the article image acquisition method for weighing equipment (e.g., a food scale with a camera) can acquire an initial article image from an integrated device via a wired or wireless connection, and can connect the initial article image to a server for processing via a wired or wireless connection. The article image can be acquired by setting a camera on the food scale. The camera can be a fixed-focus camera or a high-definition wide-angle camera, without specific limitation. It should be noted that the wireless connection method can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other currently known or future developed wireless connection methods. The initial article image can include, but is not limited to, food images or dish images, without specific limitation.

[0026] Optionally, the aforementioned entities may also perform the following steps:

[0027] The first step is to save the initial item image locally. The executing entity can temporarily save the initial item image within itself. The saving method can include, but is not limited to, local flash memory or an SD card.

[0028] The second step involves uploading the initial item images saved within a preset time period to the server. This preset time period can be set as needed; for example, it could be once a month to upload all item images collected within that month, and after uploading, all uploaded item images should be automatically cleared to prevent local memory from becoming full. The cloud server mentioned above can be a private cloud server, without specific limitations. The execution entity can receive item images uploaded from different food scales and distinguish them by food scale number. The execution entity can also categorize the received item images by time period, allowing users to download them as needed.

[0029] Step 102: Input the initial object image into the pre-trained image classification model to obtain the initial object image classification result.

[0030] In some embodiments, the execution entity may input the initial object image into a pre-trained image classification model to obtain an initial object image classification result. The image classification model is a neural network model that takes the initial object image as input and the initial object image classification result as output.

[0031] In practice, the aforementioned implementing entity can categorize the initial object images into four types. For example, the initial object images can be categorized as: normal images, images obscured by plastic bags, unclear images, and images obscured by foreign objects.

[0032] In some optional implementations of certain embodiments, the aforementioned execution entity may input the initial object image into a pre-trained image classification model through the following steps to obtain the initial object image classification result:

[0033] The first step involves inputting the initial object image into the classification convolutional network included in the pre-trained image classification model to obtain a classification feature map. The pre-trained image classification model may further include fully connected layers and output layers. The classification convolutional network may include a first set of convolutional layers, a second set of convolutional layers, a third set of convolutional layers, a fourth set of convolutional layers, and a fifth set of convolutional layers. Each convolutional layer in the first set of convolutional layers may have a first preset number of convolutional kernels. Each convolutional layer in the second set of convolutional layers may have a second preset number of convolutional kernels. The second preset number may be greater than the first preset number. Each convolutional layer in the third set of convolutional layers may have a third preset number of convolutional kernels. The third preset number may be greater than the second preset number. Each convolutional layer in the fourth set of convolutional layers may have a fourth preset number of convolutional kernels. The fourth preset number may be greater than the third preset number. Each convolutional layer in the fifth set of convolutional layers may have a fifth preset number of convolutional kernels. The fifth preset quantity mentioned above can be equal to the fourth preset quantity mentioned above. The first classification model convolutional layer set mentioned above can include two convolutional layers with 3*3 kernels and 64 channels, and one pooling layer. In practice, the execution entity can extract local features of the initial object image, such as edges and textures, through the convolutional layers in the first classification model convolutional layer set. The pooling layer in the first classification model convolutional layer set can downsample these local features, reducing the amount of data while retaining the main features and preventing overfitting. The second classification model convolutional layer set mentioned above can include two convolutional layers with 3*3 kernels and 128 channels, and one pooling layer. The convolutional layers in the second classification model convolutional layer set can be used to extract depth features, such as shape and color, and then downsampled through the pooling layer. The third classification model convolutional layer set mentioned above can include four convolutional layers with 3*3 kernels and 256 channels, and one pooling layer. The aforementioned third-classification model convolutional layer set can be used to further increase the number of channels to extract deeper features, such as shape combinations, and also performs downsampling operations through pooling layers. The aforementioned fourth-classification and fifth-classification model convolutional layer sets each include four convolutional layers with 3x3 kernels and 512 channels, and one pooling layer. The purpose of both sets is to use convolutional layers with further increased channel counts to extract deeper features, such as more shape combinations. The pooling layer of the fourth-classification model convolutional layer set is connected to the fifth-classification model convolutional layer set to maximize the feature extraction capabilities of both sets.It should be noted that the number of convolution kernels in each of the above-mentioned convolutional kernel sets and the number of convolutional layers in each of the above-mentioned convolutional layer sets can be adjusted according to needs, and no specific limit is made here.

[0034] The second step involves inputting the aforementioned classification feature map into the fully connected layer of the classification model to obtain the item anomaly detection classification information. The fully connected layer of the classification model can include two fully connected layers with 4096 neurons each and one fully connected layer with 1000 neurons.

[0035] In practice, the aforementioned execution entity can integrate and further process the features extracted by the convolutional and pooling layers through the fully connected layers in the fully connected layer of the classification model. Then, it performs nonlinear transformations and maps the features to specific categories to obtain the final classification result. For example, the above-mentioned anomaly detection classification information can be judged and classified according to the following criteria:

[0036] 1. Is there any foreign object (not a plastic bag) obstructing the view? (The value is set between 0 and 1. The closer it is to 0, the higher the probability that there is no foreign object obstructing the view.)

[0037] 2. Is there a plastic bag covering the area? (The value is set between 0 and 1. The closer to 0, the higher the probability that there is no plastic bag covering the area.)

[0038] 3. Clarity (the value is set between 0 and 1, the closer to 1, the higher the probability of a clear image).

[0039] 4. Based on the three outputs of each image, set thresholds to filter images (e.g., if the clarity value is less than 0.8, it is judged as an unclear image; if the foreign object occlusion value is greater than 0.6, it is judged as an image with foreign object occlusion; if the plastic bag occlusion value is greater than 0.7, it is judged as an image with plastic bag occlusion; the rest are normal images).

[0040] The third step involves inputting the anomaly detection and classification information of the aforementioned items into the output layer of the classification model to obtain the initial item image classification result. The output layer of the classification model can be a Sigmoid output layer. This output layer can be used to perform the final classification or prediction of the input item image and output the corresponding result. For example, it can output image 1, label it "normal," and automatically categorize it into the "normal image set."

[0041] In some optional implementations of certain embodiments, the execution entity may input the initial object image into the classification convolutional network included in the pre-trained image classification model through the following steps to obtain a classification feature map:

[0042] The first step involves inputting the initial object image into the set of convolutional layers of the first classification model to obtain the first feature map of the initial object image. The information contained in this first feature map can be local features of the initial object image, such as edges and textures.

[0043] The second step involves inputting the first feature map into the convolutional layer set of the second classification model to obtain the second feature map. The information contained in the second feature map can be relatively abstract features of the initial object image, such as shape and color.

[0044] The third step involves inputting the second feature map into the convolutional layer set of the third classification model to obtain the third feature map. This third feature map may contain information such as depth features of the initial object image, including shape combinations and relationships.

[0045] The fourth step involves inputting the third feature map into the convolutional layer set of the fourth classification model to obtain the fourth feature map. This fourth feature map may contain deeper features from the initial object image, such as shape combinations and relationships.

[0046] Fifth, the fourth feature map is input into the fifth classification model's convolutional layer set to obtain the fifth feature map as the classification feature map. The information contained in this fifth feature map can be further depth features of the initial object image, such as combinations of more shapes.

[0047] Optionally, the pre-trained image classification model described above is trained through the following steps:

[0048] The first step is to obtain a pre-labeled set of initial item image samples. Each initial item image sample in this set may correspond to image category information. This image category information may include the label values ​​of each item in the corresponding image category set.

[0049] In practice, firstly, each initial item image in the above initial item image sample set is manually judged and classified, and a category (three labels) is given for each image. For example, an unclear image with a plastic bag and no other foreign objects can be labeled as (clear: 0, foreign object occlusion: 0, plastic bag occlusion: 1).

[0050] The second step is to divide the initial set of item image samples into a training set and a test set according to a preset ratio.

[0051] In practice, after completing the above annotations, the executing entity can calculate the number of images with each of the three labels being 0 and 1. For example, the initial dataset contains 1000 object images, of which 200 are clear, 800 are unclear, 700 are occluded by foreign objects, 300 are unoccluded, 500 contain plastic bags, and 500 do not contain plastic bags. By flexibly supplementing the data with a near-balanced (1:1) ratio of images from each category, an expanded model dataset is obtained.

[0052] The third step involves training the image classification model using the training and test sets described above, resulting in a fully trained image classification model. In practice, the execution entity can train the image classification model using the training set, update parameters through forward and backward propagation, evaluate it using the test set, and adjust the parameters based on the results. Once the desired outcome is achieved, the trained image classification model is saved.

[0053] Step 103: In response to determining that the initial item image classification result characterizes the image as being obscured by a plastic bag, determine that the initial item image meets the temporary retention condition.

[0054] In some embodiments, the executing entity may determine that the initial item image meets the temporary retention condition in response to determining that the initial item image classification result indicates that the item is obscured by a plastic bag. The temporary retention condition may be that the item in the image is obscured by a plastic bag and the item image is clear and free of ghosting.

[0055] Step 104: In response to determining that the initial item image meets the temporary retention condition, plastic bag feature recognition is performed on the initial item image to obtain the plastic bag feature information corresponding to the initial item image.

[0056] In some embodiments, the executing entity may, in response to determining that the initial item image meets the temporary retention conditions, perform plastic bag feature recognition on the initial item image to obtain plastic bag feature information corresponding to the initial item image. The plastic bag feature information includes plastic bag occlusion ratio information, plastic bag color, and plastic bag transparency. The plastic bag occlusion ratio information can be a plastic bag occlusion ratio. Specifically, the plastic bag occlusion ratio can be defined as the ratio of the number of pixels containing the item and showing the plastic bag to the number of pixels containing the item.

[0057] In some optional implementations of certain embodiments, the execution entity may, in response to determining that the initial item image meets the temporary retention condition, perform plastic bag feature recognition on the initial item image to obtain plastic bag feature information corresponding to the initial item image:

[0058] The first step involves inputting the initial item image that meets the aforementioned temporary retention conditions into the extraction convolutional network of the pre-trained plastic bag feature extraction model to obtain a plastic bag information feature map. The aforementioned plastic bag feature extraction model is a neural network model that takes the initial item image that meets the aforementioned temporary retention conditions as input and outputs the plastic bag feature information. The pre-trained plastic bag feature extraction model includes an average pooling layer, a fully connected layer, and an output layer. The extraction convolutional network includes a first extraction model convolutional layer, a second set of extraction model convolutional layers, a third set of extraction model convolutional layers, a fourth set of extraction model convolutional layers, and a fifth set of extraction model convolutional layers. The first extraction model convolutional layer has a sixth preset number of convolutional kernels; the second set of extraction model convolutional layers has a seventh preset number of convolutional kernels (the seventh preset number is greater than the sixth preset number); the third set of extraction model convolutional layers has an eighth preset number of convolutional kernels (the eighth preset number is greater than the seventh preset number); the fourth set of extraction model convolutional layers has a ninth preset number of convolutional kernels (the ninth preset number is greater than the eighth preset number); and the fifth set of extraction model convolutional layers has a tenth preset number of convolutional kernels (the tenth preset number is greater than the ninth preset number). The first extraction model convolutional layer may include a convolutional layer with a 7*7 kernel, 64 channels, and a stride of 2, as well as a pooling layer with a stride of 2. In practice, the aforementioned execution entity can extract low-level features of the plastic bag in the image through the first extraction model convolutional layer, such as the plastic bag's outline and texture. Simultaneously, downsampling with a stride of 2 is performed to reduce the image size. Further downsampling is then performed through the aforementioned pooling layer to prevent overfitting. The second extraction model convolutional layer set can include six convolutional layers with 3x3 kernels and 64 channels. The third extraction model convolutional layer set can include eight convolutional layers with 3x3 kernels and 128 channels. The fourth extraction model convolutional layer set can include twelve convolutional layers with 3x3 kernels and 256 channels. The fifth extraction model convolutional layer set can include twelve convolutional layers with 3x3 kernels and 512 channels. The working principle of the aforementioned second, third, fourth, and fifth sets of extraction model convolutional layers is the same as that of the various convolutional layer sets in the aforementioned classification convolutional network, namely, continuously extracting higher-level and more abstract features from the plastic bag image. Each convolutional layer can capture features at different levels; shallow layers may capture edges, simple shapes, etc., while deeper layers can capture more complex object parts and overall structures.Residual connections can be established between the output of the second extraction model convolutional layer set and the first two convolutional layers of the third extraction model convolutional layer set, between the output of the third extraction model convolutional layer set and the first two convolutional layers of the fourth extraction model convolutional layer set, and between the output of the fourth extraction model convolutional layer set and the first two convolutional layers of the fifth extraction model convolutional layer set. These residual connections allow for effective updates to the weights of both shallow and deep convolutional layers, ensuring continuous optimization of the convolutional layers and improving feature extraction performance. It should be noted that the number of convolutional kernels and the number of convolutional layers in each of the aforementioned convolutional layer sets can be adjusted according to requirements and are not specifically limited here.

[0059] The second step involves inputting the aforementioned plastic bag information feature map into the average pooling layer of the extraction model to obtain a dimensionality-reduced plastic bag information feature map. This average pooling layer can be used to convert the plastic bag information feature map into a fixed-length feature vector, further reducing the feature dimensionality, while simultaneously providing a comprehensive representation of the entire feature map, preparing it for subsequent fully connected layers.

[0060] The third step involves inputting the dimensionality-reduced feature map of the plastic bag into the fully connected layer of the extraction model to obtain the extracted plastic bag information. This fully connected layer can be a single layer with 1000 neurons, used to convert the feature vector into a distribution for each category, ultimately for prediction in tasks such as image classification.

[0061] The fourth step is to output the extracted plastic bag information through the extraction model's output layer to obtain the plastic bag feature information. This plastic bag feature information can be any information related to the plastic bag in the initial image of the item that meets the aforementioned temporary retention conditions, such as: "The plastic bag in this image is white, has 80% transparency, and an occlusion rate of 18%, so it can be retained."

[0062] In practice, the aforementioned implementing entity can automatically save initial item images with the "can be retained" label under the aforementioned temporary retention condition and clear initial item images with the "cannot be retained" label under the aforementioned temporary retention condition.

[0063] In some optional implementations of certain embodiments, the aforementioned execution entity may input the initial item image that meets the aforementioned temporary retention conditions into the extraction convolutional network included in the pre-trained plastic bag feature extraction model through the following steps to obtain the plastic bag information feature map:

[0064] The first step involves inputting the initial item image that meets the aforementioned temporary retention conditions into the first extraction model convolutional layer of a pre-trained plastic bag feature extraction model to obtain the sixth feature map of the initial item image that meets the aforementioned temporary retention conditions. The information contained in the sixth feature map may be local features of the plastic bag in the initial item image that meets the aforementioned temporary retention conditions, such as edges and textures.

[0065] The second step involves inputting the sixth feature map into the convolutional layer set of the second extraction model to obtain the seventh feature map. The information contained in the seventh feature map can be more abstract features of the plastic bag in the initial item image under the aforementioned temporary retention conditions, such as shape and color.

[0066] The third step involves inputting the seventh feature map into the convolutional layer set of the third extraction model to obtain the eighth feature map. The information contained in this eighth feature map may include depth features of the plastic bag in the initial item image under the temporarily retained conditions, such as the combination relationship between the plastic bag and the item.

[0067] Fourth, the eighth feature map is input into the convolutional layer set of the fourth extraction model to obtain the ninth feature map. The information contained in the ninth feature map may be deeper features of the plastic bag in the initial item image with the temporarily preserved conditions, such as more detailed relationships between the plastic bag and the item, such as color difference and color fusion with the item.

[0068] Fifth, the ninth feature map is input into the fifth extraction model's convolutional layer set to obtain the tenth feature map as the plastic bag information feature map. The information contained in the plastic bag information feature map can be further-depth features of the plastic bag in the initial item image under the aforementioned temporary retention conditions, such as feature information on the relationship between multiple items and multiple plastic bags.

[0069] Step 105: Determine whether the initial item image meets the retention criteria based on the plastic bag occlusion ratio information included in the plastic bag feature information.

[0070] In some embodiments, the executing entity can determine whether the initial item image meets the retention criteria based on the plastic bag occlusion ratio information included in the plastic bag feature information. This determination can be achieved by comparing the plastic bag occlusion ratio information with a pre-set occlusion ratio threshold. The occlusion ratio threshold can be set according to actual conditions; for example, if the plastic bag is white and its transparency is not less than 70%, an occlusion ratio of no more than 20% is acceptable for retention, and vice versa.

[0071] Step 106: In response to determining that the initial item image meets the retention conditions, the initial item image is determined as the acquired item image.

[0072] In some embodiments, the executing entity may determine the initial item image as the acquired item image in response to determining that the initial item image meets the retention criteria. Item images that do not meet the retention criteria will be automatically deleted. The acquired item images may be automatically stored in a server.

[0073] Optionally, the aforementioned implementing entity may also perform the following steps:

[0074] The first step, in response to the determination that the initial item image classification result indicates occlusion and ghosting, outputs an anomaly visualization prompt. This anomaly prompt can be visualized using a visualization device. The device is not specifically limited. For example, outputting the anomaly prompt could be displaying "Error, cannot be recognized".

[0075] The second step involves performing the following steps on the collected object images:

[0076] The first sub-step involves preprocessing the acquired item images to obtain preprocessed item images. These item images can include food images or dish images, without specific limitations. The preprocessing may include adjusting the acquired item images to fit the dimensions of subsequent inputs.

[0077] The second sub-step involves inputting the preprocessed item image into the feature embedding sub-network of the item recognition model to obtain a feature recognition map. The item recognition model includes a feature embedding sub-network, a relation learning sub-network, and an output layer. This item recognition model can be a neural network model that takes the preprocessed item image as input and item recognition information as output. The feature embedding sub-network can include at least one convolutional layer with a 7x7 kernel, and two or more of these convolutional layers can form a residual block. For example, when the preprocessed item image is input into the residual block, it first passes through a convolutional layer with a 3x3 kernel, resulting in 64 output channels. Then, batch normalization and ReLU activation are performed, followed by another 3x3 convolutional layer, again with 64 output channels. This output is then element-wise added to the input feature map to achieve residual connections. This helps address the gradient vanishing and degradation problems that occur as the network deepens, allowing the network to be trained deeper and extract higher-level features. The relation learning sub-network can include a graph neural network (GNN). The aforementioned graph neural network can be a data structure composed of nodes (vertices) and edges, capable of representing many complex relational data. In practice, the aforementioned execution entity can receive the preprocessed object image through the aforementioned feature embedding subnetwork and transform the high-dimensional, redundant image data into low-dimensional, compact feature vectors rich in semantic information, thus obtaining a feature recognition map. The aforementioned relation learning subnetwork can be a neural network that mines the intrinsic relationships between different features in the feature recognition map. For example, in an image containing multiple fruits, the feature embedding subnetwork extracts the shape and color features of each fruit, while the relation learning subnetwork analyzes logical relationships such as "the red circular object is more likely an apple" and "the yellow elongated object is most likely a banana," allowing the model to not only know which isolated features exist in the image but also understand how they are related and combined, thereby improving recognition accuracy.

[0078] The third sub-step involves fusing the aforementioned feature recognition map to obtain a fused feature recognition map. This fusion can be achieved by element-wise addition of the feature maps by the residual module. The aforementioned feature map can include edges, textures, etc., in the object image.

[0079] The fourth sub-step involves inputting the fused feature recognition map into the relation learning sub-network to obtain relation learning information. This relation learning information can include the relationship between item category and price, for example, "apple" corresponds to "1 yuan".

[0080] The fifth sub-step involves inputting the aforementioned relationship learning information into the output layer to obtain item recognition information. For example, the output might be "Item type: Apple, Price: 1 yuan".

[0081] The sixth sub-step involves visualizing the aforementioned item identification information. The device used for this visualization process can be the same as the device used to visualize the aforementioned anomaly alert information in the first step.

[0082] The first and second steps and related content described above, as an inventive point of this disclosure, solve the technical problem of "lack of utilization of cleaned item data". The factors leading to this lack of utilization are often as follows: currently, most data cleaning methods use the filtered item image data primarily for subsequent model training, without further utilization, resulting in low utilization. Solving these factors can improve the utilization rate of the filtered item images. To achieve this, firstly, the first and second steps and related content described above provide a method for recognizing filtered item image data and provide relevant information to the user through visualization. Therefore, the filtered item images are further and effectively utilized, thereby improving the utilization rate of the filtered item image data.

[0083] Optionally, the aforementioned implementing entity may also perform the following steps:

[0084] The first step involves converting the initial item images, which are classified as unclear, into an array of item images to obtain an array-like representation of the images. Image sharpness can be determined using PSNR, a widely used objective metric for measuring image quality. PSNR measures the degree of image distortion by calculating the mean squared error (MSE) between the original image and the image to be evaluated. In practice, the entity performing the conversion can use techniques to transform the unclear item images from a file format into a computer-recognizable and processable digital format. This conversion technique could involve first reading the images using Python's OpenCV library, and then using the `cv2.imread()` function to convert the unclear item images into an array-like representation of the images.

[0085] The second step is to perform grayscale conversion on the array-formatted item images to obtain the grayscale-converted item images. In practice, the execution entity can perform grayscale conversion on the item images using a grayscale conversion function. This grayscale conversion function can be the `cv2.cvtColor()` function in OpenCV.

[0086] The third step involves performing the following steps on the grayscale-converted image of the item:

[0087] The first sub-step involves dividing the grayscale-converted object image into a predetermined number of non-overlapping regions, resulting in a segmented object image. This predetermined number can be set according to the image size and actual needs; for example, the grayscale-converted object image can be divided into multiple 10x10 pixel blocks. In practice, the executing entity can segment the grayscale-converted object image using a segmentation algorithm. This segmentation algorithm can include, but is not limited to, superpixel-based segmentation algorithms and image content-based adaptive segmentation.

[0088] The second sub-step generates the standard deviation of pixel values ​​in each non-overlapping region of the segmented object image, resulting in a standard deviation set. This standard deviation reflects the dispersion of pixel values ​​within the corresponding region, providing a data basis for subsequent noise level estimation. In practice, the execution entity can calculate the standard deviation of pixel values ​​in each non-overlapping region using a standard deviation function. This standard deviation function can be the `np.std()` function from Python's numpy library.

[0089] The third sub-step generates the average of the standard deviations in the aforementioned standard deviation set, yielding a noise level estimate. This noise level estimate can be a numerical indicator of the degree of noise in an image. For example, a landscape photo taken with a mobile phone in low light might have an average standard deviation of 8.6 for each pixel block, meaning a noise level estimate of 8.6. This indicates that the photo is heavily affected by noise and may contain a significant amount of noise.

[0090] The fourth step involves generating the image gradients of each pixel in the grayscale-converted image of the object, resulting in a set of image gradients along a preset direction. This set of gradients includes the image gradients of each pixel along the preset direction. The image gradient can be the rate of change and direction of change of pixel values ​​in the grayscale-converted image. The preset direction can be the horizontal (x-direction) and vertical (y-direction) of the current pixel. In practice, the execution entity can calculate the image gradient of each pixel using the Sobel operator. For example, the Sobel operator can be implemented using the cv2.Sobel() function in the OpenCV library. The horizontal gradient of a pixel in the x-direction can be denoted as gradient_x, and the vertical gradient in the y-direction as gradient_y.

[0091] The fifth step involves generating the gradient magnitude values ​​corresponding to each pixel in the grayscale-converted object image based on the image gradient set in the preset direction, thus obtaining a gradient magnitude set. The gradient magnitude can be a numerical value reflecting the degree of change in local pixel values ​​within the image. In practice, the executing entity can use a preset calculation method and the image gradient calculation to obtain the corresponding gradient magnitude. The gradient magnitude can be denoted as `gradient_magnitude`. The preset calculation method can be to sum the squares of `gradient_x` and `gradient_y` and then take the square root.

[0092] Step 6: Based on the aforementioned gradient magnitude set and a pre-set gradient magnitude threshold, the grayscale-converted object image is divided into different neighborhood blocks. These neighborhood blocks can be non-overlapping regions in the grayscale-converted object image. The pre-set gradient magnitude threshold can be a value used to distinguish different image feature regions; for example, the pre-set gradient magnitude threshold can be 50. In practice, firstly, the executing entity can traverse each pixel in the grayscale-converted object image to obtain its gradient magnitude. Then, each gradient magnitude is compared with the pre-set gradient magnitude threshold. Next, based on the comparison result, the coordinates of each pixel are added to a pre-created list of smooth regions, edges, or texture regions. Finally, the pixels corresponding to the coordinates of pixels in the same region are combined to form the aforementioned different neighborhood blocks.

[0093] Step 7: Based on the aforementioned neighboring blocks, determine the set of similar blocks. In practice, the execution entity can use a difference algorithm to calculate the difference in pixel values ​​within each neighboring block to determine similarity. This difference algorithm can be a hash table-based search method. For example, first, the zlib library can be used to generate the hash values ​​of each neighboring block. Then, the hash values ​​of each neighboring block are stored in a pre-created hash table. Finally, blocks with the same or similar hash values ​​(e.g., a difference less than 0.2) are searched in the pre-created hash table, and these blocks with similar hash values ​​are identified as similar blocks.

[0094] Step 8: Generate the structure tensor components for each pixel of the grayscale-converted object image based on the image gradient along the preset direction, thus obtaining a set of structure tensor components. Each structure tensor component can have three values, namely... , and In practice, the aforementioned execution entity can generate the aforementioned structural tensor components using a preset gradient calculation method and the aforementioned image gradient. Specifically, the preset gradient calculation method can calculate the aforementioned three values ​​using the following methods: It can be the square of the gradient_x of the current pixel. It can be the square of the gradient_y mentioned above. It can be the product of gradient_x and gradient_y.

[0095] Step nine involves generating the structure tensor matrix for each pixel in the grayscale-converted object image based on the aforementioned structure tensor component set, thus obtaining a structure tensor matrix set. This structure tensor matrix comprehensively reflects the local structural features of the image, capturing the distribution patterns of pixels and characteristics such as edges and textures. In practice, the execution entity can generate the structure tensor matrix for each pixel using the following formula. .

[0096]

[0097] in, It can represent pixel coordinates in an image. It is the neighborhood of the current pixel. The neighborhood can be a local area centered on the current pixel. The local area can include a preset number (e.g., 60) of pixels around the current pixel. It can be Gaussian weights, It can be the first Okay, number The Gaussian weights of the pixels in the column. The above could be the first... Okay, number Column of pixels . It can be the first Okay, number Column of pixels The above. It can be the first Okay, number Column of pixels The aforementioned structural tensor matrix comprehensively reflects the local structural features of the image, capturing the distribution patterns of pixels and characteristics such as edges and textures.

[0098] Step 10: Generate numerical values ​​of the degree of difference between the structural tensor matrices corresponding to each similar block in the aforementioned set of similar blocks using a matrix difference algorithm, thus obtaining a similarity value set. The matrix difference algorithm can calculate the Euclidean distance between the elements within the structural tensor matrices corresponding to each similar block. In practice, firstly, the executing entity can use the Euclidean distance formula to generate a structural similarity metric between the structural tensor matrices corresponding to each pair of similar blocks. Then, the structural similarity metrics between each pair of similar blocks can be aggregated to form the aforementioned similarity metric set.

[0099] Step 11: Assign weights to each of the aforementioned similar blocks based on the similarity value set and the pre-defined weight allocation information to obtain a preliminary weight allocation information set. The pre-defined weight allocation information can be linear information based on structural similarity metrics. This linear information based on structural similarity metrics can assume that blocks with more similar structures contribute more to the current pixel restoration and are assigned higher weights. The preliminary weight allocation information set can include the initial weight value corresponding to each similar block. In practice, the executing entity can first define a weight allocation method. This weight allocation method can be a threshold-based allocation method. The threshold can be set as needed. For example, an empty dictionary `weight_dict` is initialized to store the weights of each similar block. For each pair of similar blocks, the weight is determined based on its structural similarity metric value `d`. If `d` is less than a set small threshold (e.g., 0.1, indicating very similar structures), a higher initial weight value, such as 0.8, is assigned. If `d` is greater than a set large threshold (e.g., 0.5, indicating significant structural differences), a lower initial weight value, such as 0.2, is assigned. Weights between these two values ​​can be determined through linear interpolation. Then, the weights of each similar block are determined using the weight allocation method described above. Finally, the obtained weight values ​​can be stored, with the position coordinates of each similar block in the image as the key and the corresponding weight value as the value, to obtain a preliminary weight allocation information set.

[0100] Step 12: Adjust the preliminary weight allocation information set based on the noise level estimate to obtain the final weight distribution information set. This final weight distribution information set may include the updated weight values ​​assigned to each similar block after adjustment based on the noise level estimate. In practice, the executing entity can adjust the preliminary weight allocation information set using a weight threshold method. This method involves setting different weight thresholds based on the noise level estimate, thereby amplifying or reducing the weights in the preliminary weight allocation information set. For example, the threshold that originally distinguished between high and low weights can be increased from 0.3 to 0.4. When the noise level is low, the threshold can be decreased (e.g., to 0.2). Then, the weights of each similar block are redistributed based on the adjusted thresholds (e.g., lower weights are assigned to blocks with higher noise level estimates, and higher weights are assigned to blocks with lower noise level estimates), updating the preliminary weight allocation information set to obtain the final weight distribution information set.

[0101] Step 13: Based on the aforementioned set of similar blocks and the final determined weight distribution, the grayscale-converted object image is repaired to obtain the repaired object image. In practice, for each pixel in the grayscale-converted object image, the execution entity can generate the repaired pixel value of each pixel in the grayscale-converted object image according to each similar block and its corresponding weight using a weight algorithm, thus obtaining the preliminary repaired object image. The weight algorithm can be a non-local mean filtering algorithm. The data type is numpy.ndarray, with the same shape as the input. The preliminary repaired object image can be an image that has repaired the blurred or ghosting parts of the initial object image, which was characterized as unclear.

[0102] Step fourteen: In response to the fact that the repaired item image meets the above retention conditions, the repaired item image is retained. In practice, the executing entity can also determine whether the repaired item image meets the above retention conditions using an image sharpness determination method. If it does, the repaired item image is automatically stored. If it does not, it is discarded. The image sharpness determination method can also be PSNR.

[0103] The aforementioned steps one through fourteen and related content, as an inventive point of this disclosure, solve the technical problem of "lacking a solution for further processing of acquired unclear object images." The factors leading to the lack of a solution for further processing acquired unclear object images are often as follows: Currently, most data cleaning methods directly discard acquired object images characterized as unclear instead of performing further processing, resulting in the loss of many acquired object images. Solving these factors can further increase the number of usable acquired object images. To achieve this effect, firstly, the aforementioned steps one through fourteen and related content provide a solution for repairing unclear object images. This involves traversing each pixel of the image, dynamically dividing regions, and calculating and assigning corresponding weights to repair afterimages and blurred areas in the image. Thus, while ensuring image quality, the number of usable acquired object image data is further increased.

[0104] The above embodiments of this disclosure have the following beneficial effects: Through the image acquisition method for weighing equipment disclosed herein, the retention conditions of the acquired image can be automatically set according to a downstream model (such as item recognition) by predicting the plastic bag information in an image containing a plastic bag. This allows for flexible selection of images containing plastic bags but with high usability. Specifically, the reason why most image acquisition methods cannot flexibly select images with high usability is that most methods do not further filter images of items obscured by plastic bags, resulting in a small number and poor quality of acquired images. Based on this, some embodiments of the image acquisition method for weighing equipment disclosed herein include: First, taking a picture of an object on the weighing equipment to obtain an initial image. This yields an image of the item to be processed. Then, the initial image is input into a pre-trained image classification model to obtain an initial image classification result. This allows for an initial coarse classification of the item using the first model. Then, in response to determining that the initial image classification result indicates plastic bag obscuration, the initial image is determined to meet the temporary retention conditions. This lays the foundation for subsequent further classification. Then, in response to determining that the initial item image meets the aforementioned temporary retention conditions, plastic bag feature recognition is performed on the initial item image to obtain plastic bag feature information corresponding to the initial item image, wherein the plastic bag feature information includes plastic bag occlusion ratio information. Then, based on the plastic bag occlusion ratio information included in the plastic bag feature information, it is determined whether the initial item image meets the retention conditions. Finally, in response to determining that the initial item image meets the aforementioned retention conditions, the initial item image is determined as the acquired item image. Thus, data cleaning of the acquired item images is completed. The above-described item image acquisition method for weighing equipment uses two deep learning models to perform data cleaning on the acquired item images and uses preset conditions to filter out usable item images. Therefore, while ensuring the quality of the acquired item images, the number of acquired item images is increased as much as possible, improving the efficiency of acquiring usable item images.

[0105] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an apparatus for acquiring images of items for weighing equipment. These apparatus embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0106] like Figure 2As shown, in some embodiments, the device 200 for acquiring images of items for weighing equipment includes: a shooting unit 201, a classification unit 202, a first determining unit 203, a second determining unit 204, a third determining unit 205, and a fourth determining unit 206. The imaging unit 201 is configured to capture images of objects on a weighing device to obtain initial object images; the classification unit 202 is configured to input the initial object images into a pre-trained image classification model to obtain initial object image classification results; the first determining unit 203 is configured to determine that the initial object images meet the temporary retention conditions in response to determining that the initial object image classification results indicate that plastic bags are obstructing the images; the second determining unit 204 is configured to perform plastic bag feature recognition on the initial object images in response to determining that the initial object images meet the temporary retention conditions, to obtain plastic bag feature information corresponding to the initial object images, wherein the plastic bag feature information includes plastic bag obstruction ratio information; the third determining unit 205 is configured to determine whether the initial object images meet the retention conditions based on the plastic bag obstruction ratio information included in the plastic bag feature information; and the fourth determining unit 206 is configured to determine the initial object images as the acquired object images in response to determining that the initial object images meet the retention conditions.

[0107] It is understandable that the units described in the device 200 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here.

[0108] The following is for reference. Figure 3 It illustrates electronic devices suitable for implementing some embodiments of the present disclosure (such as...). Figure 1 The diagram shows the structure of the computing device 101) 300. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0109] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0110] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0111] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0112] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0113] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0114] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: capture an image of an object on a weighing device to obtain an initial object image; input the initial object image into a pre-trained image classification model to obtain an initial object image classification result; in response to determining that the initial object image classification result indicates that a plastic bag occlusion exists, determine that the initial object image meets a temporary retention condition; in response to determining that the initial object image meets the temporary retention condition, perform plastic bag feature recognition on the initial object image to obtain plastic bag feature information corresponding to the initial object image, wherein the plastic bag feature information includes plastic bag occlusion ratio information; determine whether the initial object image meets the retention condition based on the plastic bag occlusion ratio information included in the plastic bag feature information; and in response to determining that the initial object image meets the retention condition, identify the initial object image as the acquired object image.

[0115] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0117] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a shooting unit, a classification unit, a first determining unit, a second determining unit, a third determining unit, and a fourth determining unit. The names of these units do not necessarily limit the specific unit; for example, the shooting unit may also be described as "a unit that takes pictures of objects on a weighing device to obtain initial images of the objects."

[0118] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0119] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for acquiring images of items for weighing equipment, comprising: Take pictures of the objects on the weighing device to obtain initial images of the objects; The initial object image is input into a pre-trained image classification model to obtain the initial object image classification result; In response to determining that the initial item image classification result indicates that the item is obscured by a plastic bag, the initial item image is determined to meet the temporary retention condition, wherein the temporary retention condition is that the item in the item image is obscured by a plastic bag and the item image is clear and without afterimages; In response to determining that the initial item image meets the temporary retention condition, plastic bag feature recognition is performed on the initial item image to obtain plastic bag feature information corresponding to the initial item image, wherein the plastic bag feature information includes plastic bag occlusion ratio information; Based on the plastic bag occlusion ratio information included in the plastic bag feature information, it is determined whether the initial item image meets the retention conditions. The determination of whether the initial item image meets the retention conditions is achieved by comparing the plastic bag occlusion ratio information with a preset occlusion ratio threshold. In response to determining that the initial item image satisfies the retention condition, the initial item image is determined as the acquired item image; In response to determining that the initial item image classification result indicates occlusion and afterimage, an abnormal visual prompt message is output; The following steps are performed on the acquired image of the object: The collected object images are preprocessed to obtain preprocessed object images; The preprocessed item image is input into the feature embedding subnetwork of the item recognition model to obtain a feature recognition map. The item recognition model includes a feature embedding subnetwork, a relation learning subnetwork, and an output layer. The feature recognition map is fused to obtain a fused feature recognition map; The fused feature recognition map is input into the relation learning sub-network to obtain relation learning information; The relationship learning information is input into the output layer to obtain the item recognition information; The item identification information is then visualized.

2. The method according to claim 1, wherein, The method further includes: Save the initial item image locally; Upload the images of each initial item saved within the preset time period to the server.

3. The method according to claim 1, wherein, The step of inputting the initial item image into a pre-trained image classification model to obtain the initial item image classification result includes: An initial image of an object is input into the classification convolutional network of the pre-trained image classification model to obtain a classification feature map. The pre-trained image classification model further includes a fully connected layer and an output layer. The classification convolutional network includes a first set of classification model convolutional layers, a second set of classification model convolutional layers, a third set of classification model convolutional layers, a fourth set of classification model convolutional layers, and a fifth set of classification model convolutional layers. Each convolutional layer in the first set of classification model convolutional layers has a first preset number of convolutional kernels. Each convolutional layer in the second set of classification model convolutional layers has a second preset number of convolutional kernels, which is greater than the first preset number. Each convolutional layer in the third set of classification model convolutional layers has a third preset number of convolutional kernels, which is greater than the second preset number. Each convolutional layer in the fourth set of classification model convolutional layers has a fourth preset number of convolutional kernels, which is greater than the third preset number. Each convolutional layer in the fifth set of classification model convolutional layers has a fifth preset number of convolutional kernels, which is equal to the fourth preset number. The classification feature map is input into the fully connected layer of the classification model to obtain the item anomaly detection classification information. The anomaly detection and classification information of the items is input into the output layer of the classification model to obtain the initial item image classification result.

4. The method according to claim 3, wherein, The process of inputting the initial object image into the pre-trained image classification model, including the classification convolutional network, to obtain a classification feature map includes: The initial item image is input into the first set of convolutional layers of the classification model to obtain the first feature map of the initial item image; The first feature map is input into the set of convolutional layers of the second classification model to obtain the second feature map; The second feature map is input into the set of convolutional layers of the third classification model to obtain the third feature map; The third feature map is input into the set of convolutional layers of the fourth classification model to obtain the fourth feature map; The fourth feature map is input into the fifth classification model convolutional layer set to obtain the fifth feature map as the classification feature map.

5. The method according to claim 3, wherein, The pre-trained image classification model is obtained through the following steps: Obtain a pre-labeled initial item image sample set, wherein each initial item image sample in the initial item image sample set corresponds to image category information, and the image category information includes the label values ​​of each item in the corresponding image category set; The initial set of item image samples is divided into a training set and a test set according to a preset ratio; The image classification model is trained using the training set and the test set to obtain a trained image classification model.

6. The method according to claim 1, wherein, In response to determining that the initial item image meets the temporary retention condition, the method performs plastic bag feature recognition on the initial item image to obtain plastic bag feature information corresponding to the initial item image, including: The initial item image that meets the temporary retention condition is input into the extraction convolutional network of the pre-trained plastic bag feature extraction model to obtain the plastic bag information feature map. The pre-trained plastic bag feature extraction model further includes an extraction model average pooling layer, an extraction model fully connected layer, and an extraction model output layer. The extraction convolutional network includes a first extraction model convolutional layer, a second extraction model convolutional layer set, a third extraction model convolutional layer set, a fourth extraction model convolutional layer set, and a fifth extraction model convolutional layer set. The first extraction model convolutional layer has a sixth preset number of convolutional kernels. The second extraction model convolutional layer set has a seventh preset number of convolutional kernels, which is greater than the sixth preset number. The third extraction model convolutional layer set has an eighth preset number of convolutional kernels, which is greater than the seventh preset number. The fourth extraction model convolutional layer set has a ninth preset number of convolutional kernels, which is greater than the eighth preset number. The fifth extraction model convolutional layer set has a tenth preset number of convolutional kernels, which is greater than the ninth preset number. The plastic bag information feature map is input into the average pooling layer of the extraction model to obtain the dimensionality-reduced plastic bag information feature map. The reduced feature map of the plastic bag information is input into the fully connected layer of the extraction model to obtain the extracted plastic bag information; The extracted information from the plastic bag is output through the extraction model output layer to obtain the plastic bag feature information.

7. The method according to claim 6, wherein, The step of inputting the initial item image that meets the temporary retention condition into the pre-trained plastic bag feature extraction model includes an extraction convolutional network to obtain a plastic bag information feature map, including: The initial item image that meets the temporary retention condition is input into the first extraction model convolutional layer of the pre-trained plastic bag feature extraction model to obtain the sixth feature map of the initial item image that meets the temporary retention condition. The sixth feature map is input into the second extraction model convolutional layer set to obtain the seventh feature map; The seventh feature map is input into the set of convolutional layers of the third extraction model to obtain the eighth feature map; The eighth feature map is input into the set of convolutional layers of the fourth extraction model to obtain the ninth feature map; The ninth feature map is input into the fifth extraction model convolutional layer set to obtain the tenth feature map as the plastic bag information feature map.

8. An image acquisition device for weighing equipment, comprising: The imaging unit is configured to photograph objects on the weighing device to obtain initial images of the objects. The classification unit is configured to input the initial object image into a pre-trained image classification model to obtain the initial object image classification result; The first determining unit is configured to determine, in response to determining that the initial item image classification result indicates that the item is obscured by a plastic bag, that the initial item image meets a temporary retention condition, wherein the temporary retention condition is that the item in the item image is obscured by a plastic bag and the item image is clear and without afterimages. The second determining unit is configured to, in response to determining that the initial item image meets the temporary retention condition, perform plastic bag feature recognition on the initial item image to obtain plastic bag feature information corresponding to the initial item image, wherein the plastic bag feature information includes plastic bag occlusion ratio information; The third determining unit is configured to determine whether the initial item image meets the retention condition based on the plastic bag occlusion ratio information included in the plastic bag feature information, wherein the determination of whether the initial item image meets the retention condition is achieved by comparing the plastic bag occlusion ratio information with a preset occlusion ratio threshold. The fourth determining unit is configured to determine the initial item image as the acquired item image in response to determining that the initial item image satisfies the retention condition; The fifth unit is configured to output an abnormal visual prompt message in response to determining that the initial item image classification result represents occlusion and afterimage; The execution unit is configured to perform the following steps on the acquired image of the object: The collected object images are preprocessed to obtain preprocessed object images; The preprocessed item image is input into the feature embedding subnetwork of the item recognition model to obtain a feature recognition map. The item recognition model includes a feature embedding subnetwork, a relation learning subnetwork, and an output layer. The feature recognition map is fused to obtain a fused feature recognition map; The fused feature recognition map is input into the relation learning sub-network to obtain relation learning information; The relationship learning information is input into the output layer to obtain the item recognition information; The item identification information is then visualized.

9. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images