Method for weighing an item and food weighing system
By using deep learning regression networks and image data annotation, the problems of low efficiency and inaccuracy in weighing items are solved, achieving efficient and accurate weighing without electronic scales or manual operation, and applicable to various container conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JUNZHENG NETWORK TECH CO LTD
- Filing Date
- 2023-02-15
- Publication Date
- 2026-05-19
AI Technical Summary
Current technologies for weighing items are inefficient and inaccurate, especially when the electronic scale is tilted or there is interference, manual weighing cannot guarantee accuracy.
A deep learning regression network is used to calculate the weight of items in a container by acquiring and labeling image data. The deep learning regression network is used to calculate the weight of the items loaded in the container based on the first and second labeled image data. Image recognition and depth estimation are performed by combining an encoder-decoder algorithm model and a convolutional neural network.
It achieves accurate weighing of items even when the container is fixed or tilted, without the need for electronic scales or manual operation, reducing tool and labor costs, and is highly resistant to interference.
Smart Images
Figure CN117824802B_ABST
Abstract
Description
Technical Field
[0001] This manual relates to the field of deep learning technology, and in particular to a method for weighing items and a food weighing system. Background Technology
[0002] Currently, daily weighing mostly uses electronic scales and other weighing tools, which require manual tare and weighing. Manual weighing is inefficient and labor-intensive. Moreover, when the electronic scale is tilted or the object being weighed is supported by a bracket, the weight measurement becomes inaccurate.
[0003] There is currently no effective solution to the above problems. Summary of the Invention
[0004] This specification provides a method for weighing articles and a food weighing system to solve the problems of low efficiency and inaccuracy caused by manual weighing in daily life in the prior art.
[0005] This specification provides an embodiment of a method for weighing an article, including:
[0006] Acquire first labeled image data; the first labeled image data includes the distance of each pixel in the first image data from the image acquisition device; the first image data includes image data acquired by the image acquisition device when the container is empty;
[0007] Acquire second image data, annotate the second image data to obtain second annotated image data; the second image data includes image data acquired by the image acquisition device when the container is loaded with items; the second annotated image data includes the distance of the position of each pixel in the second image data from the image acquisition device.
[0008] Using a deep learning regression network, the weight of the items loaded in the container is calculated based on the first labeled image data and the second labeled image data.
[0009] In one embodiment, after calculating the weight of the items loaded in the container based on the first labeled image data and the second labeled image data using a deep learning regression network, the method further includes:
[0010] An item addition instruction is generated if the weight of the item is less than a first preset weight; or,
[0011] The calculated weight of the item is sent to the server, which generates an item addition instruction if the weight of the item is less than a first preset weight.
[0012] In one embodiment, the second image data is annotated to obtain second annotated image data, including:
[0013] The second image data is identified to obtain the target region; the target region includes the area where there are items in the container.
[0014] The distance of each pixel in the target area from the image acquisition device is determined to obtain the second labeled image data.
[0015] In one embodiment, the second image data is annotated to obtain second annotated image data, including:
[0016] The second image data is identified to obtain the target region; the target region includes the area where there are items in the container.
[0017] The distance of each pixel in the target area from the image acquisition device is determined to obtain the second labeled image data.
[0018] In one embodiment, the second image data is annotated using a pre-built encoder-decoder algorithm model to obtain second annotated image data, including:
[0019] The second image data is encoded to obtain the first feature map;
[0020] The first feature map is subjected to hollow space pyramid pooling to obtain the second feature map;
[0021] The second feature map is upsampled to obtain a mask map of the target region; the second feature map is decoded to obtain a second depth map corresponding to the second image data; the target region includes the area where an item exists in the container;
[0022] The depth map of the target region is determined based on the mask map of the target region and the second depth map.
[0023] In one embodiment, using a deep learning regression network, the weight of the items loaded in the container is calculated based on the first labeled image data and the second labeled image data, including:
[0024] Determine the target distance matrix based on the first labeled image data and the second labeled image data;
[0025] The target distance matrix is input into a deep learning regression network to obtain the weight of the items loaded in the container.
[0026] In one embodiment, the deep learning regression network uses the MobileNetv2 network as the backbone network; the deep learning regression network uses smoothL1Loss as the loss function of the regression algorithm.
[0027] This specification also provides an embodiment of a weighing device for an article, including:
[0028] The acquisition module is used to acquire first labeled image data; the first labeled image data includes the distance of each pixel in the first image data from the image acquisition device; the first image data includes image data acquired by the image acquisition device when the container is empty;
[0029] An annotation module is used to acquire second image data, annotate the second image data, and obtain second annotated image data; the second image data includes image data acquired by the image acquisition device when the container is loaded with items; the second annotated image data includes the distance of the position of each pixel in the second image data from the image acquisition device.
[0030] The weighing module is used to calculate the weight of the items loaded in the container based on the first labeled image data and the second labeled image data using a deep learning regression network.
[0031] This specification also provides a computer device, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the steps of the article weighing method described in any of the above embodiments.
[0032] This specification also provides a computer-readable storage medium storing computer instructions that, when executed, implement the steps of the article weighing method described in any of the above embodiments.
[0033] This specification also provides an embodiment of a food weighing system, including:
[0034] The device includes a first image acquisition device for acquiring first image data and second image data. The first image data includes image data acquired when the food box is empty; the second image data includes image data acquired by the image acquisition device when the food box is filled with food. The device is also used to annotate the first image data to obtain first annotated image data; to annotate the second image data to obtain second annotated image data; and to calculate the weight of the food in the food box based on the first and second annotated image data using a deep learning regression network. The first annotated image data includes the distance of each pixel in the first image data from the image acquisition device; the second annotated image data includes the distance of each pixel in the second image data from the image acquisition device.
[0035] The server is used to receive the calculated weight of the food sent by the device and determine whether to generate a food addition instruction based on the weight of the food.
[0036] A food adding mechanism is used to receive a food adding instruction sent by the server and add food to the food box in response to the food adding instruction.
[0037] In one embodiment, the food adding mechanism includes a grain bin with graduations for measuring the amount of food remaining in the food adding device;
[0038] Correspondingly, the device also includes: a second image device for acquiring third image data, the third image data including image data of the food adding device;
[0039] Correspondingly, the device is also used to identify the third image data in order to determine the amount of remaining food in the food adding device;
[0040] Correspondingly, the server is also used to receive the remaining food quantity sent by the device, and to generate a notification message when the remaining food quantity meets a preset condition;
[0041] Accordingly, the system also includes an operation and maintenance client, which is used to receive the notification information sent by the server.
[0042] In this embodiment of the specification, a method for weighing an item is provided, which can acquire first labeled image data; the first labeled image data includes the distance of each pixel in a plurality of pixels in the first image data from the image acquisition device; the first image data includes image data acquired by the image acquisition device when the container is empty; acquire second image data, label the second image data to obtain second labeled image data; the second image data includes image data acquired by the image acquisition device when the container is loaded with an item; the second labeled image data includes the distance of each pixel in a plurality of pixels in the second image data from the image acquisition device; and use a deep learning regression network to calculate the weight of the item loaded in the container based on the first labeled image data and the second labeled image data. In the above scheme, after obtaining the first labeled image data when the container is empty and the second labeled image data when the container is loaded with items, the weight of the items in the container can be calculated using a deep learning regression network based on the distance data in the first labeled image data and the distance data in the second labeled image data. This eliminates the need for weighing tools such as electronic scales and manual operation, reducing tool and labor costs. Furthermore, the weight of the items in the container can be accurately measured even when the container is fixed or tilted, demonstrating high accuracy and strong anti-interference capabilities. Attached Figure Description
[0043] The accompanying drawings, which are included to provide a further understanding of this specification and form part of it, do not constitute a limitation thereof. In the drawings:
[0044] Figure 1 A flowchart of an article weighing method according to one embodiment of this specification is shown;
[0045] Figure 2 A flowchart of an article weighing method according to one embodiment of this specification is shown;
[0046] Figure 3 A schematic diagram of an article weighing device according to one embodiment of this specification is shown;
[0047] Figure 4 A schematic diagram of a computer device according to one embodiment of this specification is shown;
[0048] Figure 5 A schematic diagram of a food weighing system according to one embodiment of this specification is shown. Detailed Implementation
[0049] The principles and spirit of this specification will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement this specification, and are not intended to limit the scope of this specification in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.
[0050] Those skilled in the art will recognize that the embodiments described in this specification can be implemented as a system, apparatus, method, or computer program product. Therefore, the disclosure of this specification can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.
[0051] This specification provides a method for weighing an item. The method described in this specification can be applied to a computer device or software running on that device. First image data can be acquired. The first image data may include image data acquired by an image acquisition device when the container is empty. The image acquisition device may be a monocular camera, a binocular camera, a LiDAR depth imaging device, a computer stereo vision imaging device, etc. The container is a vessel used to hold items. The items may be granular solid items or liquid items, etc. When the container is empty, the first image data can be acquired by the image acquisition device.
[0052] In one embodiment, when the image acquisition device is a lidar depth imaging device, a computer stereo vision imaging device, or the like, a depth map can be directly acquired. The depth map can include distance data from the image acquisition device to the position of each pixel in the first image data. Since the positions of the camera and the food container are relatively fixed, the area corresponding to the container in the first image data is also fixed. Therefore, the distance of each pixel in the multiple pixels corresponding to the container in the first image data to the image acquisition device can be obtained.
[0053] In one embodiment, when the image acquisition device is a monocular camera, the first image data can be input into a pre-built encoding-decoding algorithm model to obtain the distance data of the position of each pixel in the first image data from the image acquisition device.
[0054] In one embodiment, since the positions of the camera and the food container are relatively fixed, the region corresponding to the container in the first image data is also fixed. Therefore, after the container region is manually or AI-calibrated for the first time, the region corresponding to the container in the first image data can be determined. In another embodiment, an image recognition model can be pre-trained to identify the region where the container is located from the first image data.
[0055] In one embodiment, since the positions of the camera and the food container are relatively fixed, once the position and shape parameters of the camera and suitable position and shape parameters are known, the distance data of each pixel in the multiple pixels corresponding to the container in the first image data from the image acquisition device can be calculated.
[0056] The computer device can also acquire second image data. The second image data includes image data captured by the image acquisition device when the container is loaded with items. In one embodiment, the second image data can be acquired in real time. In another embodiment, the acquisition of the second image data can be triggered under specific conditions.
[0057] The second image data can be annotated to obtain second annotated image data. The second annotated image data includes the distance of each pixel in the second image data from the image acquisition device.
[0058] In one embodiment, the second image data can be input into a pre-built encoder-decoder model to obtain the second labeled image data.
[0059] After obtaining the first and second labeled image data, a deep learning regression network can be used to calculate the weight of the items loaded in the container based on the first and second labeled image data.
[0060] In one application scenario, the item weighing method described in this specification can be applied to outdoor cat houses, etc. Cat houses typically contain a cat food container and a camera. After the cat house is set up, the camera can acquire first image data and send it to a computer device. This computer device can be a client or a server. The computer device can annotate the first image data to obtain first annotated image data. Subsequently, the cat food can be weighed in real time, allowing for refilling when the food is running low. In one embodiment, upon detecting a cat entering the cat house, second image data can be acquired in real time to determine the current weight of the cat food in the container. If the weight of the cat food is less than a first preset weight, a cat food refill command can be generated and sent to a cat food refilling device. The cat food refilling device can be equipped with a switch; upon receiving the cat food refill command, the switch can be turned on to refill the cat food. During the cat food refilling process, second image data can be acquired in real time. If the weight of the cat food calculated based on the second image data is greater than or equal to the second preset weight, a stop command is generated to stop refilling the cat food.
[0061] Figure 1A flowchart of a method for weighing items according to one embodiment of this specification is shown. While this specification provides method operation steps or apparatus structures as illustrated in the following embodiments or figures, more or fewer operation steps or module units may be included in the method or apparatus based on conventional or non-inventive effort. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure described in the embodiments and figures of this specification. When the method or module structure is applied in a practical device or end product, it can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed processing environment) according to the method or module structure shown in the embodiments or figures.
[0062] Specifically, such as Figure 1 As shown, one embodiment of the article weighing method provided in this specification may include the following steps:
[0063] Step S101: Obtain first labeled image data; the first labeled image data includes the distance of each pixel in the first image data from the image acquisition device; the first image data includes image data acquired by the image acquisition device when the container is empty.
[0064] The methods described in this specification can be applied to a computer device or an application running on that device. First image data can be acquired. This first image data may include image data acquired by the image acquisition device when the container is empty. The image acquisition device may be a monocular camera, a binocular camera, a LiDAR depth imaging device, a computer stereo vision imaging device, etc. The container is a vessel used to hold items. These items may be granular solid items or liquid items, etc. When the container is empty, the first image data can be acquired by the image acquisition device.
[0065] In one embodiment, when the image acquisition device is a lidar depth imaging device, a computer stereo vision imaging device, or the like, a depth map can be directly acquired. The depth map can include distance data from the image acquisition device to the position of each pixel in the first image data. Since the positions of the camera and the food container are relatively fixed, the area corresponding to the container in the first image data is also fixed. Therefore, the distance of each pixel in the multiple pixels corresponding to the container in the first image data to the image acquisition device can be obtained.
[0066] In one embodiment, when the image acquisition device is a monocular camera, the first image data can be input into a pre-built encoding-decoding algorithm model to obtain the distance data of the position of each pixel in the first image data from the image acquisition device.
[0067] In one embodiment, since the positions of the camera and the food container are relatively fixed, the region corresponding to the container in the first image data is also fixed. Therefore, after the container region is manually or AI-calibrated for the first time, the region corresponding to the container in the first image data can be determined. In another embodiment, an image recognition model can be pre-trained to identify the region where the container is located from the first image data.
[0068] In one embodiment, since the positions of the camera and the food container are relatively fixed, once the position and shape parameters of the camera and suitable position and shape parameters are known, the distance data of each pixel in the multiple pixels corresponding to the container in the first image data from the image acquisition device can be calculated.
[0069] Step S102: Acquire second image data, annotate the second image data to obtain second annotated image data; the second image data includes image data acquired by the image acquisition device when the container is loaded with items; the second annotated image data includes the distance of the position of each pixel in the second image data from the image acquisition device.
[0070] The computer device can also acquire second image data. The second image data includes image data captured by the image acquisition device when the container is loaded with items. In one embodiment, the second image data can be acquired in real time. In another embodiment, the acquisition of the second image data can be triggered under specific conditions.
[0071] The second image data can be annotated to obtain second annotated image data. The second annotated image data includes the distance of each pixel in the second image data from the image acquisition device.
[0072] Step S103: Using a deep learning regression network, calculate the weight of the items loaded in the container based on the first labeled image data and the second labeled image data.
[0073] After obtaining the first and second labeled image data, a deep learning regression network can be used to calculate the weight of the items loaded in the container based on the first and second labeled image data. After obtaining the first and second labeled image data, the thickness information of the items can be obtained, and thus the volume data of the items can be determined. Multiplying the volume data by the known density of the items yields the weight of the items in the container.
[0074] In the above embodiments, after obtaining the first labeled image data when the container is empty and the second labeled image data when the container is loaded with items, the weight of the items in the container can be calculated using a deep learning regression network based on the distance data in the first labeled image data and the distance data in the second labeled image data. This eliminates the need for weighing tools such as electronic scales and manual operation, reducing tool and labor costs. Furthermore, the weight of the items in the container can be accurately measured even when the container is fixed or tilted, demonstrating high accuracy and strong anti-interference capabilities.
[0075] In some embodiments of this specification, after calculating the weight of the items loaded in the container based on the first labeled image data and the second labeled image data using a deep learning regression network, the method may further include: generating an item addition instruction if the weight of the items is less than a first preset weight; or, sending the calculated weight of the items to a server, the server being used to generate an item addition instruction if the weight of the items is less than the first preset weight.
[0076] Specifically, after calculating the weight of the item, the device can determine whether the calculated weight is less than a first preset weight. If the weight is less than the first preset weight, the device can generate an item addition command. The device can send the item addition command to the item addition device, causing the item addition device to add the item to the container. Conversely, if the calculated weight is greater than or equal to a second preset weight, the device can generate a stop command. The device can send the stop command to the item addition device, causing the item addition device to stop adding items to the container.
[0077] Understandably, the item addition instruction can be generated by the server. Correspondingly, the device can send the calculated weight of the item to the server. The server generates the item addition instruction when the item's weight is less than a first preset weight. The server can then send the item addition instruction to the item adding device, causing the device to add the item to the container. Conversely, if the calculated weight is greater than or equal to a second preset weight, the server can generate a stop instruction. The server can then send the stop instruction to the item adding device, causing the device to stop adding items to the container.
[0078] The above method enables automatic addition and stopping of items without manual maintenance, saving costs and ensuring that the items in the container are within the preset range.
[0079] In some embodiments of this specification, annotating the second image data to obtain second annotated image data may include: identifying the second image data to obtain a target region; the target region includes an area where an item exists in a container; determining the distance of each pixel in the target region from the image acquisition device to obtain the second annotated image data.
[0080] After obtaining the second image data, it can be used for recognition to obtain the target region. In one embodiment, a trained convolutional neural network model can be used to perform image recognition or image segmentation on the second image data to obtain the target region where the item in the container is located. The target region is the area in the container where the item exists. After obtaining the target region, the distance of each pixel in the target region from the image acquisition device can be determined to obtain the second labeled image data. Through the above method, the distance of each pixel in the target region where the item is located in the second image data from the image acquisition device can be obtained.
[0081] In some embodiments of this specification, annotating the second image data to obtain second annotated image data may include: using a pre-built encoding-decoding algorithm model to annotate the second image data to obtain second annotated image data.
[0082] Specifically, the second image data can be labeled using a pre-built encoder-decoder algorithm model to obtain the second labeled image data. In one embodiment, the backbone network of the pre-built encoder-decoder algorithm model can be a ResNet network, which can achieve greater network depth and extract more detailed features compared to ordinary networks. By inputting the second image data into the pre-built encoder-decoder algorithm model in the above manner, the second labeled image data can be obtained.
[0083] In some embodiments of this specification, the second image data is annotated using a pre-built encoder-decoder algorithm model to obtain second annotated image data. This may include: encoding the second image data to obtain a first feature map; performing hollow spatial pyramid pooling on the first feature map to obtain a second feature map; upsampling the second feature map to obtain a mask map of the target region; decoding the second feature map to obtain a second depth map corresponding to the second image data; the target region includes the region where an item exists in the container; and determining the depth map of the target region based on the mask map of the target region and the second depth map.
[0084] In this embodiment, the second image data can be input into a pre-built encoder-decoder algorithm model, which can output the distance of each pixel from the camera and a mask map of the target region. The pre-built encoder-decoder algorithm model can include an encoder and a decoder. In one embodiment, the encoder process performs four 2x downsampling operations, outputting four feature maps of different sizes. ASPP (Alternating Persistent Convolution) is performed on the last feature map to increase feature diversity while preserving some resolution. After ASPP, two branches are connected: one performs two 4x upsampling operations for semantic segmentation, outputting a mask map of the target region; the other branch performs the decoder process. The decoder process corresponds to the encoder process, performing four 2x upsampling operations, which is a process of restoring the feature maps layer by layer. The restored feature maps are subtracted from the corresponding feature maps of the original encoder to extract difference features. The same operation is performed on each layer. Additionally, features of the same size are concatenated and fused. After fusion, a convolution operation is performed, and finally, the feature maps of each layer are fused to obtain the predicted depth map. Multiplying the mask image and the depth map yields a depth map that retains only the food region, which in turn gives the distance of each pixel in the target region of the second image data from the image acquisition device.
[0085] In some embodiments of this specification, using a deep learning regression network to calculate the weight of the items loaded in the container based on the first labeled image data and the second labeled image data may include: determining a target distance matrix based on the first labeled image data and the second labeled image data; and inputting the target distance matrix into the deep learning regression network to obtain the weight of the items loaded in the container.
[0086] After obtaining the first and second labeled image data, the distances in the first and second labeled image data can be subtracted from the corresponding distances in the second labeled image data to obtain the target distance matrix. The target distance matrix represents the thickness information of the object, and the target region represents the area information of the object. The target distance matrix can be input into a deep learning regression network to obtain the weight of the object loaded in the container. The weight of the object can be obtained in this way.
[0087] In some embodiments of this specification, the deep learning regression network may use MobileNetv2 as its backbone network; the deep learning regression network may use smoothL1Loss as the loss function of the regression algorithm. The deep learning regression network may use MobileNetv2 as its backbone network. SmoothL1Loss can be used as the loss function of the regression algorithm. The regression algorithm modeling process involves regressing the weight of the food based on the volume of the food region. Through the above method, the weight of the items loaded in the container can be accurately determined.
[0088] The above method will be described below with reference to a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustration of this specification and does not constitute an improper limitation of this specification.
[0089] This specification provides a method for weighing items. This specific embodiment uses the example of weighing cat food in a cat food box in a cat house for illustration. Please refer to... Figure 2 The flowchart of the article weighing method in this specific embodiment is shown. Figure 2 As shown, the method in this specific embodiment may include the following steps.
[0090] Step 1) The camera collects data on the empty food box and marks the distance of each pixel in the image from the camera.
[0091] Step 2) Collect images of the food box when it is not empty, and use the Encoder-Decoder algorithm model built based on the intelligent algorithm of convolutional neural network to identify the areas in the food box that contain food and the distance of the corresponding areas from the camera.
[0092] Algorithm implementation: The input is a single image, and the output is the distance of each pixel from the camera and a mask image of the food area.
[0093] The input algorithm model backbone is ResNet. The advantage of ResNet is that it can achieve a deeper network depth and extract more detailed features compared to ordinary networks.
[0094] The encoder process performs four 2x downsampling operations to output four different feature maps. The last feature map is then processed by ASPP, which combines dilated convolution and SPP. The purpose of this is to increase feature diversity while preserving a certain resolution.
[0095] Following ASPP are two branches. One branch performs two 4x upsampling operations for semantic segmentation, outputting a mask image of the food region. The other branch performs the Decoder process.
[0096] The Decoder process corresponds to the Encoder process, performing four 2x upsampling operations, which is essentially a process of restoring the feature map layer by layer. The restored feature map is subtracted from the corresponding feature map in the original encoder to extract the difference features. This same operation is performed on each layer. Additionally, features of the same size are concatenated and fused. After fusion, a convolution operation is performed, and finally, the feature maps from each layer are fused to obtain the predicted depth map.
[0097] Multiply the mask map and the depth map to obtain a depth map that retains only the food region.
[0098] Step 3) Subtract the corresponding distances between the empty food box state and the non-empty area to obtain a new distance matrix.
[0099] Step 4) Input the distance matrix from the previous step into the deep learning-based regression network to calculate the weight of the food.
[0100] Algorithm implementation: Input the above matrix, output the weight of the food.
[0101] Using mobileNetv2 as the backbone
[0102] SmoothL1Loss is used as the loss function for the regression algorithm.
[0103] The regression algorithm modeling process involves regressing the weight of the food based on the volume of the food region.
[0104] Step 5) Finally, upload the results to the server. If the food in the food box is less than the preset weight, food will be added automatically.
[0105] The overall matching system architecture is shown in the diagram below. It collects video data of the food in the food box using a camera, extracts the distance between food areas using algorithms, calculates the weight of the food, and uploads the results to the server.
[0106] The weighing method described in this specific embodiment can reduce the cost of weighing tools and manpower, thus saving operating costs. Even when the food container is fixed or tilted, the weight of the food can be accurately measured, demonstrating strong resistance to interference.
[0107] Based on the same inventive concept, this specification also provides an article weighing device, as described in the following embodiments. Since the principle of the article weighing device in solving the problem is similar to that of the article weighing method, the implementation of the article weighing device can refer to the implementation of the article weighing method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated. Figure 3This is a structural block diagram of an article weighing device according to an embodiment of this specification, such as... Figure 3 As shown, it includes: acquisition module 301, labeling module 302 and weighing module 303. The structure is described below.
[0108] The acquisition module 301 is used to acquire first labeled image data; the first labeled image data includes the distance of each pixel in the first image data from the image acquisition device; the first image data includes image data acquired by the image acquisition device when the container is empty.
[0109] The annotation module 302 is used to acquire second image data, annotate the second image data, and obtain second annotated image data; the second image data includes image data acquired by the image acquisition device when the container is loaded with items; the second annotated image data includes the distance of the position of each pixel in the second image data from the image acquisition device.
[0110] The weighing module 303 is used to calculate the weight of the items loaded in the container based on the first labeled image data and the second labeled image data using a deep learning regression network.
[0111] In some embodiments of this specification, the device may further include an adding module, which may be specifically used to: after calculating the weight of the item loaded in the container based on the first labeled image data and the second labeled image data using a deep learning regression network, determine whether the weight of the item is less than a first preset weight; if it is determined that the weight of the item is less than the first preset weight, generate an item adding instruction to add the item to the container.
[0112] In some embodiments of this specification, the annotation module may be specifically used to: identify the second image data to obtain a target region; the target region includes an area where an item exists in a container; determine the distance of each pixel in the target region from the image acquisition device to obtain the second annotated image data; the second annotated image data includes the distance of the position of each pixel in the target region from the image acquisition device.
[0113] In some embodiments of this specification, the annotation module may be specifically used to: annotate the second image data using a pre-built encoding-decoding algorithm model to obtain second annotated image data.
[0114] In some embodiments of this specification, the annotation module may be specifically used to: encode the second image data to obtain a first feature map; perform hollow spatial pyramid pooling on the first feature map to obtain a second feature map; perform upsampling on the second feature map to obtain a mask map of the target region; perform decoding on the second feature map to obtain a second depth map corresponding to the second image data; the target region includes the region where an item exists in the container; and determine the depth map of the target region based on the mask map of the target region and the second depth map.
[0115] In some embodiments of this specification, the calculation module may be specifically used to: determine a target distance matrix based on the first labeled image data and the second labeled image data; input the target distance matrix into a deep learning regression network to obtain the weight of the items loaded in the container.
[0116] In some embodiments of this specification, the deep learning regression network uses the MobileNetv2 network as the backbone network; the deep learning regression network uses smoothL1Loss as the loss function of the regression algorithm.
[0117] As can be seen from the above description, the embodiments of this specification achieve the following technical effects: After obtaining the first labeled image data when the container is empty and the second labeled image data when the container is loaded with items, the weight of the items in the container can be calculated using a deep learning regression network based on the distance data in the first labeled image data and the distance data in the second labeled image data. There is no need to set up weighing tools such as electronic scales, nor is there a need for manual operation, which can reduce tool costs and labor costs. At the same time, even if the container is fixed or tilted, the weight of the items in the container can be accurately measured, with high accuracy and strong anti-interference.
[0118] This specification also provides a computer device, which can be found in the following description. Figure 4 The diagram shown illustrates the computer device structure based on the article weighing method provided in the embodiments of this specification. Specifically, the computer device may include an input device 41, a processor 42, and a memory 43. The memory 43 stores processor-executable instructions. When the processor 42 executes the instructions, it implements the steps of the article weighing method described in any of the above embodiments.
[0119] In this embodiment, the input device can specifically be one of the main devices for information exchange between the user and the computer system. The input device may include a keyboard, mouse, camera, scanner, light pen, handwriting input tablet, voice input device, etc.; the input device is used to input raw data and programs for processing these data into the computer. The input device can also receive data transmitted from other modules, units, and devices. The processor can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. The memory can specifically be a memory device used to store information in modern information technology. The memory can include multiple layers; in digital systems, anything that can store binary data can be considered memory; in integrated circuits, a circuit without physical form but with storage function is also called memory, such as RAM, FIFO, etc.; in a system, a storage device with physical form is also called memory, such as a memory stick, TF card, etc.
[0120] In this embodiment, the specific functions and effects implemented by the computer device can be explained in comparison with other embodiments, and will not be repeated here.
[0121] This specification also provides a computer storage medium based on an article weighing method, wherein the computer storage medium stores computer program instructions that, when executed, implement the steps of the article weighing method described in any of the above embodiments.
[0122] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.
[0123] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer storage medium can be explained by comparison with other embodiments, and will not be repeated here.
[0124] This specification also provides an embodiment of a grain weighing system. Please refer to... Figure 5The diagram shows a schematic representation of the food weighing system in this embodiment. Figure 5 As shown, the food weighing system in this embodiment may include: device 51, server 52 and food adding mechanism 53.
[0125] The device 51 may include a first image acquisition device for acquiring first image data and second image data. The first image data includes image data acquired when the food box is empty; the second image data includes image data acquired by the image acquisition device when the food box is filled with food.
[0126] The device 51 can also be used to annotate the first image data to obtain first annotated image data; it can also be used to annotate the second image data to obtain second annotated image data; and it can also be used to calculate the weight of the food loaded in the food box based on the first annotated image data and the second annotated image data using a deep learning regression network; the first annotated image data includes the distance of the position of each pixel in the first image data from the image acquisition device; the second annotated image data includes the distance of the position of each pixel in the second image data from the image acquisition device.
[0127] Server 52 can be used to receive the calculated weight of food sent by the device and determine whether to generate a food addition instruction based on the weight of the food.
[0128] The food adding mechanism 53 can be used to receive food adding instructions sent by the server and add food to the food box in response to the food adding instructions.
[0129] In some embodiments of this specification, the food adding mechanism may include a grain bin with graduations for measuring the amount of food remaining in the food adding device.
[0130] Correspondingly, the device may also include: a second image device for acquiring third image data, wherein the third image data includes image data of the food addition device.
[0131] Correspondingly, the device can also be used to identify the third image data in order to determine the amount of remaining food in the food adding device.
[0132] Correspondingly, the server can also be used to receive the remaining food quantity sent by the device, and to generate notification information when the remaining food quantity meets preset conditions.
[0133] Accordingly, the system also includes an operations and maintenance client, which is used to receive the notification information sent by the server. Upon receiving the notification information, the operations and maintenance personnel on the client can add grain to the grain depot as quickly as possible.
[0134] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. For details, please refer to the foregoing descriptions of the relevant processing embodiments; they will not be repeated here.
[0135] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0136] Obviously, those skilled in the art will understand that the modules or steps of the embodiments described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this specification are not limited to any particular combination of hardware and software.
[0137] It should be understood that the above description is for illustrative purposes and not for limitation. Many embodiments and applications beyond the provided examples will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this specification should not be determined by reference to the above description, but rather by reference to the foregoing claims and the full scope of their equivalents.
[0138] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to the embodiments described herein by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.
Claims
1. A method for weighing an article, characterized in that, include: Obtain the first labeled image data; The first labeled image data includes the distance of each pixel in the first image data from the image acquisition device; the first image data includes image data acquired by the image acquisition device when the container is empty; Acquire second image data, annotate the second image data to obtain second annotated image data; the second image data includes image data acquired by the image acquisition device when the container is loaded with items; the second annotated image data includes the distance of the position of each pixel in the second image data from the image acquisition device. Using a deep learning regression network, the weight of the items loaded in the container is calculated based on the first labeled image data and the second labeled image data; The method involves using a deep learning regression network to calculate the weight of the items loaded in the container based on the first and second labeled image data, including: Determine the target distance matrix based on the first labeled image data and the second labeled image data; The target distance matrix is input into a deep learning regression network to obtain the weight of the items loaded in the container.
2. The method for weighing articles according to claim 1, characterized in that, After calculating the weight of the items loaded in the container based on the first and second labeled image data using a deep learning regression network, the method further includes: An item addition instruction is generated if the weight of the item is less than a first preset weight; or, The calculated weight of the item is sent to the server, which generates an item addition instruction if the weight of the item is less than a first preset weight.
3. The method for weighing articles according to claim 1, characterized in that, The second image data is annotated to obtain second annotated image data, including: The second image data is identified to obtain the target region; the target region includes the area where there are items in the container. The distance of each pixel in the target area from the image acquisition device is determined to obtain the second labeled image data.
4. The method for weighing articles according to claim 1, characterized in that, The second image data is annotated to obtain second annotated image data, including: The second image data is labeled using a pre-built encoding-decoding algorithm model to obtain the second labeled image data.
5. The method for weighing articles according to claim 4, characterized in that, The second image data is annotated using a pre-built encoder-decoder algorithm model to obtain second annotated image data, including: The second image data is encoded to obtain the first feature map; The first feature map is subjected to hollow space pyramid pooling to obtain the second feature map; The second feature map is upsampled to obtain a mask map of the target region; the second feature map is decoded to obtain a second depth map corresponding to the second image data; the target region includes the area where an item exists in the container; The depth map of the target region is determined based on the mask map of the target region and the second depth map.
6. The method for weighing articles according to claim 1, characterized in that, The deep learning regression network uses the MobileNetv2 network as its backbone network; the deep learning regression network uses smoothL1Loss as the loss function of the regression algorithm.
7. A computer device, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 6.
8. A grain weighing system, characterized in that, include: The device includes a first image acquisition device for acquiring first image data and second image data. The first image data includes image data acquired when the food box is empty; the second image data includes image data acquired by the image acquisition device when the food box is filled with food. The device is also used to annotate the first image data to obtain first annotated image data; to annotate the second image data to obtain second annotated image data; and to calculate the weight of the food in the food box based on the first and second annotated image data using a deep learning regression network. The first annotated image data includes the distance of each pixel in the first image data from the image acquisition device; the second annotated image data includes the distance of each pixel in the second image data from the image acquisition device. The step of calculating the weight of the food in the food box based on the first and second annotated image data using a deep learning regression network includes: determining a target distance matrix based on the first and second annotated image data; and inputting the target distance matrix into the deep learning regression network to obtain the weight of the food in the food box. The server is used to receive the calculated weight of the food sent by the device and determine whether to generate a food addition instruction based on the weight of the food. A food adding mechanism is used to receive a food adding instruction sent by the server and add food to the food box in response to the food adding instruction.
9. The grain weighing system according to claim 8, characterized in that, The food addition mechanism includes a grain bin with graduations for measuring the amount of food remaining in the food addition mechanism; Correspondingly, the device also includes: a second image device for acquiring third image data, the third image data including image data of the food adding mechanism; Correspondingly, the device is also used to identify the third image data in order to determine the amount of remaining food in the food adding mechanism; Correspondingly, the server is also used to receive the remaining food quantity sent by the device, and to generate a notification message when the remaining food quantity meets a preset condition; Accordingly, the system also includes an operation and maintenance client, which is used to receive the notification information sent by the server.