Image processing method and device applied to intelligent scale

Abstract

The embodiment of the invention provides an image processing method and device applied to an intelligent scale, and the method comprises the steps: carrying out the image processing of a weighed commodity through the intelligent scale, the method comprises the following steps: firstly, carrying out calibration region segmentation on a calibration image acquired for a weighing assembly of the intelligent scale to obtain region coordinates of each calibration region, generating and displaying an acquisition guide page of a target calibration region according to an image acquisition prompt and the region coordinates of the target calibration region, and further acquiring an acquired commodity image of a to-be-input commodity; and carrying out regional position verification and angle verification on the target calibration region on the commodity image, and carrying out verification response processing on the target calibration region according to a verification result, so that image processing of the to-be-input commodity is realized through the intelligent scale.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of image processing, and in particular to an image processing method and device applied to a smart scale. BACKGROUND

[0002] With the continuous development of image recognition technology and intelligent terminals, image-based commodity identification methods have begun to be widely used in supermarkets, convenience stores and self-service retail scenarios, and this process usually relies on the collection of commodity images to achieve subsequent commodity identification. In this context, the image collection process often relies on the operator placing the commodity in a specific device area and completing the shooting through the camera. With the increasing variety of commodities and the increasing demand for commodity identification efficiency and accuracy in the retail industry, the need for standardization and accuracy of commodity image collection has also increased. SUMMARY

[0003] One or more embodiments of the present specification provide an image processing method applied to a smart scale, comprising: performing calibration region segmentation on a calibration image collected by a weighing component of the smart scale to obtain region coordinates of each calibration region. According to the image collection prompt and region coordinates of a target calibration region, a collection guide page of the target calibration region is generated and displayed. A commodity image of a commodity to be entered is obtained, and the commodity image is subjected to region position verification and angle verification of the target calibration region. According to the verification result, verification response processing of the target calibration region is performed.

[0004] One or more embodiments of the present specification provide an image processing device applied to a smart scale, comprising: a region segmentation module configured to perform calibration region segmentation on a calibration image collected by a weighing component of the smart scale to obtain region coordinates of each calibration region. A page generation module configured to generate and display a collection guide page of a target calibration region according to an image collection prompt and region coordinates of the target calibration region. An image verification module configured to obtain a commodity image of a commodity to be entered, and perform region position verification and angle verification of the target calibration region on the commodity image. A response processing module configured to perform verification response processing of the target calibration region according to the verification result.

[0005] The one or more embodiments of the specification provide an image processing device applied to a smart scale, comprising: a processor; and a memory configured to store computer executable instructions which, when executed, cause the processor to: perform calibration region segmentation on a calibration image collected by a weighing component of the smart scale, and obtain region coordinates of each calibration region. According to an image collection prompt and the region coordinates of a target calibration region, a collection guide page of the target calibration region is generated and displayed. A product image of a product to be entered is obtained, and region position verification and angle verification of the target calibration region are performed on the product image. According to the verification result, verification response processing of the target calibration region is performed.

[0006] The one or more embodiments of the specification provide a computer readable storage medium for storing computer executable instructions, which, when executed, implement the following processes: performing calibration region segmentation on a calibration image collected by a weighing component of the smart scale, and obtaining region coordinates of each calibration region. According to an image collection prompt and the region coordinates of a target calibration region, a collection guide page of the target calibration region is generated and displayed. A product image of a product to be entered is obtained, and region position verification and angle verification of the target calibration region are performed on the product image. According to the verification result, verification response processing of the target calibration region is performed. BRIEF DESCRIPTION OF DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the one or more embodiments of the specification or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor; Figure 1 A schematic diagram of an image processing method implementation environment applied to a smart scale is provided for the one or more embodiments of the specification; Figure 2 An image processing method processing flowchart applied to a smart scale is provided for the one or more embodiments of the specification; Figure 3 A schematic diagram of a first collection guide page is provided for the one or more embodiments of the specification; Figure 4 A schematic diagram of a second collection guide page is provided for the one or more embodiments of the specification; Figure 5 An image processing method processing flowchart applied to a smart scale applied to an image processing scene is provided for the one or more embodiments of the specification; Figure 6A schematic diagram illustrating an embodiment of an image processing device applied to a smart scale, provided by one or more embodiments of this specification. Figure 7 This is a schematic diagram of the structure of an image processing device applied to a smart scale, provided for one or more embodiments of this specification. Detailed Implementation

[0008] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0009] The image processing method for smart scales provided in one or more embodiments of this specification is applicable to the implementation environment of smart scales. (Refer to...) Figure 1 The implementation environment includes at least: smart scale 101; Among them, the smart scale 101 is used to segment the calibration area of ​​the calibration image collected for the weighing component, generate and display the collection guidance page of the target calibration area, and acquire the product image of the product to be entered and perform image processing on the product image. In addition, the implementation environment may also include a server 102; the server 102 is used to store the product image sequence of the recorded products, and cooperate with the smart scale 101 to perform product recognition on the products to be identified. After detecting the start command of the smart scale 101, the server 102 returns the product image sequence of the recorded products to the smart scale 101.

[0010] In this implementation environment, during the image processing of the weighed goods by the smart scale, firstly, the smart scale 101 segments the calibration image acquired by the weighing component of the smart scale to obtain the coordinates of each calibration area. Based on the image acquisition prompts and coordinates of the target calibration area, it generates and displays the acquisition guidance page for the target calibration area. Subsequently, after the smart scale 101 acquires the image of the goods to be entered, it further acquires the acquired image of the goods to be entered and performs regional position and angle verification of the target calibration area on the image. Based on the verification results, it performs verification response processing of the target calibration area, thereby realizing image processing of the goods to be entered through the smart scale.

[0011] This specification provides one or more embodiments of an image processing method for smart scales, as follows: ReferenceFigure 2 The application provides an image processing method applied to a smart scale, and the method comprises steps S202 to S208.

[0012] In step S202, a calibration region of a calibration image collected by a weighing component of the smart scale is segmented to obtain region coordinates of each calibration region.

[0013] The smart scale refers to a weighing device deployed in a merchant. For example, the smart scale can be a weighing device deployed in a merchant and integrated with functions of weighing a commodity, collecting an image of the commodity, and / or identifying the commodity. The smart scale can comprise a weighing component, an image collection component, a storage component, and / or a display screen. In addition, the smart scale can further comprise other processing components, such as a communication component. For example, the weighing component can be a scale pan for carrying a commodity, the image collection component can be a component for collecting an image of the commodity, such as a camera, and the storage component can be a component for storing an image of the commodity.

[0014] The calibration image refers to a standard image obtained by performing calibration processing on a component image collected by the weighing component of the smart scale. The calibration image can be an image collected in real time by the weighing component during image collection of the smart scale on a commodity to be entered. In this case, the calibration image can be obtained by performing image collection processing and calibration processing on the weighing component by the smart scale each time the smart scale is started to collect an image of the commodity to be entered. Alternatively, the calibration image can be an image obtained by performing image collection processing and calibration processing on the weighing component by the smart scale and stored in a server or a storage module after the smart scale is started for the first time. In this case, the calibration image can be a calibration image obtained from the server after a start instruction of the smart scale is detected.

[0015] The calibration region refers to a plurality of sub-regions obtained by segmenting the calibration image based on a preset rule. The sub-regions can have a boundary range, and each sub-region can have independent region coordinates. For example, the calibration region can be a region obtained by grid segmentation of the calibration image. The region coordinates of each calibration region refer to position parameters of each calibration region in a pixel coordinate system of the calibration image, such as a center point coordinate of the calibration region, or a key point coordinate, such as a coordinate of each vertex of the calibration region.

[0016] In practice, during the image processing of the smart scale, in order to support the diversity of subsequent image acquisition and ensure that the different placement positions of the goods to be entered on the weighing component can correspond to the specific calibration area, the image acquisition component of the smart scale can be used to acquire images of the weighing component of the smart scale to obtain calibration images. On this basis, the calibration images can be segmented into calibration areas according to preset rules to obtain each calibration area, and the area coordinates of each calibration area can be obtained further.

[0017] In the specific execution process, in order to eliminate the influence of image distortion on the calibration region segmentation and improve the accuracy of calibration region segmentation, the calibration image can be image corrected; and, in order to improve the regional positioning accuracy of the calibration region, the regional coordinates of each calibration region can be determined by inverse mapping calculation. In one optional implementation of this embodiment, the calibration image acquired by the weighing component of the smart scale is segmented into calibration regions to obtain the region coordinates of each calibration region, including: The calibration image is subjected to image correction mapping to obtain the corrected image, and the calibration region is segmented into each segmented region. Based on the mapping parameters between the calibration image and the corrected image, each segmented region is inversely mapped to obtain each calibration region, and the position coordinates of the key points in each calibration region are calculated.

[0018] Specifically, in the process of segmenting the calibration image to obtain the coordinates of each calibration region, based on the image acquisition of the weighing component of the smart scale to obtain the calibration image, the calibration image is first corrected and mapped to obtain the corrected image. Then, the calibration image is segmented according to preset rules to obtain each segmented region. Furthermore, each segmented region is inversely mapped according to the mapping parameters between the calibration image and the corrected image to obtain each calibration region in the calibration image. After that, the position coordinates of the key points in each calibration region are calculated, and the calculated position coordinates are used as the coordinates of each calibration region.

[0019] For example, after starting up, the smart scale can acquire images of the weighing pan through the image acquisition component, and calibrate the four physical corner points of the weighing pan image to obtain a calibration image. Subsequently, the smart scale can perform image correction mapping on the calibration image to obtain a calibration region segmentation, obtain each segmented region, perform inverse mapping on each segmented region to obtain each calibration region, and calculate the position coordinates of the center point of each calibration region as the region coordinates, or it can also calculate the position coordinates of the vertices of each calibration region as the region coordinates.

[0020] Step S204: Generate and display the acquisition guidance page for the target calibration area based on the image acquisition prompts and area coordinates of the target calibration area.

[0021] In practice, based on the obtained regional coordinates of each calibration area, in order to reduce the complexity of manual operation in the image processing process and ensure the accurate transmission of the image acquisition requirements of the products to be entered, thereby improving the efficiency and accuracy of image acquisition, an acquisition guidance page for guiding user operation can be generated and displayed. Specifically, during the generation of the data acquisition guidance page, the target calibration area for image acquisition can be determined, and a data acquisition guidance page for the target calibration area can be generated based on the image acquisition prompts and area coordinates of the target calibration area. The data acquisition guidance page can then be displayed for the user to refer to and execute. Afterward, the user can place the goods to be entered in the corresponding target calibration area of ​​the weighing component based on the displayed data acquisition guidance page. After the user has placed the goods to be entered, the smart scale can acquire the image of the goods to be entered in the target calibration area.

[0022] Here, the target calibration area is any one of the calibration areas mentioned above. The method for generating the acquisition guidance page for each calibration area is the same as the implementation method provided above for generating the acquisition guidance page for the target calibration area based on the image acquisition prompts and area coordinates of the target calibration area. Here, we will take any calibration area (target calibration area) as an example to explain the process of generating the acquisition guidance page for the calibration area.

[0023] In the specific execution process, in order to improve the angular diversity of the collected product images to present the different angular features of the products, an angle generation algorithm can be introduced to generate the calibration collection angle; and, in order to reduce the operational difficulty for users during product placement, operation guidance can be generated based on rotation indicators, area coordinates, and / or viewpoint keywords; in an optional implementation of this embodiment, a collection guidance page for the target calibration area is generated based on the image collection prompts and area coordinates of the target calibration area, including: A random angle of the target calibration area is generated by an angle generation algorithm as the calibration acquisition angle, and the angle mark contained in the image acquisition prompt is rotated according to the calibration acquisition angle to obtain the rotation mark. A capture guidance page is generated based on the rotation indicator, area coordinates, and viewpoint keywords corresponding to the calibration viewpoint type included in the image capture prompts.

[0024] The viewpoint keywords refer to the viewpoint prompt text corresponding to the calibrated viewpoint type, such as front or back; here, viewpoint keywords can also be replaced with viewpoint prompt text.

[0025] The image acquisition prompts refer to prompts used to guide, indicate, or standardize the placement, angle, and / or orientation of goods to ensure that the acquired product images meet preset standards. Specifically, the image acquisition prompts may include calibrating the viewing angle type, such as describing the specific viewing angle from which the product is photographed, such as front, back, left side, and right side; they may also include angle markings, such as markings indicating the product placement angle, such as arrow markings or text markings; and they may also include calibrating the acquisition angle and / or acquisition guidance text. Optionally, the image acquisition prompts may include the calibration viewpoint type, angle identifier, calibration acquisition angle, and / or acquisition guidance text.

[0026] Specifically, in the process of generating the acquisition guidance page for the target calibration area, the angle generation algorithm is first called to generate a random angle within a preset angle range as the calibration acquisition angle for the target calibration area. The standard angle mark contained in the image acquisition prompt is rotated according to the calibration acquisition angle to obtain the rotation mark. Then, the corresponding viewpoint keyword can be determined according to the current calibration viewpoint type contained in the image acquisition prompt, and the acquisition guidance page is generated according to the rotation mark, area coordinates and viewpoint keyword.

[0027] For example, based on the region coordinates and the viewpoint keyword "frontal" corresponding to the calibration viewpoint type included in the image acquisition prompts, a viewpoint like this can be generated. Figure 3 The data collection guidance page shown guides users to place products in the target area based on the page's layout. Users should place the products with the front facing up, and the placement angle should match the angle displayed on the guidance page. Furthermore, Figure 3 The data collection guide page shown may also include the text "Face up, take a picture within this area".

[0028] Here, if the image acquisition prompt includes a calibration viewpoint type, angle identifier, calibration acquisition angle, and / or acquisition guidance text, an acquisition guidance page can be generated based on any one or more of these elements. In this case, the above-mentioned method for generating an acquisition guidance page for the target calibration area can be replaced by: rotating the angle identifier according to the calibration acquisition angle to obtain a rotation identifier, and generating an acquisition guidance page based on the rotation identifier, area coordinates, and viewpoint keywords corresponding to the calibration viewpoint type; or, it can be replaced by: rotating the angle identifier according to the calibration acquisition angle to obtain a rotation identifier, and generating an acquisition guidance page based on the rotation identifier, area coordinates, viewpoint keywords corresponding to the calibration viewpoint type, and acquisition guidance text.

[0029] In practical applications, in order to eliminate the angle adjustment error caused by inaccurate angle judgment or other reasons when users manually rotate the product, reduce the probability of subsequent angle verification of the product image failing, and improve the efficiency of product image acquisition by reducing user operation steps so that users do not need to repeatedly adjust the product placement angle, the weighing component can also be rotated during image processing. In one optional implementation of this embodiment, after generating and displaying a target calibration area acquisition guidance page based on the image acquisition prompts and area coordinates of the target calibration area, the following operations are performed: The system generates a rotation command based on the calibration acquisition angle contained in the image acquisition prompt and sends it to the weighing component, so that the weighing component rotates according to the calibration acquisition angle carried by the rotation command.

[0030] Optionally, the product image is acquired after the product to be entered is placed on the weighing area corresponding to the target calibration area on the surface of the weighing component and rotated.

[0031] Specifically, the calibration acquisition angle can be extracted from the image acquisition prompts, and a rotation command can be generated based on the calibration acquisition angle and sent to the motor drive module of the weighing component. Correspondingly, upon receiving the rotation command, the weighing component rotates according to the calibration acquisition angle carried by the rotation command, thereby achieving the purpose of rotating the product to be entered carried by the weighing component to the calibration acquisition angle. Afterwards, after the product to be entered is placed on the weighing area corresponding to the target calibration area on the surface of the weighing component and the weighing component rotates according to the calibration acquisition angle, the product image of the product to be entered can be acquired.

[0032] It should be noted that during the image acquisition process of the smart scale for the goods to be entered, the smart scale can segment the calibration image acquired by the weighing component of the smart scale into calibration areas after each startup, obtain the coordinates of each calibration area, and generate and display a target calibration area acquisition guide page based on the image acquisition prompts and coordinates of the target calibration area. Furthermore, the smart scale can also segment the calibration image acquired by the weighing component of the smart scale during the first startup to obtain the coordinates of each calibration area and generate image acquisition prompts for each calibration area. In this case, the smart scale can also display the calibration image and coordinates of each calibration area obtained during the first startup. The area coordinates and / or image acquisition prompts of the target calibration area are stored on the server. Subsequently, after detecting the start command of the smart scale, the smart scale can obtain the image acquisition prompts and area coordinates of the target calibration area from the server, and generate and display the acquisition guidance page of the target calibration area based on the image acquisition prompts and area coordinates of the target calibration area. Based on this, the above steps S202 to S204 are also replaced by: obtaining the image acquisition prompts and area coordinates of the target calibration area, generating and displaying the acquisition guidance page of the target calibration area based on the image acquisition prompts and area coordinates of the target calibration area, and forming a new implementation method with other processing steps provided in this embodiment. Alternatively, upon initial startup, the smart scale can segment the calibration image collected by the weighing component of the smart scale to obtain the coordinates of each calibration area. Based on the image acquisition prompts and coordinates of each calibration area, a corresponding acquisition guidance page is generated for each calibration area, and each acquisition guidance page is stored on the server. Subsequently, upon detecting the startup command of the smart scale, the smart scale can obtain and display the acquisition guidance page of the target calibration area from the server. Based on this, the above steps S202 to S204 are replaced by: obtaining and displaying the acquisition guidance page of the target calibration area, and forming a new implementation method with other processing steps provided in this embodiment.

[0033] Step S206: Acquire the product image of the product to be entered, and perform regional position verification and angle verification of the target calibration area on the product image.

[0034] As described above, after the collection guidance page for generating the target calibration area is displayed, the user can place the product to be entered in the corresponding target calibration area in the weighing component based on the displayed collection guidance page. After the product to be entered is placed, the smart scale can collect the product image in the target calibration area and perform regional position verification and calibration angle verification on the collected product image.

[0035] In specific implementation, after acquiring product images of the products to be entered as described above, here, the acquired product images of the products to be entered are obtained. Based on this, in order to effectively eliminate invalid images caused by incorrect product placement areas or excessive placement angle deviations, and to ensure the high quality of product images obtained by image processing as training data for the visual model, the regional position and angle of the target calibration area of ​​the product image are verified.

[0036] In practical applications, during the process of verifying the region position and angle of product images, in order to improve the processing efficiency of product image verification and avoid locating and calculating the coordinates of product images with obvious viewing angle errors, it is advisable to first check whether the acquisition viewing angle type of the product image block in the product image matches the calibration viewing angle type; here, the acquisition viewing angle of the target calibration area of ​​the product image can be verified first. In one optional implementation of this embodiment, the method of performing regional position verification and angle verification of the target calibration area on the product image further includes: The acquisition viewpoint type is obtained by identifying the acquisition viewpoint of product image blocks in the product image. Check whether the acquisition viewpoint type matches the calibration viewpoint type included in the image acquisition prompt. If so, determine the product coordinates of the product image block in the product image.

[0037] Among them, the acquisition viewpoint type refers to the actual viewpoint from which the product is photographed during the process of acquiring product images, such as front, back, left side, and right side; the calibration viewpoint type refers to the preset viewpoint type from which the product image needs to be acquired.

[0038] Specifically, in the process of verifying the region position and angle of the product image, the product image is first segmented to obtain product image blocks. Then, the acquisition viewpoint is identified for each product image block to obtain the acquisition viewpoint type. Further, when the image acquisition prompt is read, the calibration viewpoint type is extracted from the image acquisition prompt, and it is checked whether the acquisition viewpoint type matches the calibration viewpoint type. If they match, that is, if the acquisition viewpoint verification result is passed, the product coordinates of the product image block in the product image are determined. If not, it indicates that the acquisition viewpoint of the currently acquired product image is inconsistent with the calibration viewpoint type, and a verification result of "verification failed" or "acquisition viewpoint verification failed" is generated.

[0039] For example, in the process of identifying the acquisition perspective type of a product image block, a perspective classification model can be called. The product image block can be input into the perspective classification model to identify the acquisition perspective type.

[0040] In the specific execution process, during the area position verification and angle verification of the product image, in order to improve the accuracy of the verification, the angle verification of the product image can be performed after the coordinate verification of the product image passes, thereby avoiding misjudgment of angle due to product misalignment; in an optional implementation method provided in this embodiment, the area position verification and angle verification of the target calibration area of ​​the product image includes: Determine the product coordinates of the product image block in the product image, and perform coordinate verification on the product coordinates based on the region coordinates of the target calibration area; If the coordinate verification passes, the product image is input into the angle recognition model to identify the product angle, and the obtained product angle is verified based on the image acquisition prompts to obtain the verification result.

[0041] The angle recognition model can be a large language model, or it can be an algorithm model, such as a model based on a decision tree algorithm, a model based on a neural network algorithm, or a model based on other algorithms; the input of the angle recognition model includes a product image, and the output includes the product angle.

[0042] Specifically, during the process of verifying the region position and angle of the product image, after acquiring the product image and segmenting it into product image blocks, the product coordinates of the product image blocks are determined. Given the region coordinates of the target calibration area, the product coordinates are verified based on these region coordinates. If the coordinate verification fails, a verification result is generated. If the coordinate verification passes, the angle recognition model is invoked, the product image is input into the angle recognition model to obtain the product angle, and the angle is verified based on the image acquisition prompts to obtain the angle verification result.

[0043] In this process, to improve the accuracy and consistency of the verification results, quantitative judgment can be performed on the coordinate verification during the coordinate verification of the product based on the regional coordinates of the target calibration area. In one optional implementation of this embodiment, the coordinate verification of the product based on the regional coordinates of the target calibration area includes: The coordinate mapping index between the product image block and the target calibration area is calculated based on the product coordinates and the area coordinates. If the coordinate mapping index is greater than the preset index threshold, the coordinate verification is confirmed to be successful.

[0044] Specifically, based on the obtained product coordinates and region coordinates, a coordinate mapping index between the product image patch and the target calibration region is calculated. Specifically, in calculating the coordinate mapping index, the positional overlap between the product image patch and the target calibration region can be calculated, and / or, the positional matching degree between the product image patch and the target calibration region can be calculated, such as calculating the mutual containment matching degree between the product image patch and the target calibration region. The calculated positional overlap and / or positional matching degree are used as the coordinate mapping index and compared with a preset index threshold. If the coordinate mapping index is greater than the preset index threshold, the coordinate verification is considered successful; otherwise, if the coordinate mapping index is less than the preset index threshold, the coordinate verification is considered unsuccessful.

[0045] Specifically, in the process of verifying the angle of the product, one optional implementation method provided in this embodiment verifies the angle of the obtained product based on image acquisition prompts, including: Calculate the angle deviation between the product angle and the calibrated acquisition angle corresponding to the angle identifier included in the image acquisition prompt; If the angle deviation is greater than or equal to the preset deviation threshold, the verification result is determined to be a failure. If the angle deviation is less than the preset deviation threshold, the verification result is determined to be a successful verification.

[0046] Specifically, in the process of verifying the angle of the acquired product based on image acquisition prompts, the calibration acquisition angle corresponding to the angle identifier contained in the image acquisition prompts is determined, and the angle deviation between the product angle of the product image and the calibration acquisition angle is calculated. The calculated angle deviation is compared with a preset deviation threshold. If the angle deviation is greater than or equal to the preset deviation threshold, the verification result is determined to be a failure; otherwise, if the angle deviation is less than the preset deviation threshold, the verification result is determined to be a success. Based on this, the verification results of regional position verification and angle verification of the product image are obtained.

[0047] In addition to the above-mentioned methods for verifying the acquisition viewpoint and angle of the product image, regional verification can also be performed on the product image during the process of verifying the region position and angle of the product image. Another optional implementation of this embodiment includes verifying the region position and angle of the target calibration area of ​​the product image, including: The product image is mapped to a calibrated region to obtain a mapped region, and the mapped region is verified based on the target calibrated region. If the region verification passes, the acquisition viewpoint type is obtained by identifying the acquisition viewpoint of the product image block in the product image. It is then checked whether the acquisition viewpoint type maps to the acquisition viewpoint type of the second product image in the target calibration area. If so, the verification result is determined to be successful.

[0048] Specifically, the product image is mapped to a calibrated region. Given a target calibrated region, the mapped region is validated based on that region. This validation can be performed by detecting the overlap and / or matching degree between the mapped region and the target calibrated region. If the overlap and / or matching degree meet a preset threshold, the region validation is considered successful. Further, the product image blocks in the product image are identified by viewing angle to obtain the viewing angle type. It is then checked whether the viewing angle type maps to the viewing angle type of the second product image in the target calibrated region. In other words, it checks whether the viewing angle type maps to the viewing angle type of the standard product image or standard template image in the target calibrated region. If yes, the validation result is considered successful; otherwise, if not, the validation result is considered unsuccessful.

[0049] It should be noted that the three implementation methods for collecting viewpoint verification, angle verification, and region verification of product images provided above can be implemented in any one or more ways according to actual needs during the specific implementation process. For example, the implementation method for collecting viewpoint verification of product images can be selected, or the implementation method for angle verification of product images can be selected, or the implementation method for region verification of product images can be selected. In addition, any two implementation methods can be combined according to the needs of the actual execution process, or any two implementation methods can be combined after adaptive modification, change, or deletion according to the needs of the actual execution process. For example, the product image can be mapped to a calibration region to obtain a mapping region, and the mapping region can be verified based on the target calibration region to obtain a region verification result. If the region verification result is that the region verification is passed, the product image is input into the angle recognition model to identify the product angle, and the obtained product angle is verified based on the image collection prompts to obtain an angle verification result. In this case, the implementation method of performing the verification response processing of the target calibration region based on the verification result provided in step S208 below can be similarly replaced by: performing the verification response processing of the target calibration region based on the region verification result and the angle verification result. Furthermore, all three implementation methods can be combined according to the needs of the actual execution process, or, after adaptive modification, change, or deletion according to the needs of the actual execution process, all three implementation methods can be combined. For example, it can be done by detecting whether the acquisition viewpoint type matches the calibration viewpoint type included in the image acquisition prompt, and obtaining the acquisition viewpoint verification result; determining the product coordinates of the product image block in the product image, and performing coordinate verification on the product coordinates based on the regional coordinates of the target calibration area; if the coordinate verification passes, inputting the product image into the angle recognition model to perform product angle recognition, and performing angle verification on the obtained product angle based on the image acquisition prompt, and obtaining the angle verification result; mapping the product image to obtain the mapping area, and performing regional verification on the mapping area based on the target calibration area, and obtaining the regional verification result; in this case, the implementation method of performing the verification response processing of the target calibration area based on the verification result provided in step S208 below can be similarly replaced by: performing the verification response processing of the target calibration area based on the acquisition viewpoint verification result, angle verification result, and regional verification result; Here, during the process of verifying the regional position and angle of the target calibration area of ​​the product image, the execution order of the various verification processing methods provided above can be arbitrary, and this embodiment does not impose any specific limitations on this.

[0050] Step S208: Perform verification response processing on the target calibration area based on the verification result.

[0051] As described above, after performing regional position verification and angle verification on the product image and obtaining the verification results, in this step, the verification response processing of the target calibration area is performed based on the verification results. That is, the corresponding verification response processing of the target calibration area is performed based on the above-mentioned collected viewing angle verification results, angle verification results and / or regional verification results.

[0052] In the specific execution process, after performing regional position and angle verification on the target calibration area of ​​the product image and obtaining the verification results, if the verification result is a failure, in order to ensure the image quality of the acquired product images and ensure that all product images entering the visual model meet the position and angle requirements, thereby improving the learning effect of the viewpoint model, an image acquisition reminder for secondary image acquisition can be generated; in an optional implementation method provided in this embodiment, the verification response processing of the target calibration area according to the verification result includes: If the verification result is that the verification fails, an image acquisition reminder is generated based on the area verification result and / or angle verification result to perform secondary image acquisition of the target calibration area.

[0053] Specifically, if the verification fails, the region verification result and / or angle verification result can be obtained. If the region verification result is a region verification failure, an image acquisition reminder is generated based on the region verification result that failed. Alternatively, if the angle verification result is a region verification failure, an image acquisition reminder is generated based on the angle verification result that failed. Or, if both the region verification and angle verification fail, an image acquisition reminder is generated based on the region verification result that failed and the angle verification result that failed, so as to guide the secondary image acquisition of the target calibration area.

[0054] Furthermore, if the verification result obtained from the above-mentioned regional position verification and angle verification of the target calibration area of ​​the product image is a pass, it is also possible to detect whether the image acquisition of the current target calibration area is complete, and to perform the next corresponding processing according to the detection result; in an optional embodiment provided in this example, the target calibration area verification response processing according to the verification result further includes: If the verification result is successful, check whether the image acquisition of the target calibration area is complete; If so, based on the image acquisition prompts and area coordinates of the next calibration area of ​​the target calibration area, generate and display the acquisition guidance page for the next calibration area; If not, generate and display a capture guidance page based on the rotation indicators, region coordinates, and viewpoint keywords included in the image capture prompts.

[0055] Optionally, the viewpoint keyword corresponds to another calibrated viewpoint type mapped to the calibrated viewpoint type of the product image; for example, if the calibrated viewpoint type is a front view, then the other calibrated viewpoint type can be a back view, and correspondingly, the viewpoint keyword corresponding to the other calibrated viewpoint type can be "back"; similarly, if the calibrated viewpoint type is a back view, then the other calibrated viewpoint type can be a front view, and correspondingly, the viewpoint keyword corresponding to the other calibrated viewpoint type can be "front".

[0056] Specifically, during the verification response processing of the target calibration area, if the verification result is successful, the image acquisition of the target calibration area is further checked. If so, it indicates that the image acquisition of the target calibration area under the calibration viewpoint type and the product image under another calibration viewpoint type has been completed. Then, the acquisition guidance page of the next calibration area is generated and displayed according to the image acquisition prompt and area coordinates of the next calibration area of ​​the target calibration area. Based on the acquisition guidance page of the next calibration area, the product image of the next calibration area is acquired. Similarly, the above-mentioned implementation method can also be used to perform area position verification and angle verification of the acquired product image of the next calibration area, and perform corresponding verification response processing according to the verification results. If not, it indicates that the image acquisition of the target calibration area or the product image under a certain viewpoint type in another calibration viewpoint type has not been completed. Then, a collection guide page is generated and displayed according to the rotation mark, area coordinates and / or viewpoint keywords contained in the image acquisition prompt, so as to continue the image acquisition of the current target calibration area according to the collection guide page, and continue to perform area position verification and angle verification based on the product image of the current target calibration area. Here, during the process of continuing image acquisition of the current target calibration area according to the acquisition guidance page, if image acquisition under the calibration view type (front view) of the target calibration area has been completed, the generated acquisition guidance page is used to guide the image acquisition processing under another calibration view type (rear view) of the target calibration area. Similarly, if image acquisition under another calibration view type (rear view) of the target calibration area has been completed, the generated acquisition guidance page is used to guide the image acquisition processing under the calibration view type (front view).

[0057] For example, during the verification response processing of the target calibration area, if the verification result is successful but image acquisition of the target calibration area is detected as incomplete, the system generates a response based on the rotation indicator, area coordinates, and viewpoint keywords included in the image acquisition prompt. Figure 4 The data collection guidance page is shown and displayed. Figure 4 The acquisition guidance page shown is used to guide the image acquisition and processing from the rear view of the target calibration area. Figure 4 The data collection guidance page shown displays a rotation indicator, the viewpoint keyword "back view," and the corresponding data collection guidance text.

[0058] Repeat the above process until each calibration area has completed the corresponding image acquisition, area position verification, and angle verification. Based on this, obtain the current product image and store it on the server of the smart scale. Here, when the product image is stored on the server, the obtained current product image is also the product image of the recorded product.

[0059] It should be noted that in the above-mentioned process of processing the verification response of the target calibration area based on the verification result, if the verification result is successful, it is not necessary to check whether the image acquisition of the target calibration area is completed. Instead, the acquisition guidance page for the next calibration area can be directly generated to guide the image acquisition of the next calibration area. In this case, the above-mentioned implementation method of processing the verification response of the target calibration area based on the verification result can be replaced by: if the verification result is successful, generating the acquisition guidance page for the next calibration area based on the image acquisition prompt and area coordinates of the next calibration area of ​​the target calibration area. Alternatively, if the verification result is successful, it is not necessary to check whether the image acquisition of the target calibration area is completed. Instead, a target calibration area acquisition guidance page can be generated to guide the image acquisition of the target calibration area. That is, an acquisition guidance page corresponding to another calibration view type mapped to the calibration view type of the target calibration area can be generated. In this case, the above-mentioned implementation method of processing the verification response of the target calibration area based on the verification result can be replaced by: if the verification result is successful, generating an acquisition guidance page based on the rotation mark, area coordinates and / or view keywords contained in the image acquisition prompt.

[0060] In specific implementation, after obtaining product images through image processing based on the above-described implementation method, the obtained product images of each product can be stored on the server side of the smart scale. Based on this, the server side of the smart scale can store product images of the recorded products. With product images of the recorded products stored on the server side of the smart scale, product identification can be performed on the product to be identified through the smart scale. In one optional implementation method provided in this embodiment, the following operations are performed: The image of the product to be identified is segmented to obtain product image blocks, and the corresponding calibration region of the product image block is determined; The similarity between the product image block and the product image of the corresponding registered product in the designated area is calculated, and the registered product that matches the product image to be identified is determined based on the calculation results.

[0061] Specifically, after the product to be identified is placed on the smart scale, the smart scale acquires an image of the product to be identified. Based on this, the image of the product to be identified is segmented to obtain product image blocks, and the corresponding calibration area is determined. The product image of the registered product in the corresponding calibration area is acquired, and the similarity between the product image block and the product image is calculated. Based on the calculation result, the registered product that matches the product image of the product to be identified is determined.

[0062] In this process, to improve matching efficiency and accuracy, during the similarity calculation between the product image block and the corresponding registered product images in the designated area, candidate product image matching can be performed based on the product angle and / or the acquisition viewpoint type, and similarity calculation can be performed based on the matched candidate product images. In one optional implementation of this embodiment, the similarity calculation between the product image block and the corresponding registered product images in the designated area includes: Perform product angle recognition and acquisition viewpoint recognition on product image blocks to obtain product angle and acquisition viewpoint type; Search for candidate product images that match the product angle and acquisition viewpoint type in the product image sequence that has been entered, and calculate the similarity between the product image block and the candidate product image. Based on the calculated similarity distribution, the recorded products that match the product image to be identified are determined.

[0063] Specifically, during the product identification process using a smart scale, after detecting the smart scale's start command, a sequence of product images of the recorded products can be obtained from the server. Based on the obtained product image sequence, during the similarity calculation process, the product image blocks of the product to be identified are first subjected to product angle and acquisition viewpoint identification to obtain the product angle and acquisition viewpoint type. Based on the product angle and acquisition viewpoint type, candidate product images that match the product angle and acquisition viewpoint type are queried in the sequence of recorded product images. After matching the candidate product images, the similarity between the product image block and the candidate product image is calculated, and the recorded product that matches the product image to be identified is determined according to the calculated similarity distribution.

[0064] Optionally, the sequence of product images that have been recorded can be obtained from the server after the smart scale's start command is detected.

[0065] Furthermore, in the process of determining the matched recorded products for the product image to be identified based on the similarity distribution, there may be situations where multiple recorded products are matched. In this case, to improve the recognition accuracy for the product to be identified, difference interference can be eliminated through region mapping and correction mapping processing. In one optional implementation of this embodiment, determining the matched recorded products for the product image to be identified based on the calculated similarity distribution includes: If there are multiple candidate product images with a similarity greater than the similarity threshold, the product image blocks are corrected and mapped according to the region mapping relationship between the calibration region corresponding to the product image block and the calibration region mapped to the candidate product image. The recorded product that matches the product image to be identified is determined based on the similarity between the obtained corrected image block and the candidate product image.

[0066] Specifically, after calculating the similarity between the product image patch and the candidate product image and obtaining the similarity distribution, if there are multiple candidate product images with a similarity greater than the similarity threshold, the region mapping relationship between the calibration region corresponding to the product image patch and the calibration region mapped to the candidate product image is first determined. Then, the product image patch is corrected and mapped according to the region mapping relationship to obtain a corrected image patch. On this basis, the recorded product that matches the product image to be identified is further determined according to the similarity between the corrected image patch and the candidate product image, thereby realizing the product identification processing for the product to be identified.

[0067] It should be noted that in the above-described process of identifying a product, the image of the product to be identified may contain multiple products of the same type, or it may contain only one product. Similarly, when multiple products of the same type exist in the image of the product to be identified, the above-described implementation method can be replaced by: performing image segmentation on the image of the product to be identified to obtain each product image block, and determining the calibration region corresponding to any product image block; calculating the similarity between the product image block and the product images of the recorded products in the corresponding calibration region, and determining the recorded products that match the image of the product to be identified based on the calculation results; or, it can be replaced by... The process involves: segmenting the image of the product to be identified to obtain individual product image blocks, and determining the calibration region corresponding to any given product image block; calculating the similarity between the product image block and the corresponding registered product image in the calibration region, and determining the registered product corresponding to the product to be identified based on the calculated similarity; here, the process of identifying multiple products of the same type in the image of the product to be identified is similar to the process of identifying the product to be identified described above, and can be referred to the process of identifying the product to be identified described above and adapted, modified, changed, or deleted as needed in the actual execution process; Similarly, in the process of determining the registered products that match the product image to be identified based on the calculation results, if there are multiple products of the same type in the product image to be identified, the above-mentioned implementation method of determining the registered products that match the product image to be identified based on the calculation results can be replaced as follows: if there is only one candidate product image with a similarity greater than the similarity threshold, then the registered product corresponding to the candidate product image is determined as the registered product corresponding to the product image to be identified; if there are multiple candidate product images with a similarity greater than the similarity threshold, the product image block is corrected and mapped according to the region mapping relationship between the calibration region corresponding to the product image block and the calibration region mapped to the candidate product image, and the registered product that matches the product image to be identified is determined according to the similarity between the obtained corrected image block and the candidate product image.

[0068] It should be added that each optional implementation method and each feasible execution method in steps S202 to S208 provided in this embodiment can be executed independently as needed, or they can be combined and referenced with each other. At the same time, each specific execution step in each optional implementation method or each feasible execution method can also be executed independently or combined as needed. Any feature in each execution step can also be deleted, or any feature in one execution step can be added to another execution step or replace any feature in another execution step. The execution conditions of "if" or "under what circumstances" involved in each step or operation can be directly deleted. This embodiment does not specifically limit the subsequent operations after the execution conditions.

[0069] In summary, the image processing method for smart scales provided in this embodiment, during the image processing of weighed goods by the smart scale, in order to support the diversity of subsequent image acquisition, ensure that different placement positions of the goods to be entered on the weighing component can correspond to specific calibration areas, and reduce the complexity of manual operation during image processing, performs calibration area segmentation on the calibration image acquired from the weighing component of the smart scale to obtain the area coordinates of each calibration area, and generates and displays the acquisition guidance page of the target calibration area based on the image acquisition prompts and area coordinates of the target calibration area. On this basis, in order to effectively eliminate invalid images caused by incorrect placement area or excessive placement angle deviation, the acquired image of the goods to be entered is further obtained, and the area position and angle of the target calibration area are verified on the goods image. Based on the verification results, the verification response processing of the target calibration area is performed. In this way, the accuracy and processing efficiency of image processing of goods to be entered are improved when image processing of goods to be entered is achieved through the smart scale. Furthermore, after obtaining product images through image processing of the products to be entered using a smart scale, the smart scale can also be used to identify the products to be identified. Specifically, similarity calculations can be performed on the product image blocks and the product images of the already entered products in the corresponding calibration areas. Based on the calculation results, the already entered products that match the product image to be identified can be determined, thereby improving the accuracy of product identification for the products to be identified.

[0070] The following example uses an image processing method for a smart scale provided in this embodiment to illustrate its application in an image processing scenario. Figure 5 The image processing method for smart scales provided in this embodiment will be further described below. (See also...) Figure 5 The image processing method for smart scales, which is applied to image processing scenarios, specifically includes the following steps.

[0071] Step S502: The calibration image collected for the weighing component of the smart scale is segmented into calibration regions to obtain the region coordinates of each calibration region.

[0072] Step S504: Generate a random angle of the target calibration area using an angle generation algorithm as the calibration acquisition angle, and perform a rotation process on the angle markers contained in the image acquisition prompt according to the calibration acquisition angle to obtain the rotation marker.

[0073] Step S506: Generate an acquisition guidance page based on the rotation marker, area coordinates, and the viewpoint keywords corresponding to the calibration viewpoint type included in the image acquisition prompt.

[0074] Step S508: The acquisition viewpoint type is obtained by identifying the acquisition viewpoint of the product image block in the product image.

[0075] Step S510: Check whether the acquisition viewpoint type matches the calibration viewpoint type included in the image acquisition prompt; if so, proceed to step S512 below.

[0076] Step S512: Acquire the product image of the product to be entered, determine the product coordinates of the product image block in the product image, and calculate the coordinate mapping index between the product image block and the target calibration area based on the product coordinates and the area coordinates.

[0077] Step S514: Check whether the coordinate mapping index is greater than the preset index threshold; if yes, determine that the coordinate verification has passed and proceed to step S516 below; if no, determine that the coordinate verification has failed and proceed to step S522 below.

[0078] Step S516: Input the product image into the angle recognition model to obtain the product angle, and calculate the angle deviation between the product angle and the calibrated acquisition angle corresponding to the angle identifier contained in the image acquisition prompt.

[0079] Step S518: Check whether the angle deviation is less than the preset deviation threshold; if yes, determine that the verification result is passed and proceed to step S520; if no, determine that the verification result is failed and proceed to step S522.

[0080] Step S520: Based on the image acquisition prompts and area coordinates of the next calibration area of ​​the target calibration area, generate and display the acquisition guidance page for the next calibration area.

[0081] Step S522: Generate an image acquisition reminder based on the coordinate verification results and angle verification results to perform secondary image acquisition of the target calibration area.

[0082] It should be noted that any one or more steps in steps S502 to S520, or steps S502 to S518 and steps S522, can be combined with any one or more steps in steps S202 to S208 to form a new implementation method according to the needs of implementation and deployment. In addition, according to the actual deployment needs, any one or more technical features in steps S502 to S520, or steps S502 to S518 and steps S522 can be selected and combined with any one or more technical features provided in steps S202 to S208 to form a new implementation method. Alternatively, any one or more technical features in steps S502 to S520, or steps S502 to S518 and steps S522 can also be replaced with any one or more technical features provided in steps S202 to S208 to form a new implementation method according to the needs of actual deployment. These will not be elaborated on here.

[0083] This specification provides an embodiment of an image processing device for use in smart scales, as follows: In the above embodiments, an image processing method for a smart scale is provided, and correspondingly, an image processing device for a smart scale is also provided, which will be described below with reference to the accompanying drawings.

[0084] Reference Figure 6 This illustration shows a schematic diagram of an image processing device embodiment for use in a smart scale, provided in this embodiment.

[0085] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0086] This embodiment provides an image processing device for use in smart scales, including: The region segmentation module 602 is configured to segment the calibration region of the calibration image acquired by the weighing component of the smart scale and obtain the region coordinates of each calibration region. The page generation module 604 is configured to generate and display a target calibration area acquisition guide page based on the image acquisition prompts and area coordinates of the target calibration area. The image verification module 606 is configured to acquire the product image of the product to be entered, and to perform regional position verification and angle verification of the target calibration area on the product image. The response processing module 608 is configured to perform verification response processing on the target calibration area based on the verification result.

[0087] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0088] This specification provides an example of an image processing device for use in a smart scale, as follows: Corresponding to the image processing method for a smart scale described above, based on the same technical concept, one or more embodiments of this specification also provide an image processing device for a smart scale, which is used to execute the image processing method for a smart scale provided above. Figure 7 This is a schematic diagram of the structure of an image processing device applied to a smart scale, provided for one or more embodiments of this specification.

[0089] This embodiment provides an image processing device for use in smart scales, comprising: like Figure 7 As shown, device 700 mainly consists of a communication interface 702, a user interface 704, a processor 706, and a data storage 708. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 710. Communication interface 702 enables device 700 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, communication interface 702 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, communication interface 702 can be a wired interface such as Ethernet, Token Ring, or USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or wide area wireless interface (e.g., WiMAX or LTE). Of course, communication interface 702 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. Communication interface 702 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide area wireless interfaces. User interface 704 includes receiving user input and providing output to the user. Therefore, user interface 704 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 704 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 704 may include software, circuitry, or other forms of logic capable of transmitting data to and receiving data from external user input / output devices. Additionally or alternatively, device 700 may support remote access from other devices via communication interface 702 or another physical interface (not shown). User interface 704 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 704 may also be configured as a display device for rendering or displaying text fragments.

[0090] Processor 706 may include one or more general-purpose processors and / or special-purpose processors. Data storage 708 may include one or more volatile and / or non-volatile storage components, and may be integrated wholly or partially with processor 706. Data storage 708 may include removable and non-removable components.

[0091] Processor 706 is capable of executing program instructions 718 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 708 to perform the various functions described herein. Data storage 708 may comprise a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 700, enable device 700 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 718 by processor 706 may result in processor 706 using data 712. For example, program instructions 718 may include an operating system 722 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 700 and one or more application programs 720 (e.g., a browser, social application, or game application). Similarly, data 712 may include operating system data 716 and application data 714. Operating system data 716 is primarily accessible to operating system 722, while application data 714 is primarily accessible to one or more application programs 720. Application data 714 may reside in a file system visible or hidden from the user of device 700. Application 720 can communicate with operating system 722 through one or more application programming interfaces (APIs). These APIs facilitate application 720 in reading and / or writing application data 714, transmitting or receiving information via communication interface 702, and receiving or displaying information on user interface 704. In some terms, application 720 may be simply referred to as "app". Furthermore, application 720 can be downloaded to device 700 through one or more online app stores or app markets. However, applications can also be installed on device 700 in other ways, such as through a web browser or a physical interface on device 700 (e.g., a USB port).

[0092] In one specific embodiment, the image processing device applied to the smart scale includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the image processing device applied to the smart scale, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following: The calibration images collected from the weighing components of the smart scale are segmented into calibration regions to obtain the region coordinates of each calibration region; Based on the image acquisition prompts and area coordinates of the target calibration area, a acquisition guidance page for the target calibration area is generated and displayed; Acquire the product image of the product to be entered, and perform regional position verification and angle verification of the target calibration area on the product image; Based on the verification results, perform verification response processing on the target calibration area.

[0093] This specification provides an embodiment of a computer-readable storage medium as follows: In accordance with the image processing method for a smart scale described above, and based on the same technical concept, one or more embodiments of this specification also provide a computer-readable storage medium.

[0094] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed, implement the following process: The calibration images collected from the weighing components of the smart scale are segmented into calibration regions to obtain the region coordinates of each calibration region; Based on the image acquisition prompts and area coordinates of the target calibration area, a acquisition guidance page for the target calibration area is generated and displayed; Acquire the product image of the product to be entered, and perform regional position verification and angle verification of the target calibration area on the product image; Based on the verification results, perform verification response processing on the target calibration area.

[0095] It should be noted that the embodiments of a computer-readable storage medium described in this specification and the embodiments of an image processing method applied to a smart scale described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0096] This specification provides an example of a computer program product as follows: Corresponding to the image processing method for a smart scale described above, based on the same technical concept, one or more embodiments of this specification also provide a computer program product.

[0097] A computer program product includes a computer program / instructions that, when executed by a processor, perform the following steps: The calibration images collected from the weighing components of the smart scale are segmented into calibration regions to obtain the region coordinates of each calibration region; Based on the image acquisition prompts and area coordinates of the target calibration area, a acquisition guidance page for the target calibration area is generated and displayed; Acquire the product image of the product to be entered, and perform regional position verification and angle verification of the target calibration area on the product image; Based on the verification results, perform verification response processing on the target calibration area.

[0098] It should be noted that the embodiments of a computer program product described in this specification and the embodiments of an image processing method applied to a smart scale described in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0099] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments. For example, the device embodiment, equipment embodiment and computer-readable storage medium embodiment are all similar to the method embodiment, so the description is relatively simple. When reading the relevant content of the device embodiment, equipment embodiment and computer-readable storage medium embodiment, please refer to the description of the method embodiment.

[0100] While one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps, and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims. This specification uses specific terms to describe embodiments of this specification. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0101] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0102] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0103] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0104] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0105] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0106] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0111] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0112] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0113] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising at least one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0115] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. An image processing method for use in smart scales, comprising: The calibration images collected from the weighing components of the smart scale are segmented into calibration regions to obtain the region coordinates of each calibration region; Based on the image acquisition prompts and area coordinates of the target calibration area, a acquisition guidance page for the target calibration area is generated and displayed; Acquire the product image of the product to be entered, and perform regional position verification and angle verification of the target calibration area on the product image; Based on the verification results, perform verification response processing on the target calibration area.

2. The image processing method for a smart scale according to claim 1, wherein the step of performing regional position verification and angle verification of the target calibration area on the product image includes: Determine the product coordinates of the product image block in the product image, and perform coordinate verification on the product coordinates based on the region coordinates of the target calibration area; If the coordinate verification passes, the product image is input into the angle recognition model to identify the product angle, and the obtained product angle is verified based on the image acquisition prompts to obtain the verification result.

3. The image processing method for a smart scale according to claim 2, wherein the step of performing regional position verification and angle verification of the target calibration area on the product image further includes: The acquisition viewpoint type is obtained by identifying the acquisition viewpoint of the product image blocks in the product image. Detect whether the acquisition viewpoint type matches the calibration viewpoint type included in the image acquisition prompt. If so, perform the operation of determining the product coordinates of the product image block in the product image.

4. The image processing method for a smart scale according to claim 2, wherein the coordinate verification of the commodity coordinates based on the region coordinates of the target calibration area includes: Based on the product coordinates and the region coordinates, calculate the coordinate mapping index between the product image block and the target calibration region. If the coordinate mapping index is greater than a preset index threshold, the coordinate verification is confirmed to be successful.

5. The image processing method for a smart scale according to claim 2, wherein the step of verifying the angle of the acquired product based on the image acquisition prompt includes: Calculate the angle deviation between the product angle and the calibrated acquisition angle corresponding to the angle identifier included in the image acquisition prompt; If the angle deviation is greater than or equal to the preset deviation threshold, the verification result is determined to be a verification failure. If the angle deviation is less than the preset deviation threshold, the verification result is determined to be a successful verification.

6. The image processing method for a smart scale according to claim 1, wherein the step of performing regional position verification and angle verification of the target calibration area on the product image includes: The product image is mapped by a calibration region to obtain a mapped region, and the mapped region is verified based on the target calibration region. If the region verification passes, the acquisition viewpoint type is obtained by identifying the acquisition viewpoint of the product image block in the product image. It is then detected whether the acquisition viewpoint type maps to the acquisition viewpoint type of the second product image in the target calibration region. If so, the verification result is determined to be successful.

7. The image processing method for a smart scale according to claim 1, wherein segmenting the calibration image acquired from the weighing component of the smart scale to obtain the region coordinates of each calibration region includes: The calibration image is subjected to image correction mapping to obtain a corrected image, and the corrected image is subjected to calibration region segmentation to obtain each segmented region; The calibration region is obtained by inverse mapping of each segmented region according to the mapping parameters between the calibration image and the corrected image, and the position coordinates of the key location points of each calibration region are calculated.

8. The image processing method for a smart scale according to claim 1, wherein generating a target calibration area acquisition guidance page based on the image acquisition prompts and area coordinates of the target calibration area includes: A random angle of the target calibration area is generated by an angle generation algorithm as the calibration acquisition angle, and the angle mark contained in the image acquisition prompt is rotated according to the calibration acquisition angle to obtain the rotation mark. The acquisition guidance page is generated based on the rotation indicator, the area coordinates, and the viewpoint keywords corresponding to the calibration viewpoint type included in the image acquisition prompt.

9. The image processing method for a smart scale according to claim 1, wherein the step of performing verification response processing on the target calibration area based on the verification result includes: If the verification result is that the verification fails, an image acquisition reminder is generated based on the region verification result and / or angle verification result to perform secondary image acquisition of the target calibration area.

10. The image processing method for a smart scale according to claim 9, wherein the step of performing verification response processing on the target calibration region based on the verification result further includes: If the verification result is successful, check whether the image acquisition of the target calibration area is complete; If so, based on the image acquisition prompts and area coordinates of the next calibration area of ​​the target calibration area, generate and display the acquisition guidance page for the next calibration area; If not, generate and display a capture guidance page based on the rotation marker, region coordinates, and viewpoint keywords included in the image capture prompt; wherein, the viewpoint keywords correspond to another calibration viewpoint type mapped from the calibration viewpoint type of the product image.

11. The image processing method for a smart scale according to claim 1, further comprising, after the step of generating and displaying a target calibration area acquisition guidance page based on the image acquisition prompts and area coordinates of the target calibration area, and before the step of acquiring the acquired product image of the product to be entered, and performing area position verification and angle verification of the product image in the target calibration area, the method further comprises: A rotation command is generated based on the calibration acquisition angle included in the image acquisition prompt and sent to the weighing component, so that the weighing component rotates according to the calibration acquisition angle carried by the rotation command; The product image is acquired by rotating the product placed on the surface of the weighing component in the weighing area corresponding to the target calibration area.

12. The image processing method for a smart scale according to claim 1, further comprising: The product image to be identified is segmented to obtain product image blocks, and the corresponding calibration region of the product image blocks is determined; The similarity between the product image block and the product images of the corresponding registered products in the designated area is calculated, and the registered products that match the product image to be identified are determined based on the calculation results.

13. The image processing method for a smart scale according to claim 12, wherein calculating the similarity between the product image block and the product image of the corresponding calibrated area of ​​the recorded product includes: Perform product angle recognition and acquisition viewpoint recognition on the product image block to obtain the product angle and acquisition viewpoint type; In the sequence of product images of the recorded products, query candidate product images that match the product angle and the acquisition viewpoint type, and calculate the similarity between the product image block and the candidate product image. The recorded products that match the image of the product to be identified are determined based on the calculated similarity distribution.

14. The image processing method for a smart scale according to claim 13, wherein determining the recorded product matching the image of the product to be identified based on the calculated similarity distribution includes: If there are multiple candidate product images with a similarity greater than the similarity threshold, the product image block is corrected and mapped according to the region mapping relationship between the calibration region corresponding to the product image block and the calibration region mapped to the candidate product image. The recorded product that matches the product image to be identified is determined based on the similarity between the obtained corrected image block and the candidate product image.

15. An image processing device for use in a smart scale, comprising: The region segmentation module is configured to segment the calibration region of the calibration image acquired by the weighing component of the smart scale and obtain the region coordinates of each calibration region. The page generation module is configured to generate and display a target calibration area acquisition guide page based on the image acquisition prompts and area coordinates of the target calibration area. The image verification module is configured to acquire the product image of the product to be entered, and to perform regional position verification and angle verification of the target calibration area on the product image; The response processing module is configured to perform verification response processing on the target calibration area based on the verification result.

16. An image processing device for use in a smart scale, comprising: processor; And, a memory configured to store computer-executable instructions, which, when executed, cause the processor to: The calibration images collected from the weighing components of the smart scale are segmented into calibration regions to obtain the region coordinates of each calibration region; Based on the image acquisition prompts and area coordinates of the target calibration area, a acquisition guidance page for the target calibration area is generated and displayed; Acquire the product image of the product to be entered, and perform regional position verification and angle verification of the target calibration area on the product image; Based on the verification results, perform verification response processing on the target calibration area.

17. A computer-readable storage medium for storing computer-executable instructions that, when executed, implement the steps of the method of claim 1.

Images