Commodity information determination method and device, electronic equipment and storage medium

Abstract

The invention provides a commodity information determination method and device, electronic equipment and a storage medium. The commodity information determination method comprises the steps that an open set detection model is adopted to determine a first commodity image corresponding to a commodity in a first image and a second commodity image corresponding to a commodity in a second image, the first image is an image of a target area at a first moment, the second image is an image of the target area at a second moment, and the first image is an image of the target area at a first moment; the first moment is before the second moment; comparing a first commodity image corresponding to the commodity in the first image with a second commodity image corresponding to the commodity in the second image, and determining a third commodity image corresponding to the out-of-stock commodity which is reduced compared with the first image in the second image; and identifying the third commodity image by adopting a visual identification model, and determining commodity information of the out-of-stock commodity corresponding to the third commodity image.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for determining commodity information. Background Art

[0002] Currently, commercial entities such as convenience stores and supermarkets usually adopt a self-service method for customers to select goods, and the goods are placed on the shelves for customers to purchase.

[0003] During the commodity sales process, the commodities on the shelves need to be counted and replenished in a timely manner to reduce out-of-stock situations such as insufficient quantity of commodities on the shelves.

[0004] Therefore, how to promptly determine the product information on the shelves, such as the name and quantity of out-of-stock products, is an urgent problem to be solved. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method, device, electronic device, and storage medium for determining product information.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for determining product information is proposed, the method comprising: Using an open set detection model, respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image, wherein the first image is an image of the target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; comparing a first product image corresponding to a product in the first image with a second product image corresponding to the product in the second image, and determining a third product image corresponding to a product that is out of stock and has a smaller number of products in the second image than in the first image; The third product image is recognized using a visual recognition model, and product information of the out-of-stock product corresponding to the third product image is determined.

[0007] In some embodiments, the step of using an open set detection model to respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image includes: Identifying the product in the first image using the open set detection model, and marking the first product image corresponding to the identified product in the first image using a predetermined identifier; Identifying the product in the second image using the open set detection model, and marking a second product image corresponding to the product identified in the second image using a predetermined identification marker; The comparing the first product image corresponding to the product in the first image with the second product image corresponding to the product in the second image includes: The first product image marked with the predetermined identifier is compared with the second product image marked with the predetermined identifier.

[0008] In some embodiments, the method further comprises: The open set detection model is trained using a first training product image of a first non-standard appearance product labeled as a predetermined product type.

[0009] In some embodiments, the step of training the open set detection model using a first training product image of a first non-standard-appearance product labeled as a predetermined product type includes at least one of the following: Using the first training product image as input to the open set detection model, using the product type of the first non-standard appearance product as labeling information of the first training product image, and training the output of the open set detection model; The first training product image is used as input of the open set detection model, and position information of the product type of the first non-standard appearance product in the image description of the first training product image is specified to train the output of the open set detection model.

[0010] In some embodiments, the method further comprises: In response to the open set detection model being able to identify the product type of the second non-standard appearance product in the product image, it is determined that the open set detection model has completed training.

[0011] In some embodiments, the method further comprises: A visual recognition model capable of recognizing a first category of commodities is trained so that the visual recognition model can recognize a second category of commodities, wherein the second category is a subcategory of the first category.

[0012] In some embodiments, the step of training a visual recognition model capable of identifying a first category of goods so that the visual recognition model can identify a second category of goods comprises: Using a second training product image as input to the visual recognition model and using a product description including a second category description of the product in the second training product image as output of the visual recognition model to train the visual recognition model; In response to the visual recognition model identifying a second category of the commodity in the unlabeled commodity image, it is determined that the visual recognition model has completed training.

[0013] According to a second aspect of an embodiment of the present disclosure, a device for determining product information is provided, the device comprising: a processing module, wherein the processing module is configured to: Using an open set detection model, respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image, wherein the first image is an image of the target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; comparing a first product image corresponding to a product in the first image with a second product image corresponding to the product in the second image, and determining a third product image corresponding to a product that is out of stock and has a smaller number of products in the second image than in the first image; The third product image is recognized using a visual recognition model, and product information of the out-of-stock product corresponding to the third product image is determined.

[0014] In some embodiments, the processing module is specifically configured to: Identifying the product in the first image using the open set detection model, and marking the first product image corresponding to the identified product in the first image using a predetermined identifier; Identifying the product in the second image using the open set detection model, and marking a second product image corresponding to the product identified in the second image using a predetermined identification marker; The first product image marked with the predetermined identifier is compared with the second product image marked with the predetermined identifier.

[0015] In some embodiments, the apparatus further comprises: a training module, The training module is used to train the open set detection model using a first training product image of a first non-standard appearance product marked as a predetermined product type.

[0016] In some embodiments, the training module is specifically used for at least one of the following: Using the first training product image as input to the open set detection model, using the product type of the first non-standard appearance product as labeling information of the first training product image, and training the output of the open set detection model; The first training product image is used as input of the open set detection model, and position information of the product type of the first non-standard appearance product in the image description of the first training product image is specified to train the output of the open set detection model.

[0017] In some embodiments, the training module is further configured to: determine that the open set detection model has completed training in response to the open set detection model being able to identify the product type of the second non-standard appearance product in the product image.

[0018] In some embodiments, the apparatus further comprises: a training module, The training module is used to train a visual recognition model that can identify a first category of goods, so that the visual recognition model can identify a second category of goods, wherein the second category is a subcategory of the first category.

[0019] In some embodiments, the training module is specifically used to: Using a second training product image as input to the visual recognition model and using a product description including a second category description of the product in the second training product image as output of the visual recognition model to train the visual recognition model; In response to the visual recognition model identifying a second category of the commodity in the unlabeled commodity image, it is determined that the visual recognition model has completed training.

[0020] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: one or more processors; The processor is used to call instructions to enable the electronic device to execute the commodity information determination method described in the first aspect.

[0021] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is proposed, which stores instructions. When the instructions are executed on an electronic device, the electronic device executes the product information determination method described in the first aspect.

[0022] According to the disclosed embodiments, the product information determination method, apparatus, electronic device, and storage medium include: using an open set detection model to determine a first product image corresponding to a product in a first image and a second product image corresponding to a product in a second image, wherein the first image is an image of a target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; comparing the first product image corresponding to the product in the first image with the second product image corresponding to the product in the second image, and determining a third product image corresponding to a product that is out of stock and has a smaller number of products in the second image than in the first image; and using a visual recognition model to identify the third product image and determine the product information of the out-of-stock product corresponding to the third product image. Thus, the open set detection model can be used to determine products of known and unknown categories in an image without requiring annotation training for all product types. Furthermore, it is possible to compare products in target area images at different moments, identify out-of-stock products, and determine information such as the product type of the out-of-stock product using the visual recognition model. This can shorten the time required to label all product types and train the recognition model, thereby saving costs. Products can be put on shelves without prior annotation and training, improving the real-time nature of product placement. On the other hand, the open-set detection model doesn't need to precisely determine the product type, which can improve its recognition speed. The visual recognition model then identifies out-of-stock products, speeding up the identification process. Furthermore, because the open-set detection model only needs to distinguish between products and non-products, it can reduce the chances of errors caused by the recognition model's fine-grained product type identification, thereby improving the accuracy of out-of-stock product identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram illustrating an application environment of a method for determining product information according to an embodiment; Figure 2 is a flow chart illustrating a method for determining product information according to an embodiment; Figure 3 is a schematic diagram of a first image of a target area according to an embodiment; Figure 4 is a schematic diagram showing a second image of a target area according to an embodiment; Figure 5 is a schematic diagram showing a first product image in a first image according to an embodiment; Figure 6 is a schematic diagram showing a second product image in a second image according to an embodiment; Figure 7 is a schematic diagram of a third product image according to an embodiment; Figure 8This is a flow chart showing a method for detecting out-of-stock items on a shelf according to an embodiment; Figure 9 This is a schematic diagram of the operation flow of a shelf commodity out-of-stock detection system according to an embodiment; Figure 10 is a structural diagram of a device for determining product information according to an embodiment; Figure 11 The figure is a schematic structural diagram of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0024] To make the technical solutions and beneficial effects of the present invention more clearly understood, the following detailed description is given by way of specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly illustrate the details of the local features. Unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application belongs.

[0025] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0026] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0027] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0028] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "said", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0029] In the embodiments of the present disclosure, “plurality” refers to two or more.

[0030] In some embodiments, the terms “at least one”, “one or more”, “a plurality of”, “multiple”, etc. can be used interchangeably.

[0031] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "in one case A, in another case B," or "in one case A, in another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The same applies when there are more branches, such as A, B, and C.

[0032] In some embodiments, "A or B" and other expressions may include the following technical solutions, depending on the circumstances: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is also applicable when there are more branches, such as A, B, and C.

[0033] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, value or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields", and "first" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the value of the description object is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the value of "device" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0034] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0035] In some embodiments, terms such as "...", "determine...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0036] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0037] In some embodiments, devices, etc. can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. Terms such as "device", "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", and "subject" can be used interchangeably.

[0038] In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any elements, any rows, and any columns can also be implemented as an independent embodiment.

[0039] In some embodiments, out-of-stock detection of shelf merchandise can be implemented using a product recognition model. The specific implementation method includes: (1) first labeling the type of each product and training the product recognition model; (2) capturing shelf images and generating a shelf display comparison diagram; (3) using the product recognition model to identify the product information (e.g., product type) of the products currently on the shelf and the product information (e.g., product type) of the products in the display comparison diagram, and then comparing them to identify out-of-stock products. This requires that the product recognition model can directly identify the product information of the products in the shelf image.

[0040] The above-mentioned out-of-stock detection method for shelf products using the product recognition model has the following main problems: 1. Difficulty adapting to the rapid introduction of new products: Existing methods rely on product recognition models, and accurate recognition networks require labeled data samples. For convenience stores and chain stores, new products are introduced quickly, and each new sample requires the creation of a training dataset and refinement of the product recognition model. This process increases labor costs and affects real-time performance.

[0041] 2. Low product generalization: Often, the same product is divided into different subcategories based on flavor and packaging. These subcategories may also have packaging differences, which can cause existing product recognition models to fail, necessitating timely data replenishment. In practice, exhaustive training on all products is impossible, resulting in a high probability of product recognition model failure.

[0042] 3. By training a specific product recognition model one-on-one, when the number of product types increases or new products need to be added, it usually takes several days or weeks of data to label or train the products.

[0043] By adopting the open set detection model and large visual model of this technical solution in conjunction with the implementation steps of this technical solution, when the number of product types increases or new products need to be recorded, the product recognition and detection cycle is greatly shortened to minutes or hours, while ensuring the quality of product recognition and greatly improving the speed of new product addition.

[0044] Figure 1 is a schematic diagram of an application environment of a commodity information determination method provided by an embodiment of the present disclosure, such as Figure 1 As shown, the application environment at least includes an information processing device 10 and an image acquisition device 20 (such as a terminal).

[0045] The information processing device 10 can be used as a service-hosting environment, processing image data transmitted by the user through the image acquisition device 20 through the services running thereon (such as data processing services). The information processing device 10 can also send the processing results obtained from processing the image data to the terminal for presentation to the user. The information processing device 10 can be an independent computing service device, or a server cluster or distributed system composed of multiple computing service devices. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0046] The image acquisition device 20 may include but is not limited to smart phones, tablet computers, laptop computers, digital assistants, smart wearable devices, cameras, and other electronic devices with image acquisition capabilities.

[0047] In addition, it should be noted that Figure 1 What is shown is only an application environment of the commodity information determination method. In actual application, other application environments may also be included, for example, more terminals, cameras, etc.

[0048] In the embodiments of this specification, the above-mentioned information processing devices 10 and / or the information processing device 10 and the image acquisition device 20 can be directly or indirectly connected through wired or wireless communication, which is not limited in this disclosure.

[0049] The embodiment of the present disclosure proposes a method for determining product information, such as Figure 2 As shown, the product information determination method includes: Step 201: Using an open set detection model, determine a first product image corresponding to a product in a first image and a second product image corresponding to a product in a second image, respectively, wherein the first image is an image of a target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; Step 202: Compare the first product image corresponding to the product in the first image with the second product image corresponding to the product in the second image, and determine a third product image corresponding to the out-of-stock product that is less in the second image than in the first image; Step 203: Use a visual recognition model to identify the third product image and determine product information of the out-of-stock product corresponding to the third product image.

[0050] Here, you can Figure 1 The information processing device shown executes the commodity information determination method.

[0051] The first image and the second image may be images captured by an image capture device. The image capture device may send the captured images to an information processing device for determining product information.

[0052] In one possible implementation, the image acquisition device and the information processing device may be the same device.

[0053] In a possible implementation, the image acquisition device and the information processing device may be different devices, and the image acquired by the image acquisition device may be transmitted to the information processing device via a wired and / or wireless manner.

[0054] In one possible implementation, the target area may include a shelf area for placing merchandise.

[0055] In one possible implementation, the first image may be a frame of an image in a video, and the second image may also be a frame of an image in the video. For example, the image acquisition device may capture a video from a first moment to a second moment, and capture a frame of an image at the first moment from the video as the first image, and capture a frame of an image at the second moment from the video as the second image.

[0056] In a possible implementation, there may be multiple target areas, and different target areas may be distinguished by different identifiers. The product information of out-of-stock products in the target area may be determined by comparing the first image and the second image of the same target area.

[0057] In one possible implementation, the out-of-stock product may be a product that is removed (eg, taken away) from the target area between the first time point and the second time point.

[0058] Exemplarily, the first image may be an initial image of the target area, and the second image may be an image of the target area after a period of time.

[0059] like Figure 3 As shown, the first image can be the initial image of the target area at the first moment. There are many commodities in the first image. Figure 4 As shown, the second image may be an image of the target area at the second moment, wherein some of the commodities in the target area are taken away. Figure 3 and Figure 4 The images of the target area can be acquired at the same angle.

[0060] The first image and the second image may be captured by the image capture device at the same position and the same angle, or may be captured by the image capture device at different positions and / or at different angles.

[0061] Here, an open set detection model may be used to identify a first product image of a product from a first image, and to identify a second product image of a product from a second image.

[0062] In one possible implementation, an open-set detection model performs coarse-grained recognition. The open-set detection model can classify objects in the first and second images into commodities and non-commodities, thereby identifying commodities from the first and second images and separating them from the first or second image. For example, the open-set detection model can distinguish commodities from non-commodities (such as shelves, people, and other objects) in the first and second images.

[0063] In one possible implementation, the open set detection model may not identify the specific category of the product.

[0064] Here, using an open-set detection model to identify products can overcome the limitations of traditional object detection methods (closed-set object detection) in open environments. Unlike traditional closed-set object detection, open-set detection models can recognize objects of known categories and can also identify and distinguish objects of unknown categories. For example, open-set detection models can identify unknown products.

[0065] By using an open-set detection model to identify products in the first and second images, we can achieve coarse-grained identification of products on the shelf, identify products outside the training sample, and detect products in non-standard conditions (such as different shapes and packaging) without the need to train models for new products. This effectively reduces the training costs required by traditional object detection methods.

[0066] For example, the same product is often divided into different subcategories based on flavor or packaging, and different subcategories may have different packaging. The open set detection model can identify products with different packaging in images, allowing the product to be separated from the first or second image without the need for training for different packaging.

[0067] In one possible implementation, the first image may contain one or more products, and thus the open set detection model may determine one or more first product images. The second image may contain one or more products, and thus the open set detection model may determine one or more second product images.

[0068] In a possible implementation, one product corresponds to one product image.

[0069] For example, Figure 5 Model identification for open set detection Figure 3 The first product image of multiple products obtained from the product (as shown by the product shadow). Figure 6Model identification for open set detection Figure 4 The second product images of the multiple products obtained from the product (as shown by the shadows of the products).

[0070] After determining the first product image in the first image and the second product image in the second image, the first product image and the second product image can be compared to determine a third product image that is missing from the second image compared to the first image. The product corresponding to the third product image that is missing from the second image compared to the first image is the product that was removed between the first time and the second time, i.e., the out-of-stock product.

[0071] After the third product image is determined, product information of the corresponding out-of-stock product may be determined based on the third product image.

[0072] In a possible implementation, image recognition technology may be used to identify out-of-stock products in the third product image.

[0073] The visual recognition model may include a machine learning model. The visual recognition model may be trained using product images and text feature descriptions, so that the visual recognition model can identify product information of out-of-stock products based on the product image feature information and text information in the third product image.

[0074] In some embodiments, the product information includes at least one of the following: the name of the out-of-stock product, and the product type of the out-of-stock product.

[0075] In one possible implementation, statistical information of out-of-stock products may be determined based on the determined product information. For example, the product information may be determined based on all third product images, and the product name and out-of-stock quantity of the out-of-stock product may be determined.

[0076] In this way, the open-set detection model can identify known and unknown categories of goods in an image without requiring labeling training for all types of goods. This allows for comparison of goods in target area images at different times, identifying out-of-stock goods, and using the visual recognition model to determine information such as the type of the out-of-stock goods. This reduces the time required to label all types of goods and train the recognition model, thereby saving costs. Goods can be put on the shelves without prior labeling and training, improving the real-time nature of product placement. Furthermore, the open-set detection model does not need to precisely determine the type of goods, thereby increasing its recognition speed. The visual recognition model identifies out-of-stock goods, speeding up the identification process. Furthermore, because the open-set detection model only needs to distinguish between goods and non-goods, it can reduce the number of recognition errors caused by the recognition model's fine-grained product type identification, thereby improving the accuracy of out-of-stock product identification.

[0077] In some embodiments, the step of using an open set detection model to respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image includes: Identifying the product in the first image using the open set detection model, and marking the first product image corresponding to the identified product in the first image using a predetermined identifier; Identifying the product in the second image using the open set detection model, and marking the second product image corresponding to the product identified in the second image using a predetermined identification marker; The comparing the first product image corresponding to the product in the first image with the second product image corresponding to the product in the second image includes: The first product image marked with the predetermined identifier is compared with the second product image marked with the predetermined identifier.

[0078] In one possible implementation, the information processing device has a display component, and the information processing device can display at least one of the first image, the second image, the first product image, the second product image, and the third product image on the display component.

[0079] In a possible implementation, each commodity is marked with the same type of predetermined identification.

[0080] In a possible implementation, each commodity is marked with a different type of predetermined identification.

[0081] For example, the first product image in the first image and the second product image in the second image are both Figure 5 and Figure 6 The shaded box shown is used to identify them.

[0082] When comparing the first image and the second image, the first product image and the second product image marked with a predetermined identifier can be compared. For example, the portions marked by shaded boxes in the first image and the second image can be compared. This can then determine whether the third product image is reduced in the second image compared to the first image.

[0083] In one possible implementation, the position of each first product image in the first image relative to the shelf and the position of each second product image in the second image relative to the shelf can be first determined. Out-of-stock locations on the shelves that have first product images in the first image but not in the second image can then be determined, and the first product images corresponding to the out-of-stock locations can be used as third product images.

[0084] For example, Figure 5 As shown, the first image has products A to G and products 1 to 6, and the open set detection model uses a shaded box to mark the first image of all products. Figure 6 In the second image shown, some products are removed. The open set detection model also uses shaded boxes to mark the second image of all products. The information processing device can compare the product images marked with shaded boxes one by one to determine the third product image that is missing from the second image relative to the first image (the missing third product image is as follows: Figure 7 shown).

[0085] By tagging the first and second product images, the tagged first and second product images can be directly compared without comparing other parts of the first and second images, improving comparison efficiency. Furthermore, by tagging the first and second product images, users can intuitively understand the processing status of the information processing device, thereby improving communication and user experience.

[0086] In some embodiments, comparing a first product image corresponding to a product in the first image with a second product image corresponding to a product in the second image, and determining a third product image corresponding to a product that is out of stock less than that in the first image, includes: A feature matching model is used to match the first product image in the first image with the second product image in the second image, and the first product image that is not matched with the corresponding second product image is determined as the third product image corresponding to the out-of-stock product.

[0087] Here, the feature matching model can be used to extract feature parameters (such as image features, etc.) from the first product image and the second product image as matching entities, and to achieve registration of the first product image and the second product image by calculating the similarity between the matching entities.

[0088] The feature matching model can accurately match the first product image and the second product image, reduce matching errors caused by differences in shooting parameters (such as shooting equipment, shooting angle and / or shooting environment) between the first and second images and / or differences in product images (such as differences between the first and second product images of the same product due to product movement), and improve the accuracy of determining the third product image.

[0089] In some embodiments, the method further includes: training a visual recognition model capable of identifying a first category of goods so that the visual recognition model can identify a second category of goods, wherein the second category is a subcategory of the first category.

[0090] Here, the visual recognition model can be adjusted so that it can perform one-step recognition of the second category. For example, the first category may be beverages of Brand 1, and further training can be used to enable the visual recognition model to perform one-step recognition of the flavor of beverages of Brand 1.

[0091] The visual recognition model can realize the recognition of the second category based on the recognition of the product image and the recognition of the text in the product image.

[0092] By identifying the subcategories of the first category of goods, it is possible to further segment the goods and refine the classification of goods, thereby providing more accurate out-of-stock goods information.

[0093] In some embodiments, the step of training a visual recognition model capable of identifying a first category of goods so that the visual recognition model can identify a second category of goods comprises: Using a second training product image as input to the visual recognition model and using a product description including a second category description of the product in the second training product image as output of the visual recognition model to train the visual recognition model; In response to the visual recognition model identifying a second category of the commodity in the unlabeled commodity image, it is determined that the visual recognition model has completed training.

[0094] Specifically, during the training of the visual recognition model, the second classification description can be added to the product description as the output of the visual recognition model. This allows the visual recognition model to fine-tune its parameters during training and thus output the second classification.

[0095] For example, the trained visual recognition model can also be trained by inputting irregular commodities into the visual recognition model, fine-tuning parameters, and training multiple batches (such as 20 batches) to improve the recognition ability of the visual recognition model.

[0096] The visual recognition model is fed with a second training product image and textual descriptions (product descriptions). For example, if an image containing Beverage A is input, the model can identify images containing other flavors of Beverage A. For example, the large visual model can analyze the text and image features in the input product image and output the product information for the product image as: Peach-flavored Beverage A.

[0097] In this way, the third product image of the out-of-stock product is fine-grainedly recognized through the visual recognition model, and the product information of the out-of-stock product (such as product name and product type, etc.) is obtained, thereby improving the accuracy of product recognition.

[0098] In some embodiments, the method further includes: training the open set detection model using a first training product image of a first non-standard appearance product labeled as a predetermined product type.

[0099] In one possible implementation, the commodity type determined by the open set detection model can be used to distinguish commodities from non-commodities. For example, the commodity type can be predetermined to include "commodity".

[0100] In one possible implementation, the commodity type determined by the open set detection model can be used to distinguish different types of commodities. For example, commodity types can be predetermined to include "beverages," "dogs," and the like.

[0101] Products with non-standard appearance can be products that break through the industry's conventional form cognition.

[0102] In one possible implementation, non-standard appearance merchandise may include at least one of the following: Products whose appearance deviates from the typical shape required for the product's inherent function (e.g. biscuits are usually round / square, but designed in the shape of animals); Products that directly imitate real-life objects; for example, imitating living things (tree-shaped pencils, dinosaur erasers), imitating abstract cultural symbols (such as star-shaped lamps).

[0103] By using the first training product image of the first non-standard appearance product to train the open set detection model, the open set detection model can be enabled to identify the non-standard appearance product, thereby improving the recognition rate of the open set detection model and further improving the accuracy of determining out-of-stock products.

[0104] In some embodiments, the method further comprises: In response to the open set detection model being able to identify the product type of the second non-standard appearance product in the product image, it is determined that the open set detection model has completed training.

[0105] Here, the open set detection model can be trained using labeled products as a training set. For example, in a retail scenario, different products labeled as “products” in the retail scenario can be used as a training set to train the open set detection model, so that the open set detection model can recognize unlabeled product images.

[0106] The open set detection model can be further trained for actual scenarios. Here, targeted training can be training for non-standard products. Non-standard products can include products with irregular shapes and other products whose image features match the product features to a greater than a threshold. For example: a pencil in the shape of a potted plant, etc. Targeted training includes (inputting images of non-standard products in the retail industry into the open set target detection model, and defining the labels as products for training iterations until the training is completed (the loss function reaches the preset conditions or the training cycle reaches the preset value). The fine-tuned open set detection model can identify non-standard product images as products. If the training category is a further subdivided beverage or "dog", the open set detection model can only select beverage products or products belonging to the "dog" category when used, and compare the missing "beverage" or "dog" selected in the previous and next frames. Subsequently, the visual recognition model is used to further identify the type and characteristics of beverages or the breed and characteristics of dogs.

[0107] For targeted training, the first non-standard appearance product image can be input into the open set detection model, and the label can be defined as "product" for training iteration until the training is completed (the loss function reaches the preset condition or the training cycle reaches the preset value), so that the open set detection model after targeted training can identify the second non-standard appearance product as "product".

[0108] By training the open set detection model, we can adapt to different product appearances, improve the accuracy of the open set detection model in identifying products, and thus improve the accuracy of determining out-of-stock products.

[0109] In some embodiments, the step of training the open set detection model using a first training product image of a first non-standard-appearance product labeled as a predetermined product type includes at least one of the following: Using the first training product image as input to the open set detection model, using the product type of the first non-standard appearance product as labeling information of the first training product image, and training the output of the open set detection model; The first training product image is used as input of the open set detection model, and position information of the product type of the first non-standard appearance product in the image description of the first training product image is specified to train the output of the open set detection model.

[0110] Here, the first training product image can be a single image or a portion of an image. During training, the location of the first non-standard-appearance product can be specified. For example, the first training product image can be specified by the location range of the first non-standard-appearance product in the image.

[0111] When training the open set detection model, the labeling information of the first training product image may be directly provided, or the labeling information of the first training product image may be specified by indicating the image description position information.

[0112] The first training product image can be used as the input of the open set detection model, and the annotation information can be used as the output of the open set detection model to train the open set detection model.

[0113] In a possible implementation, the above two methods of indicating the labeling information can be applied to the training of non-standard appearance products, and can also be applied to the training of general products.

[0114] For example, this embodiment provides two formats of two training sets in code form: Format 1: Data format for target detection: {"filename":"images.jpg", "height": 512, "width": 769, "detection": { "instances": [ {"bbox": [109.4768676992, 346.0190429696, 135.1918335098, 365.3641967616], "label": 2, "category": "goods"}, {"bbox": [58.612365705900004, 323.2281494016, 242.6005859067, 451.4166870016], "label": 8, "category": "car"} ]}} In format 1, the training product image images.jpg is provided. The positions of the first training product images of the two first non-standard appearance products in the training product image are specified through "bbox". The product types "category" of the two first non-standard appearance products are given: "goods" and "car".

[0115] Format 2: The second format is the data format: {"filename":"2405116.jpg", "height": 375, "width": 500, "grounding": {"caption":"Two surfers walking down the shore. sand on the beach.", "regions": [{"bbox": [206, 156, 282, 248], "phrase": "Two surfers", "tokens_positive": [[0, 3], [4, 11]]}, {"bbox": [303, 338, 443, 343], "phrase": "sand", "tokens_positive": [[36, 40]]}, {"bbox": [[327, 223, 421, 282], [300, 200, 400, 210]], "phrase": "beach", "tokens_positive": [[48, 53]]} ]}} In format 2, a training product image (2405116.jpg) is provided. The "regions" and "bbox" fields specify the locations of three objects to be identified within the training image. The "positive tokens" (tokens_positive) indicate the location of each object within the training image description: "Two surfers walking down the shore. Sand on the beach." For example, if "tokens_positive": [[36, 40]], the object to be identified is labeled as characters 36 to 40 in the training image description.

[0116] In some embodiments, after determining the product information of the out-of-stock product corresponding to the third product image, the method further includes: An out-of-stock indication is sent, indicating product information corresponding to the out-of-stock product.

[0117] Here, the out-of-stock indication can be used to instruct the user to replenish the out-of-stock product. The out-of-stock indication can also indicate the quantity of the out-of-stock product determined based on the product information of the out-of-stock product.

[0118] Sending an out-of-stock indication indicating the product information corresponding to the out-of-stock product may include at least one of the following: displaying an out-of-stock indication on a display unit of the information processing device to indicate product information of the out-of-stock product; Send out-of-stock indications to user terminals (such as mobile phones, etc.) to indicate product information of out-of-stock products.

[0119] The following provides multiple specific examples in combination with any of the above embodiments: This example discloses a method for detecting out-of-stock items on a shelf. Figure 8 Shown, including: Step 801: Train the open set detection model. Based on the general open set detection model, use the labeled product dataset for training to obtain an open set detection model suitable for the product task. For example, you can choose the highly real-time open-set detection model Grounding-DINO for fine-tuning, and use the publicly available pre-trained model of the Grounding-DINO detection model. The pre-trained model of Grounding-DINO is trained on the COCO, RefC, O365V2, GoldG, GRIT, Open-Images, and V3Det datasets.

[0120] Because the data categories in public datasets aren't fully adapted to retail scenarios, we use labeled retail datasets for targeted training. This training involves feeding images of non-standard retail products (with non-standard appearances) into an open-set object detection model, labeling them as goods, and iterating until training is complete (e.g., the loss function meets a preset condition or the training cycle reaches a preset value). The fine-tuned open-set detection model is sufficient to identify non-standard product images as goods.

[0121] If the training category is further subdivided into beverages or "dogs," the open-set detection model can select only beverage products or products belonging to the "dog" category, and compare the missing "beverages" or "dogs" selected in the previous and next frames. The large-scale visual model can then be used to further identify the type and characteristics of the beverage or the breed and characteristics of the dog.

[0122] Step 802: Use the open set detection model to detect the initial image. Use the trained open set detection model to detect all the products on the shelf and generate a complete product inventory reference image, such as Figure 5 shown.

[0123] Specifically, frame images from the video stream of an obliquely mounted camera device or images taken by a mobile phone or other device are obtained, and the trained open set detection model is used to obtain the detection frame and category of the product (the category is the product), and the detection result is used as the category reference image of the shelf.

[0124] Step 803: Use the open set detection model to detect the real-time image. Use the trained open set detection model to detect all the products on the shelf in real time and generate a real-time image. Figure 6 shown.

[0125] Step 804: Use feature matching algorithm to compare the distribution of goods in the initial image and the real-time image, and mark the goods that are not matched successfully as out-of-stock goods. Figure 7 shown.

[0126] Specifically: 1) At specific inspection times, use a mobile phone or other camera to capture shelf images, or use an obliquely mounted camera to obtain real-time image frames. Use the trained open-set detection model to obtain the detection frame and category of the product.

[0127] 2) Use feature matching to match the objects in the current image with those in the reference image and filter out out-of-stock objects.

[0128] Step 805: Use the large visual model to perform fine-grained recognition of out-of-stock products and obtain product name and category information.

[0129] Specifically, the out-of-stock object images are sequentially input into the visual model, which then outputs the recognized object names. The visual model can be an open-source model such as VITA-1.5.

[0130] This example also discloses a shelf commodity out-of-stock detection system. Figure 9 This is a schematic diagram of the operation of the shelf commodity out-of-stock detection system in this example, including steps 901 to 906: Step 901: Set up an image acquisition device to capture original product images (i.e., initial images, first images) and real-time product images (i.e., second images). An obliquely mounted camera device mounted in a supermarket or store captures the original products on the shelf and the real-time products on the shelf.

[0131] Step 902: Load the open set detection model, and pre-load the trained open set detection model adapted to the product task.

[0132] Before step 902, the open set detection model is trained, specifically including: Step 9021: Fine-tune the general open-set detection model and use its publicly available pre-trained model to train on a public dataset. Then, use a labeled retail scenario dataset for targeted training (supplementing the retail industry data with product data, including special products such as irregularly shaped products like potted pencils) to make the open-set detection model better suited for retail scenarios.

[0133] Step 903: Detect the original product image. Initially, use the open set detection model to detect the collected original product image of the shelf, obtain the detection frame and category of the product, and use the detection result as the reference image of the shelf.

[0134] Step 904: Detect real-time product images. Use a camera or a photographic device to obtain real-time product images on the shelf, and use the trained open-set detection model to obtain the detection frame and category of the product.

[0135] Step 905: Feature matching: Use a feature matching method to match the images and filter out out-of-stock objects.

[0136] Step 906: Use the visual big model to perform fine-grained recognition of out-of-stock products and obtain product information (product name and product category).

[0137] Specifically, you can fine-tune the currently open-source visual big model (VITA 1.5) and use its publicly available pre-trained model to train on a public dataset. Input the visual big model for training on irregular products, fine-tune the parameters, and train for approximately 20 batches to determine if training is complete. Input product images and textual descriptions of the products. For example, if an image of original beverage A is input, the visual big model will be able to recognize images of beverage A with other flavors. The visual big model analyzes the text and image features in the input image and outputs the product information in the image as: peach-flavored beverage A. By inputting images of out-of-stock products into the visual big model one by one, the recognized product names will be output.

[0138] Figure 10 The embodiment of the present application provides a device 100 for determining product information, the device comprising: a processing module 110, the processing module being configured to: Using an open set detection model, respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image, wherein the first image is an image of the target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; comparing a first product image corresponding to a product in the first image with a second product image corresponding to the product in the second image, and determining a third product image corresponding to a product that is out of stock and has a smaller number of products in the second image than in the first image; The third product image is recognized using a visual recognition model, and product information of the out-of-stock product corresponding to the third product image is determined.

[0139] In some embodiments, the processing module is specifically configured to: Identifying the product in the first image using the open set detection model, and marking the first product image corresponding to the identified product in the first image using a predetermined identifier; Identifying the product in the second image using the open set detection model, and marking a second product image corresponding to the product identified in the second image using a predetermined identification marker; The first product image marked with the predetermined identifier is compared with the second product image marked with the predetermined identifier.

[0140] In some embodiments, the apparatus further comprises: a training module 120, The training module is used to train the open set detection model using a first training product image of a first non-standard appearance product marked as a predetermined product type.

[0141] In some embodiments, the training module is specifically used for at least one of the following: Using the first training product image as input to the open set detection model, using the product type of the first non-standard appearance product as labeling information of the first training product image, and training the output of the open set detection model; The first training product image is used as input of the open set detection model, and position information of the product type of the first non-standard appearance product in the image description of the first training product image is specified to train the output of the open set detection model.

[0142] In some embodiments, the training module is further configured to: determine that the open set detection model has completed training in response to the open set detection model being able to identify the product type of the second non-standard appearance product in the product image.

[0143] In some embodiments, the apparatus further comprises: a training module, The training module is used to train a visual recognition model that can identify a first category of goods, so that the visual recognition model can identify a second category of goods, wherein the second category is a subcategory of the first category.

[0144] In some embodiments, the training module is specifically used to: Using a second training product image as input to the visual recognition model and using a product description including a second category description of the product in the second training product image as output of the visual recognition model to train the visual recognition model; In response to the visual recognition model identifying a second category of the commodity in the unlabeled commodity image, it is determined that the visual recognition model has completed training.

[0145] It should be understood that the division of the various units or modules in the above-mentioned devices is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units or modules in the device may be implemented in the form of a processor invoking software: for example, the device includes a processor connected to a memory storing instructions, and the processor invokes the instructions stored in the memory to implement any of the above-mentioned methods or the functions of the various units or modules in the above-mentioned device. The processor may be, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory may be internal or external to the device. Alternatively, the units or modules in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units or modules may be implemented by designing the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above units or modules may be implemented by designing the logical relationships between the components within the circuit. For another example, in another implementation, the hardware circuit may be implemented by a programmable logic device (PLD), such as a field programmable gate array (FPGA), which may include a large number of logic gate circuits. The connections between the logic gate circuits are configured through configuration files, thereby implementing the functions of some or all of the above units or modules. All units or modules of the above devices may be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software, with the remainder implemented by hardware circuits.

[0146] In the embodiments of the present disclosure, a processor is a circuit with data processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the above hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0147] Figure 11 is a schematic diagram of the structure of an electronic device 9100 provided in an embodiment of the present disclosure. Electronic device 9100 may be a computer device, a terminal, or a chip, a chip system, or a processor that supports implementation of any of the above methods. Electronic device 9100 may be used to implement the method for determining product information described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[0148] like Figure 10 As shown, the electronic device 9100 includes one or more processors 9101. The processor 9101 can be a general-purpose processor or a dedicated processor, etc. The processor 9101 is used to call instructions to enable the electronic device 9100 to execute any of the above product information determination methods.

[0149] In some embodiments, the electronic device 9100 further includes one or more memories 9102 for storing instructions. Optionally, all or part of the memories 9102 may be located outside the electronic device 9100.

[0150] In some embodiments, the electronic device 9100 further includes one or more transceivers 9103. When the electronic device 9100 includes one or more transceivers 9103, the steps of sending, receiving, and / or acquiring in the above method are performed by the transceiver 9103, and the other steps are performed by the processor 9101.

[0151] In some embodiments, the steps of obtaining and the like in the above method may also be executed by the processor 9101 , for example, obtaining information from the memory 9102 .

[0152] In some embodiments, a transceiver may include a receiver and a transmitter, which may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0153] Optionally, the electronic device 9100 further includes one or more interface circuits 9104, which are connected to the memory 9102. The interface circuits 9104 can be used to receive signals from the memory 9102 or other devices, and can be used to send signals to the memory 9102 or other devices. For example, the interface circuits 9104 can read instructions stored in the memory 9102 and send the instructions to the processor 9101.

[0154] The electronic device 9100 described in the above embodiments may be a network device or a terminal, but the scope of the electronic device 9100 described in the present disclosure is not limited thereto. The structure of the electronic device 9100 may not be limited thereto. Figure 10 The electronic device may be an independent device or a part of a larger device. For example, the electronic device may be: (1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data and programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a network device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0155] Those skilled in the art will appreciate that all or part of the steps of the above method embodiments may be implemented by hardware related to program commands, and the aforementioned program may be stored in a storage medium, including various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0156] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several commands for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0157] It should be understood that the above embodiments are exemplary and are not intended to include all possible implementations. Various modifications and changes may be made to the above embodiments without departing from the scope of the present disclosure. Similarly, the various technical features of the above embodiments may be arbitrarily combined to form other embodiments of the present invention that may not be explicitly described. Therefore, the above embodiments merely express several implementations of the present invention and do not limit the scope of protection of the patent of the present invention.

Claims

1. A method for determining product information, characterized in that: The method comprises: Using an open set detection model, respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image, wherein the first image is an image of the target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; comparing a first product image corresponding to a product in the first image with a second product image corresponding to a product in the second image, and determining a third product image corresponding to a product that is out of stock and has a smaller number of products in the second image than in the first image; The third product image is recognized using a visual recognition model, and product information of the out-of-stock product corresponding to the third product image is determined.

2. The commodity information determination method according to claim 1, wherein: The method of using an open set detection model to respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image includes: Identifying the product in the first image using the open set detection model, and marking the first product image corresponding to the identified product in the first image using a predetermined identifier; Identifying the product in the second image using the open set detection model, and marking the second product image corresponding to the product identified in the second image using a predetermined identification marker; The comparing the first product image corresponding to the product in the first image with the second product image corresponding to the product in the second image includes: The first product image marked with the predetermined identifier is compared with the second product image marked with the predetermined identifier.

3. The commodity information determination method according to claim 1, wherein: The method further comprises: The open set detection model is trained using a first training product image of a first non-standard appearance product labeled as a predetermined product type.

4. The commodity information determination method according to claim 3, wherein: The training of the open set detection model using a first training product image of a first non-standard appearance product labeled as a predetermined product type includes at least one of the following: Using the first training product image as input to the open set detection model, using the product type of the first non-standard appearance product as labeling information of the first training product image, and training the output of the open set detection model; The first training product image is used as input of the open set detection model, and position information of the product type of the first non-standard appearance product in the image description of the first training product image is specified to train the output of the open set detection model.

5. The commodity information determination method according to claim 3, wherein: The method further comprises: In response to the open set detection model being able to identify the product type of the second non-standard appearance product in the product image, it is determined that the open set detection model has completed training.

6. The commodity information determination method according to claim 1, wherein: The method further comprises: A visual recognition model capable of recognizing a first category of commodities is trained so that the visual recognition model can recognize a second category of commodities, wherein the second category is a subcategory of the first category.

7. The commodity information determination method according to claim 6, characterized in that: The visual recognition model capable of identifying the first category of commodities is trained so that the visual recognition model can identify the second category of commodities; include: Using a second training product image as input to the visual recognition model and using a product description including a second category description of the product in the second training product image as output of the visual recognition model to train the visual recognition model; In response to the visual recognition model identifying a second category of the commodity in the unlabeled commodity image, it is determined that the visual recognition model has completed training.

8. A commodity information determination device, characterized in that: The device comprises: a processing module, wherein the processing module is configured to: Using an open set detection model, respectively determine a first product image corresponding to the product in the first image and a second product image corresponding to the product in the second image, wherein the first image is an image of the target area at a first moment, and the second image is an image of the target area at a second moment, wherein the first moment is before the second moment; comparing a first product image corresponding to a product in the first image with a second product image corresponding to a product in the second image, and determining a third product image corresponding to a product that is out of stock and has a smaller number of products in the second image than in the first image; The third product image is recognized using a visual recognition model, and product information of the out-of-stock product corresponding to the third product image is determined.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; The processor is configured to call instructions to enable the electronic device to execute the commodity information determination method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed on the electronic device, the electronic device executes the commodity information determination method according to any one of claims 1 to 7.

Images