Product inspection method, system and apparatus, program product, and storage medium

Abstract

A product inspection method, system and apparatus, a program product, and a storage medium. The method comprises: acquiring product images corresponding to a product to be inspected, wherein the product images are foreground images obtained by photographing said product from multiple angles (S701); performing imaging quality monitoring on the product images to obtain an imaging quality monitoring result (S702); and if the imaging quality monitoring result indicates that the product images have no abnormalities, using an AI model to inspect said product on the basis of the product images to obtain a product inspection result (S703). In the present invention, the AI model is used to inspect said product, so that the efficiency and accuracy of product inspection can be improved.

Description

A product, system, apparatus and procedure for inspecting goods, and storage medium.

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese patent application CN202510321728.2, filed on March 18, 2025, entitled “A commodity inspection method, system, apparatus and program product, storage medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of image processing technology, and more specifically, to a product inspection method, system, apparatus, program product, and storage medium. Background Technology

[0004] Today, the consumer market is booming, with diverse online and offline shopping channels and a dazzling array of goods. However, with technological advancements, counterfeiting techniques are constantly "upgrading," making counterfeit goods increasingly difficult to distinguish. From brand-name clothing to electronics, from cosmetics and skincare to food and medicine, counterfeit goods have infiltrated every sector, and consumers can easily fall victim to scams. Therefore, to address these issues, adopting a "verification before shipping" shopping process is particularly important. By adding services such as authenticating products and checking for defects, consumer rights can be effectively protected.

[0005] Currently, most services for identifying authenticity and checking for defects still rely on manual inspection. Taking footwear as an example, operators need to follow procedures to remove the shoes from the shoebox and check the shoebox, shoes, and accessories for defects and authenticity. This method is inefficient, costly, and easily affected by human factors, leading to inaccurate test results. Summary of the Invention

[0006] The purpose of this application is to provide a commodity inspection method, system, device, program product, and storage medium to solve the technical problems of low efficiency and inaccurate test results caused by manual inspection methods in the prior art.

[0007] In a first aspect, embodiments of this application provide a product inspection method, comprising: acquiring a product image corresponding to the product to be inspected, wherein the product image is a foreground image obtained by taking pictures of the product to be inspected from multiple angles; performing imaging quality monitoring on the product image to obtain an imaging quality monitoring result; if the imaging quality monitoring result indicates that the product image has no abnormalities, then using an AI model to inspect the product to be inspected based on the product image to obtain a product inspection result.

[0008] In the above solution, firstly, multi-angle photographs of the product to be inspected are taken to obtain corresponding product images, thus obtaining a more complete image of the product. Then, the image quality of the product images is monitored in real time to check whether the image quality meets the requirements. Finally, if the image quality monitoring results indicate that the product images are normal, an AI model is used to inspect the product based on the product images. Compared with the manual inspection methods in the prior art, this embodiment of the application utilizes an AI model to inspect the product, which can improve the efficiency and accuracy of product inspection.

[0009] In an optional implementation, the step of monitoring the imaging quality of the product image to obtain the imaging quality monitoring result includes: analyzing the imaging quality of a single frame of the product image to obtain imaging anomaly information corresponding to the product image; determining whether the image acquisition device has factors affecting imaging by aggregating imaging anomaly information corresponding to multiple frames of product images from a certain image acquisition device within a preset time period; if the image acquisition device has the factors affecting imaging, then issuing an alarm and / or outputting an intervention prompt based on the factors affecting imaging. In the above scheme, by aggregating imaging anomaly information corresponding to multiple frames of product images from an image acquisition device within a preset time period, the image acquisition device where the anomaly occurs can be located, thereby promptly alerting on-site personnel to handle the situation when an anomaly occurs, thus improving the efficiency and accuracy of product inspection.

[0010] In an optional implementation, the step of performing imaging quality analysis on a single-frame product image to obtain imaging anomaly information corresponding to the product image includes: inputting the product image into an imaging quality analysis model to obtain the imaging anomaly information output by the imaging quality analysis model; wherein, the imaging quality analysis model is obtained by training a deep learning model using imaging anomaly samples. Before inputting the product image into the imaging quality analysis model to obtain the imaging anomaly information output by the imaging quality analysis model, the method further includes: obtaining the imaging anomaly samples using at least one of the following methods: simulating and collecting imaging anomaly cases through a laboratory setting; synthesizing the imaging anomaly samples using an image synthesis algorithm combined with normal image data; or performing semi-supervised training using a pre-trained model to obtain the imaging anomaly samples for training the imaging quality analysis model. The above methods can generate a large number of imaging anomaly samples, thereby improving the performance of the trained imaging quality analysis model.

[0011] In an optional implementation, the product inspection result includes at least one of product quality inspection result, product identification result, and product fingerprint result. The step of using an AI model to inspect the product based on the product image to obtain the product inspection result includes at least the following steps: using the AI ​​model to perform quality inspection on the product based on the product image to obtain a product quality inspection result, wherein the product quality inspection result indicates whether the product has any appearance defects; using the AI ​​model to identify the product based on the product image to obtain a product identification result, wherein the product identification result indicates whether the product is genuine; and using the AI ​​model to perform fingerprint recognition on the product based on the product image to obtain a product fingerprint result, wherein the product fingerprint result indicates whether the product exists in the product database. In the above solution, by performing quality inspection on the product to be inspected, it can be determined whether the product has any appearance defects. Compared with the manual quality inspection method in the prior art, the embodiments of this application utilize an AI model to perform quality inspection on the product to be inspected, which can improve the efficiency and accuracy of product quality inspection. By identifying the goods to be inspected, it can be determined whether the goods are genuine. Compared with the manual identification methods in the prior art, the embodiments of this application utilize AI models to identify the goods to be inspected, which can improve the efficiency and accuracy of goods identification. By performing fingerprint recognition on the goods to be inspected, it can be determined whether the goods to be inspected exist in the goods database. Compared with the manual identification methods in the prior art, the embodiments of this application utilize AI models to perform fingerprint recognition on the goods to be inspected, which can improve the efficiency and accuracy of goods fingerprint recognition.

[0012] In an optional implementation, the step of using the AI ​​model to perform quality inspection on the product to be inspected based on the product image to obtain product quality inspection results includes: using an image feature extraction model to extract features from the product image to obtain product feature information; using a text feature extraction model to extract features from the product information corresponding to the product to be inspected to obtain text feature information, wherein the product information includes product attribute information, or the product information includes product attribute information and product circulation information; and using a product quality inspection model to perform quality inspection on the product to be inspected based on the product feature information and the text feature information to obtain the product quality inspection results. In the above scheme, by extracting product feature information from the product image and combining it with the text feature information extracted from the product information to perform quality inspection on the product, the product feature information can reflect more detailed features of the product, while the text feature information can provide more background information as an aid to product quality inspection. This can reduce misjudgments that may be caused by factors such as image noise and shooting angle when relying solely on image features for quality inspection. Combining product information can provide more comprehensive verification information, reduce misjudgments, and improve the accuracy and efficiency of quality inspection.

[0013] In an optional implementation, the step of using the AI ​​model to identify the product to be inspected based on the product image and obtaining a product identification result includes: using an image feature extraction model to extract features from the product image to obtain product feature information; using a text recognition model to perform text recognition on the product image to obtain product attribute information; using a text feature extraction model to extract features from the product additional information corresponding to the product to be inspected to obtain text feature information, wherein the product additional information is information about the product to be inspected other than the product attribute information; and using a product identification model to identify the product to be inspected based on the product feature information, the product attribute information, and the text feature information to obtain the product identification result. In the above scheme, by extracting product feature information and product attribute information from the product image and combining it with the text feature information extracted from the product information to identify the product, the product feature information and product attribute information can reflect more detailed features of the product, while the text feature information can provide more background information as an aid to product identification, thereby providing more comprehensive verification information and improving the accuracy and efficiency of identification.

[0014] In an optional implementation, the step of using the AI ​​model to perform fingerprint recognition on the product to be inspected based on the product image to obtain a product fingerprint result includes: extracting features from the product image using a first image feature extraction model to obtain first product feature information; extracting features from the microscopic image corresponding to the product to be inspected using a second image feature extraction model to obtain second product feature information, wherein the microscopic image is obtained by taking a picture of the product to be inspected using an image acquisition device with a magnification greater than a preset magnification threshold; determining a product fingerprint to be matched based on the first product feature information and the second product feature information; and matching the product fingerprint to be matched in a product fingerprint database to obtain the product fingerprint result. In the above scheme, by fusing macroscopic features extracted from multi-view images and microscopic features extracted from microscopic images to obtain product fingerprint features, and then matching the product fingerprint features, macroscopic features can capture information such as the overall appearance and shape of the product, while microscopic features can capture detailed texture information of specific parts of the product. By fusing macroscopic and microscopic features, the product fingerprint can more comprehensively reflect the product characteristics, which is beneficial to improving the accuracy of product recognition.

[0015] In an optional implementation, after obtaining the product image corresponding to the product to be inspected, the method further includes: extracting features from the product image to obtain product feature information; and comparing the product feature information with product image features in the database to obtain product information corresponding to the product to be inspected. In the above scheme, the product information can provide more background information as an aid to product inspection, thereby providing more comprehensive verification information and improving the accuracy and efficiency of inspection.

[0016] Secondly, embodiments of this application provide a commodity inspection system, comprising: a commodity inspection device, the commodity inspection device including an image acquisition unit and a controller, the image acquisition unit being used to acquire commodity images corresponding to the commodity to be inspected, and the controller being used to execute the commodity inspection method as described in the first aspect; and a server, the server having a system management platform deployed thereon, the system management platform being used to manage the commodity inspection device.

[0017] In the above scheme, the controller in the commodity inspection device can execute the commodity inspection method locally and use an AI model to inspect the commodity to be inspected based on the commodity image. Compared with the manual inspection method in the prior art, the embodiment of this application uses an AI model to inspect the commodity to be inspected, which can improve the efficiency and accuracy of commodity inspection.

[0018] Thirdly, embodiments of this application provide a commodity inspection system, comprising: a commodity inspection device, the commodity inspection device including an image acquisition unit and a controller, the image acquisition unit being used to acquire commodity images corresponding to the commodity to be inspected, and the controller being used to control the image acquisition unit; and a server, the server being deployed with a system management platform and a cloud computing module, the system management platform being used to manage the commodity inspection device, and the cloud computing module being used to execute the commodity inspection method as described in the first aspect.

[0019] In the above solution, the cloud computing module deployed on the server can execute the commodity inspection method in the cloud and use an AI model to inspect the commodity based on the commodity image. Compared with the manual inspection method in the prior art, the embodiment of this application uses an AI model to inspect the commodity, which can improve the efficiency and accuracy of commodity inspection.

[0020] Fourthly, embodiments of this application provide a commodity inspection device, comprising: an image acquisition unit for acquiring commodity images corresponding to the commodity to be inspected; a controller for controlling the image acquisition unit and for executing the commodity inspection method as described in the first aspect.

[0021] In the above scheme, the controller in the commodity inspection device can execute the commodity inspection method locally and use an AI model to inspect the commodity to be inspected based on the commodity image. Compared with the manual inspection method in the prior art, the embodiment of this application uses an AI model to inspect the commodity to be inspected, which can improve the efficiency and accuracy of commodity inspection.

[0022] In an optional implementation, the product inspection device further includes: an interactive interface, and / or, an interactive device. In the above solution, on-site personnel can interact with the product inspection device based on its interactive interface or the interactive device within it, thereby increasing the interactivity between the user and the device.

[0023] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the commodity inspection method as described in the first aspect.

[0024] Sixthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a computer, cause the computer to perform the commodity inspection method as described in the first aspect.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, embodiments of this application are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 is a structural block diagram of a commodity inspection device provided in an embodiment of this application;

[0028] Figure 2 is a schematic diagram of another commodity inspection device provided in an embodiment of this application;

[0029] Figure 3 is a schematic diagram of a commodity inspection system provided in an embodiment of this application;

[0030] Figure 4 is a framework diagram of a system management platform provided in an embodiment of this application;

[0031] Figure 5 is a schematic diagram of another commodity inspection system provided in an embodiment of this application;

[0032] Figure 6 is a framework diagram of a controller provided in an embodiment of this application;

[0033] Figure 7 is a flowchart of a commodity inspection method provided in an embodiment of this application;

[0034] Figure 8 is an overall flowchart of an imaging quality monitoring module provided in an embodiment of this application;

[0035] Figure 9 is an overall flowchart of an imaging anomaly decision module provided in an embodiment of this application;

[0036] Figure 10 is an overall flowchart of an imaging anomaly intervention module provided in an embodiment of this application;

[0037] Figure 11 is a schematic diagram of an imaging anomaly sample collection framework provided in an embodiment of this application;

[0038] Figure 12 is an overall flowchart of an AI module provided in an embodiment of this application;

[0039] Figure 13 is a flowchart of a commodity quality inspection method provided in an embodiment of this application;

[0040] Figure 14 is a schematic diagram of the structure of an image feature extraction model provided in an embodiment of this application;

[0041] Figure 15 is a schematic diagram of the structure of a commodity quality inspection model provided in an embodiment of this application;

[0042] Figure 16 is a flowchart of a commodity identification method provided in an embodiment of this application;

[0043] Figure 17 is a flowchart of a product fingerprint recognition method provided in an embodiment of this application;

[0044] Figure 18 is an overall flowchart of a product fingerprint recognition method provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0046] This application provides a product inspection method. The method first obtains corresponding product images by taking pictures of the product to be inspected from multiple angles. Then, the imaging quality of the product images is monitored. Finally, if the imaging quality monitoring results indicate that the product images are normal, an AI model is used to inspect the product based on the product images, thereby improving the efficiency and accuracy of product inspection.

[0047] The following describes the commodity inspection device used in the commodity inspection method provided in this application embodiment. Please refer to Figure 1. Figure 1 is a structural block diagram of a commodity inspection device provided in this application embodiment. The commodity inspection device 100 may include: an image acquisition unit 110 and a controller 120.

[0048] Specifically, the image acquisition device 110 is used to acquire images of the goods to be inspected. As one implementation, the image acquisition device 110 can be a camera. The goods inspection device 100 may include multiple image acquisition devices 110, which can be located at different positions in the area where the goods to be inspected are placed, and are used to acquire images of the goods in the goods inspection area from different positions.

[0049] The commodity inspection area refers to the area used to place the goods to be inspected. For example, the commodity inspection area can be a commodity cabinet. When commodity inspection is required, the goods to be inspected can be placed in the commodity cabinet, such as on the wire platform at the center of the commodity cabinet along the diagonal.

[0050] Image acquisition devices 110 can be installed at different locations inside the product cabinet. For example, image acquisition devices 110 can be installed on the top, bottom, left, right, front, and back surfaces inside the product cabinet to capture images of the product to be inspected from different angles, thereby obtaining multi-angle images of the product. At this time, multiple product images can be obtained. By combining the above multiple product images, the product inspection method of this solution can be used for inspection to achieve comprehensive inspection.

[0051] The controller 120 can be connected to the image acquisition unit 110 via a network cable or a Universal Serial Bus (USB) cable. The image acquisition unit 110 can receive the image capture command from the controller 120, complete the image capture, and return the image to the controller 120. In one implementation, the controller 120 can be used solely to control the image acquisition unit 110; in this case, the product inspection method of this solution can be executed by a cloud server or an external device. In another implementation, the controller 120 can be used to control the image acquisition unit 110 and to execute the product inspection method of this solution.

[0052] The controller 120 can refer to a host computer, such as a terminal or computer. Sensors can also be installed in the product inspection area to detect whether there are any products to be inspected within the area. When a sensor detects a product, it sends a signal to the controller 120. Upon receiving the signal, the controller 120 controls multiple image acquisition units 110 to acquire images. All acquired product images are then sent to the controller 120. Once the controller 120 receives the product images, it can proceed with the subsequent inspection process.

[0053] The controller 120 can also be a processor installed on the merchandise cabinet. In this case, the controller 120 and the image acquisition device 110 can be understood as an integrated device, namely the merchandise cabinet.

[0054] In the above scheme, the controller 120 in the commodity inspection device 100 can execute the commodity inspection method locally and use an AI model to inspect the commodity to be inspected based on the commodity image. Compared with the manual inspection method in the prior art, the embodiment of this application uses an AI model to inspect the commodity to be inspected, which can improve the efficiency and accuracy of commodity inspection.

[0055] Furthermore, based on the above embodiments, the commodity inspection device 100 provided in this application embodiment may further include: an interactive interface, and / or an interactive device 130.

[0056] Specifically, the interactive interface can be located on the device where the controller 120 is located, while the interactive device 130 can be a handheld terminal device. The interactive interface and the interactive device 130 are responsible for displaying and controlling the operating status of the product inspection device 100. After installing the corresponding operating software, the interactive device 130 can control a single product inspection device 100 or multiple product inspection devices 100; this embodiment does not impose specific limitations on this.

[0057] In one implementation, the interactive device 130 can communicate with the controller 120 via wireless communication methods such as Wi-Fi or Bluetooth.

[0058] In the above scheme, on-site personnel can interact with the product inspection device 100 based on the interactive interface of the product inspection device 100 or the interactive device 130 in the product inspection device 100, so as to increase the interactivity between the user and the device.

[0059] Furthermore, based on the above embodiments, the commodity inspection device 100 provided in this application embodiment may further include: a printing module, used to print an inspection certificate after the commodity inspection is completed and the results of quality inspection, identification, fingerprint recognition, etc. are obtained.

[0060] Furthermore, based on the above embodiments, the commodity inspection device 100 provided in this application embodiment may further include: an illumination module 150. The illumination module 150 may be a light bulb or a light shield installed inside the commodity cabinet, used to illuminate the commodity to be inspected while reducing external ambient light interference and ensuring the stability of the imaging illumination.

[0061] Please refer to Figure 2, which is a schematic diagram of another commodity inspection device provided in this application embodiment. In Figure 2, the image acquisition unit 110 is used to acquire commodity images corresponding to the commodity to be inspected; the controller 120 shown in Figure 2 is used to control the image acquisition unit 110 and to execute the commodity inspection method of this solution; the display / interaction module, i.e., the aforementioned interactive interface, and the handheld interactive device 130, i.e., the aforementioned interactive device 130, shown in Figure 2, are used to display and control the operating status of the commodity inspection device 100.

[0062] The following describes the commodity inspection system used in the commodity inspection method provided in the embodiments of this application. Among them, the embodiments of this application provide two commodity inspection systems.

[0063] Please refer to Figure 3, which is a schematic diagram of a commodity inspection system provided in an embodiment of this application. The commodity inspection system 30 may include: a commodity inspection device 100 and a server.

[0064] Specifically, the aforementioned product inspection device 100 may include an image acquisition unit and a controller. The image acquisition unit is used to acquire product images corresponding to the products to be inspected, and the controller is used to execute the product inspection method of this scheme. The product inspection device 100 has been described in detail in the above embodiments and will not be repeated here. In this case, the product inspection device 100 includes a complete computing unit and is suitable for production scenarios requiring high efficiency and throughput.

[0065] The aforementioned server is equipped with a system management platform 200, which is used to manage the commodity inspection device 100. The system management platform 200 is a software system deployed on the server side, which has the capabilities to manage equipment information, maintain equipment software, monitor operating status, and handle equipment anomalies.

[0066] Please refer to Figure 4, which is a framework diagram of a system management platform provided in an embodiment of this application. After the commodity inspection device 100 is linked to the system management platform 200 through the network, the system management platform 200 has the capabilities shown in Figure 4.

[0067] The system management platform 200 can manage equipment information, which mainly includes equipment information (manufacturer, hardware information, etc.), identity authentication (equipment number, Internet Protocol (IP) address), certificate management (remote connection key, remote connection certificate, system program certificate), etc. This information is generated either during the equipment production stage, during registration when the equipment is put into production, or when it is distributed when the equipment is connected to the equipment management platform.

[0068] The system management platform 200 can manage device programs, including gray-scale / upgrade management (updating software systems, etc.), business program management (business-related functions such as artificial intelligence (AI) recognition functions, etc.), configuration change management (changes to device IP addresses, production areas, and imaging configuration parameters), and operation and maintenance program management (operation and maintenance programs deployed on inspection devices).

[0069] The system management platform 200 can monitor operational status, including device connection status (whether it is online, the status of core components of the device), production data management (whether it is being used, production capacity data, etc.), performance monitoring (CPU, memory, video memory, etc.), and AI recognition data management (imaging quality monitoring, identification, quality inspection, fingerprint and other AI recognition information).

[0070] The system management platform 200 can perform device maintenance, including remote system reset, one-click recovery capability by inserting a USB flash drive to read settings and run specified scripts to reset when remote system reset is unavailable, a log system that records all system data and operation records, and remote command issuance capabilities.

[0071] In the above scheme, the controller in the commodity inspection device 100 can execute the commodity inspection method locally and use an AI model to inspect the commodity to be inspected based on the commodity image. Compared with the manual inspection method in the prior art, the embodiment of this application uses an AI model to inspect the commodity to be inspected, which can improve the efficiency and accuracy of commodity inspection.

[0072] Please refer to Figure 5, which is a schematic diagram of another commodity inspection system provided in this application embodiment. The commodity inspection system 30 may include: commodity inspection device 100 and server.

[0073] Specifically, the aforementioned product inspection device 100 may include an image acquisition unit and a controller. The image acquisition unit is used to acquire product images corresponding to the products to be inspected, and the controller is used to control the image acquisition unit. The product inspection device 100 has been described in detail in the above embodiments and will not be repeated here. In this case, the product inspection device 100 can only handle basic functions such as taking photos and printing reports. Data storage and computation rely on a cloud server, and the local controller can use low-cost hardware, making it suitable for retail scenarios with sparse workloads.

[0074] The aforementioned server is equipped with a system management platform 200 and a cloud computing module 300. The system management platform 200 manages the commodity inspection device 100, and the cloud computing module 300 executes the commodity inspection method of this solution. The system management platform 200 is a server-side software system with capabilities for managing device information, maintaining device software, monitoring operational status, and handling device anomalies. The cloud computing module 300 receives data and instructions generated by the commodity inspection device 100 and returns the commodity inspection results.

[0075] As one implementation method, a cloud storage module can also be deployed on the server to store the data generated by the commodity inspection system 30. It can be self-built or use services provided by cloud vendors. Self-built modules can be deployed locally or remotely, and include information such as images and operation records.

[0076] In the above scheme, the cloud computing module 300 deployed on the server can execute the commodity inspection method in the cloud and use an AI model to inspect the commodity to be inspected based on the commodity image. Compared with the manual inspection method in the prior art, the embodiment of this application uses an AI model to inspect the commodity to be inspected, which can improve the efficiency and accuracy of commodity inspection.

[0077] The following example illustrates an inspection application running on a controller, which can control the entire product inspection system. Please refer to Figure 6, which is a framework diagram of a controller provided in an embodiment of this application.

[0078] The inspection application can run on the smallest inspection unit (i.e., the product inspection device) or on an interactive device. If running on an interactive device, it can interact with the product inspection device via Bluetooth or a wireless network. Commands are issued through a visual interface; upon system startup, the controller is initialized by obtaining "system configuration parameters" from the system management platform; Over-The-Air (OTA) service operates as a service, interacting with the system management platform via Hypertext Transfer Protocol (HTTP) to sequentially start, manage, and update the entire controller's software system; the camera module (i.e., the image acquisition unit) controls the product inspection device to take photos and controls the computing and image quality services to complete corresponding system functions.

[0079] The entire controller connects to the cloud computing module via HTTP to upload image data and obtain calculation results, which are then returned to the verification application. It also connects to the system management platform via HTTP to report the system's operating status and data.

[0080] The following details the implementation process of the commodity inspection method executed by the controller or cloud computing module.

[0081] Please refer to Figure 7, which is a flowchart of a commodity inspection method provided in an embodiment of this application. The commodity inspection method may specifically include the following steps:

[0082] Step S701: Obtain the product image corresponding to the product to be inspected, wherein the product image is a foreground image obtained by taking pictures of the product to be inspected from multiple angles.

[0083] Step S702: Perform imaging quality monitoring on the product image to obtain the imaging quality monitoring results.

[0084] Step S703: If the imaging quality monitoring result indicates that there are no abnormalities in the product image, then the AI ​​model is used to inspect the product based on the product image to obtain the product inspection result.

[0085] Specifically, in step S701 above, the product image, as in the above embodiment, can be obtained by multiple image acquisition devices to collect images of the product to be inspected from multiple angles. For example, the product to be inspected can be placed in the product inspection area, and cameras can be installed at multiple angles in the product inspection area. Through these cameras, images of the product to be inspected can be collected from multiple angles. For example, six cameras can be used to take images of the front, back, left, right, top, and bottom of the shoe, and each collected image is used as the product image.

[0086] It should be noted that the embodiments of this application do not specifically limit the specific implementation methods for acquiring product images described above, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the image acquisition device and the controller are integrated into one device, so product images can be directly captured; or, product images sent by the image acquisition device can be received; or, pre-stored product images can be read or received, etc.

[0087] In step S702, the imaging quality of the product image can be monitored to obtain the imaging quality monitoring result. It should be noted that the embodiments of this application do not specifically limit the specific implementation of the above imaging quality monitoring method, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, an AI model can be used for imaging quality monitoring; or, the product image can be compared with standard images in an image database, etc.

[0088] In step S703 above, if the imaging quality monitoring result indicates that the product image is normal, the AI ​​model can be used to inspect the product based on the product image, thereby obtaining the product inspection result. The above-mentioned imaging quality monitoring result indicating that the product image is normal can have several scenarios, such as: every frame of the product image is normal; or, some product images are normal, and the number of normal product images is greater than a preset threshold, etc.

[0089] It should be noted that the embodiments of this application do not impose specific limitations on the specific implementation of the above-described AI model, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the above-described AI model may include an image feature extraction model, a text feature extraction model, a product quality inspection model, a product identification model, etc.

[0090] In the above solution, firstly, multi-angle photographs of the product to be inspected are taken to obtain corresponding product images, thus obtaining a more complete image of the product. Then, the image quality of the product images is monitored in real time to check whether the image quality meets the requirements. Finally, if the image quality monitoring results indicate that the product images are normal, an AI model is used to inspect the product based on the product images. Compared with the manual inspection methods in the prior art, this embodiment of the application utilizes an AI model to inspect the product, which can improve the efficiency and accuracy of product inspection.

[0091] Furthermore, based on the above embodiments, step S702 may specifically include the following steps:

[0092] Step 1) Perform imaging quality analysis on a single frame of the product image to obtain imaging anomaly information corresponding to the product image.

[0093] Step 2) By aggregating imaging anomaly information corresponding to multiple frames of product images from a certain image acquisition device within a preset time period, it is determined whether the image acquisition device has any factors affecting imaging.

[0094] Step 3): If there are factors affecting the imaging of the image acquisition device, then an alarm and / or an intervention prompt will be issued based on the factors affecting the imaging.

[0095] Specifically, please refer to Figure 8, which is an overall flowchart of an imaging quality monitoring module provided in an embodiment of this application. In step 1) above, the single-image imaging quality analysis module can use image processing and analysis algorithms (such as detection, segmentation, classification, etc.) to perform imaging quality analysis on the product image and obtain imaging anomaly information in a single image.

[0096] As one implementation method, deep learning models can be used to determine whether there are any anomalies in the image that may affect the quality inspection results, including but not limited to: out-of-focus, occlusion, target truncation, abnormal pose, abnormal color, abnormal lighting, abnormal part, abnormal placement of accessories, etc. The deep learning tasks involved in step 1) above can include detection, classification, etc.

[0097] In step 2) above, the imaging anomaly decision module can aggregate the imaging quality information of multiple images taken by the same camera within a continuous period of time (i.e., a preset time period) and then make a decision to determine whether there are factors affecting the imaging of the current camera.

[0098] As one implementation method, please refer to Figure 9, which is an overall flowchart of an imaging anomaly decision module provided in an embodiment of this application. When making a decision, the corresponding imaging anomaly problem sequence can be located according to the ID of the commodity inspection device and the ID of the image acquisition device. Combined with preset parameters, it is determined whether an alarm is required. If an alarm is required, an alarm message is output.

[0099] In step 3) above, if there are factors affecting the image acquisition device, the imaging anomaly intervention module can issue an alarm and / or output an intervention prompt based on these factors. As one implementation, the imaging anomaly decision result can be used to determine whether the anomaly requires adjustment / replacement of the equipment. If intervention is needed, the operation of the current equipment can be blocked via software.

[0100] As one implementation method, please refer to Figure 10, which is an overall flowchart of an imaging anomaly intervention module provided in an embodiment of this application. For some serious or long-term anomalies, it may be necessary to adjust the equipment or train personnel to avoid causing a large-scale impact. Therefore, based on the imaging anomaly information, a judgment is made according to predefined rules. If it is necessary to stop the operation, the software locks the current equipment, and on-site personnel can intervene according to the pop-up reminder; otherwise, only a pop-up reminder is given to the operator to pay attention to the operating procedures.

[0101] In the above solution, by aggregating the imaging anomaly information corresponding to multiple frames of product images within a preset time period, the image acquisition device that has an anomaly can be located. This allows on-site personnel to be promptly alerted to handle any anomalies, thereby improving the efficiency and accuracy of product inspection.

[0102] Furthermore, based on the above embodiments, the step of performing imaging quality analysis on a single frame of a product image to obtain imaging anomaly information corresponding to the product image may specifically include the following steps:

[0103] The product image is input into the imaging quality analysis model to obtain imaging anomaly information output by the imaging quality analysis model.

[0104] The aforementioned imaging quality analysis model is obtained by training a deep learning model using imaging anomaly samples. Therefore, please refer to Figure 11, which is a schematic diagram of an imaging anomaly sample collection framework provided in an embodiment of this application. Before the step of inputting the product image into the imaging quality analysis model to obtain the imaging anomaly information output by the imaging quality analysis model, the product inspection method provided in this embodiment of the application may further include the following steps:

[0105] Imaging anomaly samples can be obtained using at least one of the following methods: collecting imaging anomaly cases through laboratory scenarios; synthesizing imaging anomaly samples by combining normal image data with an image synthesis algorithm; or obtaining imaging anomaly samples by performing semi-supervised training using a pre-trained model.

[0106] In the third approach mentioned above, a pre-trained model can be semi-supervised by combining it with a small amount of abnormal sample data to obtain a model with a certain ability to recognize abnormal images. Then, this model can be used to infer the image results on a large amount of real-world generated environment data. By combining certain rules, a portion of real-quality abnormal data with a high probability of abnormal results can be screened out, namely, imaging abnormal samples.

[0107] In the above scheme, imaging anomaly cases can be collected through simulated laboratory scenarios, or by combining image synthesis algorithms with normal image data, or by using pre-trained models for semi-supervised training to obtain imaging anomaly samples for training the imaging quality analysis model. These methods can generate a large number of imaging anomaly samples, thereby improving the performance of the trained imaging quality analysis model.

[0108] Furthermore, based on the above embodiments, the product inspection results may include at least one of the following: product quality inspection results, product identification results, and product fingerprint results.

[0109] When the product inspection results include product quality inspection results, the above step S703 may specifically include the following steps:

[0110] AI models are used to perform quality inspections on products based on product images, and product quality inspection results are obtained. The product quality inspection results indicate whether the products have appearance defects.

[0111] In the above solution, by conducting quality inspection on the goods to be inspected, it can be determined whether the goods have any appearance defects. Compared with the manual quality inspection method in the prior art, the embodiments of this application use AI models to conduct quality inspection on the goods to be inspected, which can improve the efficiency and accuracy of the quality inspection.

[0112] When the product inspection result includes the product identification result, the above step S703 may specifically include the following steps:

[0113] An AI model is used to identify the product under inspection based on the product image, and the product identification result is obtained. The product identification result indicates whether the product under inspection is genuine.

[0114] In the above solution, by identifying the goods to be inspected, it can be determined whether the goods are genuine. Compared with the manual identification method in the prior art, the embodiments of this application use AI models to identify the goods to be inspected, which can improve the efficiency and accuracy of goods identification.

[0115] When the product inspection result includes the product fingerprint result, the above step S703 may specifically include the following steps:

[0116] An AI model is used to perform fingerprint recognition on the product image to obtain the product fingerprint result, which indicates whether the product exists in the product database.

[0117] In the above solution, by performing fingerprint recognition on the product to be inspected, it can be determined whether the product exists in the product database. Compared with the manual identification method in the prior art, the embodiment of this application uses an AI model to perform fingerprint recognition on the product to be inspected, which can improve the efficiency and accuracy of product fingerprint recognition.

[0118] It is understood that one, two, or all of the above-mentioned quality inspection, identification, and fingerprint recognition steps can be performed. Please refer to Figure 12, which is an overall flowchart of an AI module provided in an embodiment of this application. The AI ​​module can simultaneously perform the above-mentioned quality inspection, identification, and fingerprint recognition steps to obtain the final product inspection result.

[0119] Among them, the product images shown in Figure 12 are all the images captured by the image acquisition device. The number of images varies for different products. Taking shoes as an example, the number of product images is N=16. There are two shoes, left and right. Each shoe contains 8 images, namely the upper side, front side, back side, left side, right side, sole, front of insole, and back of insole.

[0120] The product information shown in Figure 12 can include the brand, item number, size, color, material, style, and other information of the current product. The product database shown in Figure 12 includes basic information about the products after verification and the verification results, and may also include the product's unique identification number, corresponding circulation information, and other information.

[0121] The product foreground extraction module refers to using image segmentation algorithms (such as FCN, Segnet, maskformer, etc.) to segment the product region in the product image to extract the product region image. This image can then be used as the image for subsequent product inspection, thus avoiding the impact of excessive background areas in the original acquired image on the product inspection results.

[0122] The quality inspection module is an AI algorithm module that determines whether a product has any appearance defects based on the product image and product information. The module will output the following quality inspection results: 1. Quality inspection passed: The product has no appearance defects; 2. Quality inspection failed: The module will output a description of the appearance defects of the product. Taking shoes as an example, it will output the defect information of the product (such as sole stain - 1.5 square centimeters, upper scratch - 1 centimeter, etc.).

[0123] The authentication module determines whether a product is genuine based on its image and information. If the product is genuine, it outputs "genuine" and related information; otherwise, it outputs "not genuine" and the reason for the non-genuine status, such as the main product being fake, non-original accessories, fake accessories, questionable origin, or incompatible products.

[0124] The fingerprint module determines whether a product already exists in the product database based on its appearance characteristics. There are two scenarios for product entry: 1. If the fingerprint result indicates the product is already in the database, the product information in the database is updated; 2. If the fingerprint result indicates the product is not in the database, a unique product code is created using an auto-incrementing method, along with related product information and verification information.

[0125] The following sections will introduce the specific implementation methods of commodity quality inspection, commodity identification, and commodity fingerprint recognition.

[0126] Please refer to Figure 13, which is a flowchart of a commodity quality inspection method provided in an embodiment of this application. The steps described above, which utilize an AI model to perform quality inspection on the commodity to be inspected based on a commodity image and obtain the commodity quality inspection result, may specifically include the following steps:

[0127] Step S1301: Use an image feature extraction model to extract features from the product image to obtain product feature information.

[0128] Step S1302: Use a text feature extraction model to extract features from the product information corresponding to the product to be inspected, and obtain text feature information, wherein the product information includes product attribute information, or the product information includes product attribute information and product circulation information.

[0129] Step S1303: Use the commodity quality inspection model to perform quality inspection on the commodity to be inspected based on commodity feature information and text feature information, and obtain the commodity quality inspection results.

[0130] Specifically, in step S1301 above, an image feature extraction model can be used to extract features from the product image to obtain product feature information. Please refer to Figure 14, which is a schematic diagram of the structure of an image feature extraction model provided in an embodiment of this application. The image feature extraction model includes a mask decoder, an image encoder, a pixel decoder, and a feature extractor.

[0131] The feature extractor may include a multilayer perceptron, an averaging unit, and a multiplier. In obtaining product feature information, a pixel decoder can be used to process image features to obtain pixel features. A multilayer perceptron can be used to process defect mask features to obtain defect mask location features. Then, a multiplier is used to multiply the defect mask location features and pixel features to obtain defect location prediction features. An averaging unit is used to average the defect mask features to obtain image query features. Finally, a multilayer perceptron is used to process the defect mask features separately to obtain image category prediction features and defect category prediction features. The product feature information includes the aforementioned defect location prediction features, image query features, image category prediction features, and defect category prediction features.

[0132] In step S1302 above, a text feature extraction model can be used to extract features from the product information corresponding to the product to be inspected, thereby obtaining text feature information.

[0133] Product information includes product attribute information or product circulation information. Product attribute information may include, but is not limited to, basic product information such as SPU, SKU, size, item number, color, material, style, etc. Product circulation information may include, but is not limited to, product circulation records, sales channels, sales records, production date, batch number, historical quality inspection information, etc.

[0134] Among them, the attribute information of the product can be obtained by comparing the product image. For example, the product image taken from the front of the product is compared with the product images stored in the product database, and the most similar product image in the product database is matched. The product database also records the product images corresponding to various products and the attribute information of the product. Therefore, after matching the product image, the attribute information of the product can be obtained from the product database.

[0135] In some implementations, product attribute information can also be obtained directly from the product database. For example, based on the basic information of the product that needs to be inspected uploaded by the user, more product attribute information can be retrieved from the product database.

[0136] The product database can also store the circulation information of each product. After obtaining the aforementioned product attribute information, the circulation information of that product can be retrieved from the product database. Alternatively, after determining the product attribute information, the circulation information of that product can be found in other databases that store product circulation information based on the product attribute information. Or, the product circulation information can also be retrieved from the product database based on the basic information of the product requiring quality inspection uploaded by the user.

[0137] In step S1303 above, after obtaining the product feature information and text feature information, the product quality inspection model can be used to combine this information to conduct quality inspection on the product. For example, if the product is shoes, this information can be used to detect the defect category, defect score, defect location, and other information of each defect on the shoes. Each defect can correspond to a defect category, defect score, and defect location. Then, this information can be output as the product quality inspection result for quality inspectors to refer to.

[0138] Please refer to Figure 15, which is a schematic diagram of the structure of a commodity quality inspection model provided in an embodiment of this application. Image query features and image category features can be input into the image feature processing module for feature fusion and normalization processing. The image feature processing module may include a fully connected layer and a normalization layer, namely the LayerNorm layer.

[0139] Fully connected layers can be used to transform input features into output features for feature mixing and transformation. Fully connected layers linearly combine the input image query features and image category features through a weight matrix, and then introduce non-linearity through an activation function, thereby transforming the input features into a higher-level feature representation.

[0140] Normalization layers are used to normalize features. This normalization process can reduce scale differences between different features and prevent certain features from dominating the training process.

[0141] The dimension of the global feature of the product image output after the image feature processing module is (M+1)×C5, where M is the number of image query features and C5 can be 256, which is the length of the image feature encoding.

[0142] The defect location prediction feature and defect category prediction feature can be input into the text encoder for encoding to obtain the encoded vector. The product information can also be input into the text encoder for encoding to obtain the encoded vector. The two encoded vectors can be input into the text feature processing module for feature fusion and normalization processing.

[0143] The text feature processing module includes a fully connected layer and a normalization layer. Through the fully connected layer and the normalization layer, the two encoding vectors can be fused and normalized, thereby mapping the features to the same dimension. The dimension of the product defect feature output by the text feature processing module is (K+1)×C5, where C5 can take the value 256, which is the length of the image feature encoding.

[0144] After obtaining the global features of the product image and the product defect features, the two features can be combined to conduct quality inspection of the product. The global features of the product image can locate the defective area in the defective image, while the product defect features can locate the specific defect location, defect type, and other information. Therefore, combining these features can accurately detect whether there are defects in the product and the defect type.

[0145] The defect score can be understood as a representation of the severity of the defect. The higher the defect score, the more serious the defect, and the lower the defect score, the less serious the defect.

[0146] In some implementations, the product quality inspection result may include a quality inspection score, which may be obtained by combining information such as defect category and defect score. For example, the defect scores of each defect category may be weighted or summed to obtain the quality inspection score. Then, the quality inspection score is output as the product quality inspection result to the quality inspector. The lower the quality inspection score, the more defects the product has, and the higher the quality inspection score, the fewer defects the product has.

[0147] In some implementations, the product quality inspection results may include whether the quality inspection is qualified. For example, the quality inspection results may be determined based on information such as the defect category, defect score, and defect location of the shoes (specific judgment rules may be set according to actual needs), and the quality inspection results may be output to the quality inspectors.

[0148] In some implementations, product quality inspection can be performed based on predefined rules and thresholds. These rules can be formulated based on product information and product characteristic information. For example, if a certain area on a product has obvious texture, it might be judged as a defect based solely on product characteristic information. However, considering the product information, if the area is crocodile skin, which naturally has texture, it wouldn't be considered a defect. Another example is that some product defects may be unclear whether they are due to wear or deliberate aging. However, considering the product's circulation information, if it has changed hands many times, it can be inferred that the defect is due to wear. Furthermore, the same defect may manifest differently on different colors (e.g., white or black products) and different materials. In other words, if the product attribute information is different, the quality inspection result for the same defect may also differ. Thus, quality inspection can be performed based on product characteristic information and predefined rules, outputting the product quality inspection result.

[0149] In the above scheme, product feature information is extracted from product images and combined with text feature information extracted from product information to conduct quality inspection of products. Product feature information can reflect more detailed features of products, while text feature information can provide more background information as an aid to product quality inspection. This can reduce misjudgments that may be caused by factors such as image noise and shooting angle when relying solely on image features for quality inspection. Combining product information can provide more comprehensive verification information, reduce misjudgments, and improve the accuracy and efficiency of quality inspection.

[0150] Furthermore, based on the above embodiments, please refer to Figure 16, which is a flowchart of a product identification method provided by an embodiment of this application. The steps of using an AI model to identify the product to be inspected based on the product image and obtaining the product identification result may specifically include the following steps:

[0151] Step S1601: Use an image feature extraction model to extract features from the product image to obtain product feature information.

[0152] Step S1602: Use a text recognition model to perform text recognition on the product image to obtain product attribute information.

[0153] Step S1603: Use a text feature extraction model to extract features from the additional information of the product to be inspected, and obtain text feature information.

[0154] Step S1604: Use the product identification model to identify the product to be inspected based on product feature information, product attribute information and text feature information, and obtain the product identification result.

[0155] Specifically, in step S1601 above, the product feature information is a feature vector extracted from the product image, which may include information such as color, texture, shape, or edges. Image feature extraction methods include: using an encoder to convert the product image into lower-dimensional product feature information; or using a pre-trained image feature extraction model to process the product image to obtain product feature information.

[0156] In one implementation, the product image may include a foreground image. For example, multiple cameras, either fixed or non-fixed, can be used to capture images of the product from different angles. Each camera is responsible for capturing images of the product from different angles, such as front, back, left, right, top, and bottom, thereby acquiring multiple product images. Then, an image segmentation algorithm is used to process the product images, separating the product from the background to obtain the corresponding foreground image.

[0157] Optionally, the shooting light source can adopt a symmetrical design, with multiple light sources installed on the upper and lower parts of the product respectively, so that the internal light source provides sufficiently uniform brightness. Diffuse light can also be used to supplement the internal lighting of the device, reducing interference from external ambient light and improving the stability of imaging lighting.

[0158] As another implementation method, a "genuine product score" can be obtained based on the aforementioned new product characteristic information, representing the probability that the product is genuine. This genuine product score can be a numerical value, characterizing the probability that the product is genuine; for example, a larger value indicates a higher probability that the product is genuine. Alternatively, the genuine product score can be a combination of a label and a numerical value, such as "genuine 0.9" or "counterfeit 0.8," where "genuine 0.9" represents a 90% probability that the product is genuine, and "counterfeit 0.8" represents an 80% probability that the product is counterfeit (i.e., the probability of the product being counterfeit is higher, and the probability of it being genuine is lower). This method requires combining the label and the numerical value to determine the probability of the product being genuine, rather than solely considering the magnitude of the numerical value.

[0159] One way to obtain a product authenticity score is to use a deep learning model (such as a convolutional neural network, CNN) to classify product features and output the probability that the product is authentic. For example, a large number of images of authentic and counterfeit products can be collected, and image features can be extracted. The extracted sample features are labeled with authentic product scores, and the deep learning model is trained using these sample features to obtain a trained authentic product score model. Then, this trained model is used to predict product features and obtain the authentic product score. This method can automatically learn the complex relationship between product features and the probability of authenticity, offering high efficiency and convenience.

[0160] It can also calculate the distance between the features of the product image to be inspected and the classification centers of genuine and counterfeit products, and convert the distance into a genuine product score. For example, multiple features are extracted from multiple known genuine product images, and a center vector is calculated using these features as the genuine product classification center; similarly, multiple features are extracted from multiple counterfeit product images, and a center vector is calculated using these features as the counterfeit product classification center. Then, the distances between the feature images and the genuine and counterfeit product classification centers are calculated, and the distance values ​​are converted into genuine product scores. This method is simple to calculate and saves computational resources.

[0161] Of course, one can also use models that calculate probability distributions to obtain the authenticity score corresponding to product feature information. For example, the image features of the product to be inspected can be input into an estimated probability density model, and the probability that it belongs to the authenticity distribution can be calculated as the authenticity score. This method can capture the distribution characteristics of features and is applicable even for cases with complex feature distributions, with a high accuracy rate for authenticity scores.

[0162] In step S1602 above, product attribute information includes information such as the product's origin, item number, size, color, material, brand, and / or model. This product attribute information can be extracted from the product image using Optical Character Recognition (OCR) technology, or a deep learning-based text detection model can be used to detect and recognize text regions in the product image to obtain the product attribute information.

[0163] In one scenario, the product image includes a product label, and the product image can be considered a product label image. As one implementation, the product information may include a foreground image of the product label.

[0164] In step S1603 above, the additional product information refers to information about the product to be inspected, excluding product attribute information. This additional product information can originate from product order information or information recorded in the inventory system for the product to be identified. The additional product information includes the order time, weight, color, place of origin, size, etc.

[0165] It should be noted that product supplementary information and product attribute information can contain the same content, but their sources are different. For example, product attribute information includes the product's color, which is obtained through label image recognition; while product supplementary information also includes the product's color, which is obtained through system records (such as inventory records or user reviews). Of course, product supplementary information can also include content not included in product attribute information.

[0166] In an optional embodiment, consistency checks can be performed on common information in product attribute information and product additional information, and the check results can be used for product identification.

[0167] Textual features can be extracted from product information to obtain textual feature information. For example, natural language processing techniques can be used to segment text in product order information or inventory system records and extract keywords, phrases, etc., as textual feature information. Alternatively, the text can be input into a pre-trained machine learning model, which can automatically learn and extract textual feature information.

[0168] In step S1604 above, the product identification result is used to characterize the authenticity category label, the probability of authenticity (counterfeit), and / or the reasons for authenticity (for genuine products) or counterfeit (for counterfeit products). The authenticity score and product attribute information can be input into a large language model. For example, the large language model can identify the product to be inspected and obtain the product identification result. The large language model can pre-learn knowledge of identifying product authenticity and attribution.

[0169] Other machine learning models, such as logistic regression, support vector machines, and random forests, can be pre-trained and used to predict new products to be inspected. Using the authenticity score and product attribute information as input features, the model will output the probability or category label that the product is authentic.

[0170] Of course, product authentication results can also be based on preset rules. For example, if a shoe was released a long time ago and is showing significant wear, the required score for authenticity can be appropriately lowered. The rules can be determined according to the actual situation.

[0171] In the above scheme, product feature information and product attribute information are extracted from the product image, and combined with text feature information extracted from the product information to identify the product. The product feature information and product attribute information can reflect more detailed features of the product, while the text feature information can provide more background information as an aid to product identification, thereby providing more comprehensive verification information and improving the accuracy and efficiency of identification.

[0172] Furthermore, based on the above embodiments, please refer to Figure 17, which is a flowchart of a product fingerprint recognition method provided by an embodiment of this application. The steps of using an AI model to perform fingerprint recognition on the product to be inspected based on the product image to obtain the product fingerprint result may specifically include the following steps:

[0173] Step S1701: Use the first image feature extraction model to extract features from the product image to obtain the first product feature information.

[0174] Step S1702: Use the second image feature extraction model to extract features from the microscopic image corresponding to the product to be inspected, and obtain the second product feature information.

[0175] Step S1703: Determine the fingerprint of the product to be matched based on the first product feature information and the second product feature information.

[0176] Step S1704: Match the fingerprint of the product to be matched in the product fingerprint database to obtain the product fingerprint result.

[0177] Specifically, please refer to Figure 18, which is an overall flowchart of a product fingerprint recognition method provided in an embodiment of this application. In step S1701 above, a first image feature extraction model can be used to extract features from the product image to obtain first product feature information. The first product feature information can be obtained based on a trained first image feature extraction model. The input to the first image feature extraction model is the product image, and the first image feature extraction model is used to encode the input image into a fixed-length feature vector. The first image feature extraction model can be an open-source network architecture such as ResNet, VIT network, Swin-Transformer network, or Convnext network.

[0178] In step S1702 above, the second image feature extraction model can be used to extract features from the microscopic image corresponding to the product to be inspected, so as to obtain the second product feature information.

[0179] The aforementioned microscopic images are obtained using image acquisition devices with a magnification greater than a preset magnification threshold. In addition, the preset magnification threshold of the image acquisition devices for acquiring microscopic images varies for different categories and characteristics of goods. For example, for categories such as footwear and apparel, a higher preset magnification threshold is required to obtain clearer textures of footwear and apparel.

[0180] The aforementioned second product feature information can be obtained based on a trained second image feature extraction model. The input to the second image feature extraction model is the aforementioned microscopic image, and the second image feature extraction model is used to encode the input image into a fixed-length feature vector. The second image feature extraction model can be an open-source network architecture such as ResNet, VIT network, Swin-Transformer network, or Convnext network.

[0181] It is understood that the first image feature extraction model and the second image feature extraction model mentioned above can use the same image feature extraction model, that is: the first commodity feature information and the second commodity feature information mentioned above can be extracted using the same image feature extraction model.

[0182] In step S1703 above, the fingerprint of the product to be matched can be determined based on the first product feature information and the second product feature information. The aforementioned product fingerprint refers to a set of feature information used to uniquely identify and describe a product. This feature information may include the product's physical attributes, visual features, brand information, etc., and can be used for product identification, classification, retrieval, and management. The composition of the product fingerprint in this embodiment is described below: 1. A fusion feature of the first and second product feature information; 2. Product context information, which may include at least one of the following: SKU (Stock Keeping Unit), product image, microscopic image, historical order information, and product production batch.

[0183] In addition, the fingerprint of the aforementioned product can be represented as:

[0184] F = (f_1, f_2, ..., f_n);

[0185] Where n is the dimension of the feature, which can be of any length.

[0186] In step S1704 above, the fingerprints of the products to be matched can be performed in the product fingerprint database to obtain the product fingerprint results. It can be understood that the product fingerprints of products already existing in the product fingerprint database can be represented as:

[0187] D = (D_1, D_2, ..., D_m);

[0188] By comparing features of the same type in the product fingerprints F=(f_1,f_2,…,f_n) and D=(D_1,D_2,…,D_m) one by one, such as comparing feature sources and fused features respectively, the product fingerprints with an overall similarity less than a preset similarity threshold are determined as the fingerprints that match the product fingerprints to be matched.

[0189] In the above scheme, product fingerprint features are obtained by fusing macro features extracted from multi-view images and micro features extracted from micro images. Then, the product fingerprint features are matched. Macro features can capture information such as the overall appearance and shape of the product, while micro features can capture detailed texture information of specific parts of the product. By fusing macro and micro features, the product fingerprint can more comprehensively reflect the product characteristics, which is conducive to improving the accuracy of product recognition.

[0190] Furthermore, based on the above embodiments, after step S701, the commodity inspection method provided in this application embodiment may further include the following steps:

[0191] Step 1) Extract features from the product image to obtain product feature information.

[0192] Step 2) Compare the product feature information with the product image features in the database to obtain the product information corresponding to the product to be inspected.

[0193] In the above scheme, product information can provide more background information to assist in product inspection, thereby providing more comprehensive verification information and improving the accuracy and efficiency of inspection.

[0194] This application also provides a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes computer program instructions. When the computer program instructions are executed by a computer, the computer can perform the steps of the commodity inspection method described in the above embodiments, such as: Step S701: Obtain a commodity image corresponding to the commodity to be inspected, wherein the commodity image is a foreground image obtained by taking pictures of the commodity to be inspected from multiple angles. Step S702: Perform image quality monitoring on the commodity image to obtain an image quality monitoring result. Step S703: If the image quality monitoring result indicates that the commodity image has no abnormalities, then use an AI model to inspect the commodity to be inspected based on the commodity image to obtain a commodity inspection result.

[0195] This application also provides a computer-readable storage medium that stores computer program instructions. When the computer program instructions are executed by a computer, the computer performs the commodity inspection method described in the foregoing method embodiments.

[0196] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0197] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0199] It should be noted that if the function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0201] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for inspecting goods, characterized in that, include: Obtain a product image corresponding to the product to be inspected, wherein the product image is a foreground image obtained by taking pictures of the product to be inspected from multiple angles; The imaging quality of the product image is monitored to obtain the imaging quality monitoring results. If the imaging quality monitoring result indicates that the product image is normal, then the AI ​​model is used to inspect the product based on the product image to obtain the product inspection result.

2. The commodity inspection method according to claim 1, characterized in that, The step of performing image quality monitoring on the product image to obtain image quality monitoring results includes: Image quality analysis is performed on a single frame of a product image to obtain imaging anomaly information corresponding to the product image; By aggregating imaging anomaly information corresponding to multiple frames of product images from a certain image acquisition device within a preset time period, it can be determined whether the image acquisition device has factors affecting imaging. If the image acquisition device has the factors that affect imaging, then an alarm and / or an intervention prompt will be issued based on the factors affecting imaging.

3. The commodity inspection method according to claim 2, characterized in that, The step of performing imaging quality analysis on a single frame of a product image to obtain imaging anomaly information corresponding to the product image includes: The product image is input into the imaging quality analysis model to obtain the imaging anomaly information output by the imaging quality analysis model; The imaging quality analysis model is obtained by training a deep learning model using imaging anomaly samples. Before inputting the product image into the imaging quality analysis model to obtain the imaging anomaly information output by the imaging quality analysis model, the method further includes: The imaging anomaly sample is obtained using at least one of the following methods: The imaging anomaly samples were obtained by simulating and collecting imaging anomaly cases in a laboratory setting. The abnormal imaging sample is synthesized by combining normal image data with an image synthesis algorithm; The imaging anomaly samples are obtained by semi-supervised training using a pre-trained model.

4. The commodity inspection method according to any one of claims 1-3, characterized in that, The product inspection result includes at least one of the following: product quality inspection result, product identification result, and product fingerprint result. The step of using an AI model to inspect the product based on the product image to obtain the product inspection result includes at least one of the following steps: The AI ​​model is used to perform quality inspection on the product to be inspected based on the product image to obtain the product quality inspection result, wherein the product quality inspection result indicates whether the product to be inspected has any appearance defects; The AI ​​model is used to identify the product to be inspected based on the product image to obtain a product identification result, wherein the product identification result indicates whether the product to be inspected is genuine. The AI ​​model is used to perform fingerprint recognition on the product image to obtain a product fingerprint result, wherein the product fingerprint result indicates whether the product to be inspected exists in the product database.

5. The commodity inspection method according to claim 4, characterized in that, The step of using the AI ​​model to perform quality inspection on the product to be inspected based on the product image, and obtaining the product quality inspection result, includes: The product image is used to extract features using an image feature extraction model to obtain product feature information; The text feature extraction model is used to extract features from the product information corresponding to the product to be inspected, thereby obtaining text feature information. The product information includes product attribute information, or the product information includes product attribute information and product circulation information. The product quality inspection model is used to perform quality inspection on the product to be inspected based on the product feature information and the text feature information, and the product quality inspection result is obtained.

6. The commodity inspection method according to claim 4, characterized in that, The step of using the AI ​​model to identify the product to be inspected based on the product image and obtaining the product identification result includes: The product image is used to extract features using an image feature extraction model to obtain product feature information; The product image is subjected to text recognition using a text recognition model to obtain product attribute information; The text feature extraction model is used to extract features from the product additional information corresponding to the product to be inspected, and the text feature information is obtained. The product additional information is information about the product to be inspected other than the product attribute information. The product identification model is used to identify the product to be inspected based on the product feature information, the product attribute information, and the text feature information, and the product identification result is obtained.

7. The commodity inspection method according to claim 4, characterized in that, The step of using the AI ​​model to perform fingerprint recognition on the product to be inspected based on the product image to obtain the product fingerprint result includes: The product image is used to extract features using a first image feature extraction model to obtain first product feature information; The second image feature extraction model is used to extract features from the microscopic image corresponding to the product to be inspected to obtain the second product feature information. The microscopic image is obtained by taking pictures of the product to be inspected using an image acquisition device with a magnification greater than a preset magnification threshold. The fingerprint of the product to be matched is determined based on the first product feature information and the second product feature information; The product fingerprint to be matched is matched in the product fingerprint database to obtain the product fingerprint result.

8. The commodity inspection method according to any one of claims 1-3, characterized in that, After obtaining the product image corresponding to the product to be inspected, the method further includes: Product feature information is obtained by extracting features from the product image; The product feature information is compared with the product image features in the database to obtain the product information corresponding to the product to be inspected.

9. A commodity inspection system, characterized in that, include: A commodity inspection device, comprising an image acquisition unit and a controller, wherein the image acquisition unit is used to acquire commodity images corresponding to the commodity to be inspected, and the controller is used to execute the commodity inspection method as described in any one of claims 1-8; The server has a system management platform deployed on it, which is used to manage the product inspection device.

10. A commodity inspection system, characterized in that, include: A commodity inspection device, comprising an image acquisition unit and a controller, wherein the image acquisition unit is used to acquire commodity images corresponding to the commodity to be inspected, and the controller is used to control the image acquisition unit; The server is equipped with a system management platform and a cloud computing module. The system management platform is used to manage the commodity inspection device, and the cloud computing module is used to execute the commodity inspection method as described in any one of claims 1-8.

11. A commodity inspection device, characterized in that, include: Image acquisition device, used to acquire product images corresponding to the products to be inspected; A controller for controlling the image acquisition device and for performing the commodity inspection method as described in any one of claims 1-8.

12. The commodity inspection device according to claim 11, characterized in that, Also includes: Interactive interface, and / or interactive device.

13. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the commodity inspection method as described in any one of claims 1-8.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a computer, cause the computer to perform the commodity inspection method as described in any one of claims 1-8.

Images