A machine vision-based container camera data management system and method

Abstract

This invention relates to the field of container camera data technology. Specifically, it is a container camera data management system and method based on machine vision. The system includes a data acquisition module, a data analysis module, a camera data management module, and a transaction monitoring module. The data acquisition module is used to collect camera data from the built-in camera in the container and gravity change data in the container sensed by a gravity sensor. The data analysis module performs preliminary analysis of user purchase data and analyzes the accuracy of identifying user purchase information based on camera data. The camera data management module analyzes and judges whether there are defects in the camera data and performs labeling management. The transaction monitoring module analyzes the actual goods purchased by the user and monitors the transaction process, sending an alarm to the backend when the user's payment fails. This invention avoids discrepancies between the actual transaction amount and the amount paid by the user by analyzing camera data defects, thus preventing increased workload in subsequent transactions.

Description

Technical Field

[0001] This invention relates to the field of container camera data management technology, specifically to a container camera data management system and method based on machine vision. Background Technology

[0002] Currently, unmanned commercial refrigerated vending machines on the market primarily use two methods to identify the types and quantities of beverages inside, thereby replenishing sold items. One method uses RFID (Radio Frequency Identification) technology, and the other uses gravity sensor technology. RFID is susceptible to issues such as labels falling off due to fogging in the refrigerated cabinet, loose adhesion, or human-caused damage, making accurate label identification impossible. Gravity sensor technology offers advantages over RFID, allowing for more precise location and assessment of beverage sales and more efficient replenishment. However, it also has drawbacks. For example, if a customer purchases a beverage and then places other items of the same weight inside, the gravity sensor might not register the missing items, resulting in unnecessary losses for the merchant. Both methods require manual verification, significantly increasing labor costs. Therefore, a more intelligent method is needed for better identification and processing. Therefore, by identifying the goods, the problem of uncontrollable factors such as product labels and perceived damage is solved. This avoids the risk of the gravity sensor mistaking an item of equal weight for a replacement, thus failing to accurately determine the cause of product loss. Placing a built-in camera on the refrigerated vending machine can better address the possibility of the gravity sensor mistaking an item of equal weight for a replacement.

[0003] With existing technology, due to blind spots in the built-in cameras of the vending machines, incorrect camera data is stored in the vending machine's backend system when goods are tilted, leading to incorrect product identification by users. To avoid discrepancies between the actual transaction amount and the amount paid by the user, which increases the workload of subsequent services and causes deviations between the vending machine's inventory in the backend system and the actual inventory, thus affecting the efficiency of vending machine replenishment, it is necessary to analyze the camera data collected by the built-in cameras of the vending machines to reduce identification errors. Summary of the Invention

[0004] The purpose of this invention is to provide a container camera data management system and method based on machine vision to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, the present invention provides the following technical solution: a container camera data management method based on machine vision, wherein the container camera data management method specifically includes the following steps:

[0006] S100: Collect video data of the built-in camera scanning and photographing the goods in the cabinet before and after the user makes a purchase; analyze and identify the user's purchase data based on the video data before and after the purchase; collect gravity change data of the user before and after the purchase through the cabinet's built-in gravity sensor.

[0007] S200: Extract the weight comparison table of goods in the container through the container's back-end system, and evaluate the accuracy of the camera data analysis and recognition based on the weight comparison table and gravity change data; when the accuracy of the camera data analysis and recognition is greater than or equal to a preset accuracy threshold, store the camera data in the back-end database; when the accuracy of the camera data analysis and recognition is less than the preset accuracy threshold, it indicates that there are goods in the camera's blind spot, resulting in defects in the camera data, so it is necessary to analyze the defects in the camera data;

[0008] S300: Based on the comparison of video data before and after the user purchases the goods, the storage location of the purchased goods is obtained; by analyzing the similarity values ​​of the video data before and after the user purchases the goods, the correlation value between the storage locations of the goods in the cabinet is obtained; based on the correlation value between the storage locations of the goods in the cabinet, the defect location of the video data is obtained; after marking the defect in the video data, the new video data is stored in the background database.

[0009] S400: The new camera data in the background database is compared again with the camera data before the user purchased the goods to obtain the actual goods purchased by the user. The actual goods purchased by the user are then transmitted to the back-end system of the vending machine to complete the transaction.

[0010] Furthermore, the specific method for analyzing and identifying user purchase data based on camera data before and after the user's purchase in S100 is as follows:

[0011] S101. Collect camera datasets S and S' before and after the user purchases the product, respectively, where S = {S...} i1 S i2 S i3 ...S ij ...S iJ}, S′={S′ i1 S′ i2 S′ i3 ...S′ ij ...S′ iJ}; where i = 1, 2, 3...I, I represents the number of rows of shelves in the container, j = 1, 2, 3...J, J represents the number of columns of shelves in the container, S ij S′ represents the camera data of the i-th row and j-th column area before a user purchases a product.ij This represents the camera data of the i-th row and j-th column area after the user purchases the goods; the gravity change data of the user before and after purchasing the goods is collected by the gravity sensor built into the container, which is denoted as m.

[0012] S102. Map the elements in the camera dataset to space to obtain the corresponding position information of the camera dataset in space, S. ij →(X ij Y ij ), S′ ij →(X′ ij ,Y′ ij ); where (X ij Y ij (X′) represents the spatial location information of the camera data in the i-th row and j-th column area before the user purchases the product. ij ,Y′ ij Let d represent the spatial location information of the camera data in the i-th row and j-th column region after a user purchases a product. Based on the camera datasets before and after the purchase, the spatial location information of the camera data in both regions is compared to calculate the similarity d between the camera data in different regions. ij =[(X ij -X′ ij ) 2 +(Y ij -Y′ ij ) 2 ] 1 / 2 ;

[0013] S103, when the similarity d of the camera data from the different regions ij When the similarity is greater than or equal to a preset similarity threshold D, it is determined that the product purchased by the user does not belong to the product in the i-th row and j-th column region; when the similarity d of the camera data from the different regions is greater than or equal to a preset similarity threshold D, it is determined that the product purchased by the user does not belong to the product in the i-th row and j-th column region. ij If the similarity is less than the preset similarity threshold D, it is preliminarily determined that the product purchased by the user is from the i-th row and j-th column area. Based on the preliminary determination result, the purchased product data is extracted and accumulated to obtain a set G, G = {G1, G2, G3...G...} z ...G Z}, z = 1, 2, 3...Z, where Z represents the quantity of goods purchased by the user, G z This represents the data information for the z-th item purchased by the user.

[0014] Furthermore, the specific method for analyzing the accuracy of camera data analysis and recognition in S200 is as follows:

[0015] S201. The weight comparison table of goods in the container is extracted through the back-end system of the container and compared one by one with the set G to obtain the weight information of the goods purchased by the user as m. zm z This represents the weight information of the z-th item purchased by the user; according to the formula: The accurate value f is calculated based on the analysis and identification of user purchase data information using camera data; where a represents the weight error value.

[0016] S202. When the accuracy value f of the data analysis and identification of user purchase data information is greater than or equal to the preset accuracy threshold F, the camera data is stored in the background database; when the accuracy f of the camera data analysis and identification is less than the preset accuracy threshold F, it indicates that there is a product in the camera's blind spot, resulting in defects in the camera data, so it is necessary to analyze the defects in the camera data.

[0017] Furthermore, the specific method for analyzing the defect location of the camera data based on the correlation value between the storage locations of goods in the container in step S300 is as follows:

[0018] S301. Randomly obtain the storage location of a user's purchased goods as g. a′(uv) Based on the storage location of the user's purchased goods, the similarity of the camera data of adjacent areas of the storage location of the user's purchased goods is extracted to obtain d. a′n ; where g a′(uv) Let d represent the a'-th item purchased by the user and stored in the u-th row and v-th column of the shelf, where a'∈{1, 2, 3...Z}, u∈{1, 2, 3...I}, v∈{1, 2, 3...J}; a′n Let d represent the similarity of the camera data between the nth adjacent storage locations of the storage location where the user purchased the a′th item. a′n The similarity threshold is greater than or equal to the preset similarity threshold D; n = 1, 2, 3...N, where N represents the number of adjacent storage locations for the purchased item; according to the formula: p a′n =k / d a′n The correlation value between the location where the user's purchased goods are stored and the adjacent areas is calculated, where p a′n This represents the correlation value between the nth adjacent area and the storage location of the user's purchased item a', where k is a constant. The smaller the similarity between adjacent areas before and after the user purchases the item, the greater the influence the user has on the adjacent area when taking the item, indicating a stronger correlation between the storage location of the user's purchased item and the adjacent area.

[0019] S302, traverse the association values ​​between the storage location of the user's purchased goods and adjacent areas to obtain p. zn p zn Represented as the association value between the nth adjacent region and the storage location of the zth item purchased by the user, according to... Sort the weights of the items purchased by the user in descending order, and then sum the weights of the items purchased by the user according to the sorting results to M. When the time comes, the storage location corresponding to the product whose weight is sequentially accumulated according to the sorting result is determined to be the camera data defect location, the camera data defect location is marked and stored in the background database.

[0020] Furthermore, S400 includes: comparing the new camera data in the background database with the camera data before the user purchased the goods to obtain the actual goods purchased by the user, and transmitting the actual goods purchased by the user to the vending machine's background system for transaction; when the user's transaction time exceeds a preset time, determining that the user's payment was unsuccessful and sending an alarm reminder to the background.

[0021] A machine vision-based container camera data management system includes a data acquisition module, a data analysis module, a camera data management module, and a transaction monitoring module. The output of the data acquisition module is connected to the input of the data analysis module, the output of the data analysis module is connected to the input of the camera data management module, and the output of the camera data management module is connected to the input of the transaction monitoring module. The data acquisition module collects video data from the built-in camera in the container and gravity change data sensed by a gravity sensor. The data analysis module performs preliminary analysis of user purchase data and analyzes the accuracy of identifying user purchase information based on camera data. The camera data management module analyzes and determines whether there are defects in the camera data and labels and manages defective camera data. The transaction monitoring module analyzes the actual goods purchased by the user and monitors the transaction process, sending an alert to the backend when the user's payment fails.

[0022] Furthermore, the data acquisition module includes a camera data acquisition unit and a gravity change data acquisition unit. The camera data acquisition unit is used to collect camera data from the built-in camera in the container. By comparing the similarity of the camera data before and after the user purchases the goods, the data on the goods purchased by the user can be preliminarily analyzed. The gravity change data acquisition unit is used to collect gravity change data in the container sensed by the gravity sensor. By comparing the gravity change data in the container with the preliminarily analyzed data on the goods purchased by the user according to the goods weight comparison table, the accuracy of identifying the user's purchased goods information based on the camera data can be analyzed and judged.

[0023] Furthermore, the data analysis module includes a preliminary analysis unit for purchased goods and a camera data recognition accuracy analysis unit. The preliminary analysis unit for purchased goods is necessary because the built-in camera in the vending machine has blind spots, so it is necessary to perform a preliminary analysis of the goods purchased by the user based on the camera data. The camera data recognition accuracy analysis unit is used to analyze and determine the accuracy of the user's purchased goods based on the camera data.

[0024] Furthermore, the camera data management module includes a camera data defect analysis unit and a camera data defect annotation unit; the camera data defect analysis unit is used to analyze defects in the camera data; the camera data defect annotation unit is used to annotate defective areas in the camera data.

[0025] Furthermore, the transaction monitoring module includes an actual purchased goods analysis unit, a transaction success monitoring unit, and an alarm reminder module; the actual purchased goods analysis unit compares the camera data marked with defects in the background database with the camera data before the user purchased the goods to obtain the actual purchased goods; the transaction success monitoring unit monitors the transaction process; and the alarm reminder module sends an alarm reminder to the background when the user's payment is unsuccessful.

[0026] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention analyzes the similarity comparison of video data before and after a user's purchase to initially identify the purchased goods; it analyzes and judges the blind spots of the built-in camera in the vending machine based on the magnitude of the correlation value between adjacent areas of the purchased goods, and marks the defective areas in the video data; by comparing the marked video data with the video data before the user's purchase, it obtains the actual purchased goods, avoiding errors in the vending machine's backend system in identifying the purchased goods, preventing discrepancies between the actual transaction amount and the amount paid by the user, reducing the workload of subsequent services, and improving the efficiency of vending machine replenishment. Attached Figure Description

[0027] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0028] Figure 1 This is a schematic diagram of the structure of a container camera data management system based on machine vision according to the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 The present invention provides a technical solution: a method for managing container camera data based on machine vision, wherein the method specifically includes the following steps:

[0031] S100: Collect video data of the built-in camera scanning and photographing the goods in the cabinet before and after the user makes a purchase; analyze and identify the user's purchase data based on the video data before and after the purchase; collect gravity change data of the user before and after the purchase through the cabinet's built-in gravity sensor.

[0032] S200: Extract the weight comparison table of goods in the container through the container's back-end system, and evaluate the accuracy of the camera data analysis and recognition based on the weight comparison table and gravity change data; when the accuracy of the camera data analysis and recognition is greater than or equal to a preset accuracy threshold, store the camera data in the back-end database; when the accuracy of the camera data analysis and recognition is less than the preset accuracy threshold, it indicates that there are goods in the camera's blind spot, resulting in defects in the camera data, so it is necessary to analyze the defects in the camera data;

[0033] S300: Based on the comparison of video data before and after the user purchases the goods, the storage location of the purchased goods is obtained; by analyzing the similarity values ​​of the video data before and after the user purchases the goods, the correlation value between the storage locations of the goods in the cabinet is obtained; based on the correlation value between the storage locations of the goods in the cabinet, the defect location of the video data is obtained; after marking the defect in the video data, the new video data is stored in the background database.

[0034] S400: The new camera data in the background database is compared again with the camera data before the user purchased the goods to obtain the actual goods purchased by the user. The actual goods purchased by the user are then transmitted to the back-end system of the vending machine to complete the transaction.

[0035] Furthermore, the specific method for analyzing and identifying user purchase data based on camera data before and after the user's purchase in S100 is as follows:

[0036] S101. Collect camera datasets S and S' before and after the user purchases the product, respectively, where S = {S...} i1 S i2 Si3 ...S ij ...S iJ}, S′={S' i1 S′ i2 S′ i3 ...S′ ij ...S′ iJ}; where i = 1, 2, 3...I, I represents the number of rows of shelves in the container, j = 1, 2, 3...J, J represents the number of columns of shelves in the container, S ij S′ represents the camera data of the i-th row and j-th column area before a user purchases a product. ij This represents the camera data of the i-th row and j-th column area after the user purchases the goods; the gravity change data of the user before and after purchasing the goods is collected by the gravity sensor built into the container, which is denoted as m.

[0037] S102. Map the elements in the camera dataset to space to obtain the corresponding position information of the camera dataset in space, S. ij →(X ij Y ij ), S′ ij →(X′ ij ,Y′ ij ); where (X ij Y ij (X′) represents the spatial location information of the camera data in the i-th row and j-th column area before the user purchases the product. ij ,Y′ ij Let d represent the spatial location information of the camera data in the i-th row and j-th column region after a user purchases a product. Based on the camera datasets before and after the purchase, the spatial location information of the camera data in both regions is compared to calculate the similarity d between the camera data in different regions. ij =[(X ij -X′ ij ) 2 +(Y ij -Y′ ij ) 2 ] 1 / 2 ;

[0038] S103, when the similarity d of the camera data from the different regions ij When the similarity is greater than or equal to a preset similarity threshold D, it is determined that the product purchased by the user does not belong to the product in the i-th row and j-th column region; when the similarity d of the camera data from the different regions is greater than or equal to a preset similarity threshold D, it is determined that the product purchased by the user does not belong to the product in the i-th row and j-th column region. ij If the similarity is less than the preset similarity threshold D, it is preliminarily determined that the product purchased by the user is from the i-th row and j-th column area. Based on the preliminary determination result, the purchased product data is extracted and accumulated to obtain G, where G = {G1, G2, G3...G...} z ...GZ}, z = 1, 2, 3...Z, where Z represents the quantity of goods purchased by the user, G z This represents the data information for the z-th item purchased by the user.

[0039] Furthermore, the specific method for analyzing the accuracy of camera data analysis and recognition in S200 is as follows:

[0040] S201. The weight comparison table of goods in the container is extracted through the back-end system of the container and compared one by one with the set G to obtain the weight information of the goods purchased by the user as m. z m z This represents the weight information of the z-th item purchased by the user; according to the formula: The accurate value f is calculated based on the analysis and identification of user purchase data information using camera data; where a represents the weight error value.

[0041] S202. When the accuracy value f of the data analysis and identification of user purchase data information is greater than or equal to the preset accuracy threshold F, the camera data is stored in the background database; when the accuracy f of the camera data analysis and identification is less than the preset accuracy threshold F, it indicates that there is a product in the camera's blind spot, resulting in defects in the camera data, so it is necessary to analyze the defects in the camera data.

[0042] Furthermore, the specific method for analyzing the defect location of the camera data based on the correlation value between the storage locations of goods in the container in step S300 is as follows:

[0043] S301. Randomly obtain the storage location of a user's purchased goods as g. a′(uv) Based on the storage location of the user's purchased goods, the similarity of the camera data of adjacent areas of the storage location of the user's purchased goods is extracted to obtain d. a′n ; where g a′(uv) Let d represent the a'-th item purchased by the user and stored in the u-th row and v-th column of the shelf, where a'∈{1, 2, 3...Z}, u∈{1, 2, 3...I}, v∈{1, 2, 3...J}; a′n Let d represent the similarity of the camera data between the nth adjacent storage locations of the storage location where the user purchased the a'th item. a′n Greater than or equal to the preset similarity threshold D; n = 1, 2, 3...N, where N represents the number of adjacent storage locations for the purchased item; according to formula p a′n =k / d a′n The correlation value between the location where the user's purchased goods are stored and the adjacent areas is calculated, where p a′n This represents the correlation value between the nth adjacent region and the storage location of the user's purchased item a′, where k is a constant.

[0044] S302, traverse the association values ​​between the storage location of the user's purchased goods and adjacent areas to obtain p. zn p zn Represented as the association value between the nth adjacent region and the storage location of the zth item purchased by the user, according to... Sort the weights of the items purchased by the user in descending order, and then sum the weights of the items purchased by the user according to the sorting results to M. When the time comes, the storage location corresponding to the product whose weight is sequentially accumulated according to the sorting result is determined to be the camera data defect location, the camera data defect location is marked and stored in the background database.

[0045] Furthermore, S400 includes: comparing the new camera data in the background database with the camera data before the user purchased the goods to obtain the actual goods purchased by the user, and transmitting the actual goods purchased by the user to the vending machine's background system for transaction; when the user's transaction time exceeds a preset time, determining that the user's payment was unsuccessful and sending an alarm reminder to the background.

[0046] A machine vision-based container camera data management system includes a data acquisition module, a data analysis module, a camera data management module, and a transaction monitoring module. The output of the data acquisition module is connected to the input of the data analysis module, the output of the data analysis module is connected to the input of the camera data management module, and the output of the camera data management module is connected to the input of the transaction monitoring module. The data acquisition module collects video data from the built-in camera in the container and gravity change data sensed by a gravity sensor. The data analysis module performs preliminary analysis of user purchase data and analyzes the accuracy of identifying user purchase information based on camera data. The camera data management module analyzes and determines whether there are defects in the camera data and labels and manages defective camera data. The transaction monitoring module analyzes the actual goods purchased by the user and monitors the transaction process, sending an alert to the backend when the user's payment fails.

[0047] Furthermore, the data acquisition module includes a camera data acquisition unit and a gravity change data acquisition unit. The camera data acquisition unit is used to collect camera data from the built-in camera in the container. By comparing the similarity of the camera data before and after the user purchases the goods, the data on the goods purchased by the user can be preliminarily analyzed. The gravity change data acquisition unit is used to collect gravity change data in the container sensed by the gravity sensor. By comparing the gravity change data in the container with the preliminarily analyzed data on the goods purchased by the user according to the goods weight comparison table, the accuracy of identifying the user's purchased goods information based on the camera data can be analyzed and judged.

[0048] Furthermore, the data analysis module includes a preliminary analysis unit for purchased goods and a camera data recognition accuracy analysis unit. The preliminary analysis unit for purchased goods is necessary because the built-in camera in the vending machine has blind spots, so it is necessary to perform a preliminary analysis of the goods purchased by the user based on the camera data. The camera data recognition accuracy analysis unit is used to analyze and determine the accuracy of the user's purchased goods based on the camera data.

[0049] Furthermore, the camera data management module includes a camera data defect analysis unit and a camera data defect annotation unit; the camera data defect analysis unit is used to analyze defects in the camera data; the camera data defect annotation unit is used to annotate defective areas in the camera data.

[0050] Furthermore, the transaction monitoring module includes an actual purchased goods analysis unit, a transaction success monitoring unit, and an alarm reminder module; the actual purchased goods analysis unit compares the camera data marked with defects in the background database with the camera data before the user purchased the goods to obtain the actual purchased goods; the transaction success monitoring unit monitors the transaction process; and the alarm reminder module sends an alarm reminder to the background when the user's payment is unsuccessful.

[0051] In this embodiment:

[0052] The camera datasets collected before and after a user purchases a product are S and S', respectively, where S = {S} i1 S i2 S i3 ...S ij ...S iJ}, S′={S′ i1 S′ i2 S′ i3 ...S′ ij ...S′ iJ The gravity sensor on the container collects data on the user's weight change before and after purchasing goods, with a value of m = 2 kg. The elements in the camera dataset are mapped into space to obtain the corresponding spatial location information of the camera dataset, S.ij →(X ij , Y ij ), S′ ij →(X′ ij , Y′ ij ); where (X ij , Y ij ) represents the position information corresponding to the camera data in the area of the i-th row and j-th column before the user purchases the commodity in space, and (X′ ij , Y′ ij ) represents the position information corresponding to the camera data in the area of the i-th row and j-th column after the user purchases the commodity in space; according to the camera data sets before and after the user purchases the commodity, by comparing the corresponding position information of the two camera data in space, the similarity of the camera data in different areas is calculated as d ij = [(X ij - X′ ij ) 2 + (Y ij - Y′ ij ) 2 1 / 2 = {0.2, 1, 0.8...}

[0053] When the similarity d of the camera data in the different areas ij is greater than or equal to the preset similarity threshold D = 5, it is determined that the commodity purchased by the user does not belong to the commodity in the area of the i-th row and j-th column; when the similarity d of the camera data in the different areas ij is less than the preset similarity threshold D = 5, it is preliminarily determined that the commodity purchased by the user is the commodity in the area of the i-th row and j-th column. Based on the preliminary determination result, the purchased commodity data is extracted and accumulated to obtain the set G, G = {G1, G2, G3... G z ... G Z}, z = 1, 2, 3... Z, Z represents the number of commodities purchased by the user, and G z represents the data information of the z-th commodity purchased by the user.

[0054] Example 1: The weight comparison table of the commodities in the container is extracted through the background system of the container and compared with the set G one by one to obtain the weight information of the commodities purchased by the user as m z = {20g, 1kg, 500g...}, according to the formula: f = 0.0275 < F = 0.03, the camera data is stored in the background database to complete this transaction;

[0055] Example 2: The weight comparison table of the commodities in the container is extracted through the background system of the container and compared with the set G one by one to obtain the weight information of the commodities purchased by the user as m z ​={50g, 1kg, 600g...}, according to the formula: The fact that f = 0.3537 > F = 0.03 indicates that the product is in the blind spot of the camera, resulting in defects in the camera data. Therefore, it is necessary to analyze the defects in the camera data.

[0056] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0057] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for managing container camera data based on machine vision, characterized in that: The container camera data management method specifically includes the following steps: S100: Collect video data of the built-in camera scanning and photographing the goods in the cabinet before and after the user makes a purchase; analyze and identify the user's purchase data based on the video data before and after the purchase; collect gravity change data of the user before and after the purchase through the cabinet's built-in gravity sensor. S200: Extract the weight comparison table of goods in the container through the container's back-end system; evaluate the accuracy of the camera data analysis and recognition based on the weight comparison table and gravity change data; when the accuracy of the camera data analysis and recognition is greater than or equal to a preset accuracy threshold, store the camera data in the back-end database; when the accuracy of the camera data analysis and recognition is less than the preset accuracy threshold, analyze the defects in the camera data. S300: Based on the comparison of video data before and after the user purchases the goods, the storage location of the purchased goods is obtained; by analyzing the similarity values ​​of the video data before and after the user purchases the goods, the correlation value between the storage locations of the goods in the cabinet is obtained; based on the correlation value between the storage locations of the goods in the cabinet, the defect location of the video data is obtained; after marking the defect in the video data, the new video data is stored in the background database. S400: The new camera data in the background database is compared again with the camera data before the user purchased the goods to obtain the actual goods purchased by the user. The actual goods purchased by the user are then transmitted to the back-end system of the vending machine to complete the transaction.

2. The method for managing container camera data based on machine vision according to claim 1, characterized in that: The specific method for analyzing and identifying user purchase data based on camera data before and after the user's purchase in S100 is as follows: S101. Collect camera data sets before and after the user purchases the product. and , , ;in , This refers to the number of rows of shelves in the container. , This represents the number of shelf rows in the container. This indicates the number of times a user purchases a product. Ranked Camera data in column areas, This indicates the number of times a user purchases a product. Ranked The system collects video data from the column area; and uses the container's built-in gravity sensor to collect data on the user's weight changes before and after purchasing goods. ; S102. Map the elements in the camera dataset to space to obtain the corresponding location information of the camera dataset in space. , ;in This indicates the number of times a user purchases a product. Ranked The spatial location information of the camera data in the column area. This indicates the number of times a user purchases a product. Ranked The similarity between the camera data of different regions is calculated by comparing the spatial location information of the camera data before and after the user's purchase, based on the camera data before and after the purchase. ; S103, when the similarity of the camera data from the different regions Greater than or equal to the preset similarity threshold When determining that the user's purchase of the goods does not belong to the category of Ranked Products in the column area; when the similarity of the camera data from the different areas... Less than the preset similarity threshold It was initially determined that the product purchased by the user was the [number missing]. Ranked For the products in the column area, the purchased product data is extracted based on the preliminary judgment results and accumulated to obtain a set. , , , This represents the quantity of goods purchased by the user. This indicates the number of purchases made by the user. Data information for each product.

3. The method for managing container camera data based on machine vision according to claim 2, characterized in that: The specific method for analyzing the accuracy of camera data analysis and recognition in S200 is as follows: S201. Extract the weight comparison table of goods in the container and the set through the back-end system of the container. By comparing each item one by one, the weight information of the goods purchased by the user can be obtained. , This indicates the number of purchases made by the user. The weight information of each item; according to the formula: The accurate value of the user's purchased goods data was calculated based on the analysis of the camera data. ;in This is expressed as a weight error value; S202, when the data analysis identifies the accurate value of the user's purchased goods data information. Greater than or equal to the preset accurate threshold At that time, the camera data is stored in the background database; When the accuracy of the camera data analysis and identification Less than the preset accurate threshold At that time, the defects in the camera data were analyzed.

4. The method for managing container camera data based on machine vision according to claim 3, characterized in that: The specific method for analyzing the defect location of the camera data based on the correlation value between the storage locations of goods in the container in S300 is as follows: S301, arbitrarily obtain the storage location of a user's purchased goods. Based on the storage location of the user's purchased goods, the similarity of the camera data of adjacent areas of the storage location of the purchased goods is extracted to obtain... ;in This indicates that the user has purchased the first item. The item was stored in the container. Ranked List, , , ; This indicates that the user has purchased the first item. The first product storage location The similarity of camera data from adjacent storage locations Greater than or equal to the preset similarity threshold ; 1, 2, 3...N, where N represents the number of adjacent storage locations for the purchased goods; according to the formula: The correlation value between the location where the user's purchased goods are stored and the adjacent areas is calculated, where Represented as the first Adjacent areas and user purchases The associated values ​​between storage locations It is a constant; S302, according to Sort the weight of the items purchased by the user in descending order. Represented as the first The adjacent areas and the user's purchase The association values ​​between the storage locations of each product are calculated, and the weight of the products purchased by the user is accumulated sequentially according to the sorting results. to When the time comes, the storage location corresponding to the product whose weight is sequentially accumulated according to the sorting result is determined to be the camera data defect location, the camera data defect location is marked and stored in the background database.

5. The method for managing container camera data based on machine vision according to claim 4, characterized in that: The S400 includes: comparing the new camera data in the background database with the camera data before the user purchased the goods to obtain the actual goods purchased by the user, and transmitting the actual goods purchased by the user to the back-end system of the vending machine for transaction; when the user's transaction time exceeds the preset time, determining that the user's payment is unsuccessful and sending an alarm reminder to the background.

6. A machine vision-based container camera data management system, used to implement the machine vision-based container camera data management method as described in any one of claims 1-5, characterized in that: The container camera data management system includes a data acquisition module, a data analysis module, a camera data management module, and a transaction monitoring module. The output of the data acquisition module is connected to the input of the data analysis module, the output of the data analysis module is connected to the input of the camera data management module, and the output of the camera data management module is connected to the input of the transaction monitoring module. The data acquisition module is used to collect video data from the built-in cameras in the container and gravity change data in the container sensed by the gravity sensor. The data analysis module performs preliminary analysis of user purchase data and analyzes the accuracy of identifying user purchase information based on camera data. The camera data management module analyzes and judges whether there are defects in the camera data and marks and manages defective camera data. The transaction monitoring module analyzes the actual goods purchased by the user and monitors the transaction process, sending an alarm to the backend when the user's payment fails.

7. A container camera data management system based on machine vision according to claim 6, characterized in that: The data acquisition module includes a camera data acquisition unit and a gravity change data acquisition unit; the camera data acquisition unit is used to acquire camera data from the built-in camera in the container; the gravity change data acquisition unit is used to acquire gravity change data in the container sensed by the gravity sensor.

8. A container camera data management system based on machine vision according to claim 7, characterized in that: The data analysis module includes a preliminary analysis unit for purchased goods and a camera data recognition accuracy analysis unit. The preliminary analysis unit for purchased goods performs a preliminary analysis of the goods purchased by the user based on camera data. The camera data recognition accuracy analysis unit determines the accuracy of the user's purchased goods based on the analysis of camera data.

9. A container camera data management system based on machine vision according to claim 8, characterized in that: The camera data management module includes a camera data defect analysis unit and a camera data defect annotation unit; the camera data defect analysis unit is used to analyze defects in the camera data; the camera data defect annotation unit is used to annotate defective areas in the camera data.

10. A container camera data management system based on machine vision according to claim 9, characterized in that: The transaction monitoring module includes an actual purchased goods analysis unit, a transaction success monitoring unit, and an alarm reminder module; The actual purchased goods analysis unit compares the camera data marked with defects in the backend database with the camera data before the user made the purchase to determine the actual purchased goods; the transaction success monitoring unit monitors the transaction process; and the alarm reminder module sends an alarm reminder to the backend when the user's payment fails.

Images