Image detection model processing method, storage medium and system

Abstract

This invention provides an image detection model processing method, storage medium, and system. The method includes: acquiring an object image; acquiring a ground truth bounding box A. i ; Obtain the image detection model to generate the detection box B p From detection box B p Obtain the detection box b to be judged from the middle k From the real frame A i Obtain the detection box b to be judged from the middle k The corresponding matching real bounding box A kx ; Obtain the detection box b to be judged k Area Sb k and the detection box b to be judged k Each corresponding real bounding box A kx area SA kx ; Obtain the detection box b to be judged k With each corresponding real bounding box A kx The overlapping area Sb k ∩SA kx ; Calculate the detection box b to be judged k With each corresponding real bounding box A kx The overlap ratio bA kx Overlap ratio to determine the detection box b k Does there exist an overlap ratio bA greater than the first threshold? kx If so, then the detection box b to be judged will be... k The judgment result is marked as correct. The storage medium and system are capable of implementing the above method. This setup enables automatic determination of whether the detection boxes generated by the image detection model are correct and ensures the accuracy of the judgment results.

Description

Technical Field

[0001] This invention relates to the field of household appliances, and more particularly to an image detection model processing method, storage medium, and system for refrigeration equipment. Background Technology

[0002] With technological advancements, users have increasingly higher demands for refrigeration equipment, making intelligent transformation a new research and development direction for refrigeration equipment. Existing intelligent refrigeration equipment is generally equipped with image detection models. After acquiring an image of an object, the image detection model generates bounding boxes around the object, each corresponding to an item in the image. These bounding boxes then form candidate regions for object recognition. Therefore, the accuracy of the bounding boxes generated by the image detection model in selecting the object has a significant impact on the object recognition results. Currently, the correctness of the generated bounding boxes is often determined manually; that is, the bounding boxes generated by the image detection model are obtained, and staff determine whether the bounding boxes are incorrect.

[0003] However, this design has the following drawbacks: it increases labor costs and the efficiency of manual judgment is relatively low. Summary of the Invention

[0004] The purpose of this invention is to provide an image detection model processing method, storage medium, and system. By calculating the overlap ratio between the detection box to be judged and each corresponding matching real box, it is possible to automatically determine whether the detection box generated by the image detection model is correct, and to ensure the accuracy and reliability of the judgment result.

[0005] To achieve the above-mentioned objective, one embodiment of the present invention provides an image detection model processing method, comprising:

[0006] Get the item image;

[0007] Obtain the ground truth bounding box A of each item in the input item image. i Let i = {1, 2, ..., n1}, where n1 is the number of items in the item image, and each item corresponds to a real bounding box A. i The real frame A i The bounding box corresponding to the maximum outline of each item in the item image;

[0008] The image of the object is input into the image detection model, and the rectangular detection box B of the object generated by the image detection model for the object image is obtained. p p = {1, 2, ..., n2}, where n2 is the detection box B p The number of;

[0009] From the detection box B p Obtain the detection box b to be judged from the middlek k = {1, 2, ..., n3}, where n3 is the detection box b to be judged. k The number of elements, n3≤n2;

[0010] From the real frame A i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx x = {1, 2, ..., n4}, n4 ≤ n1, n4 is the detection box b to be judged. k The corresponding matching real box A kx The number of;

[0011] Obtain the detection box b to be judged k The area Sb formed k and the detection box b to be judged k Each corresponding real bounding box A kx The area SA formed kx ;

[0012] Obtain the detection box b to be judged k With each of the corresponding real bounding boxes A kx The overlapping area Sb k ∩SA kx ;

[0013] Calculate the detection box b to be judged k With each of the corresponding real bounding boxes A kx The overlap ratio bA kx The overlap ratio

[0014] Determine the detection box b to be judged k Does the overlap ratio bA exceed the first threshold? kx ;

[0015] If so, then the detection box to be judged b will be... k The judgment result is marked as correct.

[0016] As a further improvement to one embodiment of the present invention, wherein, "from the rectangular detection box B" p Obtain the detection box b to be judged from the middle k Specifically, it includes:

[0017] From the real frame A i Obtain the detection box B from the middle p There is at least a partially overlapping initial matching ground truth box A py y = {1, 2, ..., n5}, n5 ≤ n1, n5 is the value of the detection box B p At least partially overlapping true bounding boxes A exist.i The number of;

[0018] Obtain the detection box B p The area formed is SB p and the detection box B p Each of the corresponding initial matching ground truth boxes A py The area SA formed py ;

[0019] Obtain the detection box B p Each of the corresponding initial matching real boxes A py The overlapping area SB p ∩SA py ;

[0020] Obtain the detection box B p Each of the corresponding initial matching real boxes A py The area of ​​the merged region SB p ∪SA py ;

[0021] Calculate the detection box B p Each of the corresponding initial matching real boxes A py The intersection and comparison of BA py The crossover ratio

[0022] Determine the detection box B p Does the intersection-union ratio (BA) exceed the second threshold? py ;

[0023] If so, then the detection box B... p The judgment result is marked as correct;

[0024] If not, then the detection box B will be... p The box to be detected is marked as b. k .

[0025] As a further improvement to one embodiment of the present invention, it further includes:

[0026] If the detection box B p The determination result is correct, from the B. p The corresponding initial matching real box A py Obtain the maximum intersection-union ratio (BA) in the middle. py The corresponding best real bounding box ZA p ;

[0027] "From the real frame A" i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx"Including from the real frame A" i Excluding all obtained optimal true bounding boxes ZA p Then, the detection box b to be judged is obtained. k The corresponding matching real box A kx .

[0028] As a further improvement to one embodiment of the present invention, it further includes:

[0029] Obtain the real frame A i Item category labels;

[0030] Obtain the detection box b to be judged k Item category labels;

[0031] "From the real frame A" i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "Including from the detection box b to be judged" k The real frame A with the same item type i Obtain the detection box b to be judged from the middle. k The corresponding matching real box A kx .

[0032] As a further improvement to one embodiment of the present invention, the image detection model includes, "from the real bounding box A..." i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "include:

[0033] From the detection box b to be judged k At least partially overlapping true bounding boxes A exist. i Obtain the detection box b to be judged from the middle. k The corresponding matching real box A kx .

[0034] As a further improvement to one embodiment of the present invention, each detection box b to be judged is obtained sequentially in ascending order of k value. k The judgment results also include:

[0035] Obtain the detection box b to be judged k-m The overlap ratio bA is greater than the first threshold. (k-m)x The quantity, m={1,2,…,n6}, k-n6≥1;

[0036] If the overlap ratio bA is greater than the first threshold (k-m)x If the quantity is 1, then the overlap ratio bA is... (k-m)x The corresponding matching real box A(k-m)x Marked as the best true frame ZA k-m ;

[0037] "From the real frame A" i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "Including from the real frame A" i Excluding all obtained optimal true bounding boxes ZA k-m Then, the detection box b to be judged is obtained. k The corresponding matching real box A kx .

[0038] As a further improvement to one embodiment of the present invention, it further includes:

[0039] If the overlap ratio bA is greater than the first threshold (k-m)x If the number is greater than or equal to 2, then the detection box to be judged b is obtained. k-m The four vertices;

[0040] Obtain the overlap ratio bA that is greater than the first threshold. (k-m)x Each corresponding real bounding box A (k-m)x The four vertices;

[0041] Calculate the detection box b to be judged k-m Matching the real box A as described above (k-m)x The vertex distance and,

[0042] The method for calculating the vertex distance sum is as follows: The detection boxes b to be judged that are in the same position are... k-m The four vertices match the real bounding box A. (k-m)x The four vertices are matched one-to-one to obtain the detection box b to be judged. k-m The matching real bounding box A for each vertex and its corresponding orientation (k-m)x The vertex distances are calculated by summing the four vertex distances.

[0043] Obtain the minimum vertex distance and the corresponding matching ground truth box A. (k-m)x And marked as the best true bounding box ZA k-m .

[0044] As a further improvement to one embodiment of the present invention, it further includes:

[0045] The object images are input into several different image detection models M. j j = {1, 2, ..., n7};

[0046] Obtain each of the image detection models Mj The corresponding judgment result is the correct detection box B. p and the detection box b to be judged k Total quantity sumM j ;

[0047] Get the maximum sumM j The corresponding image detection model M j It was then marked as the best image detection model.

[0048] As a further improvement of one embodiment of the present invention, it further includes: the first threshold is any value between 0.75 and 0.9, and the second threshold is any value between 0.5 and 0.6.

[0049] To achieve the above-mentioned objectives, one embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the image detection model processing method as described in any of the above embodiments.

[0050] To achieve the above-mentioned objectives, one embodiment of the present invention provides a system for processing image detection models, wherein the system includes a memory and a processor, the memory stores a computer program that can run on the processor, and when the processor executes the computer program, it implements the steps in the image detection model processing method as described in any of the above embodiments.

[0051] Compared with the prior art, the present invention calculates the overlap ratio between the detection box to be judged and each corresponding matching real box. Its advantages are: it can automatically determine whether the detection box generated by the image detection model is correct, and ensure the accuracy and reliability of the judgment result. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the structure of a refrigeration device according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of an article image and a detection frame according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of an article image and a real frame according to an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the object image detection box, the matching real box, and the overlapping area according to an embodiment of the present invention;

[0056] Figure 5 This is a flowchart of an image detection model processing method according to an embodiment of the present invention;

[0057] Figure 6 This is a flowchart of another embodiment of the image detection model processing method of the present invention;

[0058] Figure 7 This is a schematic diagram of an object image detection frame, an initial matching real frame, and an overlapping area according to an embodiment of the present invention;

[0059] Figure 8 This is a schematic diagram of an object image detection frame, an initial matching ground truth frame, and a merging region according to an embodiment of the present invention;

[0060] Figure 9 This is a schematic diagram of the object image detection box to be judged, the real bounding box to be matched, and the vertices of an embodiment of the present invention. Detailed Implementation

[0061] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0062] Reference Figure 1 and Figure 2 In one embodiment of the present invention, the refrigeration equipment 100, such as a refrigerator, freezer, or commercial display cabinet, may include a cabinet body 1 and a door body 2. The cabinet body 1 may have a storage compartment for storing items 6 inside, and the door body 2 may be used to open and close the storage compartment. The door body 2 may also be provided with a bottle holder structure for storing items 6, or the door body 2 may also have a door compartment for storing items 6 inside, etc.

[0063] The refrigeration equipment 100 may be equipped with an image recognition module 3, which may include an image detection model and an image acquisition module. The image acquisition module may include a camera, which can be used to acquire images 7 of the items 6 stored in the refrigeration equipment 100. The image detection model can be used to perform target detection and category recognition on the item images 7. The refrigeration equipment 100 can obtain information such as the category of the items stored in the refrigeration equipment 100 by knowing the detection results output by the image detection model, so as to manage the items 6 stored in the refrigeration equipment 100.

[0064] Reference Figure 2 In processing the object image 7, the image detection model can first perform target detection on the object image 7 and generate several rectangular detection boxes 4 around the target of interest in the image. (Refer to...) Figure 2 (The rectangle marked with a solid line)

[0065] The detection box 4 is a box calculated and output by the image detection model. The location of the detection box 4 can be identified by marking its upper left and lower right coordinates, i.e., (x... 左上 y 左上 ) and (x 右下 y 右下 Alternatively, its location can be identified by marking the center coordinates of its rectangle and the rectangle's width and height, i.e., (x... 中心 y 中心 ),w 宽 and h 高 .

[0066] After generating detection boxes 4, the image detection model can obtain the item category label corresponding to each detection box 4 by sending the region formed by each detection box 4 into the classification and recognition network, or by extracting features from the region formed by each detection box 4 and comparing them with pre-stored feature data.

[0067] The item category label can refer to the name information representing a specific item category, such as milk or yogurt. The image detection model can be an algorithm model built using deep convolutional neural networks, etc.

[0068] Whether the detection box 4 generated by the image detection model can correctly select the item 6 has a significant impact on the accuracy and efficiency of item recognition. For example, if the detection box 4 does not select the item 6 but instead selects the device holding the item 6, the detection box 4 obviously cannot achieve the purpose of item recognition and will only consume computing resources. When the detection box 4 correctly selects the item 6, it can ensure the accuracy of the recognition results of the region formed by the detection box 4.

[0069] To ensure the accuracy of image recognition results for the refrigeration equipment 100 and improve its efficiency, a high-accuracy image detection model needs to be selected. To select such a model, its accuracy needs to be known.

[0070] In order to determine the accuracy of the image detection model, certain processing operations can be performed on the image detection model to determine whether the generated detection box 4 is correct, that is, whether the detection box 4 is correctly selected on the item 6. Then, the number of correct detection boxes 4 can be counted, and the accuracy of the image detection model can be determined based on the number of correct detection boxes 4.

[0071] Reference Figures 2 to 5 In one embodiment of the present invention, an image detection model processing method is provided, comprising:

[0072] Get item image 7;

[0073] Obtain the true bounding box 5A of each item 6 in the input item image 7. i i = {1, 2, ..., n1}, where n1 is the number of items 6 in the item image 7, and each item 6 corresponds to a real bounding box 5A. i The real frame 5A i The rectangular frame corresponding to the maximum outline of each item 6 in the item image 7; (refer to...) Figure 3 (The rectangle marked with a dashed line)

[0074] The image of the item 7 is input into the image detection model, and the rectangular detection box 4B of the item 6 generated by the image detection model for the image of the item 7 is obtained. p p = {1, 2, ..., n2}, where n2 is the detection box 4B p The number of;

[0075] From the detection frame 4B p Obtain the detection box to be judged 4b from the middle k k = {1, 2, ..., n3}, where n3 is the detection box to be judged 4b k The number of elements, n3≤n2;

[0076] From the actual frame 5A i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real bounding box is 5A. kx x = {1, 2, ..., n4}, n4 ≤ n1, n4 is the detection box to be judged 4b k The corresponding matching real box 5A kx The number of;

[0077] Obtain the detection box 4b to be judged k The area Sb formed k and the detection box 4b to be judged k Each corresponding real bounding box 5A kx The area SA formed kx ;

[0078] Obtain the detection box 4b to be judged k Each of the corresponding real-world matching boxes 5A kx The overlapping area Sb k ∩SA kx ;(refer to Figure 4 (Middle shaded area)

[0079] Calculate the detection box 4b to be judged k Each of the corresponding real-world matching boxes 5A kx The overlap ratio bA kx The overlap ratio

[0080] Determine the detection box 4b to be judged k Does the overlap ratio bA exceed the first threshold? kx ;

[0081] If so, then the detection box to be judged 4b will be... k The judgment result is marked as correct.

[0082] In this embodiment, the real bounding box 5 is a box pre-annotated by a person on the object image 7. The data information of the real bounding box 5 can include its location information and object type label information.

[0083] Staff can identify the area where a rectangle is located by marking its top-left and bottom-right coordinates, i.e., (x... 左上 y 左上 ) and (x 右下 y 右下 Alternatively, its area can be identified by marking the center coordinates of its rectangle and the rectangle's width and height, i.e., (x... 中心 y 中心 ),w 宽 and h 高 .

[0084] The relevant data information of the ground truth bounding box 5 can be stored in the annotation file in advance. During the processing of the image detection model, the relevant data information of the ground truth bounding box 5 of the object image 7 can be obtained by accessing the annotation file.

[0085] The detection box 4 to be judged can refer to the detection box 4 that meets specific conditions selected from the detection boxes 4 generated by the image detection model.

[0086] A matching ground truth box 5 can refer to a ground truth box 5 selected from manually labeled ground truth boxes 5 that meets specific conditions and is used to match a specific detection box 4 to determine whether the detection box 4 is correct. One detection box 4 can correspond to several matching ground truth boxes 5. The same ground truth box 5 can be a matching ground truth box 5 for different detection boxes 4.

[0087] The area of ​​the region can refer to the area of ​​the rectangular region selected by the ground truth box 5 or the detection box 4, and can be calculated by multiplying the length and width of the rectangle.

[0088] The overlapping area can refer to the part of the area where the selected area of ​​the real bounding box 5 and the selected area of ​​the detection box 4 overlap.

[0089] The overlap ratio can be the ratio of the area of ​​the overlapping region between the detection box 4 and the ground truth box 5 to the smaller area of ​​the detection box 4 and the ground truth box 5.

[0090] The first threshold can be a pre-set specific value used as a comparison benchmark for the overlap ratio. When the overlap ratio is greater than the first threshold, the corresponding detection box 4 can be judged as correct; when the overlap ratio is less than the first threshold, the corresponding detection box 4 can be judged as incorrect.

[0091] In actual image recognition processes, there may be situations where the detection box 4 is located within the ground truth box 5 and the ground truth box 5 is much larger than the detection box 4, or where the ground truth box 5 is located within the detection box 4 and the detection box 4 is much larger than the ground truth box 5, or where the ground truth box 5 is much larger than the detection box 4 and their overlapping area. Here, "much larger than" can mean that the area differs by one time or more.

[0092] In this embodiment, the overlapping area of ​​the detection frame 4 and the real frame 5 is compared with the smaller area of ​​the two frames. Even if the areas of the detection frame 4 and the real frame 5 differ significantly, the overlap ratio calculated using this embodiment can accurately reflect the degree of overlap between the detection frame 4 and the real frame 5. When the overlap ratio is large, it indicates that the detection frame 4 and the real frame 5 are highly overlapping, meaning that the detection frame 4 has correctly selected the item 6, and the generated detection frame 4 is correct.

[0093] This setting can accurately and realistically reflect the degree of overlap between the detection box 4 and the real box 5, and can automatically determine whether the detection box 4 generated by the image detection model for the object image 7 is correct, and ensure the accuracy and reliability of the judgment result. It is simple, efficient, and highly reliable, reducing the workload of manual comparison and selection.

[0094] Reference Figures 6 to 8 Furthermore, in another embodiment of the present invention, the image detection model processing method, wherein, "from the rectangular detection box 4B p Obtain the detection box to be judged 4b from the middle k Specifically, it includes:

[0095] From the actual frame 5A i Obtain the detection frame 4B from the middle p There are at least partially overlapping initial matching ground truth boxes 5A py y = {1, 2, ..., n5}, n5 ≤ n1, n5 is the value of the detection box 4B p At least partially overlapping true frame 5A exists. i The number of;

[0096] Obtain the detection frame 4B p The area formed is SB p and the detection frame 4B p The corresponding initial matching real bounding box 5A py The area SA formedpy ;

[0097] Obtain the detection frame 4B p Each of the corresponding initial matching real boxes 5A py The overlapping area SB p ∩SA py ;(refer to Figure 7 (Middle shaded area)

[0098] Obtain the detection frame 4B p Each of the corresponding initial matching real boxes 5A py The area of ​​the merged region SB p ∪SA py ;(refer to Figure 8 (Middle shaded area)

[0099] Calculate the detection frame 4B p Each of the corresponding initial matching real boxes 5A py The intersection and comparison of BA py The crossover ratio

[0100] Determine the detection frame 4B p Does the intersection-union ratio (BA) exceed the second threshold? py ;

[0101] If so, then the detection box 4B p The judgment result is marked as correct;

[0102] If not, then the detection box 4B p The detection box to be judged is marked as 4b. k .

[0103] In this embodiment, the initial matching ground truth box 5 can refer to a ground truth box 5 selected from manually labeled ground truth boxes 5 that at least partially overlaps with a specific detection box 4. One detection box 4 can correspond to several initial matching ground truth boxes 5. The same ground truth box 5 can become the initial matching ground truth box 5 of different detection boxes 4.

[0104] The merged region can refer to the region formed by merging the selected area of ​​the real bounding box 5 and the selected area of ​​the detection box 4.

[0105] The intersection-union ratio (IU) is the ratio of the area of ​​the overlapping region between the ground truth bounding box 5 and the detection bounding box 4 to the area of ​​the merged region between the ground truth bounding box 5 and the detection bounding box 4. The IU can be used as a numerical measure of the degree of overlap between the detection bounding box 4 and the ground truth bounding box 5.

[0106] The second threshold can be a pre-set specific value used as a comparison benchmark for the cross-union ratio (CUR). When the CUR is greater than the first threshold, the corresponding detection box 4 can be directly determined as correct. When the CUR is less than the first threshold, the corresponding detection box 4 can be marked as a detection box to be judged, and then a second judgment can be made on the detection box to be judged.

[0107] In this embodiment, all detection boxes 4 generated by the image detection model can be matched with their corresponding initial matching ground truth boxes 5 and the cross-union ratio can be calculated. The cross-union ratio is used to measure the degree of overlap between the detection boxes 4 and the ground truth boxes 5, so as to filter out the correct detection boxes 4 and the detection boxes 4 that need to be judged for secondary judgment.

[0108] This setup allows for the initial screening of correct detection boxes 4 by calculating the intersection-union ratio (IU), followed by the screening of correct detection boxes 4 from those whose IU does not meet the IU requirement by calculating the overlap ratio. This effectively prevents the correct detection boxes 4 from being misjudged, accurately reflects the degree of overlap between the detection boxes 4 and the real boxes 5, and enables automatic determination of whether the detection boxes 4 generated by the image detection model for the object image 7 are correct. It also ensures the accuracy and reliability of the determination results, is simple and efficient, highly reliable, and reduces the workload of manual comparison and selection.

[0109] Furthermore, in another embodiment of the present invention, the image detection model processing method further includes: the first threshold being any value between 0.75 and 0.9, and the second threshold being any value between 0.5 and 0.6.

[0110] Reference Figure 6 Furthermore, in another embodiment of the present invention, the image detection model processing method further includes:

[0111] If the detection box 4B p The determination result is correct, from the B. p The corresponding initial matching real frame 5A py Obtain the maximum intersection-union ratio (BA) in the middle. py The corresponding best real frame is 5ZA. p ;

[0112] "From the actual frame 5A" i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real bounding box is 5A. kx "Including from the real frame 5A" i Excluding all obtained best true frames 5ZA p Then, the detection box 4b to be judged is obtained. k The corresponding matching real box 5A kx .

[0113] In this embodiment, the best ground truth box 5 can refer to the ground truth box 5 among the manually labeled ground truth boxes that has the highest degree of overlap with the correct detection box 4. For the correct detection box 4 obtained by calculating the cross-union ratio, the best ground truth box 5 can be the ground truth box 5 that corresponds to the ground truth box 4 with the maximum cross-union ratio calculated from the correct detection box 4.

[0114] This setting, by excluding the best true box 5 corresponding to the correct detection box 4 obtained by calculating the intersection-union ratio, can prevent the best true box 5 from participating in the subsequent judgment process of the detection box 4 to be judged, prevent interference with the judgment of the detection box 4 to be judged, reduce the amount of calculation, improve the calculation efficiency, ensure the accuracy of the judgment result of the detection box 4 to be judged, is simple, efficient, and highly reliable, and reduces the workload of manual comparison and selection.

[0115] Reference Figure 6 Furthermore, in another embodiment of the present invention, the image detection model processing method further includes:

[0116] Obtain the real frame 5A i Item category labels;

[0117] Obtain the detection box 4b to be judged k Item category labels;

[0118] "From the actual frame 5A" i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real bounding box is 5A. kx "Including from the detection box 4b to be judged" k The real frame 5A of the same item type i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real box 5A kx .

[0119] This configuration, by selecting the matching real box 5 corresponding to the object to be judged 4 from the real boxes 5 with the same object type as the object to be judged 4, can prevent real boxes 5 with different object types from participating in the subsequent judgment process of the object to be judged 4, prevent interference with the judgment of the object to be judged 4, reduce the amount of calculation, improve the calculation efficiency, ensure the accuracy of the judgment result of the object to be judged 4, and is simple, efficient, and reliable, reducing the workload of manual comparison and selection.

[0120] Reference Figure 6 Furthermore, in another embodiment of the present invention, the image detection model processing method, wherein, "from the real bounding box 5A..." i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real bounding box is 5A. kx "include:

[0121] From the detection box 4b to be judged k At least partially overlapping true frame 5A exists. i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real box 5A kx .

[0122] This configuration, by selecting the matching real box 5 corresponding to the detection box 4 from the real boxes 5 that at least partially overlap with the detection box 4, can prevent real boxes 5 that do not overlap with the detection box 4 from participating in the subsequent judgment process of the detection box 4, thus preventing interference with the judgment of the detection box 4, reducing the amount of computation, improving computational efficiency, ensuring the accuracy of the judgment result of the detection box 4, and is simple, efficient, and highly reliable, reducing the workload of manual comparison and selection.

[0123] Reference Figure 6 Furthermore, in another embodiment of the present invention, the image detection model processing method wherein each of the detection boxes to be judged (4b) is obtained sequentially in ascending order of k value. k The judgment results also include:

[0124] Obtain the detection box 4b to be judged k-m The overlap ratio bA is greater than the first threshold. (k-m)x The quantity, m={1,2,…,n6}, k-n6≥1;

[0125] If the overlap ratio bA is greater than the first threshold (k-m)x If the quantity is 1, then the overlap ratio bA is... (k-m)x The corresponding matching real box 5A (k-m)x Marked as best real frame 5ZA k-m ;

[0126] "From the actual frame 5A" i The detection box 4b to be judged is obtained from the middle. k The corresponding matching real bounding box is 5A. kx "Including from the real frame 5A" i Excluding all obtained best true frames 5ZA k-m Then, the detection box 4b to be judged is obtained. k The corresponding matching real box 5A kx .

[0127] In this embodiment, each detection box 4b to be judged can be obtained sequentially in ascending order of k value. k The judgment result is that each detection box to be judged has 4 bytes. k The corresponding K value can be randomly labeled.

[0128] Preferably, the matching real box 5 can be the real box 5 remaining after excluding all the best real boxes 5 obtained from the manually labeled real boxes 5, which has the same item type as the detection box 4 to be judged and partially overlaps with it.

[0129] This setting, by excluding the best true box 5 corresponding to the correct detection box 4 obtained by calculating the overlap ratio, can prevent the best true box 5 from participating in the subsequent calculation process of the overlap ratio of the detection box 4, prevent interference with the judgment of the detection box 4, reduce the amount of calculation, improve the calculation efficiency, ensure the accuracy of the judgment result of the detection box 4, and is simple, efficient, and reliable, reducing the workload of manual comparison and selection.

[0130] Reference Figure 6 and Figure 9 Furthermore, in another embodiment of the present invention, the image detection model processing method further includes:

[0131] If the overlap ratio bA is greater than the first threshold (k-m)x If the number is greater than or equal to 2, then the detection box to be judged 4b is obtained. k-m The four vertices; (refer to) Figure 9 (x) b左上 y b左上 ), (x b左下 y b左下 ), (x b右上 y b右上 ), (x b右下 y b右下 ))

[0132] Obtain the overlap ratio bA that is greater than the first threshold. (k-m)x Each corresponding real bounding box 5A (k-m)x The four vertices; (refer to) Figure 9 (x) A左上 y A左上 ), (x A左下 y A左下 ), (x A右上 y A右上 ), (x A右下 y A右下 ))

[0133] Calculate the detection box 4b to be judged k-m Matching the real frame 5A as described above (k-m)x The vertex distance and,

[0134] The method for calculating the vertex distance sum is as follows: The detection boxes 4b to be judged that are in the same position are... k-m The four vertices match the real bounding box 5A. (k-m)xThe four vertices are matched one-to-one to obtain the detection box 4b to be judged. k-m The matching real bounding box 5A for each vertex and its corresponding orientation (k-m)x The vertex distances are calculated by summing the four vertex distances.

[0135] Obtain the minimum vertex distance and the corresponding matching ground truth box 5A (k-m)x And marked as the best true frame 5ZA k-m .

[0136] This setup allows for the determination of the optimal ground truth box 5 by calculating the sum of vertex distances between the target detection box 4 and the matching ground truth box 5. The smaller the sum of vertex distances, the closer the vertices of the target detection box 4 and the matching ground truth box 5 are, indicating a higher degree of matching. This ensures that the optimal ground truth box 5 is the ground truth box 5 with the highest degree of matching to the target detection box 4, avoiding the impact of incorrect labeling of the optimal ground truth box 5 on the accuracy of subsequent judgment results of the target detection box 4. It is simple, efficient, and highly reliable, reducing the workload of manual comparison and selection.

[0137] Furthermore, in another embodiment of the present invention, the image detection model processing method further includes:

[0138] The item image 7 is input into several different image detection models M. j j = {1, 2, ..., n7};

[0139] Obtain each of the image detection models M j The corresponding judgment result is the correct detection box 4B. p and the detection box 4b to be judged k Total quantity sumM j ;

[0140] Get the maximum sumM j The corresponding image detection model M j It was then marked as the best image detection model.

[0141] In this embodiment, the total number of correct detection boxes 4 generated by different image detection models can be obtained using the same object image 7 and the processing method described above. The accuracy of different image detection models can be compared based on the total number of correct detection boxes 4 generated, thereby obtaining the best image detection model.

[0142] Alternatively, the total number of detection boxes 4 generated by each image detection model can be counted, and the accuracy of the image detection model can be determined and compared based on the proportion of the total number of correct detection boxes 4 to the total number of all detection boxes 4.

[0143] This setup allows for the automatic and accurate generation of the highest-accuracy image detection model from different image detection models. It is simple, efficient, and highly reliable, reducing the workload of manual comparison and selection.

[0144] In this embodiment, the best image detection model with the highest accuracy can be applied to the refrigeration device 100 to ensure the accuracy of the image recognition results of the refrigeration device 100 and improve the image recognition efficiency of the refrigeration device 100.

[0145] Furthermore, in one embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps in the image detection model processing method as described in any of the above embodiments.

[0146] Furthermore, in one embodiment of the present invention, a system for processing image detection models is provided, wherein the system includes a memory and a processor, the memory storing a computer program that can run on the processor, and the processor executing the computer program to implement the steps in the image detection model processing method as described in any of the above embodiments.

[0147] In summary, the image detection model processing method, storage medium, and system of the present invention can solve the problems of the prior art, which increases labor costs and has low efficiency, by calculating the overlap ratio between the detection box 4 to be judged and each corresponding matching real box 5, and by manually judging whether the detection box generated by the image detection model is correct.

[0148] By adopting the technical solution in this application, the correct detection box 4 can be initially screened out by calculating the intersection-union ratio (IU), and then the correct detection box 4 can be screened out again from the detection boxes 4 whose IU do not meet the requirements by calculating the overlap ratio. This can effectively prevent the correct detection box 4 from being misjudged, accurately reflect the degree of overlap between the detection box 4 and the real box 5, and automatically determine whether the detection box 4 generated by the image detection model for the object image 7 is correct, and ensure the accuracy and reliability of the judgment result. In addition, it can also accurately and automatically obtain the image detection model with the highest accuracy from different image detection models. It is simple, efficient, and highly reliable, reducing the workload of manual comparison and selection.

[0149] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0150] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image detection model processing method, characterized in that, include: Get the item image; Obtain the ground truth bounding box A of each item in the input item image. i Let i = {1, 2, ..., n1}, where n1 is the number of items in the item image, and each item corresponds to a real bounding box A. i The real frame A i The bounding box corresponding to the maximum outline of each item in the item image; The image of the object is input into the image detection model, and the rectangular detection box B of the object generated by the image detection model for the object image is obtained. p p={1,2,…,n2}, where n2 is the detection box B p The number of; From the detection box B p Obtain the detection box b to be judged from the middle k k={1,2,…,n3}, where n3 is the detection box b to be judged. k The number of elements, n3≤n2; From the real frame A i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx x={1,2,…,n4},n4≤n1,n4 is the detection box b to be judged. k The corresponding matching real box A kx The number of; Obtain the detection box b to be judged k The area Sb formed k and the detection box to be judged b k Each corresponding real bounding box A kx The area SA formed kx ; Obtain the detection box b to be judged k With each of the corresponding real bounding boxes A kx The overlapping area Sb k ∩SA kx ; Calculate the detection box b to be judged k With each of the corresponding real bounding boxes A kx The overlap ratio bA kx The overlap ratio bA kx = ; Determine the detection box b to be judged k Does the overlap ratio bA exceed the first threshold? kx ; If so, then the detection box to be judged b will be... k The judgment result is marked as correct; "From the rectangular detection box B" p Obtain the detection box b to be judged from the middle k Specifically, it includes: From the real frame A i Obtain the detection box B from the middle p There is at least a partially overlapping initial matching ground truth box A py y={1,2,…,n5},n5≤n1,n5 is the value of the detection box B p At least partially overlapping true bounding boxes A exist. i The number of; Obtain the detection box B p The area formed is SB p and the detection box B p Each of the corresponding initial matching ground truth boxes A py The area SA formed py ; Obtain the detection box B p Each of the corresponding initial matching real boxes A py The overlapping area SB p ∩SA py ; Obtain the detection box B p Each of the corresponding initial matching real boxes A py The area of ​​the merged region SB p ∪SA py ; Calculate the detection box B p Each of the corresponding initial matching real boxes A py The intersection and comparison of BA py The intersection and union ratio BA py = ; Determine the detection box B p Does the intersection-union ratio (BA) exceed the second threshold? py ; If so, then the detection box B... p The judgment result is marked as correct; If not, then the detection box B will be... p The box to be detected is marked as b. k .

2. The image detection model processing method as described in claim 1, characterized in that, Also includes: If the detection box B p The determination result is correct, from the B. p The corresponding initial matching real box A py Obtain the maximum intersection-union ratio (BA) in the middle. py The corresponding best real bounding box ZA p ; From the real frame A i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "Including from the real frame A" i Excluding all obtained optimal true bounding boxes ZA p Then, the detection box b to be judged is obtained. k The corresponding matching real box A kx .

3. The image detection model processing method as described in claim 1, characterized in that, Also includes: Obtain the real frame A i Item category labels; Obtain the detection box b to be judged k Item category labels; From the real frame A i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "Including from the detection box b to be judged" k The real frame A with the same item type i Obtain the detection box b to be judged from the middle. k The corresponding matching real box A kx .

4. The image detection model processing method as described in claim 1, characterized in that, From the real frame A i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "include: From the detection box b to be judged k At least partially overlapping true bounding boxes A exist. i Obtain the detection box b to be judged from the middle. k The corresponding matching real box A kx .

5. The image detection model processing method as described in claim 1, characterized in that, Each detection box b to be judged is obtained sequentially according to the k value from small to large. k The judgment results also include: Obtain the detection box b to be judged k-m The overlap ratio bA is greater than the first threshold. （k-m）x The quantity, m={1,2,…,n6}, k-n6≥1; If the overlap ratio bA is greater than the first threshold （k-m）x If the quantity is 1, then the overlap ratio bA is... （k-m）x The corresponding matching real box A （k-m）x Marked as the best true frame ZA k-m ; From the real frame A i Obtain the detection box b to be judged from the middle. k The corresponding matching real bounding box A kx "Including from the real frame A" i Excluding all obtained optimal true bounding boxes ZA k-m Then, the detection box b to be judged is obtained. k The corresponding matching real box A kx .

6. The image detection model processing method as described in claim 5, characterized in that, Also includes: If the overlap ratio bA is greater than the first threshold （k-m）x If the number is greater than or equal to 2, then the detection box to be judged b is obtained. k-m The four vertices; Obtain the overlap ratio bA that is greater than the first threshold. （k-m）x Each corresponding real bounding box A （k-m）x The four vertices; Calculate the detection box b to be judged k-m Matching the real box A as described above （k-m）x The vertex distance and, The method for calculating the vertex distance sum is as follows: The detection boxes b to be judged that are in the same position are... k-m The four vertices match the real bounding box A. （k-m）x The four vertices are matched one-to-one to obtain the detection box b to be judged. k-m The matching real bounding box A for each vertex and its corresponding orientation （k-m）x The vertex distances are calculated by summing the four vertex distances. Obtain the minimum vertex distance and the corresponding matching ground truth box A. （k-m）x And marked as the best true bounding box ZA k-m .

7. The image detection model processing method as described in claim 1, characterized in that, Also includes: The object images are input into several different image detection models M. j j={1,2,…,n7}; Obtain each of the image detection models M j The corresponding judgment result is the correct detection box B. p and the detection box b to be judged k Total quantity sumM j ; Get the maximum sumM j The corresponding image detection model M j It was then marked as the best image detection model.

8. The image detection model processing method as described in claim 1, characterized in that, Also includes: The first threshold is any value between 0.75 and 0.9, and the second threshold is any value between 0.5 and 0.

6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the image detection model processing method as described in any one of claims 1-8.

10. A system for processing image detection models, characterized in that, The system includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the image detection model processing method as described in any one of claims 1-8.

Images