Image detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image detection method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring image data; analyzing the image data to obtain article information; if the article information does not belong to the preset range, determining that the article has at least one of a placement problem and an overtime problem of the foreign article; and if the article has the placement problem, responding to the first instruction, and prompting a user to process the placement problem. According to the embodiment of the invention, intelligent article management is realized, and the work efficiency and accuracy of article management are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image detection method and device, an electronic device and a storage medium. Background Art

[0002] Machine vision is an important part of the interaction between digital systems and the real world. Through machine vision products, the automation system can view the factory status at high speed and high accuracy, providing users with real-time and accurate image data for the detection of disorderly placement in the factory.

[0003] With the rapid development of the manufacturing industry, the number of factories has increased rapidly and their scale has also become larger and larger. The requirements for the placement of internal items have become more and more sophisticated. However, due to the large area and complex personnel within the factory, internal items are often placed randomly.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose an image detection method and device, an electronic device and a storage medium, aiming to improve the work efficiency and accuracy of item management.

[0006] To achieve the above object, an embodiment of the present application provides an image detection method, which includes the following steps:

[0007] Collecting image data;

[0008] Analyzing the image data to obtain item information;

[0009] If the item information does not fall within the preset range, it is determined that the item has at least one of a placement problem and a foreign item timeout problem;

[0010] If there is a placement problem with the item, in response to the first instruction, the user is prompted to solve the placement problem.

[0011] In some embodiments, the image data includes regional item image data, item stacking scale image data, and item stacking point image data, and the collected image data includes:

[0012] The image data of the items in the area, the image data of the object stacking scale and the image data of the object stacking points are collected and detected by a machine vision product.

[0013] In some embodiments, the item information includes detection area item information, item stacking scale information, and detection area item stacking point information, and the analyzing the image data to obtain the item information includes:

[0014] Analyzing the image data of the object in the detection area to obtain the object information in the detection area;

[0015] Analyzing the object stacking scale image data to obtain the object stacking scale information;

[0016] The object stacking point image data is analyzed to obtain the object stacking point information in the detection area.

[0017] In some embodiments, the preset range includes a preset minimum similarity, the item information includes detection area item information, and if the item information does not belong to the preset range, determining that the item has at least one of a placement problem and a foreign item timeout problem, includes:

[0018] Get internal item information;

[0019] Calculating a first similarity between the object in the detection area and the object inside the detection area according to the object information in the detection area and the object information inside the detection area;

[0020] If the first similarity is less than the preset minimum similarity, it is determined that there is a placement problem of the objects in the detection area and a foreign object timeout problem.

[0021] In some embodiments, the preset range includes a preset stacking point location, the item information includes the stacking point information of the items in the detection area, and if the item information does not belong to the preset range, determining that the item has at least one of a placement problem and a foreign item timeout problem, includes:

[0022] If the object stacking point information in the detection area does not belong to the preset stacking point position, it is determined that there is a problem with the placement of the object.

[0023] In some embodiments, the preset range includes a preset maximum stacking height, the item information includes item stacking scale information, and if the item information does not fall within the preset range, determining that the item has at least one of a placement problem and a foreign item timeout problem, includes:

[0024] Determine the height at which the items are placed according to the item stacking scale information;

[0025] If the placement height of the items is greater than the preset maximum stacking height, it is determined that there is a problem with the placement of the items.

[0026] In some embodiments, the method further comprises:

[0027] If the item has a foreign item timeout problem, in response to the second instruction, the user is prompted to handle the foreign item timeout problem.

[0028] In some embodiments, if the first similarity is less than the preset minimum similarity, determining that there is a placement problem of the object in the detection area and a foreign object timeout problem includes:

[0029] If the first similarity is less than the preset minimum similarity, it is determined that there is a placement problem of the objects in the detection area, and the stop start time is obtained;

[0030] Determine whether the stay start time exists, and if the stay start time does not exist, record the current time as the stay start time;

[0031] If the stay start time exists, subtract the stay start time from the current time to obtain the stay time of the external object, obtain the preset maximum stay time, and determine whether the stay time of the external object is greater than the preset maximum stay time;

[0032] If the residence time of the external object is greater than the preset maximum residence time, it is determined that there is a placement problem of the object in the detection area and a foreign object timeout problem;

[0033] If the residence time of the external object is less than the preset maximum residence time, it is determined that the object in the detection area only has a placement problem.

[0034] To achieve the above object, another aspect of an embodiment of the present application provides an image detection device, the device comprising:

[0035] An acquisition module, used for acquiring image data;

[0036] An analysis module, used for analyzing the image data to obtain item information;

[0037] A judgment module, configured to judge that the item has at least one of a placement problem and a foreign item timeout problem if the item information does not fall within a preset range;

[0038] The prompt module is used to prompt the user to solve the placement problem in response to the first instruction if there is a placement problem of the item.

[0039] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0040] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0041] The embodiments of the present application include at least the following beneficial effects: the present application provides an image detection method and device, an electronic device and a storage medium, which collects image data; analyzes the image data to obtain item information; if the item information does not fall within a preset range, determines that the item has at least one of a placement problem and a foreign item timeout problem, thereby reducing the frequency of manual inspections, improving work efficiency, and improving the degree of automation of item management; if the item has a placement problem, in response to a first instruction, prompts the user to deal with the placement problem, thereby improving the work efficiency and accuracy of item management, improving the level of civilization in the item placement area, and improving the user experience, which is conducive to users solving item placement problems in a timely manner and reducing the risk of loss and safety risks caused by item placement problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the image detection method provided in an embodiment of the present application;

[0043] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0044] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0045] Figure 4 yes Figure 3 Flow chart of step S303 in FIG.

[0046] Figure 5 yes Figure 1 Flow chart of step S103 in FIG.

[0047] Figure 6 yes Figure 1 Flow chart of step S103 in FIG.

[0048] Figure 7 is a structural diagram of an article disorderly stacking detection and alarm system provided in an embodiment of the present application;

[0049] Figure 8 This is a flow chart of the image detection method provided in the embodiment of the present application applied to the object disorderly stacking detection alarm system;

[0050] Fig. 9 is a schematic diagram of the structure of an image detection device provided in an embodiment of the present application;

[0051] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0053] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0054] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0056] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0057] 1) Mechanical vision products refer to automated equipment or systems that use computers and image processing technology to simulate the human visual system to achieve functions such as object recognition, measurement, positioning and detection.

[0058] Machine vision is an important part of the interaction between digital systems and the real world. Through machine vision products, the automation system can view the factory status at high speed and high accuracy, providing users with real-time and accurate image data for the detection of disorderly placement in the factory.

[0059] With the rapid development of the manufacturing industry, the number of domestic factories has increased rapidly and their scale has become larger and larger. The requirements for the placement of internal items have become more and more sophisticated. However, due to the large area and complex personnel within the factory, internal items are often placed randomly.

[0060] In summary, the technical problems existing in the relevant technologies need to be improved.

[0061] In view of this, an image detection method, device, equipment and medium are provided in the embodiments of the present application. The scheme collects image data; analyzes the image data to obtain item information; if the item information does not fall within a preset range, it is determined that the item has at least one of a placement problem and a foreign item timeout problem, thereby reducing the frequency of manual inspections, improving work efficiency, and improving the degree of automation of item management; if the item has a placement problem, in response to a first instruction, the user is prompted to deal with the placement problem, thereby improving the work efficiency and accuracy of item management, improving the level of civilization in the item placement area, and improving the user experience, which is conducive to users solving item placement problems in a timely manner and reducing the risk of loss and safety risks caused by item placement problems.

[0062] The image detection method provided in the embodiment of the present application relates to the field of image processing technology. The image detection method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the image detection method, etc., but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0064] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0065] Figure 1 is an optional flow chart of the image detection method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.

[0066] Step S101, collecting image data.

[0067] Specifically, the image data includes regional item image data, item stacking scale image data, and item stacking point image data.

[0068] Optionally, a camera is used to periodically capture images of the detection area to periodically determine whether there are any placement problems with the items.

[0069] In some embodiments, image data of objects in the inspection area, image data of the object stacking scale, and image data of the object stacking points are collected by a machine vision product.

[0070] In step S101 of some embodiments, the machine vision products include but are not limited to fixed cameras and smart control balls. The fixed cameras and smart control balls can detect multiple item placement areas in the factory at high speed and high accuracy.

[0071] In this embodiment, collecting image data is helpful to timely discover problems with the placement of items and potential safety hazards. At the same time, high-quality images can provide more information and improve the accuracy of subsequent processing.

[0072] Step S102: Analyze the image data to obtain item information.

[0073] Specifically, the item information includes the item information in the detection area, the item stacking scale information, and the item stacking point information in the detection area.

[0074] Optionally, the detection area object information is obtained by analyzing the detection area object image data.

[0075] In some embodiments, the object stacking scale image data is analyzed to obtain the object stacking scale information.

[0076] In other embodiments, the object stacking point image data is analyzed to obtain the object stacking point information in the detection area.

[0077] Optionally, the images are processed using image processing and computer vision techniques.

[0078] In some embodiments, objects in an image are identified through a machine learning model, and information about objects in the detection area, information about the scale of object stacking, and information about object stacking points in the detection area are extracted.

[0079] In this embodiment, analyzing the image data to obtain the object information can help determine whether there is a problem with the placement of the objects, thereby improving work efficiency.

[0080] Step S103: If the item information does not belong to the preset range, it is determined that there is at least one of a placement problem and a foreign item timeout problem.

[0081] Specifically, the preset range includes a preset minimum similarity, a preset stacking point position, and a preset maximum stacking height.

[0082] In some embodiments, internal item information is obtained; a first similarity between items in the detection area and internal items is calculated based on the detection area item information and the internal item information; if the first similarity is less than a preset minimum similarity, it is determined that there is a placement problem with items in the detection area and a foreign item timeout problem.

[0083] It can be understood that internal items are items in the internal area, and internal items may include items within the factory or items within the warehouse.

[0084] It is understandable that the first similarity can be a single similarity or multiple similarities. If there is a first similarity that is less than the preset minimum similarity (or all first similarities are greater than or equal to the preset minimum similarity), it is determined that the object in the detection area is a foreign object, and it is determined that there is a placement problem and a foreign object timeout problem in the detection area.

[0085] Optionally, if the stacking point information of the items in the detection area does not belong to the preset stacking point location, it is determined that there is a problem with the placement of the items.

[0086] In some embodiments, the placement height of the items is determined based on the item stacking scale information; if the item placement height is greater than a preset maximum stacking height, it is determined that there is a placement problem with the items.

[0087] In this embodiment, if the item information does not fall within the preset range, it is determined that the item has at least one of a placement problem and a foreign item timeout problem, thereby automatically determining the placement problem, reducing the frequency of manual inspections, improving work efficiency, and increasing the degree of automation in item management.

[0088] Step S104: If there is a placement problem with the item, in response to the first instruction, the user is prompted to solve the placement problem.

[0089] Specifically, the first instruction is an alarm instruction for placement problems. The first instruction is triggered when the system determines that there is a placement problem with the item, and is used to prompt the user to deal with the placement problem.

[0090] In some embodiments, the user may be notified via text message, App notification, or email to handle the placement problem, or may be notified via an alarm page to handle the placement problem, but is not limited thereto.

[0091] Optionally, provide processing suggestions to the user and recommend item placement locations to the user based on item information.

[0092] When the system determines that there is a placement problem, it will prompt the user to organize the items through a system alarm and recommend a placement location for the user.

[0093] Furthermore, if the item has a foreign item timeout problem, in response to the second instruction, the user is prompted to handle the foreign item timeout problem. The second instruction is an alarm instruction for the foreign item timeout problem, and the second instruction is triggered when the system determines that the item has a foreign item timeout problem, and is used to prompt the user to handle the foreign item timeout problem.

[0094] In some embodiments, the user may be notified of a timeout for handling a foreign object via text message, App notification, or email, or may be notified of a timeout for handling a foreign object via an alarm page, without limitation.

[0095] When the system determines that there is a timeout problem of foreign objects, the system alarms to prompt the user to clean up or register the foreign objects.

[0096] Furthermore, the area of ​​the placement area near the internal object N is obtained by edge feature extraction, and the suitable value of the position near the internal object suitable for accommodating foreign objects (i.e., objects in the detection area) is analyzed. _ :

[0097]

[0098] Among them, M A_q is the suitable value for accommodating foreign objects in the location near the internal objects _ , m is an arbitrary number set by the system, K1 is the appropriate value M A_q The influence coefficient of the distance between the qth position and the position of the foreign object, X N_P is the horizontal coordinate of the pth position of the internal item N, Y N_P is the ordinate of the pth position of the internal object N, X is the abscissa of the position of the foreign object, Y is the ordinate of the position of the foreign object, K2 is the appropriate value M A_q The coefficient affected by the height at which item N needs to be placed, S A is the stacking area of ​​items, H A h is the height of the items placed. m is the optimal placement height of item M.

[0099] Furthermore, the maximum value of the appropriate value is determined, and the position corresponding to the maximum value is prompted to the user as a recommended placement position.

[0100] In this embodiment, if there is a placement problem with the item, in response to the first instruction, the user is prompted to deal with the placement problem, which improves the work efficiency and accuracy of item management, improves the user experience, and helps the user to solve the item placement problem in a timely manner.

[0101] In steps S101 to S104 shown in the embodiment of the present application, image data is collected; item information is obtained by analyzing the image data; if the item information does not fall within a preset range, it is determined that the item has at least one of a placement problem and a foreign item timeout problem, thereby reducing the frequency of manual inspections, improving work efficiency, and improving the degree of automation of item management; if the item has a placement problem, in response to the first instruction, the user is prompted to deal with the placement problem, thereby improving the work efficiency and accuracy of item management, improving the level of civilization in the item placement area, and improving the user experience, which is conducive to users solving item placement problems in a timely manner and reducing the risk of loss and safety risks caused by item placement problems.

[0102] See also Figure 2In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0103] Step S201, analyzing the image data of objects in the detection area to obtain information about the objects in the detection area.

[0104] Specifically, the detection area object information includes the image grayscale value.

[0105] In step S201 of some embodiments, the object image is analyzed and the image grayscale value is obtained by graying the image.

[0106] In this embodiment, the image data of objects in the detection area are analyzed to obtain the object information in the detection area, which prepares for the subsequent judgment of whether there are placement problems and foreign object timeout problems, and is conducive to discovering and solving the problem of foreign objects entering the factory.

[0107] Step S202: Analyze the object stacking scale image data to obtain object stacking scale information.

[0108] Specifically, the item stacking scale information includes the item stacking area and the item placement height.

[0109] In step S202 of some embodiments, the object stacking scale image data is analyzed, and the object stacking area and the object placement height are obtained by edge feature extraction.

[0110] Step S203, analyzing the object stacking point image data to obtain the object stacking point information in the detection area.

[0111] Specifically, the information of the object stacking point in the detection area includes the position coordinates of the object stacking point.

[0112] In step S203 of some embodiments, the image data of the object stacking point is analyzed to establish a two-dimensional coordinate system and the position coordinates of the object stacking point in the image.

[0113] See also Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S303:

[0114] Step S301, obtaining internal item information.

[0115] Specifically, the internal items are items that can be stored in the internal area, and the internal item information includes the grayscale value of the internal item image. Exemplarily, the internal items can be factory items or warehouse items, but are not limited thereto.

[0116] In step S301 of some embodiments, the grayscale value of the internal object image is obtained from a database.

[0117] Step S302: Calculate a first similarity between the objects in the detection area and the objects inside the detection area according to the object information in the detection area and the object information inside the detection area.

[0118] In step S302 of some embodiments, the first similarity between the object in the detection area and the internal object is calculated according to the gray value of the image of the object in the detection area and the gray value of the image of the internal object, and the expression is as follows:

[0119]

[0120] Where i = 1, 2, ..., n, W Ai To detect the similarity between the regional object image and the i-th internal object image, grey Ai is the gray value of the i-th internal item image, grey A is the gray value of the object image in the detection area.

[0121] Further, the first similarities {W A1 ,W A2 ,…,W An}, and get the maximum value W AN .

[0122] Step S303: If the first similarity is less than a preset minimum similarity, it is determined that there is a placement problem of the objects in the detection area and a foreign object timeout problem.

[0123] In step S303 of some embodiments, a preset minimum similarity W is obtained.

[0124] Optionally, if the maximum value of the first similarity W AN If the similarity is less than the preset minimum similarity W, the items in the detection area are determined to be foreign items (i.e., external items), and it is determined that there is a problem with the placement of the items. Foreign items are items that should not be stored in the internal area. For example, if apples are already stored in the internal area or apples have been registered for storage, the apples in the detection area are not foreign items, while bananas in the detection area are foreign items.

[0125] Furthermore, to determine whether there is a foreign object timeout problem, first obtain the stop start time and determine whether the stop start time exists.

[0126] It is understandable that if the stop start time does not exist, the current time is recorded as the stop start time, and there is no foreign item timeout problem for the item.

[0127] In some embodiments, if the stay start time exists, the stay start time is subtracted from the current time to obtain the stay time of the external object, and the preset maximum stay time is obtained to determine whether the stay time of the external object is greater than the preset maximum stay time.

[0128] If the residence time of the external object is greater than the preset maximum residence time, it is determined that the object T in the detection area has both a placement problem and a foreign object timeout problem.

[0129] Otherwise, if the external object stays for less than the preset maximum stay time, it is determined that the object only has a placement problem.

[0130] See also Figure 4 In some embodiments, step S303 may include but is not limited to steps S401 to S409:

[0131] Step S401, obtaining a preset minimum similarity.

[0132] In step S401 of some embodiments, the preset minimum similarity may be set by a user or determined by the system.

[0133] Step S402: If the maximum value of the first similarity is less than the preset minimum similarity, the object in the detection area is determined to be an external object.

[0134] In step S402 of some embodiments, a maximum value of the first similarity is determined, and if the maximum value of the first similarity is less than a preset minimum similarity, the object in the detection area is determined to be an external object.

[0135] It is understandable that if the items in the detection area are determined to be external items, then there is a problem with the placement of the items.

[0136] Step S403, determining whether there is a stay start time.

[0137] In step S403 of some embodiments, it is determined whether there is a foreign object timeout problem.

[0138] It is understandable that judging whether there is a dwell start time is to judge whether the item is deposited in the detection area for the first time.

[0139] If the stay start time does not exist, step S404 is executed; if the stay start time exists, step S405 is executed.

[0140] Step S404, recording the current time as the stop start time.

[0141] In step S404 of some embodiments, if there is no stay start time, the current time is recorded as the stay start time, so as to prepare for the subsequent foreign object timeout problem judgment.

[0142] There is no stop start time, that is, there is no foreign object timeout problem, so step S408 is executed after step S404.

[0143] Step S405, subtracting the stay start time from the current time to obtain the stay time of the external object.

[0144] Step S406, obtaining a preset maximum stay time.

[0145] Specifically, the preset maximum stay time is the maximum stay time of the external object.

[0146] In step S406 of some embodiments, the preset maximum stay time may be set by the user or determined by the system.

[0147] Step S407, determining whether the external object's residence time is greater than a preset maximum residence time.

[0148] In step S407 of some embodiments, if the residence time of the external object is greater than the preset maximum residence time, step S409 is executed; otherwise, step S408 is executed.

[0149] Step S408, determine that the item only has a placement problem.

[0150] In step S408 of some embodiments, if the external object stays for a time less than or equal to a preset maximum stay time, it is determined that the object only has a placement problem.

[0151] Step S409, determining whether there is a problem with the placement of the items and a timeout problem for the foreign items.

[0152] In step S409 of some embodiments, if the residence time of the external object is greater than the preset maximum residence time, it is determined that there are both placement problems and foreign object timeout problems in the detection area.

[0153] See also Figure 5 In some embodiments, step S103 may also include but is not limited to steps S501 to S503:

[0154] Step S501, obtaining internal object coordinates.

[0155] In step S501 of some embodiments, internal item information is obtained, wherein the internal item information includes internal item coordinates.

[0156] It will be appreciated that the interior items may include a plurality of items.

[0157] Step S502, calculating the minimum distance between the objects in the detection area and the internal objects according to the information of the stacking points of the objects in the detection area and the coordinates of the internal objects.

[0158] In step S502 of some embodiments, the position coordinates of a internal objects are obtained {(XN_1, YN_1), (XN_2, YN_2), ..., (XN_a, YN_a)}, the distances between the object stacking points in the detection area and the a positions of the internal objects are calculated one by one {LN_1, LN_2, ..., LN_a} respectively, and the minimum distance between the object stacking points in the detection area and the internal objects is determined.

[0159] Step S503: If the minimum distance between the object in the detection area and the internal objects is greater than a preset position difference threshold, it is determined that there is a placement problem with the object.

[0160] In step S503 of some embodiments, a preset position difference threshold is obtained, and if the minimum distance between the object in the detection area and the internal objects is greater than the preset position difference threshold, it is determined that there is a problem with the placement of the object.

[0161] See also Figure 6 In some embodiments, step S103 may also include but is not limited to steps S601 to S602:

[0162] Step S601, determining the height of the items according to the item stacking scale information.

[0163] In step S601 of some embodiments, the object placement height of the object stacking scale information is obtained.

[0164] Step S602: If the placement height of the items is greater than the preset maximum stacking height, it is determined that there is a placement problem with the items.

[0165] Specifically, the preset maximum stacking height is the maximum stacking height of the internal items.

[0166] In step S602 of some embodiments, a preset maximum stacking height is obtained, and if the placement height of the items is greater than the preset maximum stacking height, it is determined that there is a placement problem with the items.

[0167] In some embodiments, the image detection method provided in the embodiments of the present application can be applied to a disorderly stacking detection and alarm system for objects. The structure diagram of the disorderly stacking detection and alarm system for objects is shown in FIG. Figure 7 As shown, the system includes: a collection module, a database, an analysis module and a judgment module.

[0168] Figure 8 This is an optional flow chart of the image detection method provided in the embodiment of the present application applied to an object disorderly stacking detection and alarm system. Figure 8 The method may include but is not limited to steps S1 to S4.

[0169] Step S1, collecting image data of objects in the detection area, image data of the object stacking scale and image data of the object stacking points through a machine vision product.

[0170] Specifically, the image data includes regional item image data, item stacking scale image data, and item stacking point image data.

[0171] In some embodiments, the acquisition module acquires the detection area object image data, the object stacking scale image data and the object stacking point image data.

[0172] Among them, the acquisition module includes a machine vision product. The system acquires image data through the machine vision product and transmits the image data to the database through the network.

[0173] Furthermore, the image data of the objects in the inspection area, the image data of the object stacking scale and the image data of the object stacking point are stored in the database, and the object information of the objects in the factory is stored;

[0174] Optionally, machine vision products include but are not limited to fixed cameras and smart control balls.

[0175] Step S2, analyzing the image data of the objects in the detection area to obtain the information of the objects in the detection area, analyzing the image data of the object stacking scale to obtain the information of the object stacking scale, and analyzing the image data of the object stacking point to obtain the information of the object stacking point in the detection area.

[0176] Specifically, the object information in the detection area includes the image features of the object and the stacking position of the object, and the stacking scale information of the object in the detection area includes the stacking height and the stacking area of ​​the object.

[0177] In some embodiments, the analysis module analyzes the image data of the objects in the detection area to obtain the information of the objects in the detection area, analyzes the image data of the object stacking scale to obtain the information of the object stacking scale in the detection area, and analyzes the image data of the object stacking point to obtain the information of the object stacking point in the detection area;

[0178] Optionally, the system analyzes any object image A (i.e., regional object image data) and obtains the grayscale value of image A by graying image A. A .

[0179] In some embodiments, the system analyzes the object stacking scale image data and obtains the object stacking area S by edge feature extraction. A And the height of the items H A .

[0180] In some other embodiments, the system analyzes the image data of the object stacking point and establishes a two-dimensional coordinate system. The coordinates of the position of the object stacking point in image A are (X A ,Y A ).

[0181] Among them, the analysis module compares the item information of the detection area with the item information of the items in the factory, analyzes the similarity between the items in the detection area and any item in the factory; compares the stacking position of any item in the factory in the detection area with the established stacking point position of any item in the factory, analyzes whether any item in the factory in the detection area is placed at the established stacking point position; compares the stacking height of any item in the factory in the detection area with the maximum stacking height of any item in the factory, analyzes whether any item in the factory is lower than the maximum stacking height of any item in the factory.

[0182] Step S3, when the similarity between the items in the detection area and any items in the factory is lower than the set minimum similarity, the system determines that the items in the detection area are items outside the factory and that there are items placed randomly; when any item in the factory is not placed at the predetermined stacking point, the system determines that the item is placed randomly; when it is determined that the placement height of any item in the factory exceeds the predetermined maximum stacking height of items in the factory, the system determines that the item is placed randomly.

[0183] In some embodiments, the judgment module comprehensively considers the information of any item and the stacking scale information of any item to judge whether any item is an item within the factory. When it is judged that any item is not an item within the factory, the judgment module alarms and prompts the user to handle it; when any item is an item within the factory, it judges whether any item is placed in a disorderly manner. When it is judged that any item is placed in a disorderly manner, the judgment module alarms and recommends a placement location.

[0184] Furthermore, when the judgment module determines that the similarity between the items in the detection area and any items in the factory is lower than the set minimum similarity, it is determined that there are items placed randomly; when the judgment module determines that any item in the factory is not placed at a predetermined stacking point, it is determined that the items are placed randomly; when the judgment module determines that any item in the factory is higher than the maximum stacking height of any item in the factory, it is determined that the items are placed randomly; when the judgment module determines that the items are placed randomly, it recommends a placement location for the user.

[0185] It should be noted that the database contains n grayscale values ​​of factory object images {grey1, grey2,…, grey n}, calculate the similarity between image A (i.e., regional item image data) and the i-th in-plant item image:

[0186]

[0187] Where i = 1, 2, ..., n, W Ai is the similarity between image A and the i-th in-plant item image, grey i is the gray value of the i-th factory item image, grey A is the gray value of image A.

[0188] Furthermore, a one-to-one comparison {W A1 ,WA2 ,…,W An}, and get the maximum value W AN .

[0189] It is understandable that when W AN When <W, the system records the current time T A1 And keep the location of the detected image A, when T is satisfied A2 -T A1 ≥T, the system determines that there are items in image A that are randomly placed (i.e., there is a placement problem) and the items are from outside the factory (i.e., there is a timeout problem for foreign items). Among them, W is the similarity threshold set by the system to determine whether the item is an item in the factory (i.e., the preset minimum similarity), T A2 To maintain another time point during the detection of image A, T is a set threshold of the time that the out-of-factory object stays (ie, a preset maximum time that the object stays).

[0190] When W AN ≥W, the system determines that the item in image A is item N in the factory, and then calls the a position coordinates of the established item N {(X N_1 ,Y N_1 ),(X N_2 ,Y N_2 ),…,(X N_a ,Y N_a )}, calculate the distances between the stacking point of the items in image A and the a positions of the predetermined items N respectively as {L N_1 ,L N_2 ,…,L N_a}, where the minimum distance between the stacking point of the item in image A and the a positions of the predetermined placed items N is L N_p , when L N_p >L, the system determines that the object N in the image A is placed randomly (i.e., there is a placement problem), and L is the placement position difference tolerance threshold set by the system (i.e., the preset position difference threshold);

[0191] When L N_p ≤L, the system calls the height H of an item N in the database N The system compares the height H of the items in image A with the maximum stacking height H of the items in the factory (i.e. the preset maximum stacking height). A and the maximum stacking height H of the items in the factory, when H A > H, the system determines that there are items in image A that are placed randomly (i.e., there is a placement problem); when H A When ≤H, the system determines that there are no objects placed randomly in image A (that is, there is no placement problem).

[0192] Step S4, when the system determines that there are foreign objects, the system alarms to prompt the user to clean up or register the foreign objects; when the system determines that there are objects placed in a disorderly manner, the system alarms to prompt the user to organize the objects and recommends a placement location for the user.

[0193] In some embodiments, when the system determines that the item is an item outside the factory, the system issues an alarm for the occupation of items outside the factory, and recommends that the user clear the item out of the factory or register it in the database as soon as possible; when the system determines that the item N in the image A is placed randomly, the system obtains the area of ​​the available placement area near the qth position of the item N as S by edge feature extraction N_q , analyze the appropriate value M of the a position where the given object N is placed to accommodate the object N in the image A A_q :

[0194]

[0195] Among them, m is an arbitrary number set by the system, and K1 is the appropriate value M A_q The influence coefficient of the distance between the qth position and the position of object N in image A, X N_P is the horizontal coordinate of the pth position of the internal item N, Y N_P is the ordinate of the pth position of the internal object N, X is the abscissa of the position of the foreign object, Y is the ordinate of the position of the foreign object, K2 is the appropriate value M A_q The coefficient affected by the height at which item N needs to be placed, S A is the stacking area of ​​items, H A h is the height of the items placed. m is the optimal placement height of item M.

[0196] Furthermore, the system calculates {M A_1 ,M A_2 ,…,M A_a}, where M A_g The value is the largest, and the system recommends the gth placement among a placement positions to the user.

[0197] For example, the coefficient of influence of the distance between the qth position and the position of the object N in the image A is K1=2, the coefficient of influence of the height at which the object N needs to be placed is K2=1, and the optimal placement height of the object N is h m =4m, the coordinates of the position of item N in image A are (4, 7). When the system determines that item N in image A is randomly placed, it calls the image information of 10 predetermined locations where item N is stored. The system obtains the area of ​​the available placement area near the first location of item N through edge feature extraction, which is 20m 2 , the coordinates are (6, 8), and the appropriate value M of the object N in the image A is analyzed to accommodate a given position of the object N. A_1 :

[0198]

[0199] The system calculates {M A_1 ,M A_2 ,…,M A_10}, where M A_5 The value is the largest, and the system recommends the fifth of the 10 placement positions to the user.

[0200] The embodiment of the present application uses machine vision technology to detect multiple item placement areas in the factory at high speed and accuracy, providing a high-accuracy database for the system. The present invention takes into account the situation where foreign items enter the factory or the items in the factory are placed in the wrong position and height, and alarms the user according to the situation, so that the user can handle the problem more accurately; on the one hand, the present invention detects the height of the items placed to prevent factory losses and safety problems caused by the items being too high; on the other hand, the system comprehensively considers the distance from the detection location to the multiple established placement locations and the area of ​​the placement locations, and recommends the best placement location for the user when a disorderly placement is detected.

[0201] See also Fig. 9 The present application also provides an image detection device that can implement the above-mentioned image detection method. The device includes:

[0202] The acquisition module 901 is used to acquire image data;

[0203] An analysis module 902 is used to analyze image data to obtain item information;

[0204] A judgment module 903 is used to judge that the item has at least one of a placement problem and a foreign item timeout problem if the item information does not belong to a preset range;

[0205] The prompt module 904 is used to prompt the user to solve the placement problem in response to the first instruction if there is a placement problem of the item.

[0206] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0207] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned image detection method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0208] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0209] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0210] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0211] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the image detection method of the embodiment of this application;

[0212] Input / output interface 1003, used to implement information input and output;

[0213] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0214] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0215] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0216] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program implements the above-mentioned image detection method when executed by a processor.

[0217] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0218] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0219] The image detection method, image detection device, electronic device and storage medium provided in the embodiments of the present application collect image data; analyze the image data to obtain item information; if the item information does not fall within a preset range, determine that the item has at least one of a placement problem and a foreign item timeout problem, thereby reducing the frequency of manual inspections, improving work efficiency, and increasing the degree of automation in item management; if the item has a placement problem, respond to a first instruction to prompt the user to deal with the placement problem, thereby improving the work efficiency and accuracy of item management, improving the level of civilization in the item placement area, and improving the user experience, which is conducive to users promptly resolving item placement problems and reducing the risk of loss and safety risks caused by item placement problems.

[0220] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0221] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0222] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0223] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0224] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0225] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0227] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0230] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An image detection method, characterized in that: The method comprises the following steps: Collecting image data; Analyzing the image data to obtain item information; If the item information does not fall within the preset range, it is determined that the item has at least one of a placement problem and a foreign item timeout problem; If there is a placement problem with the item, in response to the first instruction, the user is prompted to solve the placement problem.

2. The method according to claim 1, characterized in that The image data includes regional item image data, item stacking scale image data and item stacking point image data, and the collected image data includes: The image data of the items in the area, the image data of the object stacking scale and the image data of the object stacking points are collected and detected by a machine vision product.

3. The method according to claim 2, characterized in that The item information includes item information in the detection area, item stacking scale information, and item stacking point information in the detection area. The analyzing the image data to obtain the item information includes: Analyzing the image data of the object in the detection area to obtain the object information in the detection area; Analyzing the object stacking scale image data to obtain the object stacking scale information; The object stacking point image data is analyzed to obtain the object stacking point information in the detection area.

4. The method according to claim 1, characterized in that: The preset range includes a preset minimum similarity, the item information includes item information in the detection area, and if the item information does not belong to the preset range, determining that the item has at least one of a placement problem and a foreign item timeout problem, includes: Get internal item information; Calculating a first similarity between the object in the detection area and the object inside the detection area according to the object information in the detection area and the object information inside the detection area; If the first similarity is less than the preset minimum similarity, it is determined that there is a placement problem of the objects in the detection area and a foreign object timeout problem.

5. The method according to claim 1, characterized in that The preset range includes a preset stacking point location, the item information includes the stacking point information of the items in the detection area, and if the item information does not belong to the preset range, determining that the item has at least one of a placement problem and a foreign item timeout problem, includes: If the object stacking point information in the detection area does not belong to the preset stacking point position, it is determined that there is a problem with the placement of the object.

6. The method according to claim 1, characterized in that The preset range includes a preset maximum stacking height, the item information includes item stacking scale information, and if the item information does not belong to the preset range, determining that the item has at least one of a placement problem and a foreign item timeout problem, includes: Determine the height at which the items are placed according to the item stacking scale information; If the placement height of the items is greater than the preset maximum stacking height, it is determined that there is a problem with the placement of the items.

7. The method according to claim 1, characterized in that The method further comprises: If the item has a foreign item timeout problem, in response to the second instruction, the user is prompted to handle the foreign item timeout problem.

8. An image detection device, characterized in that: The device comprises: An acquisition module, used for acquiring image data; An analysis module, used for analyzing the image data to obtain item information; A judgment module, configured to judge that the item has at least one of a placement problem and a foreign item timeout problem if the item information does not fall within a preset range; The prompt module is used to prompt the user to solve the placement problem in response to the first instruction if there is a placement problem of the item.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images