Image recognition method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to a method and device for image recognition, equipment and a storage medium. The method comprises the steps that a first machine learning model is used for determining a recognition area related to a recognition object to be recognized in an input image, and the area defined by the recognition area in the input image does not exceed the area defined by the edge of the recognition object in the input image; and identifying whether the identification object in the input image is a target object at least based on the content in the identification area. In this way, the problems of misrecognition and congestion missing detection caused by shielding in a package dense scene can be avoided.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to a method, apparatus, device, and storage medium for image recognition. Background Art

[0002] Conveyor belts are crucial in enterprise logistics, improving transportation automation and management efficiency. However, package congestion has become a common problem that affects the effective operation of conveyor belts, which may cause package damage and reduced productivity. Traditional congestion detection methods have low accuracy, especially in scenarios with dense packages. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for image recognition is provided. The method comprises: using a first machine learning model, determining a recognition region related to an object to be recognized in an input image, wherein an area enclosed by the recognition region in the input image does not exceed an area enclosed by edges of the object to be recognized in the input image; and identifying whether the object to be recognized in the input image is a target object based at least on the content in the recognition region.

[0004] In a second aspect of the present disclosure, a device for image recognition is provided. The device includes: a recognition region determination module configured to determine a recognition region related to an object to be recognized in an input image using a first machine learning model, wherein an area enclosed by the recognition region in the input image does not exceed an area enclosed by the edge of the object to be recognized in the input image; and a target object recognition module configured to identify whether the object to be recognized in the input image is a target object based at least on the content in the recognition region.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes computer executable instructions, and when the instructions are executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A flowchart showing an example process of a method for image recognition according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing an example of an identification area according to some embodiments of the present disclosure;

[0013] Figure 4A A schematic diagram showing an example of a training image sample according to some embodiments of the present disclosure;

[0014] Figure 4B A schematic diagram showing an example of an original image sample according to some embodiments of the present disclosure;

[0015] Figure 5 A schematic diagram showing an example process of congestion detection according to an embodiment of the present disclosure;

[0016] Figure 6 A schematic structural block diagram of an apparatus for image recognition according to some embodiments of the present disclosure is shown; and

[0017] Figure 7 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0021] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0022] In this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or it is necessary to perform a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0023] As briefly described above, in a scenario with dense packages, the accuracy of traditional congestion detection methods is low. Specifically, in one solution, a machine learning model can be used to detect whether the identified object is a target object (such as a stationary package) by detecting the circumscribed rectangle of the identified object to be identified, and then determine whether congestion occurs.

[0024] In this detection method, the circumscribed rectangle takes the package and the background around the package (such as the conveyor belt) as the detection basis. When the packages are sparse, this method can still achieve good recognition results, but when the packages are dense, the dense packages block the conveyor belt, which will cause the machine learning model to identify the packages with blocked background as non-packages, resulting in missed detection of congestion.

[0025] In view of this, an embodiment of the present disclosure provides a solution for image recognition. According to the solution, firstly, a recognition area related to an object to be recognized in an input image is determined using a first machine learning model, wherein an area enclosed by the recognition area in the input image does not exceed an area enclosed by the edge of the object to be recognized in the input image. Then, based at least on the content in the recognition area, it is identified whether the object to be recognized in the input image is a target object.

[0026] It will be more clearly understood through the description below that the image recognition scheme of the present disclosure uses a first machine learning model to determine a recognition area related to the object to be recognized, and the recognition area closely surrounds the recognized object and does not exceed its edge. Subsequently, the target object is identified based on this precisely defined recognition area. In this way, the image recognition scheme of the present disclosure effectively avoids the problems of misidentification and missed detection of congestion due to occlusion in scenes with dense packages. The image recognition scheme of the present disclosure not only significantly improves the accuracy of target recognition in complex scenes, but is also particularly suitable for conveyor belt environments that are prone to occlusion. Through precise identification, the image recognition scheme of the present disclosure can avoid the error of premature cancellation of congestion alarms due to missed detection when packages are dense, thereby ensuring more timely and accurate detection and processing of package congestion, and improving the efficiency and reliability of logistics transportation.

[0027] Various example implementations of the solution will be described in detail below in conjunction with the accompanying drawings.

[0028] Figure 1 A schematic diagram of an example environment 100 is shown in which embodiments of the present disclosure can be implemented. Figure 1 As shown, the example environment 100 may include N image acquisition devices 110 - 1 , 110 - 2 , . . . , 110 -N, where N is a positive integer. For clarity, the N image acquisition devices 110 - 1 , 110 - 2 , . . . , 110 - 3 are collectively referred to as image acquisition devices 110 hereinafter.

[0029] In the example environment 100 , the image acquisition device 110 may acquire images in a corresponding target scene, and send the acquired images as input images to the electronic device 120 .

[0030] After receiving the input image, the electronic device 120 can identify the object to be identified therein to determine whether the object is a target object. As an example, the electronic device 120 can be used in the scenario of package transportation, and the target object can be, for example, a package. In the process of identifying the identified object, the electronic device 120 can determine whether the identified object is a target object based on the first machine learning model by identifying the content in the object identification area. As an example, the first machine learning model can use the inscribed graph of the identified image as the identification area, so that the identification area can remove the background image near the identified image to reduce the missed detection rate in the package dense scene. After the electronic device 120 determines the package from the input image, it can further track the determined package, etc., so as to determine whether a situation such as package congestion occurs in the target scene. Next, when it is determined that package congestion occurs in the target scene, the electronic device 120 can alert the user 140 with the help of the terminal device 130, thereby prompting the user 140 to handle it.

[0031] exist Figure 1 In the environment 100 , the terminal device 130 may present an interface 150 , and the warning information may be displayed to the user 140 through the interface 150 .

[0032] In some embodiments, the terminal device 130 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 130 can also support any type of interface for the user (such as "wearable" circuits, etc.).

[0033] The electronic device 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The electronic device 120 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. The electronic device 120 may provide background services for applications that support content presentation in the terminal device 130.

[0034] A communication connection may be established between the electronic device 120 and the terminal device 130. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a universal serial bus connection, a wireless fidelity connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the electronic device 120 and the terminal device 130 may implement signaling interaction through the communication connection between the two.

[0035] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0036] Figure 2 A flowchart is shown of an example process 200 of a method for image recognition according to some embodiments of the present disclosure, which may be implemented at the electronic device 120 . Figure 3 A schematic diagram of an example 300 of identifying regions according to some embodiments of the present disclosure is shown. Figure 3 The process 200 is described.

[0037] Figure 3 Schematically shows three recognition objects 301-1, 301-2 and 301-3 and recognition areas 302-1, 302-2 and 302-3 of the three recognition objects 301-1, 301-2 and 301-3 in the input image 310. For the sake of clarity, unless otherwise specified, the recognition objects 301-1, 301-2 and 301-3 are collectively referred to as recognition objects 301, and the recognition areas 302-1, 302-2 and 302-3 are collectively referred to as recognition areas 302.

[0038] It should be noted that, according to actual conditions, the input image 310 may include more identification objects 301, and the positions of these identification objects 301 in the input image 310 are not limited to Figure 3 The form shown.

[0039] In box 210, the electronic device 120 uses a first machine learning model to determine a recognition area 302 related to the object 301 to be recognized in the input image 310.

[0040] In some embodiments, the electronic device 120 may be used in a scenario related to package transportation. As an example, the electronic device 120 may be used to monitor a conveyor belt 320 when a package is being transported. In this scenario, the object 301 to be identified may refer to a portion of the input image 310 that may contain a package.

[0041] In some embodiments, the input image 310 may be a real-time image captured by the image acquisition device 110. As an example, the image acquisition device 110 may be a surveillance camera used to monitor the state of the conveyor belt 320. In this way, the embodiments of the present disclosure do not need to introduce an additional image acquisition device 110, thereby reducing the modification cost.

[0042] As an example, the electronic device 120 can further determine whether the identified object 301 is a target object (such as the "package" described above) by using the first machine learning model to identify the content in the area 302. In addition, when one or more packages are found to be stationary, the electronic device 120 can determine that a congestion event occurs on the conveyor belt 320. Then, the electronic device 120 can issue a corresponding congestion warning message to prompt relevant personnel to intervene and handle it.

[0043] In the embodiment of the present disclosure, the first machine learning model determines whether the identified object 301 is a target object by identifying the content in the area 302. Figure 4B The second indication box 404 shown in the figure shows that the area enclosed by the recognition area 302 in the input image 310 does not exceed the area enclosed by the edge of the recognition object 301 in the input image 310. In this way, the recognition area 302 removes the background image (such as the conveyor belt 320) near the recognition object 301. Based on such a recognition area 302, when the recognition object 301 is recognized, the background image such as the conveyor belt 320 can be prevented from interfering with the recognition process, which is conducive to improving the problem of missed detection.

[0044] In some embodiments, the electronic device 120 uses the first machine learning model to determine the inscribed graph of the recognition object 301 in the input image 310. Then, the first machine learning model determines the recognition area 302 related to the recognition object 301 based on the inscribed graph of the recognition object 301.

[0045] In some embodiments, an inscribed figure may refer to any appropriate geometric figure that is completely located inside another geometric figure (eg, the area enclosed by the edges of the identification object 301), and the boundary of the inscribed figure touches but does not exceed the boundary of the external figure.

[0046] In some embodiments, the electronic device 120 may use the edge of the inscribed figure as the edge of the recognition area 302. Since the inscribed figure can fit the outline of the recognition object 301 well, using its edge as the boundary of the recognition area 302 can make the shape of the recognition area 302 close to the actual shape of the recognition object 301. In this way, the recognition area 302 can more effectively reflect the image features of the recognition object 301.

[0047] In some embodiments, the inscribed figure includes the largest inscribed figure of the identified object 301, and the identified area 302 related to the identified object 301 coincides with the largest inscribed figure. As an example, the largest inscribed figure refers to the largest figure inside a shape (e.g., the area enclosed by the edges of the identified object 301), and each of its edges is in contact with the boundary of the shape. In an embodiment of the present disclosure, the largest inscribed figure coincides with the edge of the identified object 301.

[0048] As an example, the specific shape of the maximum inscribed figure can be determined according to actual needs. For example, the maximum inscribed figure can be any suitable figure such as a maximum inscribed triangle or rectangle.

[0049] In some embodiments, the maximum inscribed figure can cover the internal area of ​​the identified object 301 to the maximum extent, ensuring that the identified area 302 does not miss any important part of the identified object 301. This improves the integrity of the identification, allowing subsequent processing to be based on more comprehensive data. In addition, using the maximum inscribed figure as the basis for the identification area 302 also enhances the robustness of the first machine learning model. In practical applications, the shape and size of the identified object 301 may change due to various factors, such as shooting angle, distance, and lighting conditions. The maximum inscribed figure can adapt to these changes because it always fits closely to the actual outline of the identified object 301. This means that no matter how the identified object 301 changes, the identification area 302 can keep matching it, thereby ensuring the stability and reliability of the identification.

[0050] In some embodiments, the first machine learning model includes multiple layers of neural networks connected in sequence, at least one layer of the neural network includes at least one convolutional layer, at least one pooling layer, and at least one normalization layer.

[0051] In some embodiments, the first machine learning model can be any appropriate model capable of performing target detection. The embodiments of the present disclosure can construct a deeper network structure by controlling the number of layers of the neural network, thereby enhancing the feature extraction capability and complexity of the first machine learning model, so that it can capture and identify targets more accurately.

[0052] In some embodiments, an input image 310 with a higher resolution (e.g., 720P, 1080P or higher) may be used. In this way, the recall rate of the first machine learning model may be improved, thereby enhancing the performance of the first machine learning model.

[0053] Figure 4A A schematic diagram of an example 400A of a training image sample 410 is shown, in accordance with some embodiments of the present disclosure.

[0054] Figure 4AThe diagram schematically shows two training recognition objects 401-1 and 401-2 and first indicator frames 402-1 and 402-2 of the two training recognition objects 401-1 and 401-2 in the training image sample 410. For clarity, the training recognition objects 401-1 and 401-2 are collectively referred to as training recognition objects 401, and the first indicator frames 402-1 and 402-2 are collectively referred to as first indicator frames 402 unless otherwise specified.

[0055] It should be noted that, according to actual conditions, the training image sample 410 may include more training recognition objects 401, and the positions of these training recognition objects 401 in the training image sample 410 are not limited to Figure 4A The form shown.

[0056] In some embodiments, the model parameters related to the recognition of the recognition area 302 by the first machine learning model are trained based on at least a set of training image samples annotated with the first indicator frame 402. Furthermore, in at least one training image sample 410 of the set of training image samples, each training recognition object 401 is annotated with a first indicator frame 402 associated with each training recognition object 401, and the area enclosed by the first indicator frame 402 in the training image sample 410 does not exceed the area enclosed by the edge of the training recognition object 401 associated with it in the training image sample 410.

[0057] As described above, the electronic device 120 can be applied to monitor the conveyor belt 320. In this scenario, the training image sample 410 can be a real image of the conveyor belt 320 and the packages on the conveyor belt 320 collected in history, and the training recognition object 401 can refer to the package on the conveyor belt 320.

[0058] As an example, the first indication frame 402 may overlap with the edge of the inscribed figure of the training recognition object 401 associated therewith. For example, the first indication frame 402 overlaps with the edge of the maximum inscribed figure of the training recognition object 401 associated therewith.

[0059] It should be noted that the maximum inscribed figure of the training recognition object 401 can be determined by referring to the above description of the maximum inscribed figure of the recognition object 301, so the embodiments of the present disclosure will not be described in detail here.

[0060] Through the first indication box 402 , the first machine learning model can fully learn how to determine the maximum inscribed figure of the identified object 301 , and then obtain the identification area 302 of the identified object 301 .

[0061] Figure 4B A schematic diagram showing an example 400B of an original image sample 420 according to some embodiments of the present disclosure is shown.

[0062] Figure 4B The schematic diagram shows two original recognition objects 403-1 and 403-2 and second indicator frames 404-1 and 404-2 of the two original recognition objects 403-1 and 403-2 in the original image sample 420. For the sake of clarity, the original recognition objects 403-1 and 403-2 are collectively referred to as original recognition objects 403, and the second indicator frames 404-1 and 404-2 are collectively referred to as second indicator frames 404 unless otherwise specified.

[0063] It should be noted that, according to actual conditions, the original image sample 420 may include more original recognition objects 403, and the positions of these original recognition objects 403 in the original image sample 420 are not limited to Figure 4B The form shown.

[0064] In some embodiments, the first indicator box 402 in a set of training image samples is annotated using a second machine learning model based on a set of original image samples annotated with a second indicator box 404. In at least one original image sample 420 in a set of original image samples, each original recognition object 403 is annotated with a second indicator box 404 associated with each original recognition object 403, and the area enclosed by the second indicator box 404 in the original image sample 420 exceeds the area enclosed by the edge of the original recognition object 403 associated with it in the original image sample 420.

[0065] As described above, the electronic device 120 can be applied to monitoring the conveyor belt 320. In this scenario, the original image sample 420 can be a real image containing the conveyor belt 320 and the package on the conveyor belt 320 that was collected historically, and the original identification object 403 can refer to the package on the conveyor belt 320. Different from the training image sample 410, the second indicator frame 404 on the original image sample 420 is determined based on the external graphic of the original identification object 403 associated therewith.

[0066] Reference Figure 4B As an example, the second indication frame 404 may overlap with the edge of the circumscribed graph of the original recognized object 403 associated therewith. For example, the second indication frame 404 overlaps with the edge of the minimum circumscribed graph of the original recognized object 403 associated therewith.

[0067] based on Figure 4B As described above, the minimum circumscribed figure may include background images such as the conveyor belt 320 near the original recognition object 403. Therefore, the embodiment of the present disclosure processes the original image sample 420 through the second machine learning model to obtain an indication frame for removing the background image.

[0068] In some embodiments, the second machine learning model can be any appropriate model that can perform image segmentation tasks (e.g., separate a specific area or object in an image from a background or other objects) based on a prompt word. As an example, the prompt word can be, for example, "inscribed figure" or "inscribed rectangle", etc., which can be determined according to actual needs.

[0069] Combined with reference Figure 4A and Figure 4B It can be seen that the second machine learning model separates the original recognition object 403 from the second indicator frame 404, thereby obtaining the training recognition object 401. In addition, by finding the largest inscribed figure, the second machine learning model marks the first indicator frame 402 on each training recognition object 401.

[0070] In this way, the embodiment of the present disclosure omits the manual labeling process for the first indication box 402, thereby improving the efficiency of model training.

[0071] In some embodiments, during the model training process, the annotation of the second indicator box 404 can be retained on the training image sample 410; or a group of training image samples can be divided into two, and the first indicator box 402 is annotated on one part of the training image samples 410, and the second indicator box 404 is annotated on the other part of the training image samples 410. In this case, a first label can be set for the first indicator box 402, and a second label can be set for the second indicator box 404. By inputting such training image samples 410 and labels into the first machine learning model, the first machine learning model can learn the knowledge of detecting target objects through two indicator boxes. This helps to improve the recall rate of the first machine learning model, that is, to ensure that as many target objects as possible are detected.

[0072] In some embodiments, a first training image sample 410 in a set of training image samples is obtained based on performing a sample expansion operation on a second training image sample 410 in a set of training image samples, and the density of the training recognition objects 401 in the first training image sample 410 is greater than the density of the training recognition objects 401 in the second training image sample 410.

[0073] In some embodiments, the electronic device 120 may generate dense scene data (e.g., first training image samples) based on sparse scene data (e.g., second training image samples) to enhance the generalization ability of the first machine learning model and its ability to cope with complex scenes.

[0074] As an example, the second training image sample may be one or more candidate training image samples randomly selected from a group of candidate training image samples. For the second training image sample, the electronic device 120 may quickly obtain the first training image sample by copying the training recognition object 401 therein to the area on the second training image sample where the density is expected to be increased.

[0075] In some embodiments, in both the first training image sample and the second training image sample, a conveying mechanism (e.g., conveyor belt 320) for conveying the training recognition object 401 is presented. And the carrying surface (e.g., the upper surface of the conveyor belt 320) of the conveying mechanism in the first training image sample for carrying the training recognition object 401 is covered by the training recognition object 401 in the first training image sample.

[0076] As an example, in the replication process described above, the electronic device 120 can continuously repeat this operation until the entire upper surface of the conveyor belt 320 is covered by the training recognition object 401. In this way, a scene with densely arranged packages can be generated, thereby providing richer and more challenging training data for the training of the first machine learning model.

[0077] In order to ensure that the training recognition objects 401 in the first training image sample can be closely arranged without leaving gaps, the electronic device 120 can arrange them according to the largest inscribed figure of each training recognition object 401. In this way, other elements such as the conveyor belt 320 can be reduced in the generated dense scene picture, so that the first machine learning model can focus more on the detection of the package.

[0078] In this way, the embodiments of the present disclosure use sparsely packaged images as input and automatically generate densely packaged images, which not only improves detection accuracy but also eliminates the process of manually constructing training data, further improving training efficiency.

[0079] Return to reference Figure 2 In box 220, the electronic device 120 identifies whether the identification object 301 in the input image 310 is a target object based on at least the content in the identification area 302.

[0080] In some embodiments, if the recognition result related to the recognition area 302 indicates that the recognition object 301 is a target object and the confidence is higher than the recognition confidence related to the target scene, the recognition object 301 is determined to be a target object. If the recognition result related to the recognition area 302 indicates that the recognition object 301 is a target object and the confidence is lower than the recognition confidence related to the target scene (or the recognition result indicates that the recognition object 301 is a non-target object), it is determined that the recognition object 301 is not a target object.

[0081] In some embodiments, according to actual needs, the electronic device 120 can also comprehensively determine whether the identified object 301 is a target object in combination with the content outside the identification area 302. As an example, the electronic device 120 can also comprehensively determine whether the identified object 301 is a target object in combination with the external graphics of the identified object 301.

[0082] As described above, in the embodiment of the present disclosure, after the electronic device 120 identifies the identification object 301, it can further determine whether a congestion event occurs in the target scene.

[0083] Figure 5 A schematic diagram of an example process 500 of congestion detection according to an embodiment of the present disclosure is shown. Figure 5 The process of determining a congestion event in an embodiment of the present disclosure is described. It should be noted that, for the sake of clarity, the following description is given by taking the application of the electronic device 120 to the monitoring scene of the conveyor belt as an example, but this does not constitute a limitation on the application scenario of the embodiment of the present disclosure. In this scenario, the target object can be a package.

[0084] Reference Figure 5 In box 501, the electronic device 120 can identify one or more identification objects 301 in the input image 310 in the manner described above to determine a package from the input image 310.

[0085] In some embodiments, the input image 310 may include a "hot zone" in the target scene. As an example, a "hot zone" may refer to an area in the target scene that is prone to jamming or congestion, such as a sorting area or a merging area. The identification of the "hot zone" can be learned by the first machine learning model through appropriate training methods. As an example, the electronic device 120 focuses on the delivery of packages in the "hot zone" in the input image 310. In other words, the electronic device 120 uses the first machine learning model to detect and identify the part in the "hot zone" of the input image 310, thereby determining the package in the "hot zone" and determining whether there is a congestion event in the "hot zone".

[0086] In block 502, the electronic device 120 can track the determined package. In this way, the electronic device 120 can reflect the real-time location and status of the package to the user 140 so that the user 140 can react quickly when a problem occurs.

[0087] In block 503 , the electronic device 120 determines that a congestion event occurs in the target scene related to the input image 310 in response to the parcel in the input image 310 staying in the target scene related to the input image 310 for longer than a predetermined time.

[0088] In some embodiments, the electronic device 120 may time the residence time of each package. If the residence time of the package in the target scene exceeds a predetermined time, the electronic device 120 may determine that a congestion event has occurred in the target scene.

[0089] It should be noted that, according to actual needs, the electronic device 120 can combine more sensors to comprehensively determine whether a congestion event has occurred in the target scene. As an example, the electronic device 120 can further use a photoelectric sensor to detect the interval between two packages, and determine whether a congestion event has occurred based on the interval between the two packages. As another example, the electronic device 120 can further use a pressure sensor to obtain the weight of the package squeezed on the conveyor belt 320 by collecting the pressure on the conveyor belt 320, and then determine whether a congestion event has occurred.

[0090] In box 504, after determining that a congestion event occurs in the target scene, the electronic device 120 can analyze the stagnation cause of the package in the target scene through a predetermined strategy. As an example, the stagnation cause can indicate whether the package is stuck because of a jam (stuck part) or because of congestion (blocked part). As an example, the electronic device 120 can determine whether the package is stuck because of a jam or because of congestion based on information such as the dwell time and / or position change of the package. Then, the electronic device 120 can take corresponding measures according to the stagnation cause of the package, such as issuing an alarm or notifying the user 140 to intervene.

[0091] In block 505 , the electronic device 120 may also determine the area ratio occupied by the stagnant packages in the target scene based on the input image 310 , thereby determining the density of the stagnant packages in the target scene.

[0092] In block 506, the electronic device 120 may determine the congestion coefficient of the congestion event occurring in the target scene based on the stagnation reason of the package and the area ratio occupied by the stagnant package in the target scene. In this way, the congestion degree of the target scene may be quantified, thereby helping the user 140 to more intuitively understand the operating status of the conveyor belt 320, so as to timely adjust and optimize the logistics process.

[0093] In box 507, the electronic device 120 can also determine whether the package has fallen from the conveyor belt 320 based on the movement trajectory of the package (for example, including but not limited to the movement trajectory in the target scene), thereby determining whether a package drop event has occurred.

[0094] In some embodiments, after determining that a congestion event or a dropped item event has occurred, the electronic device 120 may present relevant video images to the user 140, informing the user 140 of the content of the event and presenting information such as the time (and / or cause) of the event, so as to prompt the user 140 to handle it.

[0095] In some embodiments, as described above, if the confidence of the recognition result related to the recognition area 302 is higher than the recognition confidence related to the target scene, the electronic device 120 determines that the recognition object 301 is a target object (e.g., a package). On this basis, in response to determining that a congestion event occurs in the target scene, the electronic device 120 can reduce the recognition confidence related to the target scene, thereby detecting more packages.

[0096] In some embodiments, in response to determining that a congestion event occurs in the target scene, the electronic device 120 expands the detection area for the identified object 301 .

[0097] In the above manner, when a congestion event is detected in the target scene, the electronic device 120 will intelligently expand the detection area of ​​the identification object 301 (or expand the area of ​​the "hot zone" in the input image 310), so that more areas in the input image 310 are included in the detection range. At the same time, by appropriately reducing the recognition confidence, the electronic device 120 can identify more packages to avoid missed detection. When the congestion situation is alleviated, the electronic device 120 can restore the original recognition confidence and reduce the detection area to reduce the occurrence of false alarms.

[0098] In this way, the electronic device 120 can respond more flexibly and efficiently when faced with complex and changing logistics scenarios, ensuring the accuracy and efficiency of congestion detection.

[0099] According to the various embodiments described above, it can be clearly understood that in the embodiment of the present disclosure, the electronic device 120 uses the first machine learning model to determine the identification area 302 related to the object to be identified 301, and the identification area 302 closely surrounds the identification object 301 and does not exceed its edge. Subsequently, the electronic device 120 performs target object recognition based on this precisely defined identification area 302. In this way, the embodiment of the present disclosure effectively avoids the problems of misidentification and missed detection of congestion due to occlusion in a dense parcel scene. The embodiment of the present disclosure not only significantly improves the accuracy of target recognition in complex scenes, but is also particularly suitable for the conveyor belt 320 environment prone to occlusion. Through accurate recognition, the embodiment of the present disclosure can avoid the error of the congestion alarm being released in advance due to missed detection when the parcels are dense, thereby ensuring more timely and accurate detection and processing of parcel congestion, and improving the efficiency and reliability of logistics transportation.

[0100] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 6A schematic structural block diagram of an apparatus 600 for image recognition according to some embodiments of the present disclosure is shown. The apparatus 600 may be implemented as or included in the electronic device 120. Each module / component in the apparatus 600 may be implemented by hardware, software, firmware or any combination thereof.

[0101] Reference Figure 6 , the device 600 includes a recognition area determination module 610 and a target object recognition module 620. In some embodiments, the recognition area determination module 610 is configured to use a first machine learning model to determine a recognition area related to the recognition object to be recognized in the input image, wherein the area enclosed by the recognition area in the input image does not exceed the area enclosed by the edge of the recognition object in the input image. The target object recognition module 620 is configured to identify whether the recognition object in the input image is a target object based at least on the content in the recognition area.

[0102] In some embodiments, the recognition area determination module 610 is further configured to: determine the inscribed graph of the recognition object in the input image using the first machine learning model; and determine the recognition area related to the recognition object based on the inscribed graph of the recognition object.

[0103] In some embodiments, the inscribed figure includes a maximum inscribed figure of the recognition object, and the recognition area associated with the recognition object coincides with the maximum inscribed figure.

[0104] In some embodiments, the apparatus 600 further includes a congestion event determination module configured to determine that a congestion event occurs in the target scene related to the input image in response to the target object in the input image staying in the target scene related to the input image for a duration greater than a predetermined duration.

[0105] In some embodiments, the device 600 further includes a confidence adjustment module. The target object recognition module 620 is configured to: if the recognition result related to the recognition area indicates that the recognized object is a target object and the confidence is higher than the recognition confidence related to the target scene, determine that the recognized object is a target object. The confidence adjustment module is configured to: in response to determining that a congestion event occurs in the target scene, reduce the recognition confidence related to the target scene.

[0106] In some embodiments, the first machine learning model includes multiple layers of neural networks connected in sequence, at least one layer of the neural network includes at least one convolutional layer, at least one pooling layer, and at least one normalization layer.

[0107] In some embodiments, model parameters related to the recognition of the recognition area by the first machine learning model are trained based on at least a first indicator box annotated on a set of training image samples; and in at least one training image sample in a set of training image samples, each training recognition object is annotated with a first indicator box associated with each training recognition object, and the area enclosed by the first indicator box in the training image sample does not exceed the area enclosed by the edge of the training recognition object associated with it in the training image sample.

[0108] In some embodiments, a first indicator box in a set of training image samples is annotated using a second machine learning model based on a second indicator box annotated with a set of original image samples; and in at least one original image sample in a set of original image samples, each original recognition object is annotated with a second indicator box associated with each original recognition object, and the area enclosed by the second indicator box in the original image sample exceeds the area enclosed by the edge of the original recognition object associated with it in the original image sample.

[0109] In some embodiments, a first training image sample in a set of training image samples is obtained based on performing a sample expansion operation on a second training image sample in a set of training image samples; and a density of training recognition objects in the first training image sample is greater than a density of training recognition objects in the second training image sample.

[0110] In some embodiments, a conveying mechanism for conveying a training recognition object is presented in both the first training image sample and the second training image sample; and a bearing surface of the conveying mechanism for carrying the training recognition object in the first training image sample is covered by the training recognition object in the first training image sample.

[0111] Figure 7 1 is a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented. The electronic device 700 may be used to implement, for example, Figure 1 The electronic device 120 shown. It should be understood that Figure 7 The electronic device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

[0112] Reference Figure 7, the electronic device 700 is in the form of a general electronic device. The components of the electronic device 700 may include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 700.

[0113] The electronic device 700 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 can be a volatile memory (e.g., register, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 730 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 700.

[0114] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0115] The communication unit 740 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0116] The input device 750 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 may also communicate with one or more external devices (not shown) through the communication unit 740 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 700, or communicate with any device that allows the electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0117] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0118] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0120] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0121] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0122] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The determination of the terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for image recognition, comprising: Determine, by using a first machine learning model, a recognition region related to an object to be recognized in an input image, wherein an area enclosed by the recognition region in the input image does not exceed an area enclosed by an edge of the object to be recognized in the input image; as well as Based at least on the content in the recognition area, it is identified whether the recognition object is a target object.

2. The method according to claim 1, wherein determining the identification area related to the object to be identified in the input image by using the first machine learning model comprises: Determine an inscribed figure of the identified object in the input image using the first machine learning model; as well as Based on the inscribed figure of the recognition object, the recognition area related to the recognition object is determined. 3 . The method according to claim 2 , wherein the inscribed figure comprises a maximum inscribed figure of the recognition object, and the recognition area related to the recognition object coincides with the maximum inscribed figure.

4. The method according to claim 1, further comprising: In response to the target object in the input image staying in a target scene related to the input image for a period longer than a predetermined period, it is determined that a congestion event occurs in the target scene related to the input image.

5. The method according to claim 4, wherein identifying whether the identified object is a target object based at least on the content in the identification area comprises: If the recognition result related to the recognition area indicates that the recognition object is the target object and the confidence level is higher than the recognition confidence level related to the target scene, determining that the recognition object is the target object; and The method further comprises: In response to determining that the congestion event occurs in the target scene, the recognition confidence associated with the target scene is reduced.

6. The method according to claim 1, wherein the first machine learning model comprises a multi-layer neural network connected in sequence, at least one layer of the neural network comprising at least one convolutional layer, at least one pooling layer and at least one normalization layer.

7. The method according to claim 1, wherein the model parameters related to the recognition of the recognition area by the first machine learning model are trained based on at least a first indicator box annotated by a set of training image samples; and In at least one training image sample of the group of training image samples, each training recognition object is marked with the first indicator box associated with each training recognition object, and the area enclosed by the first indicator box in the training image sample does not exceed the area enclosed by the edge of the training recognition object associated with it in the training image sample.

8. The method according to claim 7, wherein the first indicator box in the set of training image samples is annotated based on a second indicator box annotated in a set of original image samples using a second machine learning model; and In at least one original image sample of the group of original image samples, each original recognition object is marked with the second indicator box associated with each original recognition object, and the area enclosed by the second indicator box in the original image sample exceeds the area enclosed by the edge of the original recognition object associated with it in the original image sample.

9. The method according to claim 7, wherein a first training image sample in the set of training image samples is obtained based on performing a sample expansion operation on a second training image sample in the set of training image samples; and The density of the training recognition objects in the first training image sample is greater than the density of the training recognition objects in the second training image sample.

10. The method according to claim 9, wherein a transmission mechanism for transmitting the training recognition object is presented in both the first training image sample and the second training image sample; and The bearing surface of the conveying mechanism in the first training image sample for bearing the training recognition object is covered by the training recognition object in the first training image sample.

11. A device for image recognition, comprising: a recognition region determination module configured to determine, by using a first machine learning model, a recognition region related to an object to be recognized in an input image, wherein an area enclosed by the recognition region in the input image does not exceed an area enclosed by an edge of the object to be recognized in the input image; as well as The target object recognition module is configured to recognize whether the recognition object is a target object based at least on the content in the recognition area.

12. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 10.

14. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images