Article identification method, embedded terminal and storage medium

Abstract

The invention provides an article identification method, an embedded terminal and a storage medium, and the method comprises the steps: obtaining an article image in response to an article identification instruction, and displaying at least one to-be-identified article in the article image; and identifying the at least one to-be-identified article by using a preset lightweight article identification model, and determining the type to which the at least one to-be-identified article belongs. Wherein a feature map output module in the lightweight article recognition model performs output through an attention mechanism module; the attention mechanism module comprises a channel attention module and a space attention module which are parallel, and the channel attention module comprises a pooling layer, a one-dimensional convolutional network and a connection layer which are connected in sequence. Compared with the prior art in which an intelligent shelf realizes AI identification of article types through a server, the embedded terminal is endowed with an article identification capability, so that the intelligent shelf can still carry out article identification in an off-network state, and the reliability is improved.

Description

Technical Field

[0001] The present application relates to the field of smart shelves, and in particular to an item identification method, an embedded terminal and a storage medium. Background Art

[0002] Smart shelves, also known as smart cabinets, etc., combine the Internet of Things (IoT) technology and artificial intelligence (AI) technology, and can monitor and analyze product information on the shelves in real time, thereby realizing functions such as accurate goods management.

[0003] Smart shelves are essentially embedded terminals. Under normal circumstances, on the one hand, the computing resources and storage space of embedded terminals are very limited, and on the other hand, there is also a requirement for low power consumption of embedded terminals. Therefore, current smart shelves are usually connected to servers, and the servers replace the smart shelves to perform AI identification of item types, which consumes the most computing resources and storage space. In other words, current smart shelves need to be connected to servers, and the servers are equipped with AI models for item identification.

[0004] However, it can also be seen that the networking stability of smart shelves is very important. Once the smart shelves are off the network, they will lose functions such as cargo management. Therefore, the reliability of existing smart shelves is still insufficient. Summary of the invention

[0005] To this end, the present application provides an item identification method, an embedded terminal and a storage medium, so that the embedded terminal has the ability to identify items, so that the smart shelf can still identify items when it is off-grid, thereby improving reliability.

[0006] In a first aspect, the present application provides an object identification method, which is applied to an embedded terminal, and the method includes:

[0007] In response to the object recognition instruction, acquiring an object image; the object image shows at least one object to be recognized;

[0008] Using a preset lightweight object recognition model, the at least one object to be recognized is recognized, and the category of each of the at least one object to be recognized is determined;

[0009] Wherein, the feature map output module in the lightweight object recognition model is output through the attention mechanism module;

[0010] The attention mechanism module includes a parallel channel attention module and a spatial attention module, and the channel attention module includes a pooling layer, a one-dimensional convolutional network, and a connection layer connected in sequence.

[0011] In some embodiments, the lightweight object recognition model is a YOLOv5s model; the YOLOv5s model includes an intermediate layer network, and the CBS module in the intermediate layer network is the feature map output module.

[0012] In some embodiments, the feature map output module outputs through an attention mechanism module, including:

[0013] Inputting the feature map to be processed output by the feature map output module into the channel attention module and the spatial attention module respectively for processing;

[0014] Multiply the processing result of the channel attention module and the processing result of the spatial attention module to obtain an attention feature map.

[0015] In some embodiments, the channel attention module processes the feature map to be processed, including:

[0016] The feature map to be processed is processed by a first maximum pooling layer and a first average pooling layer respectively to obtain a corresponding first pooling feature map and a second pooling feature map; the pooling layer in the channel attention module includes the first maximum pooling layer and the first average pooling layer in parallel;

[0017] Inputting the first pooling feature map and the second pooling feature map into the one-dimensional convolutional network respectively to obtain a corresponding first convolutional feature map and a second convolutional feature map;

[0018] The first convolutional feature map and the second convolutional feature map are input into the connection layer to obtain the processing result of the channel attention module.

[0019] In some embodiments, the spatial attention module processes the feature map to be processed, including:

[0020] The feature map to be processed is processed by a second maximum pooling layer and a second average pooling layer respectively to obtain a corresponding third pooling feature map and a fourth pooling feature map; the pooling layer in the spatial attention module includes the second maximum pooling layer and the second average pooling layer in parallel;

[0021] Perform channel fusion on the third pooling feature map and the fourth pooling feature map to obtain a corresponding fifth pooling feature map;

[0022] The fifth pooled feature map is convolved through a convolution layer to obtain a processing result of the spatial attention module.

[0023] In some embodiments, in response to the object identification instruction, acquiring the object image includes:

[0024] In response to the object recognition instruction, an image of the object taken by an image acquisition device connected to the embedded terminal is acquired.

[0025] In some embodiments, the number of the image acquisition devices is multiple;

[0026] Before acquiring the image of the object taken by the image acquisition device connected to the embedded terminal in response to the object recognition instruction, the method further includes:

[0027] An image acquisition area is set for each image acquisition device, so that the object image acquired by the embedded terminal only displays the objects within the image acquisition area corresponding to the image acquisition device;

[0028] There is no overlap between the image acquisition areas and all the image acquisition areas can cover the smart shelf.

[0029] In some embodiments, after acquiring the object image in response to the object identification instruction, the method further includes:

[0030] Detecting the networking status of the embedded terminal;

[0031] If the networking status of the embedded terminal is online, the object image is sent to the server to obtain the server's recognition result of the at least one object to be identified; otherwise, the lightweight object recognition model is used to determine the category of each of the at least one object to be identified.

[0032] In a second aspect, the present application provides an embedded terminal, comprising:

[0033] at least one processor; and,

[0034] a memory communicatively coupled to the at least one processor;

[0035] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the object identification method as described in the first aspect.

[0036] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the object identification method as described in the first aspect.

[0037] Compared with the related art in which the smart shelf realizes AI identification of the types of items through the server, the present application gives the embedded terminal the ability to identify items. On the one hand, due to the limited computing resources and storage space of the embedded terminal, the present application sets a lightweight item recognition model in the embedded terminal and improves the model structure of the model. Specifically, the attention mechanism module is embedded in the output end of the feature map output module in the lightweight item recognition model, which reduces the calculation amount of the lightweight item recognition model. On the other hand, the present application also improves the aforementioned attention mechanism module, not only improving the channel attention module in the attention mechanism module, but also improving the data processing method of the attention mechanism module, and improving the calculation speed of the attention mechanism module. It can be seen from this that the lightweight item recognition model in the present application requires less computing resources and storage space, so the smart shelf equipped with the lightweight item recognition model can still perform item recognition when it is off-grid, which improves reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0039] Figure 1 A schematic diagram of an exemplary scenario for applying an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a flow chart of an object identification method provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the model structure of the YOLOv5s model in traditional technology;

[0042] Figure 4 To apply the embodiment of the present application Figure 3 A schematic diagram of improving the YOLOv5s model shown;

[0043] Figure 5 A schematic diagram of a model structure of an attention mechanism module in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of a flow chart of a channel attention module processing a feature map to be processed in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of a flow chart of processing a feature map to be processed by a spatial attention module in an embodiment of the present application;

[0046] Figure 8A schematic diagram of the structure of an embedded terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than the module division in the device schematic diagram or the order in the flow chart.

[0049] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0050] To facilitate understanding of this embodiment, a method for identifying an object disclosed in an embodiment of the present application is first introduced in detail. The executor of the method for identifying an object provided in the embodiment of the present application is generally an electronic device with certain computing capabilities, such as an embedded terminal. In some possible implementation methods, the method for identifying an object can be implemented by a processor calling computer-readable instructions stored in a memory.

[0051] The object identification method provided in the embodiment of the present application can be applied to an embedded terminal. As an exemplary application scenario, please refer to Figure 1 , Figure 1 The embedded terminal shown in can be a smart shelf 100. In this application scenario, the smart shelf 100 may include a processor 10, an image acquisition device 20, and an item to be identified 30. Specifically, the image acquisition device 20 is used to photograph the item to be identified 30 to obtain an image of the item; the processor 10 is respectively connected to each image acquisition device 20 to obtain the image of the item. Based on this, when the smart shelf 100 needs to perform an item identification task, the processor 10 can execute the item identification method provided in the embodiment of the present application to obtain the corresponding item identification result. It can be understood that the smart shelf 100 can still perform item identification when it is off-grid, with high reliability. It should be noted that Figure 1The implementation of the processor 10, the image acquisition device 20 and the object to be identified 30 is merely exemplary. For example, the number and setting positions of these three components are all exemplary. In actual implementation, they can be reasonably adjusted according to the actual scenario.

[0052] See also Figure 2 The object identification method in the embodiment of the present application may include but is not limited to steps S1 to S2.

[0053] S1. In response to an object recognition instruction, an object image is acquired.

[0054] Exemplarily, the item recognition instruction can be triggered by a timer, can be triggered by the user, or can be triggered when the items on the smart shelf change, and the embodiments of the present application do not limit this. In an exemplary application scenario, the user can put the newly arrived items into the warehouse, and when all the items are correctly placed on the smart shelf, the user can trigger the item recognition instruction autonomously, so that the smart shelf executes the method provided in the embodiments of the present application.

[0055] It is understandable that the acquired item image is related to the smart shelf, for example, the item image is related to a certain area in the smart shelf, or the item image is related to the entire smart shelf. Therefore, at least one item to be identified is displayed in the item image. Exemplarily, the acquired item image can be edited, such as cropped, etc.; of course, it can also be an unprocessed original image.

[0056] In some embodiments, step S1 may specifically include: in response to the object recognition instruction, obtaining an image of the object taken by an image acquisition device connected to the embedded terminal. That is, the source of the object image may be an image acquisition device, which may include a camera, etc. Exemplarily, the image acquisition device and the embedded terminal may be connected by wire, such as through a network cable, or may be connected wirelessly, such as through an intranet.

[0057] In some embodiments, the number of image acquisition devices may be multiple. The implementation of multiple image acquisition devices allows the image acquisition of smart shelves to be less affected by environmental factors. For example, the smart shelf can be divided into multiple areas, and the number of areas is consistent with the number of image acquisition devices. Then, the position of each image acquisition device can be reasonably arranged so that each image acquisition device can shoot its corresponding area to avoid interference caused by people moving around to the greatest extent. On the contrary, in the implementation of one image acquisition device capturing the entire smart shelf, the movement of people will have a greater impact on image acquisition.

[0058] Based on this, before step S1, the embodiment of the present application may further include: setting an image acquisition area for each image acquisition device, so that the object image acquired by the embedded terminal only displays the objects within the image acquisition area corresponding to the image acquisition device. There is no overlap between the image acquisition areas and all image acquisition areas can cover the smart shelf.

[0059] For example, the image acquisition area of ​​each image acquisition device can be reasonably divided according to the structural characteristics of the items placed on the smart shelf. For example, a physical boundary can be set for each image acquisition area, and the area within the physical boundary is the corresponding image acquisition area.

[0060] In this way, after the image acquisition area is set for the image acquisition device, when the embedded terminal responds to the object recognition instruction and acquires the object image, it can first receive the original image output by the image acquisition device, and then the embedded terminal performs clipping operations such as cropping on the original image according to the image acquisition area of ​​the image acquisition device, thereby acquiring the object image. It can be understood that the object displayed in the object image is the object within the image acquisition area of ​​the image acquisition device.

[0061] It can be understood that the multiple item images corresponding to the multiple image acquisition devices acquired by the embedded terminal fully cover the entire smart shelf. It can also be understood that this embodiment can avoid the instability problem caused by the focus adjustment of the image acquisition device. Specifically, since the image acquisition area is set, no matter how the image acquisition device is focused, the item images acquired by the embedded terminal are only related to the image acquisition area. Therefore, the instability problem caused by the focus adjustment of the image acquisition device can be avoided.

[0062] S2. Use a preset lightweight object recognition model to recognize at least one object to be recognized, and determine the category of each of the at least one object to be recognized.

[0063] Lightweight refers to a small size and consumes less system resources to complete the same computing task. Therefore, the storage space occupied by the lightweight object recognition model and the computing resources required are relatively small. To this end, the embodiment of the present application presets a lightweight object recognition model in the embedded terminal, which can meet the limited computing resources and storage space of the embedded terminal, and also meet the demand for low power consumption. Exemplarily, the object recognition model can be a target detection model of the YOLO series, for example, the object recognition model can be a YOLOv5n model, a YOLOv5l model, a YOLOv5m model or a YOLOv5s model.

[0064] Furthermore, the embodiment of the present application also improves the model structure of the lightweight object recognition model, further reduces the computing resources occupied by the lightweight object recognition model, and further reduces power consumption. As a first aspect of the improvement, the feature map output module in the lightweight object recognition model is output through the attention mechanism module. As a second aspect of the improvement, the attention mechanism module includes a parallel channel attention module and a spatial attention module, and the channel attention module includes a pooling layer, a one-dimensional convolutional network, and a connection layer connected in sequence. Exemplarily, the embodiment of the present application can use a trained lightweight object recognition model and then improve its model structure. Exemplarily, the embodiment of the present application can use an untrained lightweight object recognition model, improve its model structure, and then train it.

[0065] First, for deep learning, the deeper the model, the more parameters it has. Correspondingly, the stronger the model's learning ability is, but the more information and computation required, which can easily lead to information overload. For this reason, the attention mechanism can be introduced. The attention mechanism refers to using limited computing power to focus on local information to solve problems, thereby avoiding information overload. In other words, the attention mechanism is like a weight vector, giving a higher weight to an important content. Simply put, the attention mechanism is to make the model focus on more important information, reduce the processing of redundant information, and improve the model's inference speed and accuracy.

[0066] On this basis, since the lightweight object recognition model in the embodiment of the present application recognizes the objects displayed in the object image, the inventors found that in the process of the model processing the object image, it is necessary to output the feature map multiple times and then process it (such as compression and dimensionality increase, etc.). Therefore, the embodiment of the present application uses an attention mechanism module embedded in the output end of the feature map output module in the lightweight object recognition model, so as to achieve the purpose of allowing other network modules to pay more attention to key features (i.e., the objects displayed in the object image), thereby reducing the computational complexity of the lightweight object recognition model and reducing the computing resources occupied by the lightweight object recognition model.

[0067] Secondly, common attention mechanisms include attention mechanism modules that only target channels, and attention mechanism modules that target both channels and spaces, such as CBAM. Since the lightweight object recognition model in the embodiment of the present application processes object images, an attention mechanism module that targets both channels and spaces can be used.

[0068] In related technologies, the internal data processing method of the attention mechanism module containing the channel attention module and the spatial attention module is usually serial processing, that is, the feature map is processed by the spatial attention module and the channel attention module, or by the channel attention module and the spatial attention module, and then the corresponding attention feature map is output. However, in this serial processing method, the subsequent weights will be affected by the previous weights. In simple terms, the input feature map of the subsequent module is "modified" by the previous module.

[0069] It can be understood that the serial processing method has a relatively slow calculation speed and occupies relatively large system resources. To this end, the embodiment of the present application improves the data processing method of the attention mechanism module. Specifically, the attention mechanism module in the embodiment of the present application includes a parallel spatial attention module and a channel attention module. In this way, the feature map will be input to the spatial attention module and the channel attention module at the same time, so that these two modules are processed separately; and the attention mechanism module in the embodiment of the present application can combine the processing results of these two modules to output an attention feature map. It can be seen that the attention mechanism module in the embodiment of the present application adopts a parallel processing method, so that the channel attention module and the spatial attention module do not affect each other, which improves the calculation speed and reduces the occupied system resources.

[0070] In addition, in the related art, the channel attention module usually includes a pooling layer, a shared fully connected network, and a connection layer connected in sequence. However, the inventors found that such a channel attention module has relatively low detection accuracy and relatively slow calculation speed when processing feature maps. Specifically, after the feature map is pooled by the pooling layer, the input feature input to the shared fully connected network is 1×1×C (C is the number of channels). The shared fully connected network first reduces the input feature to 1×1×C / r (r is the reduction rate), and then restores it to 1×1×C.

[0071] It is understandable that the calculation speed is slow in this way, because the parameters of the shared fully connected network are positively correlated with the number of channels of the feature map; and the dimensionality reduction operation is not conducive to capturing the detailed features of the image, which affects the detection accuracy. To this end, the embodiment of the present application improves the channel attention module. Specifically, the channel attention module includes a pooling layer, a one-dimensional convolutional network, and a connection layer connected in sequence. In this way, after the feature map is pooled by the pooling layer, the input feature input to the one-dimensional convolutional network is 1×1×C (C is the number of channels); then, the one-dimensional convolutional network upgrades the input feature to 1×1×C×r, and then restores it to 1×1×C. It can be seen that the embodiment of the present application replaces the shared fully connected network in the related art with a one-dimensional convolutional network, which can reduce the number of parameters of the channel attention module, improve the calculation speed, and, because there is no dimensionality reduction operation, improve the detection accuracy.

[0072] In summary, compared with the AI ​​recognition of item types by the server in the smart shelf in the related art, the embodiment of the present application gives the embedded terminal the ability to recognize items. On the one hand, due to the limited computing resources and storage space of the embedded terminal, the embodiment of the present application sets a lightweight item recognition model in the embedded terminal and improves the model structure of the model. Specifically, the output end of the feature map output module in the lightweight item recognition model is embedded in the attention mechanism module, which reduces the calculation amount of the lightweight item recognition model. On the other hand, the embodiment of the present application also improves the aforementioned attention mechanism module, not only improving the channel attention module in the attention mechanism module, but also improving the data processing method of the attention mechanism module, and improving the calculation speed of the attention mechanism module. It can be seen that the lightweight item recognition model in the embodiment of the present application requires less computing resources and storage space, so the smart shelf equipped with the lightweight item recognition model can still perform item recognition when it is off-grid, which improves reliability.

[0073] In some embodiments, the lightweight object recognition model can be a YOLOv5s model, which may include a backbone network (Backhone), an intermediate layer network (Neck), a detection head (Head) and a loss function (Prediction). The backbone network is used to extract image features of the target image. The intermediate layer network is mainly responsible for compressing and upgrading the image features extracted by the backbone network to facilitate target detection by the detection head. The detection head is mainly used to predict the category and location information of the target image. The loss function is mainly used to evaluate the accuracy of the network output results. Based on this, combined with the previous discussion, the CBS module in the intermediate layer network is the feature map output module, and the CBS module is the Conv module, Batch Normalization layer, and SiLU activation function connected in sequence. The feature map output by the CBS module needs to be compressed and upgraded in dimension. Exemplarily, the model structure of the YOLOv5s model in traditional technology can be as follows Figure 3 As shown, the specific structure of the model is already in the prior art and will not be discussed in detail here. Based on the embodiments of the present application, Figure 3 The YOLOv5s model shown in the figure is improved, as shown in Figure 4 As shown, the improved attention mechanism module (represented by En_CBAM in the figure) is embedded into the output end of the CBS module of the intermediate layer network. As shown in the figure, the figure includes three CBS modules, which are embedded using En_CBAM1, En_CBAM2, and En_CBAM3 respectively.

[0074] In some embodiments, the feature map output module performs output through an attention mechanism module, which may include but is not limited to steps S21 to S22.

[0075] S21. Input the feature map to be processed output by the feature map output module into the channel attention module and the spatial attention module respectively for processing.

[0076] S22. Multiply the processing result of the channel attention module and the processing result of the spatial attention module to obtain an attention feature map.

[0077] The feature map to be processed output by the feature map output module in the lightweight object recognition model is input to the channel attention module in the attention mechanism module on the one hand, so as to obtain the corresponding processing result, that is, the output channel attention feature map; on the other hand, it is input to the spatial attention module in the attention mechanism module to obtain the corresponding processing result, that is, the output spatial attention feature map. Then, the processing structures of these two modules are multiplied, that is, the channel attention feature map is multiplied with the spatial attention feature map, to obtain the attention feature map. It can be understood that the attention feature map output by the attention mechanism module makes the subsequent network pay more attention to the key features (that is, the items displayed in the object image), thereby reducing the amount of calculation.

[0078] In the embodiment of the present application, the attention mechanism module can be as follows Figure 5 As shown. The upper network in the figure is a channel attention module, which may include a pooling layer, a one-dimensional convolutional network, and a connection layer connected in sequence. The lower network in the figure is a spatial attention module, which may include a pooling layer, a channel fusion layer, and a convolutional layer connected in sequence. For ease of distinction, the embodiment of the present application uses the first and second to distinguish the pooling layer in the channel attention module from the pooling layer in the spatial attention module. It should be noted that all sub-modules in the attention mechanism module are already prior art, and the specific implementation of the sub-modules will not be discussed in detail here.

[0079] Based on this, in some embodiments, the channel attention module processes the feature map to be processed, such as Figure 6 As shown, it may include but is not limited to steps S211 to S213.

[0080] S211, processing the feature map to be processed by the first maximum pooling layer and the first average pooling layer respectively to obtain the corresponding first pooling feature map and the second pooling feature map.

[0081] Among them, the pooling layer in the channel attention module includes a parallel first maximum pooling layer and a first average pooling layer.

[0082] S212. Input the first pooling feature map and the second pooling feature map into the one-dimensional convolutional network respectively to obtain the corresponding first convolutional feature map and the second convolutional feature map.

[0083] S213, input the first convolution feature map and the second convolution feature map into the connection layer to obtain the processing result of the channel attention module, that is, to obtain the channel attention feature map. Exemplarily, the first convolution feature map and the second convolution feature map can be "added" and then mapped by a function (such as Sigmoid) to obtain the channel attention feature map.

[0084] In addition, in some embodiments, the spatial attention module processes the feature map to be processed, such as Figure 7 As shown, it may include but is not limited to steps S214 to S216.

[0085] S214, processing the feature map to be processed by the second maximum pooling layer and the second average pooling layer respectively to obtain a corresponding third pooling feature map and a fourth pooling feature map.

[0086] Among them, the pooling layer in the spatial attention module includes a parallel second maximum pooling layer and a second average pooling layer.

[0087] S215, performing channel fusion on the third pooling feature map and the fourth pooling feature map, that is, performing channel splicing, to obtain a corresponding fifth pooling feature map.

[0088] S216, convolution processing is performed on the fifth pooled feature map through a convolution layer to obtain a processing result of the spatial attention module, that is, to obtain a spatial attention feature map. Exemplarily, after the fifth pooled feature map is convolutionally processed, it can be first processed by an activation function, and then the spatial attention feature map is output.

[0089] In some embodiments, after step S1, the object identification method may further include the following contents: detecting the networking status of the embedded terminal; if the networking status of the embedded terminal is online, sending the object image to the server to obtain the server's identification result for at least one object to be identified; otherwise, using a lightweight object identification model to determine the category of at least one object to be identified.

[0090] As can be seen from the above discussion, the embodiment of the present application gives the embedded terminal the ability to identify items. In this way, when identifying items, a server with stronger identification capabilities can be used for identification when there is a network, and the item identification can be performed by itself when there is no network, making the smart shelf more reliable.

[0091] See also Figure 8 The embedded terminal 700 provided in the embodiment of the present application may include: one or more processors 710 and a memory 720, Figure 8 A processor 710 is taken as an example.

[0092] The processor 710 and the memory 720 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0093] The memory 720 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the object identification method in the embodiment of the present application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 720, that is, the object identification method in the above method embodiment is implemented.

[0094] The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the embedded terminal 700, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the embedded terminal 700 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0095] The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, the object identification method in any of the above method embodiments is executed.

[0096] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0097] The embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, such as Figure 8 A processor 710 in the embodiment may enable the one or more processors to execute the object identification method in any of the above method embodiments.

[0098] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by the embedded terminal 700, the embedded terminal 700 is enabled to execute the object identification method in any of the above method embodiments.

[0099] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for object identification, characterized in that: Applied to an embedded terminal, the method comprises: In response to the object recognition instruction, acquiring an object image; the object image shows at least one object to be recognized; Using a preset lightweight object recognition model, the at least one object to be recognized is recognized, and the category of each of the at least one object to be recognized is determined; Wherein, the feature map output module in the lightweight object recognition model is output through the attention mechanism module; The attention mechanism module includes a parallel channel attention module and a spatial attention module, and the channel attention module includes a pooling layer, a one-dimensional convolutional network, and a connection layer connected in sequence.

2. The object identification method according to claim 1, characterized in that: The lightweight object recognition model is a YOLOv5s model; The YOLOv5s model includes an intermediate layer network, and the CBS module in the intermediate layer network is the feature map output module.

3. The object identification method according to claim 1 or 2, characterized in that: The feature map output module is output through the attention mechanism module, including: Inputting the feature map to be processed output by the feature map output module into the channel attention module and the spatial attention module respectively for processing; Multiply the processing result of the channel attention module and the processing result of the spatial attention module to obtain an attention feature map.

4. The object identification method according to claim 3, characterized in that: The channel attention module processes the feature map to be processed, including: The feature map to be processed is processed by a first maximum pooling layer and a first average pooling layer respectively to obtain a corresponding first pooling feature map and a second pooling feature map; the pooling layer in the channel attention module includes the first maximum pooling layer and the first average pooling layer in parallel; Inputting the first pooling feature map and the second pooling feature map into the one-dimensional convolutional network respectively to obtain a corresponding first convolutional feature map and a second convolutional feature map; The first convolutional feature map and the second convolutional feature map are input into the connection layer to obtain the processing result of the channel attention module.

5. The object identification method according to claim 3, characterized in that: The spatial attention module processes the feature map to be processed, including: The feature map to be processed is processed by a second maximum pooling layer and a second average pooling layer respectively to obtain a corresponding third pooling feature map and a fourth pooling feature map; the pooling layer in the spatial attention module includes the second maximum pooling layer and the second average pooling layer in parallel; Perform channel fusion on the third pooling feature map and the fourth pooling feature map to obtain a corresponding fifth pooling feature map; The fifth pooled feature map is convolved through a convolution layer to obtain a processing result of the spatial attention module.

6. The object identification method according to any one of claims 1, 2, 4 and 5, characterized in that: The step of acquiring an image of an object in response to an object recognition instruction includes: In response to the object recognition instruction, an image of the object taken by an image acquisition device connected to the embedded terminal is acquired.

7. The object identification method according to claim 6, characterized in that: The number of the image acquisition devices is multiple; Before acquiring the image of the object taken by the image acquisition device connected to the embedded terminal in response to the object recognition instruction, the method further includes: An image acquisition area is set for each image acquisition device, so that the object image acquired by the embedded terminal only displays the objects within the image acquisition area corresponding to the image acquisition device; There is no overlap between the image acquisition areas and all the image acquisition areas can cover the smart shelf.

8. The object identification method according to claim 1, characterized in that: After acquiring the object image in response to the object recognition instruction, the method further includes: Detecting the networking status of the embedded terminal; If the networking status of the embedded terminal is online, the object image is sent to the server to obtain the server's recognition result of the at least one object to be identified; otherwise, the lightweight object recognition model is used to determine the category of each of the at least one object to be identified.

9. An embedded terminal, characterized in that: include: at least one processor; as well as, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the object identification method described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device executes the object identification method according to any one of claims 1 to 8.

Images