Method, device, electronic device and storage medium for identifying live goods on street stalls

Abstract

The embodiment of the present invention provides a method for identifying live commodities on a street stall, the method comprising: obtaining a captured image and an infrared image of a target street stall, the captured image including a target stall owner and a target commodity, the captured image and the infrared image having the same size resolution; performing image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target commodity; performing image recognition processing on the infrared image to obtain the infrared information of the target commodity; based on the behavior information of the target stall owner, the appearance information of the target commodity and the infrared information of the target commodity, judging whether the target stall owner has engaged in the sale of live commodities. The present invention can automatically identify stalls that have engaged in the sale of live commodities, improve the management efficiency of stalls that sell live animals, and improve the recognition accuracy of the sale of live commodities by combining the infrared information of the target commodity.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, device, electronic equipment and storage medium for identifying live goods on street stalls. Background Art

[0002] With the rise of the street stall economy, more and more people are participating in the street stall economy, resulting in the emergence of a variety of goods on the street stalls. Since live animals need to meet strict epidemic prevention standards, and most of the live animals on the street stalls have not undergone strict epidemic prevention, in some areas of the city, live animals can only be sold in designated sales locations, and selling live animals on street stalls is not allowed. However, due to the high mobility of street stalls, they are generally inspected by urban management personnel, which requires a large amount of manpower, resulting in low management efficiency of live animal sales stalls. Summary of the invention

[0003] The embodiment of the present invention provides a method for identifying live commodities on street stalls, aiming to solve the problem of low management efficiency of existing live pet sales stalls. By performing image recognition processing on the captured image and infrared image of the target street stall respectively, combining the identified behavior information of the target stall owner, the appearance information of the target commodity and the infrared information of the target commodity to determine whether the target street stall has the behavior of selling live commodities, the stall with the behavior of selling live commodities can be automatically identified, and the management efficiency of the live animal sales stalls is improved. Due to the combination of the infrared information of the target commodity, the recognition accuracy of the behavior of selling live commodities is improved.

[0004] In a first aspect, an embodiment of the present invention provides a method for identifying live goods on a street stall, the method comprising:

[0005] Acquire a captured image and an infrared image of a target street stall, wherein the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution;

[0006] Performing image recognition processing on the captured image to obtain behavior information of the target stall owner and appearance information of the target product;

[0007] Performing image recognition processing on the infrared image to obtain infrared information of the target product;

[0008] Based on the behavior information of the target stall owner, the appearance information of the target commodity and the infrared information of the target commodity, it is determined whether the target stall owner is engaged in the sale of living commodities.

[0009] Optionally, performing image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target product includes:

[0010] Performing behavior detection on the captured image to obtain behavior information of the target stall owner;

[0011] Performing target detection on the captured image to obtain detection frame information of the target commodity, wherein the detection frame information includes appearance information of the target commodity.

[0012] Optionally, performing behavior detection on the captured image to obtain behavior information of the target stall owner includes:

[0013] Performing human posture detection on the captured image to obtain posture information of the target stall owner and posture information of the customer;

[0014] The behavior information of the target stall owner is determined according to the posture information of the target stall owner and the posture information of the customer.

[0015] Optionally, performing image recognition processing on the infrared image to obtain infrared information of the target commodity includes:

[0016] Performing image segmentation processing on the infrared image to obtain infrared image blocks;

[0017] Screening out human body infrared image blocks in the infrared image blocks to obtain candidate infrared image blocks;

[0018] Target recognition is performed based on the candidate infrared image blocks to obtain infrared information of the target product.

[0019] Optionally, the step of screening out human body infrared image blocks in the infrared image blocks to obtain candidate infrared image blocks includes:

[0020] Performing human body detection on the captured image to obtain human body detection information;

[0021] Matching the human body detection information with the infrared image block to obtain the human body infrared image block;

[0022] The human body infrared image blocks in the infrared image blocks are screened out to obtain candidate infrared image blocks.

[0023] Optionally, performing target recognition according to the candidate infrared image block to obtain infrared information of the target commodity includes:

[0024] Mapping the candidate infrared image block to the captured image to obtain an infrared mapping image block;

[0025] Perform channel fusion on the candidate infrared image block and the infrared mapping image block to obtain a fused image block;

[0026] Target recognition is performed on the fused image block to obtain infrared information of the target product.

[0027] Optionally, judging whether the target stall owner has engaged in the sale of living goods based on the behavior information of the target stall owner, the appearance information of the target goods, and the infrared information of the target goods includes:

[0028] Determining whether the target stall owner has any selling behavior based on the behavior information of the target stall owner;

[0029] Determining whether the target product has a preset animal appearance according to the appearance information of the target product;

[0030] Determining whether the target product has a living feature according to the infrared information of the target product;

[0031] If the target stall owner is engaged in trafficking, the target commodity has a preset animal appearance, and the target commodity has a living feature, then it is determined that the target stall is engaged in the trafficking of living commodities.

[0032] In a second aspect, an embodiment of the present invention provides a device for identifying live goods on a street stall, the device comprising:

[0033] An acquisition module, used to acquire a captured image and an infrared image of a target stall, wherein the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution;

[0034] A first processing module is used to perform image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target product;

[0035] A second processing module is used to perform image recognition processing on the infrared image to obtain infrared information of the target product;

[0036] The third processing module is used to determine whether the target stall owner sells living goods based on the behavior information of the target stall owner, the appearance information of the target goods and the infrared information of the target goods.

[0037] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for identifying live street goods provided in an embodiment of the present invention are implemented.

[0038] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for identifying live street goods provided in an embodiment of the invention are implemented.

[0039] In an embodiment of the present invention, a captured image and an infrared image of a target stall are obtained, the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution; the captured image is subjected to image recognition processing to obtain the behavior information of the target stall owner and the appearance information of the target commodity; the infrared image is subjected to image recognition processing to obtain the infrared information of the target commodity; based on the behavior information of the target stall owner, the appearance information of the target commodity, and the infrared information of the target commodity, it is determined whether the target stall owner sells live commodities. By performing image recognition processing on the captured image and the infrared image of the target stall respectively, and combining the identified behavior information of the target stall owner, the appearance information of the target commodity, and the infrared information of the target commodity to determine whether the target stall sells live commodities, it is possible to automatically identify stalls that sell live commodities, improve the management efficiency of stalls that sell live animals, and improve the recognition accuracy of live commodity sales due to the combination of the infrared information of the target commodity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is a flow chart of a method for identifying live goods on a street stall provided by an embodiment of the present invention;

[0042] Figure 2 It is a structural schematic diagram of a live commodity identification device for a street stall provided by an embodiment of the present invention;

[0043] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1 , Figure 1 is a flow chart of a method for identifying live goods on a street stall provided by an embodiment of the present invention. Figure 1 As shown, the method for identifying live goods on a street stall includes the following steps:

[0046] 101. Obtain a captured image and an infrared image of the target street stall.

[0047] In an embodiment of the present invention, the captured image may be an image captured by a surveillance camera set in a designated area, the target stall may be one or more stalls in the designated area, the surveillance camera may simultaneously capture RGB images and infrared images, and the infrared image may include temperature information of each target. The captured image includes the target stall owner and the target product, the captured image and the infrared image have the same acquisition area, and the captured image and the infrared image have the same size resolution, so that the captured image and the infrared image can be aligned.

[0048] The target stall owner may be a person who wanders around the stall, and the target commodity may be an object that initially appears within the stall. When the stall is detected, the person who wanders around and the object that initially appears within the stall are detected by an image tracking algorithm, so that the person who wanders around is determined as the target stall owner, and the object that initially appears within the stall is determined as the target commodity.

[0049] The captured image may be a static image or a dynamic image. Correspondingly, the infrared image may be a static image or a dynamic image. The static image may be a separate frame image, and the dynamic image may be a continuous frame image.

[0050] 102. Perform image recognition processing on the captured image to obtain behavior information of the target stall owner and appearance information of the target product.

[0051] In the embodiment of the present invention, the above-mentioned image recognition processing may be an existing image recognition processing method, such as an existing image recognition processing method such as image detection, image classification, and image tracking.

[0052] The captured image is an RGB image, which includes three color channels: red, green, and blue. Specifically, the captured image is processed by image recognition, and the captured image can be input into a pre-trained behavior recognition model for behavior recognition to obtain the behavior information of the target stall owner. The captured image can be input into a pre-trained target detection model for detection and classification to obtain the appearance information of the target product.

[0053] The target stall owner's behavior information can be used to describe whether the target stall owner has a sales behavior. Sales behavior refers to selling goods to customers. The target stall owner's behavior information can include goods introduction, goods display, money collection, goods delivery to customers, etc. The target goods' appearance information can be used to describe whether the target goods have the appearance of a living animal, such as whether they have the appearance of a cat, whether they have the appearance of a dog, whether they have the appearance of a chicken, etc. The target goods' appearance information includes appearances such as shape, size, and color.

[0054] 103. Perform image recognition processing on the infrared image to obtain infrared information of the target product.

[0055] In the embodiment of the present invention, the infrared image includes temperature information of each target, which may include temperature information of the customer, temperature information of the target stall owner, and temperature information of the target commodity. The above-mentioned image recognition processing may be an existing image recognition processing method, such as an existing image detection, image classification, image segmentation, and other image recognition processing methods.

[0056] Specifically, the temperature information in the infrared image can be represented by blocks or regions. For example, if the temperature of a block or region is higher than the surrounding temperature, it can be determined that the block or region has a corresponding target, which can be a customer, a stall owner, or an animal product. The above-mentioned animal products can be pets or poultry and livestock, among which pets can be cats, dogs, and other animals, and poultry and livestock can be chickens, ducks, and the like.

[0057] The above-mentioned infrared image is aligned with the above-mentioned snapshot image, and according to the location of the block with higher temperature in the infrared image, it is possible to search whether there is a corresponding target in the corresponding position in the snapshot image. If there is a corresponding target, the block with higher temperature in the infrared image is associated with the corresponding target, which can be a customer, a stall owner or an animal product. Furthermore, if the target is an animal product, the infrared information of the target product can be determined.

[0058] 104. Based on the behavior information of the target stall owner, the appearance information of the target commodity, and the infrared information of the target commodity, determine whether the target stall owner is engaged in the sale of live commodities.

[0059] In the embodiment of the present invention, it is possible to determine whether the target stall owner has the behavior of selling goods through the behavior information of the target stall owner, and whether the target goods are living goods through the appearance information of the target goods and the infrared information of the target goods. It is possible to determine whether the target stall owner has the behavior of selling goods and whether the target goods are living goods, and whether the target stall owner has the behavior of selling living goods. When the target stall owner has the behavior of selling goods and the target goods are living goods, it can be determined that there are living goods in the target stall, and then it can be determined that the target stall owner has the behavior of selling living goods.

[0060] It should be noted that, by using the infrared information of the target product, it is possible to avoid misidentification of whether the target product is an animal toy by only judging whether it is an animal based on the appearance information of the target product.

[0061] In an embodiment of the present invention, a captured image and an infrared image of a target stall are obtained, the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution; the captured image is subjected to image recognition processing to obtain the behavior information of the target stall owner and the appearance information of the target commodity; the infrared image is subjected to image recognition processing to obtain the infrared information of the target commodity; based on the behavior information of the target stall owner, the appearance information of the target commodity, and the infrared information of the target commodity, it is determined whether the target stall owner has engaged in the sale of live commodities. By performing image recognition processing on the captured image and the infrared image of the target stall respectively, and combining the identified behavior information of the target stall owner, the appearance information of the target commodity, and the infrared information of the target commodity to determine whether the target stall has engaged in the sale of live commodities, it is possible to automatically identify stalls that have engaged in the sale of live commodities, thereby improving the management efficiency of stalls that sell live pets, and by combining the infrared information of the target commodity, the recognition accuracy of the sale of live commodities is improved.

[0062] Optionally, in the step of performing image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target product, behavior detection can be performed on the captured image to obtain the behavior information of the target stall owner; target detection can be performed on the captured image to obtain detection frame information of the target product, and the detection frame information includes the appearance information of the target product.

[0063] In an embodiment of the present invention, the captured image can be subjected to behavior detection by an existing behavior detection algorithm to obtain the behavior information of each person in the captured image, including customers and target stall owners. When a stall is detected, an image tracking algorithm can be used to detect a person with wandering behavior, thereby determining the target stall owner as the person with wandering behavior. Specifically, in the image tracking algorithm, each person corresponds to a tracking ID, and the behavior information of the target stall owner can be determined from the behavior information of each person according to the tracking ID of the target stall owner. The behavior detection algorithm can be a behavior detection algorithm such as a TSN network and a C3D network.

[0064] The captured image can be subjected to target detection by an existing target detection algorithm to obtain detection frame information of each target in the captured image. The targets include customers, target stall owners, and animal products. The detection frame information of each target can include customer detection frame information, target stall owner detection frame information, and animal product detection frame information. The target detection algorithm can be an algorithm network such as the YOLO series and the R-CNN series. The detection frame information can be represented by (x, y, w, h, u, v), where (x, y) represents the center position of the detection frame, w represents the width of the detection frame, h represents the height of the detection frame, u represents the category of the target in the detection frame, and v represents the confidence of the detection frame. The detection frame information of the target product can be determined in the detection frame information of each target according to the category u of the target, and the appearance information of the target product can be determined according to (x, y, w, h) in the detection frame information of the target product.

[0065] Furthermore, the corresponding detection frame image can be determined in the captured image according to the (x, y, w, h) in the detection frame information of the target product, and the detection frame image can be segmented to obtain the outline of the target product. The shape and size of the target product can be determined according to the outline of the target product. The color of the target product can be determined according to the position of the outline of the target product in the captured image, and then the appearance information of the target product can be determined according to the shape, size and color of the target product.

[0066] Optionally, in the step of performing behavior detection on the captured image to obtain the behavior information of the target stall owner, human posture detection can be performed on the captured image to obtain the posture information of the target stall owner and the posture information of the customer; the behavior information of the target stall owner is determined based on the posture information of the target stall owner and the posture information of the customer.

[0067] In an embodiment of the present invention, human body detection can be performed on the captured image first to obtain the customer's human body image and the target stall owner's human body image, extract the human body key points in the customer's human body image, perform posture estimation based on the human body key points in the customer's human body image, and obtain the customer's posture information; extract the human body key points in the target stall owner's human body image, perform posture estimation based on the human body key points in the target stall owner's human body image, and obtain the target stall owner's posture information. Posture estimation can be two-dimensional human body posture estimation, which performs human body posture estimation by locating the anatomical key points (such as elbows, wrists, etc.) or parts of the human body. Specifically, the above-mentioned posture estimation can be performed using CPN (Cascaded Pyramid Network, CPN for short) and HRNet (High-Resolution Net, HRNet for short) networks.

[0068] The customer's posture information may include postures such as asking, watching, paying, and reaching out to receive, and the target stall owner's posture information may include postures such as talking, showing, collecting money, and reaching out to hand over, etc. After obtaining the customer's posture information and the target stall owner's state information, the target stall owner's behavior information may be determined based on the customer's posture information and the target stall owner's posture information. For example, when the customer's state is asking and the target stall owner's posture is talking, the target stall owner's behavior may be determined as the behavior of introducing the product; when the customer's posture is watching and the target stall owner's posture is showing, the target stall owner's behavior may be determined as the behavior of displaying the product; when the customer's posture is paying and the target stall owner's posture is collecting money, the target stall owner's behavior may be determined as the behavior of collecting money; when the customer's posture is reaching out to receive and the target stall owner's posture is reaching out to hand over, the target stall owner's behavior may be determined as the behavior of handing the product to the customer.

[0069] Optionally, in the step of performing image recognition processing on the infrared image to obtain infrared information of the target product, image segmentation processing can be performed on the infrared image to obtain infrared blocks; infrared blocks of human bodies in the infrared blocks can be screened out to obtain candidate infrared blocks; target recognition is performed based on the candidate infrared blocks to obtain infrared information of the target product.

[0070] In the embodiment of the present invention, the infrared image can be segmented by an image segmentation algorithm to segment the area with high temperature in the infrared image to obtain an infrared block. A high temperature indicates thermal radiation, and whether the object is alive can be determined based on the thermal radiation.

[0071] It should be noted that the infrared image blocks may be the infrared image blocks of the customer, the infrared image blocks of the target stall owner and the infrared image blocks of the target commodity. In the embodiment of the present invention, only the infrared image blocks of the target commodity need to be used to determine whether the target commodity is alive. Therefore, it is necessary to screen out the infrared image blocks of the customer, the infrared image blocks of the target stall owner and the infrared image blocks of the target commodity. After screening out the infrared image blocks of the customer and the infrared image blocks of the target stall owner, the remaining infrared image blocks are candidate infrared image blocks, thereby obtaining the infrared information of the target commodity.

[0072] In a possible embodiment, the candidate infrared image block is used as the infrared image block of the target commodity, and the infrared image block of the target commodity is used as the infrared information of the target commodity.

[0073] Optionally, in the step of screening out human infrared blocks in infrared blocks to obtain candidate infrared blocks, human body detection can be performed on the captured image to obtain human body detection information; the human body detection information is matched with the infrared blocks to obtain human body infrared blocks; and human body infrared blocks in infrared blocks are screened out to obtain candidate infrared blocks.

[0074] In an embodiment of the present invention, the infrared image is aligned with the captured image, and the captured image can be subjected to human body detection by an existing target detection algorithm to obtain customer detection frame information and target stall owner detection frame information, which can be represented by (x, y, w, h, u, v). The position of the infrared block is projected into the captured image. If the infrared block falls into the customer detection frame or the target stall owner detection frame, it means that the temperature corresponding to the infrared block is the human body temperature, and the corresponding infrared block is a human body infrared block. The human body infrared blocks are screened out from all infrared blocks, and the remaining infrared blocks are candidate infrared blocks.

[0075] Optionally, in the step of performing target recognition based on candidate infrared blocks to obtain infrared information of the target product, the candidate infrared blocks can be mapped to the captured image to obtain infrared mapping blocks; the candidate infrared blocks and the infrared mapping blocks can be channel-fused to obtain fused blocks; and target recognition can be performed on the fused blocks to obtain infrared information of the target product.

[0076] In an embodiment of the present invention, after obtaining the candidate infrared image blocks, the influence of the human body infrared image blocks has been eliminated from the candidate infrared image blocks, but there may be some non-human and non-commodity influences. Therefore, target recognition can be performed on the candidate infrared image blocks again, so that the infrared information of the target commodity can be obtained more accurately.

[0077] Specifically, the infrared image is aligned with the captured image, and the candidate infrared tiles can be mapped into the captured image to obtain infrared mapping tiles, which are image regions in the captured image. The candidate infrared tiles are channel-fused with the infrared mapping tiles to obtain fused tiles of four channels: red, green, blue, and infrared. The fused tiles of the four channels are input into a pre-trained recognition model for target recognition to obtain infrared information of the target product. The recognition model can be a model built based on a deep convolutional neural network. The training data of the recognition model includes sample images and sample labels. The channels of the sample images are red, green, blue, and infrared. The sample labels include 0 and 1, where 0 can represent non-target products and 1 can represent target products, so that the trained recognition model can identify whether it is a target product based on the red, green, blue, and infrared channel images. After obtaining the trained recognition model, the fused image block containing the four channels of red, green, blue and infrared is input into the trained recognition model, and the output category of the fused image block containing the four channels of red, green, blue and infrared is the target product or non-target product. If it is the target product, the corresponding candidate infrared image block is used as the infrared information of the target product.

[0078] By determining whether a target product is a target product through the fusion block containing the four channels of red, green, blue and infrared, objects that are not target products can be filtered out more accurately, making the infrared information of the target product more accurate.

[0079] Optionally, in the step of judging whether the target stall owner is engaged in trafficking of live commodities based on the target stall owner's behavior information, the appearance information of the target commodity and the infrared information of the target commodity, it can be determined whether the target stall owner is engaged in trafficking based on the target stall owner's behavior information; determine whether the target commodity has a preset animal appearance based on the appearance information of the target commodity; determine whether the target commodity has live characteristics based on the infrared information of the target commodity; if the target stall owner is engaged in trafficking, the target commodity has a preset animal appearance, and the target commodity has live characteristics, then it is determined that the target stall is engaged in trafficking of live commodities.

[0080] In an embodiment of the present invention, the behavior information of the target stall owner may include behaviors such as product introduction, product display, collecting money, and handing the product to the customer. When at least one of the product introduction, product display, collecting money, and handing the product to the customer appears in the behavior information of the target stall owner, it can be determined that the target stall owner has a selling behavior. The appearance of the above-mentioned target product is compared with the preset animal appearance for similarity. If the similarity between the appearance of the target product and the preset animal appearance is greater than the preset similarity, it can be determined that the target product has a preset animal appearance. The preset animal appearance can be the appearance of a pet or poultry and livestock, wherein the pet can be a cat, dog, or other animal, and the poultry and livestock can be a chicken, duck, or the like. According to the infrared information of the target product, it is determined whether the temperature of the target product is greater than the preset temperature. If it is greater than the preset temperature, it can be determined that the target product has a living body characteristic. The above-mentioned preset temperature can be determined according to the category of the target product, for example, cats, dogs, chickens, and ducks have different corresponding preset temperatures.

[0081] If the target stall owner is engaged in trafficking, the target product has a preset animal appearance, and the target product has live characteristics, then it is determined that the target stall is engaged in the trafficking of live products. Since the realization of the trafficking behavior is taken into consideration, non-commodity animals are avoided from being misidentified. Since the live characteristics of the target product are taken into consideration, the accuracy of identifying the trafficking of live products can be improved.

[0082] It should be noted that the method for identifying live street goods provided in the embodiments of the present invention can be applied to devices such as smart cameras, smart phones, computers, servers, etc. that can perform the method for identifying live street goods.

[0083] Optional, see Figure 2 , Figure 2 is a schematic diagram of the structure of a live commodity identification device for a street stall provided by an embodiment of the present invention. Figure 2 As shown, the device comprises:

[0084] An acquisition module 201 is used to acquire a captured image and an infrared image of a target street stall, wherein the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution;

[0085] The first processing module 202 is used to perform image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target product;

[0086] The second processing module 203 is used to perform image recognition processing on the infrared image to obtain infrared information of the target product;

[0087] The third processing module 204 is used to determine whether the target stall owner sells live goods based on the behavior information of the target stall owner, the appearance information of the target goods and the infrared information of the target goods.

[0088] Optionally, the first processing module 202 is further used to perform behavior detection on the captured image to obtain behavior information of the target stall owner; perform target detection on the captured image to obtain detection frame information of the target product, and the detection frame information includes appearance information of the target product.

[0089] Optionally, the first processing module 202 is further used to perform human posture detection on the captured image to obtain posture information of the target stall owner and posture information of the customer; and determine the behavior information of the target stall owner based on the posture information of the target stall owner and the posture information of the customer.

[0090] Optionally, the second processing module 203 is also used to perform image segmentation processing on the infrared image to obtain infrared blocks; screen out human infrared blocks in the infrared blocks to obtain candidate infrared blocks; perform target recognition based on the candidate infrared blocks to obtain infrared information of the target product.

[0091] Optionally, the second processing module 203 is also used to perform human body detection on the captured image to obtain human body detection information; match the human body detection information with the infrared image block to obtain the human body infrared image block; and filter out the human body infrared image blocks in the infrared image blocks to obtain candidate infrared image blocks.

[0092] Optionally, the second processing module 203 is also used to map the candidate infrared block into the captured image to obtain an infrared mapping block; perform channel fusion on the candidate infrared block and the infrared mapping block to obtain a fused block; perform target recognition on the fused block to obtain infrared information of the target product.

[0093] Optionally, the third processing module 204 is further used to determine whether the target stall owner is engaged in trafficking based on the behavior information of the target stall owner; determine whether the target commodity has a preset animal appearance based on the appearance information of the target commodity; determine whether the target commodity has living characteristics based on the infrared information of the target commodity; if the target stall owner is engaged in trafficking, the target commodity has a preset animal appearance, and the target commodity has living characteristics, then it is determined that the target stall is engaged in trafficking in living commodities.

[0094] It should be noted that the device for identifying live street goods provided in the embodiments of the present invention can be applied to devices such as smart cameras, smart phones, computers, servers, etc. that can perform the method for identifying live street goods.

[0095] The device for identifying live goods at street stalls provided by the embodiment of the present invention can implement each process implemented by the method for identifying live goods at street stalls in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0096] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program of a method for identifying live street goods stored in the memory 302 and running on the processor 301, wherein:

[0097] The processor 301 is used to call the computer program stored in the memory 302 and execute the following steps:

[0098] Acquire a captured image and an infrared image of a target street stall, wherein the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution;

[0099] Performing image recognition processing on the captured image to obtain behavior information of the target stall owner and appearance information of the target product;

[0100] Performing image recognition processing on the infrared image to obtain infrared information of the target product;

[0101] Based on the behavior information of the target stall owner, the appearance information of the target commodity and the infrared information of the target commodity, it is determined whether the target stall owner is engaged in the sale of living commodities.

[0102] Optionally, the processor 301 performs image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target product, including:

[0103] Performing behavior detection on the captured image to obtain behavior information of the target stall owner;

[0104] Performing target detection on the captured image to obtain detection frame information of the target commodity, wherein the detection frame information includes appearance information of the target commodity.

[0105] Optionally, the processor 301 performs behavior detection on the captured image to obtain behavior information of the target stall owner, including:

[0106] Performing human posture detection on the captured image to obtain posture information of the target stall owner and posture information of the customer;

[0107] The behavior information of the target stall owner is determined according to the posture information of the target stall owner and the posture information of the customer.

[0108] Optionally, the processor 301 performs image recognition processing on the infrared image to obtain infrared information of the target commodity, including:

[0109] Performing image segmentation processing on the infrared image to obtain infrared image blocks;

[0110] Screening out human body infrared image blocks in the infrared image blocks to obtain candidate infrared image blocks;

[0111] Target recognition is performed based on the candidate infrared image blocks to obtain infrared information of the target product.

[0112] Optionally, the processor 301 performs the step of screening out infrared image blocks of human bodies in the infrared image blocks to obtain candidate infrared image blocks, including:

[0113] Performing human body detection on the captured image to obtain human body detection information;

[0114] Matching the human body detection information with the infrared image block to obtain the human body infrared image block;

[0115] The human body infrared image blocks in the infrared image blocks are screened out to obtain candidate infrared image blocks.

[0116] Optionally, the processor 301 performs target recognition according to the candidate infrared image block to obtain infrared information of the target commodity, including:

[0117] Mapping the candidate infrared image block to the captured image to obtain an infrared mapping image block;

[0118] Perform channel fusion on the candidate infrared image block and the infrared mapping image block to obtain a fused image block;

[0119] Target recognition is performed on the fused image block to obtain infrared information of the target product.

[0120] Optionally, the processor 301 determines whether the target stall owner sells live goods based on the behavior information of the target stall owner, the appearance information of the target goods, and the infrared information of the target goods, including:

[0121] Determining whether the target stall owner has any selling behavior based on the behavior information of the target stall owner;

[0122] Determining whether the target product has a preset animal appearance according to the appearance information of the target product;

[0123] Determining whether the target product has a living feature according to the infrared information of the target product;

[0124] If the target stall owner is engaged in trafficking, the target commodity has a preset animal appearance, and the target commodity has a living feature, then it is determined that the target stall is engaged in the trafficking of living commodities.

[0125] The electronic device provided in the embodiment of the present invention can implement each process implemented by the method for identifying live goods at street stalls in the above method embodiment, and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0126] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the method for identifying live street goods provided by an embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0127] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a magnetic disk, an optical disk, a read-only storage memory (RON) or a random access memory (RAN).

[0128] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for identifying live goods on street stalls, characterized in that: The following steps are involved: Acquire a captured image and an infrared image of a target street stall, wherein the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution; Performing image recognition processing on the captured image to obtain behavior information of the target stall owner and appearance information of the target product; Performing image recognition processing on the infrared image to obtain infrared information of the target product; Determining whether the target stall owner has any selling behavior based on the behavior information of the target stall owner; Determining whether the target product has a preset animal appearance according to the appearance information of the target product; Determining whether the target product has a living feature according to the infrared information of the target product; If the target stall owner is engaged in trafficking, the target commodity has a preset animal appearance, and the target commodity has living characteristics, it is determined that the target stall owner is engaged in trafficking in living commodities.

2. The method for identifying live goods on street stalls as claimed in claim 1, characterized in that: The performing image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target commodity includes: Performing behavior detection on the captured image to obtain behavior information of the target stall owner; Performing target detection on the captured image to obtain detection frame information of the target commodity, wherein the detection frame information includes appearance information of the target commodity.

3. The method for identifying live goods on street stalls as claimed in claim 2, characterized in that: The performing behavior detection on the captured image to obtain the behavior information of the target stall owner includes: Performing human posture detection on the captured image to obtain posture information of the target stall owner and posture information of the customer; The behavior information of the target stall owner is determined according to the posture information of the target stall owner and the posture information of the customer.

4. The method for identifying live goods on street stalls as claimed in claim 3, characterized in that: The performing image recognition processing on the infrared image to obtain infrared information of the target commodity includes: Performing image segmentation processing on the infrared image to obtain infrared image blocks; Screening out human body infrared image blocks in the infrared image blocks to obtain candidate infrared image blocks; Target recognition is performed based on the candidate infrared image blocks to obtain infrared information of the target product.

5. The method for identifying live goods on street stalls as claimed in claim 4, characterized in that: The step of screening out human body infrared image blocks in the infrared image blocks to obtain candidate infrared image blocks includes: Performing human body detection on the captured image to obtain human body detection information; Matching the human body detection information with the infrared image block to obtain the human body infrared image block; The human body infrared image blocks in the infrared image blocks are screened out to obtain candidate infrared image blocks.

6. The method for identifying live goods on street stalls as claimed in claim 5, characterized in that: The performing target recognition according to the candidate infrared image blocks to obtain infrared information of the target commodity includes: Mapping the candidate infrared image block to the captured image to obtain an infrared mapping image block; Perform channel fusion on the candidate infrared image block and the infrared mapping image block to obtain a fused image block; Target recognition is performed on the fused image block to obtain infrared information of the target product.

7. A device for identifying live goods on street stalls, characterized in that: The device comprises: An acquisition module, used to acquire a captured image and an infrared image of a target stall, wherein the captured image includes a target stall owner and a target commodity, and the captured image and the infrared image have the same size resolution; A first processing module is used to perform image recognition processing on the captured image to obtain the behavior information of the target stall owner and the appearance information of the target product; A second processing module is used to perform image recognition processing on the infrared image to obtain infrared information of the target product; The third processing module is used to determine whether the target stall owner is engaged in trafficking based on the behavior information of the target stall owner; determine whether the target commodity has a preset animal appearance based on the appearance information of the target commodity; determine whether the target commodity has living characteristics based on the infrared information of the target commodity; if the target stall owner is engaged in trafficking, the target commodity has a preset animal appearance, and the target commodity has living characteristics, then it is determined that the target stall owner is engaged in trafficking in living commodities.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the method for identifying live street goods as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for identifying live street goods according to any one of claims 1 to 6 are implemented.

Images