Image acquisition method, device, electronic device and computer storage medium

Abstract

An embodiment of the present invention provides a method for collecting a shelf image, which includes: obtaining a shelf image collected according to the indication of first guiding information, where the shelf is used to carry commodities, and the first guiding information is used to indicate an image acquisition path of the shelf; obtaining an edge detection result of performing shelf edge detection on the shelf image; if the edge detection result indicates that the shelf edge is included in the shelf image, obtaining second guiding information indicating a new image acquisition path or obtaining third guiding information indicating the end of acquisition. Through the embodiment of the present invention, the quality of shelf image collection can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and particularly to an image acquisition method, apparatus, electronic device, and computer storage medium. Background Art

[0002] In the prior art, in some image acquisition scenarios, due to the large length and / or height of the target object to be acquired, a single image acquisition operation cannot obtain a complete image of the target object, so that the user needs to perform multiple image acquisition operations to acquire images of different parts of the target object, so as to achieve comprehensive image acquisition of the target object.

[0003] For example, when digitizing the shelf display information of an offline store, it is necessary to first take a complete image of the shelf, and then use artificial intelligence technology to identify these shelf images, so as to generate digitized display information according to the identification results. When taking a shelf image, since the shelf spacing is narrow and the length of a whole section of the shelf is generally long, it is very difficult to take a complete picture of the entire shelf and obtain clear product information in one photo. To solve this problem, it is necessary to take multiple photos of the shelf.

[0004] During the multiple shooting process, due to reasons such as non-standard user operations, some areas of the target object may be missed during image acquisition, and a complete image and information of the target object cannot be obtained. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an image acquisition solution to solve some or all of the above problems.

[0006] According to a first aspect of embodiments of the present invention, there is provided a method for acquiring a shelf image, which includes: acquiring a shelf image acquired according to an indication of first guiding information, where the shelf is used to carry goods, and the first guiding information is used to indicate an image acquisition path of the shelf; acquiring an edge detection result of performing shelf edge detection on the shelf image; if the edge detection result indicates that the shelf edge is included in the shelf image, acquiring second guiding information indicating a new image acquisition path or acquiring third guiding information indicating the end of acquisition.

[0007] According to a second aspect of an embodiment of the present invention, there is provided a method for processing commodity information, which includes: collecting image data of a shelf according to acquired first guiding information, wherein the first guiding information is used to indicate an image acquisition path of the shelf; identifying the image data and obtaining commodity information on the shelf and information on whether the shelf edge is included; if it is determined that the information on the shelf edge is included in the image data, then judging whether all the acquired image data includes all the commodity information according to the commodity information, and obtaining second guiding information indicating a new image acquisition path or third guiding information indicating the end of acquisition according to the judgment result.

[0008] According to a third aspect of an embodiment of the present invention, there is provided a method for collecting shelf images, which includes: displaying first acquisition prompt information for shelf commodities, wherein the first acquisition prompt information is used to indicate an acquisition position when collecting images of shelf commodities along an image acquisition path; acquiring an image collected according to the first acquisition prompt information and identifying the acquired image; if the identification result indicates that the shelf edge is included in the image, then displaying second acquisition prompt information for indicating a new image acquisition path and indicating to continue image acquisition.

[0009] According to a fourth aspect of an embodiment of the present invention, there is provided a client, which includes: a display interface for displaying first acquisition prompt information for indicating image acquisition of a target object along an image acquisition path; the display interface is further used to display second acquisition prompt information, which is information for indicating image acquisition of the target object along a new image acquisition path when the edge of the target object is included in the acquired image.

[0010] According to a fifth aspect of an embodiment of the present invention, there is provided a method for processing commodity information, which includes: collecting image data of a shelf; processing the image data to identify the commodity information on the shelf; determining the commodity statistics information of the shelf according to the identified commodity information.

[0011] According to a sixth aspect of an embodiment of the present invention, there is provided a method for processing commodity information, which includes: in response to a shooting operation initiated by a user, calling an image acquisition device of a client to shoot image data of a shelf; processing the image data to identify the commodity information on the shelf; determining the commodity statistics information of the shelf according to the identified commodity information.

[0012] According to a seventh aspect of an embodiment of the present invention, there is provided a method for processing commodity replenishment, which includes: in response to a replenishment operation initiated by a user, invoking an image acquisition device to capture image data of a shelf; performing recognition processing on the image data to recognize commodity information on the shelf; and determining commodities to be replenished according to the commodity information on the shelf.

[0013] According to an eighth aspect of an embodiment of the present invention, there is provided an image acquisition method, which includes: obtaining a detection result of performing real-time edge detection of a target object on a captured image, where partial image information of the target object is included in the captured image; if the detection result indicates that the edge of the target object is detected in the image, acquiring attitude data of an image acquisition device that captured the image; and generating corresponding guiding information according to the attitude data, and guiding a user to perform continuous image acquisition of the target object through the guiding information, so as to form complete image information of the target object by using a plurality of captured images.

[0014] According to a ninth aspect of an embodiment of the present invention, there is provided an image acquisition method, which includes: during the process of performing image acquisition on a target object, acquiring attitude data of an image acquisition device; and generating corresponding guiding information according to the attitude data, and guiding a user to perform continuous image acquisition of the target object through the guiding information.

[0015] According to a tenth aspect of an embodiment of the present invention, there is provided an image acquisition device, which includes: a detection module, configured to obtain a detection result of performing real-time edge detection of a target object on a captured image, where partial image information of the target object is included in the captured image; a first acquisition module, configured to, if the detection result indicates that the edge of the target object is detected in the image, acquire attitude data of an image acquisition device that captured the image; and a generation module, configured to generate corresponding guiding information according to the attitude data, and guide a user to perform continuous image acquisition of the target object through the guiding information, so as to form complete image information of the target object by using a plurality of captured images.

[0016] According to an eleventh aspect of an embodiment of the present invention, there is provided an electronic device, which includes: a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute an operation corresponding to the method according to any one of the first aspect to the third aspect and the fifth aspect to the ninth aspect.

[0017] According to a twelfth aspect of the embodiments of the present invention, there is provided a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any one of the first aspect to the third aspect and the fifth aspect to the ninth aspect is implemented. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a flowchart of the steps of an image acquisition method according to Embodiment 1 of the present invention;

[0020] Figure 2 It is a flowchart of the steps of an image acquisition method according to Embodiment 2 of the present invention;

[0021] Figure 3 It is a flowchart of the steps of an image acquisition method according to Embodiment 3 of the present invention;

[0022] Figure 4 It is a flowchart of the steps of an image acquisition method according to Embodiment 4 of the present invention;

[0023] Figure 5a It is a flowchart of the steps of Use Scenario 1 of the present invention;

[0024] Figure 5b It is a flowchart of the steps of Use Scenario 2 of the present invention;

[0025] Figure 5c It is a schematic diagram of the segmentation path of Use Scenario 2 of the present invention;

[0026] Figure 5d It is a schematic diagram of the shooting interface of Use Scenario 2 of the present invention;

[0027] Figure 5e It is a flowchart of the steps of Use Scenario 3 of the present invention;

[0028] Figure 5f It is a flowchart of the steps of Use Scenario 4 of the present invention;

[0029] Figure 5g It is a schematic diagram of the display interface of the client in Use Scenario 5 of the present invention;

[0030] Figure 5h It is a flowchart of the steps of Use Scenario 6 of the present invention;

[0031] Figure 5iIt is the flowchart of the steps of the seventh usage scenario of the present invention;

[0032] Figure 5j It is the flowchart of the steps of the eighth usage scenario of the present invention;

[0033] Figure 5k It is the information interaction diagram of the user, the image acquisition device and the server in the ninth usage scenario of the present invention;

[0034] Figure 6 It is the flowchart of the steps of an image acquisition method according to Embodiment 5 of the present invention;

[0035] Figure 7 It is the flowchart of the steps of an image acquisition method according to Embodiment 6 of the present invention;

[0036] Figure 8 It is the structural block diagram of an image acquisition device according to Embodiment 7 of the present invention;

[0037] Figure 9 It is the structural block diagram of an image acquisition device according to Embodiment 8 of the present invention;

[0038] Figure 10 It is the structural schematic diagram of an electronic device according to Embodiment 9 of the present invention. Specific embodiments

[0039] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art shall fall within the scope of protection of the embodiments of the present invention.

[0040] The following further illustrates the specific implementation of the embodiments of the present invention with reference to the accompanying drawings of the embodiments of the present invention.

[0041] Embodiment 1

[0042] Refer to Figure 1 , which shows the flowchart of the steps of an image acquisition method according to Embodiment 1 of the present invention.

[0043] The image acquisition method of this embodiment includes the following steps:

[0044] Step S102: Obtain the detection result of real-time target object edge detection for the acquired image.

[0045] Among them, the acquired image contains partial image information of the target object, and the target object edge detection is used to detect whether the acquired image contains the edge of the target object.

[0046] The detection of the edge of the target object can be performed on the client side, or can be performed on the server side and then the detection result is sent to the client. It can be implemented by using any suitable model or algorithm or other means. For example, a trained neural network model capable of performing edge detection on the target object, such as a Convolutional Neural Network (CNN), is used to perform edge detection on the collected image.

[0047] Again, for example, a feature extraction algorithm is used to extract features from the collected image, and based on the extracted features, it is determined whether the edge of the target object is included in the image, and then a detection result is generated.

[0048] According to the detection result, it can be judged whether the user has collected an image of the edge of the target object, and then it provides a reference for generating appropriate guiding information in the subsequent stage to guide the user, avoid incorrect operations when the user collects the image, and ensure that complete image information can be collected.

[0049] For example, if the detection result indicates that the edge of the target object is detected in the image, then step S104 is executed; otherwise, guiding information for instructing the user to continue moving along the current moving direction and take a picture can be directly generated.

[0050] Step S104: If the detection result indicates that the edge of the target object is detected in the image, then obtain the attitude data of the image acquisition device that collected the image.

[0051] The attitude data of the image acquisition device is used to characterize its current state. The attitude data of the image acquisition device includes, but is not limited to, acceleration information and / or angular velocity information in a spatial coordinate system. For example, it is determined that the image acquisition device is currently in an up-tilt state at a 45-degree angle through the acceleration information and / or angular velocity information, and so on.

[0052] According to this attitude data, it can be determined whether the user has the intention to continue taking pictures of different positions of the target object, and then the generated guiding information matches this intention to guide the user to perform accurate image acquisition and ensure that complete image information of the target object can be collected.

[0053] Step S106: Generate corresponding guiding information according to the attitude data, and guide the user to perform continuous image acquisition of the target object through the guiding information, so as to form complete image information of the target object using the collected multiple images.

[0054] In a specific implementation manner, a correspondence relationship between attitude data and guiding information or guiding keywords can be set in the image acquisition device. Accordingly, corresponding guiding information can be determined and generated based on the attitude data, or corresponding guiding keywords can be determined based on the attitude data, and then corresponding guiding information can be generated based on the guiding keywords. For example, if the attitude data corresponds to the guiding keyword "move up", guiding information such as "Please move up one grid for shooting" can be generated. Through the guiding information, users can be effectively guided to perform continuous image acquisition, so that a complete image information of the target object can be formed based on multiple acquired images subsequently.

[0055] It should be noted that in the process of obtaining the complete image information, steps S102 to S106 can be repeatedly executed multiple times until it is determined according to the attitude data of the image acquisition device obtained in step S104 that the user has no intention of continuous acquisition, and guiding information indicating the end of image acquisition for the user can be generated. After the acquisition is completed, a complete image information of the target object can be formed using the multiple acquired images.

[0056] It should be noted that for the case where the target object is a shelf, the image acquisition method in this embodiment is particularly applicable to the usage scenario where the goods on the shelf are not placed in a standard manner. For example, in a small retail store shelf, such shelves have problems such as being placed closely and the goods being messy and not standardized. The image acquisition method of this embodiment can overcome these problems, achieve the acquisition of a complete image of the shelf, and can obtain clear product information, so that the products on the shelf can be determined by recognizing the complete image subsequently.

[0057] For large shopping mall scenarios or large-scale shooting scenarios, drones may be used, but the technical means of drone shooting are not applicable to the usage scenario of this application, and it is also difficult to transfer its technical means to the usage scenario of this application. Similarly, the technical means in the electronic price tag scenario are also difficult to implement in this usage scenario.

[0058] Through this embodiment, edge detection of the target object is performed on the acquired images in real time. When the edge of the target object is included in the acquired images, the attitude data of the image acquisition device is obtained, and then corresponding guiding information is generated based on the attitude data, so as to guide users to perform image acquisition in a standard manner through the guiding information, achieve the final completion of the image acquisition of multiple parts included in the entire target object, avoid omission, and obtain a complete image of the target object.

[0059] The image acquisition method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones, etc.) and PC machines, etc.

[0060] Embodiment 2

[0061] Reference Figure 2 shows a flowchart of steps of an image acquisition method according to Embodiment 2 of the present invention.

[0062] The image acquisition method of this embodiment includes the aforementioned steps S102 to S106.

[0063] Wherein, before step S102, the method further includes:

[0064] Step S100: Obtain a lightweight neural network model that is dynamically sent to the image acquisition device and used for edge detection of the target object.

[0065] It should be noted that this step is an optional step. If this step is executed, it can be executed at any appropriate time before step S102.

[0066] In this embodiment, in order to ensure that it can be timely detected whether the acquired image contains the edge of the target object to ensure the accuracy of the generated guidance information, the image acquisition device locally has a dynamically sent lightweight neural network model. Using the lightweight neural network model, the image can be real-time detected locally on the image acquisition device without transmitting it to the server, thereby greatly improving the detection speed and efficiency.

[0067] The lightweight neural network model is also called a micro neural network model, which refers to a neural network model that requires fewer parameters and less computational cost. Due to its small computational overhead, it can be deployed on image acquisition devices with relatively limited computing resources.

[0068] Specifically, the lightweight neural network model can be a pre-trained lightweight convolutional neural network model. The convolutional neural network model has an input layer, a hidden layer, and an output layer. When training the convolutional neural network model, a large number of pre-collected images containing target objects (such as shelves, large mechanical equipment, large containers, etc.) are used, and these images are labeled, mainly labeling the edges of the target objects, and then using these labeled images to train the convolutional neural network model. So that the trained convolutional neural network model can correctly identify whether the image contains the edge of the shelf. After that, the trained convolutional neural network model can be dynamically sent to the image acquisition device.

[0069] In the form of a lightweight neural network model, on the premise of ensuring that the target object edge detection of the image can be accurately performed, the requirements for computing power and storage space are reduced, so that this solution can adapt to more image acquisition devices, especially small image acquisition devices, such as mobile terminal devices such as mobile phones and tablets.

[0070] In this embodiment, when step S100 is executed, step S102 can be implemented as: using the lightweight neural network model to perform real-time edge detection of the target object on the acquired image to obtain a detection result. Whether the current image contains the edge of the target object is indicated by the detection result.

[0071] Thus, it is possible to quickly, efficiently, and accurately perform edge detection of the target object locally on the image acquisition device, thereby ensuring the timeliness of generating the guidance information.

[0072] Through this embodiment, real-time edge detection of the acquired image is performed on the target object. When the acquired image contains the edge of the target object, the attitude data of the image acquisition device is obtained, and then the corresponding guidance information is generated according to the attitude data, so as to guide the user to perform image acquisition in a standardized manner through the guidance information, achieving the purpose of finally completing the image acquisition of multiple parts included in the entire target object, avoiding omissions, and obtaining a complete image of the target object.

[0073] In addition, by dynamically distributing the lightweight neural network model to the image acquisition device, it can perform real-time edge detection of the target object on the acquired image locally. On the premise of ensuring detectability, the timeliness of detection is improved, and thus the timeliness of subsequent guidance information generation is ensured. Compared with the previous method of sending the acquired image to the background server for detection and then returning the detection result, detecting locally on the image acquisition device is not restricted by the network transmission speed, has better reliability, and higher speed and efficiency.

[0074] The image acquisition method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones, etc.), and PC machines, etc.

[0075] Embodiment III

[0076] Refer to Figure 3 , which shows the step flowchart of an image acquisition method according to Embodiment III of the present invention.

[0077] The image acquisition method of this embodiment includes the aforementioned steps S102 to S106.

[0078] Among them, the method may or may not include step S100. When step S100 is included, step S102 can be implemented in the implementation manner of Embodiment II.

[0079] In this embodiment, step S104, that is, obtaining the attitude data of the image acquisition device that acquires the image, can be implemented as: obtaining the acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system.

[0080] For example, a spatial coordinate system includes an X-axis, a Y-axis, and a Z-axis. The acceleration information on these three axes can be obtained through an accelerometer configured in the image acquisition device. The angular velocity information on these three axes can be obtained through a gyroscope configured in the image acquisition device.

[0081] Of course, for image acquisition devices with different structures, different methods can be used to obtain the acceleration information and / or angular velocity information in the spatial coordinate system, and this embodiment does not limit this.

[0082] Based on the acceleration information and / or angular velocity information of the image acquisition device in the spatial coordinate system, it can be determined whether the image acquisition device has an upward or downward tilt, and further, it can be determined whether the user has the intention to take a newline shot.

[0083] In one case, if the image acquisition device maintains a certain inclination angle or quickly tends to the posture of retracting the image acquisition device within a period of time after capturing the edge of the target object, it indicates that the user has captured the complete image information of the target object and has no intention of taking a newline shot, and guiding information for guiding the user to end the image acquisition can be generated.

[0084] In another case, if the image acquisition device has an upward or downward inclination angle change within a period of time after capturing the edge of the target object, it indicates that the user has the intention to take a newline shot. Therefore, step S106 can be executed to generate guiding information for guiding the user to perform continuous image acquisition.

[0085] Optionally, when the foregoing implementation manner is adopted in step S104, step S106 includes the following sub-steps:

[0086] Sub-step S1061: Determine the current posture of the image acquisition device according to the acceleration information and / or angular velocity information.

[0087] According to the acceleration information of the image acquisition device on the X-axis, Y-axis, and Z-axis, it can be determined whether the image acquisition device moves and / or rotates. According to its angular velocity information on the X-axis, Y-axis, and Z-axis, the inclination angle deviating from the horizontal or vertical state can be determined. The current posture of the image acquisition device can be determined according to the acceleration information and / or angular velocity information.

[0088] For example, in the gyroscope, the X-axis points horizontally to the right, the Y-axis points vertically forward, and the Z-axis points directly above the screen of the image acquisition device.

[0089] When it is determined according to the angular velocity information that the image acquisition device has an angular velocity on the X-axis, its tilted state can be determined, and whether the current posture is tilted upward or downward can be determined according to the value of the angular velocity. Similarly, when it is determined according to the acceleration information that there is an acceleration on the Z-axis, it can be determined that the image acquisition device has a tendency to move upward.

[0090] Sub-step S1062: Generate guiding information according to the current posture to instruct the user to move in the direction matching the current posture for continuous image acquisition.

[0091] In the image acquisition device, a correspondence between the current posture and guiding information or guiding keywords can be set. After determining the current posture, the matching guiding information or guiding keywords can be determined according to the current posture and the set correspondence, and then guiding information for moving in the direction matching the current posture for continuous image acquisition can be generated.

[0092] For example, if the current posture is tilted upward and its matching guiding keyword is "move up", guiding information can be generated to instruct the user to move up and perform continuous image acquisition.

[0093] For another example, if the current posture is tilted downward and its matching guiding keyword is "move down", guiding information can be generated to instruct the user to move down and perform continuous image acquisition.

[0094] Of course, guiding information such as "Please move up to take a picture" or "Please move down to take a picture" can also be directly matched.

[0095] From the above, accurate guiding information can be generated to instruct the user to continue taking pictures through the guiding information, so as to ensure that complete image information of the target object can be collected.

[0096] In this embodiment, edge detection of the target object is performed on the collected images in real time. When the edge of the target object is included in the collected images, the posture data of the image acquisition device is obtained, and then corresponding guiding information is generated according to the posture data, so as to guide the user to perform image acquisition in a standardized manner through the guiding information, achieving the purpose of finally completing the image acquisition of multiple parts included in the entire target object, avoiding omission, and obtaining a complete image of the target object.

[0097] The image acquisition method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones, etc.) and PC machines, etc.

[0098] Embodiment Four

[0099] Refer to Figure 4 , which shows a flowchart of the steps of an image acquisition method according to Embodiment Four of the present invention.

[0100] The image acquisition method of this embodiment includes the aforementioned steps S102 to S106.

[0101] Among them, the method may or may not include step S100. When step S100 is included, step S102 may be implemented in the implementation manner of Embodiment 2. Step S104 may be implemented in the implementation manner of Embodiment 3 or other implementation manners. When step S104 adopts the implementation manner of Embodiment 3, step S106 may be implemented in the implementation manner of Embodiment 3.

[0102] In this embodiment, if it is determined according to the attitude data of the image acquisition device obtained in step S104 that the user has no intention of photographing other parts of the target object, then the multiple collected images can be used to form the complete image information of the target object.

[0103] In a feasible manner, using the multiple collected images to form the complete image information of the target object includes: splicing the multiple collected images to obtain a complete image containing the complete image information of the target object.

[0104] Since the sub - sampling method is adopted in the process of image acquisition of the target object, therefore, by splicing the multiple collected images, a complete image containing the complete image information of the target object can be obtained. Moreover, the complete image at the splicing part enables the user to observe the target object more intuitively.

[0105] Specifically, the splicing of the multiple collected images to obtain a complete image containing the complete image information of the target object includes the following steps:

[0106] Step S108: Determine multiple groups of images with an image overlapping relationship from the multiple collected images.

[0107] In this embodiment, each group of images includes two images. The overlapping relationship between the images means that these two images are adjacent in spatial position. Therefore, the relative position relationship between the images can be inferred based on this overlapping relationship, and then the images can be spliced according to the relative position relationship. Using this splicing method can not only ensure the accuracy of splicing but also achieve fast splicing.

[0108] One way to determine multiple groups of images with an image overlapping relationship is: extract features from each of the multiple collected images to obtain the feature points corresponding to each image; for any two images, match the feature points of the two images, and determine the multiple groups of images with the image overlapping relationship based on the matching result.

[0109] Feature extraction of images can be performed using any suitable algorithm such as the HOG (Histogram of Oriented Gradient) feature extraction algorithm, the LBP (Local Binary Pattern) feature extraction algorithm, or the Haar-like feature extraction algorithm.

[0110] When matching any two images based on the feature points of the two images, the matching result can be determined by comparing whether the similarity of the two images meets a certain threshold (specifically, for example, calculating the distance between feature points to determine the similarity). If the distance between the feature points of the two images is less than the preset value, the matching result indicates that the two images have an overlapping relationship; conversely, if the distance between the feature points of the two images is greater than or equal to the set value, the matching result indicates that the two images do not have an overlapping relationship.

[0111] Using this method can accurately judge the images with an overlapping relationship, thus ensuring the accuracy of stitching.

[0112] Step S110: Stitch the multiple collected images according to the image overlapping relationship, and obtain a complete image containing the complete image information of the target object based on the stitching result.

[0113] In a specific implementation, according to the overlapping relationship between images, determine the adjacent images of each image, and determine the relative position relationship according to the position of the overlapping part in two adjacent images, and then stitch the multiple images to obtain a complete image. The complete image contains the complete image information of the target object.

[0114] For example, if it is determined according to the overlapping relationship that image A and image B have an overlapping part, and the overlapping part is located on the left side of image A and the right side of image B, then image A can be stitched on the right side of image B.

[0115] For another example, if there is an overlapping part between the upper side of image A and the lower side of image C, then image C is stitched above image A.

[0116] Subsequently, the user can analyze and / or process the complete image as needed to obtain the required information. For example, if the target object is a shelf, by analyzing the complete image of the shelf, it can be determined whether a certain type of product is placed in a convenient position for picking up, etc.

[0117] It should be noted that the stitching of images can also be completed by the server. For example, the image acquisition device uploads the multiple collected images to the background server, and the server performs corresponding recognition and stitching operations, and then sends the completed complete image back to the image acquisition device to reduce the data processing burden of the image acquisition device.

[0118] Through this embodiment, the edge of the target object in the captured image is detected in real time. When the edge of the target object is included in the captured image, the attitude data of the image acquisition device is obtained, and then the corresponding guiding information is generated according to the attitude data, so as to guide the user to perform image acquisition in a standardized manner through the guiding information, and finally complete the image acquisition of multiple parts included in the entire target object, avoid omission, and obtain a complete image of the target object.

[0119] In addition, by dynamically distributing the lightweight neural network model to the image acquisition device, it can perform real-time edge detection of the target object on the captured image locally. On the premise of ensuring detectability, the timeliness of detection is improved, and then the timeliness and accuracy of subsequent guiding information generation are ensured. Compared with the previous method of sending the captured image to the backend server for detection by the server and then returning the detection result, detecting locally on the image acquisition device is not restricted by the network transmission speed and has better reliability.

[0120] In addition, by splicing multiple captured images to form a complete image containing complete image information, users can observe the target object more directly, and can analyze and process the complete image as needed to obtain the required analysis results.

[0121] The image acquisition method of this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablets, mobile phones, etc.) and PC machines, etc.

[0122] Usage scenario 1:

[0123] Refer to Figure 5a , which shows the flowchart of the steps of the image acquisition method in this usage scenario 1.

[0124] In this usage scenario, taking the image acquisition device as a mobile phone and the target object as a shelf as an example, the image acquisition method is described. Specifically, the image acquisition method includes the following steps:

[0125] Step A1: The user starts shooting the shelf by taking pictures.

[0126] In this scenario, the user takes pictures of the shelf by taking one image at a time. Since only a part of the shelf can be captured at a time, multiple shots are required. And for an image captured at a certain time, there is a certain image overlap between it and the image captured previously, the image captured later, the image at the corresponding position above the shelf, and the image at the corresponding position below the shelf. Optionally, the overlap can be set to be greater than or equal to 20% to ensure the effective recognition and splicing of subsequent images.

[0127] Step B1: During the process of photographing the shelf, perform real-time shelf edge detection on the captured image. If a shelf edge is detected in the image, proceed to Step C1; if no shelf edge is detected in the image, directly generate guiding information to prompt the user to continue photographing, and repeat Step B1.

[0128] Step C1: Calculate the acceleration and angular velocity of the mobile phone on the spatial coordinate system (i.e., the X-axis, Y-axis, and Z-axis) respectively through the mobile phone's accelerometer and gyroscope, and determine whether the mobile phone has an upward or downward angle based on the calculation results, so as to analyze whether the user intends to continue photographing other parts of the shelf.

[0129] If there is no intention to continue photographing, execute Step D1; if there is an intention to continue photographing, execute Step E1.

[0130] Step D1: If the mobile phone maintains a certain angle and there is almost no change in direction, it indicates that the user has finished photographing the entire section of the shelf and has no intention to continue photographing. Therefore, generate guiding information indicating the end of photographing, and this guiding information can be displayed on the mobile phone screen to guide the user. After Step D1 is completed, execute Step F1.

[0131] Step E1: If the mobile phone suddenly changes its direction to shoot upward or downward, it indicates that the user intends to photograph the upper or lower part of the shelf in a new row. Therefore, generate guiding information indicating that the user should move upward or downward and continue photographing, and this guiding information can be displayed on the mobile phone screen to guide the user.

[0132] Optionally, after generating the guiding information, the attitude data of the image acquisition device can be obtained again, and it can be determined whether the user has operated according to the instructions of the guiding information based on the attitude data. If the user has not operated according to the instructions of the guiding information, a warning message can be generated to prompt the user; if the user has operated according to the instructions of the guiding information, no action is required.

[0133] After detecting the newly captured image, return to Step B1 and continue to execute.

[0134] Step F1: End the photographing and splice multiple shelf images captured by the user to generate a complete image containing an entire section of the shelf.

[0135] Through this process, perform shelf edge detection on the captured image, analyze whether the user has photographed the shelf edge based on the detection results, and if the shelf edge is photographed, analyze whether the user intends to photograph the remaining parts of the shelf through the acceleration sensor and gyroscope on the mobile phone, so as to better guide the user to photograph a complete section of the shelf according to the analyzed intention, ensuring the photographing quality and guaranteeing the acquisition of complete image information of the shelf.

[0136] Usage Scenario 2

[0137] Refer to Figure 5b , which shows the flowchart of the steps of the shelf image acquisition method in this usage scenario.

[0138] In this usage scenario, the image acquisition device can be a mobile phone, a pad, a camera, etc. Through this method, a complete image of the shelf can be obtained, and then the commodity information can be analyzed to prompt replenishment or adjust the placement position of the commodity according to the commodity information.

[0139] Specifically, taking the image acquisition device as a mobile phone and the target object as a shelf as an example, the shelf image acquisition process includes:

[0140] Step A2: Obtain a shelf image collected according to the indication of the first guiding information.

[0141] Among them, the shelf is used to carry commodities. The shelf can be a shelf for displaying commodities in places such as shopping malls and supermarkets, or a shelf for placing commodities in a warehouse. The shelf image contains partial information of the shelf.

[0142] The first guiding information is used to indicate the image acquisition path of the shelf. The image acquisition path is a path generated by dividing the shelf according to the shelf structure information, and the shelf structure information is determined according to at least one of the overall floor plan, three-dimensional diagram, and preset shelf virtual model of the shelf.

[0143] Among them, the shelf structure information can be the overall image of the shelf taken in advance by the user. Since the overall image taken in advance is only used to obtain the shelf structure information, one or several overall images of the shelf from different perspectives can be taken, so that the server or the client can analyze the shelf structure information of the shelf from the overall image and generate an image acquisition path through the shelf structure information.

[0144] Of course, other methods can also be used to obtain the shelf structure information, such as pre-establishing virtual models of different specifications of shelves. The user can pre-select the virtual model of the shelf for which the image needs to be collected and generate an image acquisition path according to the virtual model.

[0145] The specific implementation of the analysis of the shelf structure can be realized by those skilled in the art in any appropriate way or algorithm according to actual needs, including but not limited to connected component analysis, using a neural network model for analysis, etc. The first guiding information can be generated locally or obtained by the client from the server after being generated by the server.

[0146] For example, the schematic diagram of the image acquisition path is as shown in 5c. The dotted line indicated by 001 in the figure is the segmentation path for dividing the shelf. Generating the segmentation path based on the shelf structure information can be implemented by the server or locally on the image acquisition device. When generating the image acquisition path, different segmentation paths can be generated according to the structure of the same shelf. The specific segmentation strategy can be preset in the server or the client, such as using straight-line segmentation, S-shaped segmentation, U-shaped segmentation, rectangular segmentation, spiral segmentation, etc. Or a neural network model that has been trained can output the segmentation path according to the shelf structure information.

[0147] The line indicated by 002 in the figure is the image acquisition path corresponding to the first guiding information in the segmentation path. Usually, the image acquisition path can be part or all of the segmentation path. The area indicated by 003 in the figure is the shooting area for one image acquisition of the image acquisition device, and this area covers at least part of the image acquisition path. The shooting areas corresponding to two adjacent acquisitions partially overlap.

[0148] Based on the first guiding information, the user can shoot the corresponding part of the shelf along the corresponding path according to its guiding indication and obtain the corresponding shelf image.

[0149] Step B2: Obtain the edge detection result of performing shelf edge detection on the shelf image.

[0150] After obtaining the currently captured shelf image, shelf edge detection can be performed on this shelf image. This detection can be executed locally on the image acquisition device to directly obtain the edge detection result; or the shelf image can be sent to the server, and the server performs shelf edge detection and sends the edge detection result to the image acquisition device.

[0151] If shelf edge detection is performed locally on the image acquisition device, a lightweight neural network model trained for performing shelf edge detection can be used for detection to reduce the amount of calculation and ensure that the computing power of the image acquisition device can meet the detection requirements.

[0152] If shelf edge detection is performed on the server, a neural network model with a relatively deep depth trained for performing shelf edge detection can be used for detection to improve the detection accuracy.

[0153] If the obtained edge detection result indicates that the shelf image includes a shelf edge, then step C2 is executed; otherwise, fourth guiding information indicating moving a certain distance along the image acquisition path to continue shooting is generated.

[0154] Step C2: If the edge detection result indicates that the shelf image includes a shelf edge, then obtain second guiding information indicating a new image acquisition path or obtain third guiding information indicating the end of acquisition.

[0155] Among them, the new image acquisition path can be part of the segmentation path, which can be determined according to the actual detection results and the previous path segmentation results. For example, the new image acquisition path is the path indicated by the dotted line in the lower part of Figure 5c In the subsequent steps, the user can move the image acquisition device to the position corresponding to the new image acquisition path at the viewfinder position (such as the dotted line shooting area in Figure 5c ) and continue shooting.

[0156] A feasible implementation of step C2 includes: if the edge detection result indicates that the shelf image includes a shelf edge, identify commodity information from the acquisition result image generated from all the acquired shelf images, and obtain a commodity information result; according to the commodity information result, obtain second guidance information indicating a new image acquisition path or obtain third guidance information indicating the end of acquisition.

[0157] Among them, the acquisition result image generated from all the acquired shelf images can be generated locally by the image acquisition device, or after each shelf image is acquired, the shelf image is sent to the server, and the server generates the acquisition result image and sends it to the image acquisition device.

[0158] The process of generating the acquisition result image locally by the image acquisition device can be: obtaining the acquisition result image generated by stitching all the acquired shelf images.

[0159] For example, the image acquisition device overlays the overlapping parts of two images according to the overlapping parts in the shelf images to form the stitched acquisition result image. For example, if the right side of image 1 coincides with the left side of image 2, the overlapping parts of image 1 and image 2 are overlaid to form the acquisition result image.

[0160] For the convenience of the user to view, a preview box (such as shown at 005 in Figure 5d ) can be configured in the display interface, and the stitched acquisition result image is displayed in the preview box.

[0161] The obtaining of the second guidance information indicating a new image acquisition path or the third guidance information indicating the end of acquisition according to the commodity information result includes: if the commodity information result indicates that all the commodities on the shelf are not included in the acquisition result image, obtain the second guidance information indicating to switch the shooting row in the shooting path; or, if the commodity information result indicates that all the commodities on the shelf are included in the acquisition result image, obtain the third guidance information indicating the end of shooting.

[0162] This method can generate accurate guiding information to guide the user to collect the shelf image multiple times, ensuring that the shelf image collected each time can include clear product information for identification, thereby solving the problem that the existing shelf is too long, and obtaining the overall image of the shelf through one-time collection will make the product information too small to be recognized.

[0163] Another feasible implementation method of step C2 includes: if the edge detection result indicates that the shelf edge is included in the shelf image, obtain the attitude data of the image acquisition device; obtain the second guiding information indicating a new image acquisition path or the third guiding information indicating the end of the acquisition according to the attitude data.

[0164] Among them, the attitude data includes the acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system. According to the attitude data, it can be determined whether the user has the intention to continue image acquisition. Furthermore, when there is the intention to continue image acquisition, the intended shooting direction can be determined, and then the corresponding second guiding information can be generated; when there is no intention to continue image acquisition, the third guiding information can be generated.

[0165] This method can also generate accurate guiding information to ensure that a shelf image including recognizable product information can be obtained, thereby ensuring that corresponding processing can be performed according to the recognized product information in the subsequent steps.

[0166] Optionally, during the image acquisition process, before other acquisitions after the first acquisition, it can also be executed: obtain the reserved area corresponding to the current image acquisition path from the latest acquired shelf image and display the reserved area in the set area of the display interface, so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area.

[0167] By displaying the reserved area in the set area, it is convenient for the user to align the reserved area with the corresponding position of the shelf during the next image acquisition, so as to ensure that the overlapping part of the shelf images collected by the user in two adjacent acquisitions is sufficient for splicing, and at the same time prevent too much overlapping part from resulting in too many acquisitions required to collect the complete information of the shelf.

[0168] The reserved area is as Figure 5d shown at 006. The reserved area in the latest acquired shelf image and the set area in the display interface are both determined according to the image acquisition path.

[0169] For example, for the latest captured shelf image, if the subsequent shelf image is obtained by moving a certain distance to the right along the image capture path, the retained area is the rightmost partial area of the latest captured shelf image, and the area of this partial area can be 1 / 6 to 1 / 5 of the total area. Correspondingly, the set area is the leftmost partial area of the display interface.

[0170] Again, for example, for the latest captured shelf image, if the subsequent shelf image is obtained by moving down to a new image capture path, the retained area is the lowermost partial area of the latest captured shelf image, and the area of this partial area can be 1 / 6 to 1 / 5 of the total area. Correspondingly, the set area is the uppermost partial area of the display interface.

[0171] In this way, when the user performs the next image capture operation, the retained area in the display interface can be aligned with the corresponding area in the actual shelf, so that there is enough overlapping part between the captured goods image and the previous goods image to determine the positional relationship between the two goods images, and at the same time, it will not cause too much overlapping part, resulting in a large amount of useless data.

[0172] Optionally, at any appropriate time after obtaining the captured result image, the recognition step can also be performed, that is: perform commodity information recognition and / or commodity position recognition on the captured result image, and obtain a commodity information result and / or a commodity position result.

[0173] For example, after each image capture operation, this recognition step can be performed after generating the captured result image based on the captured shelf image, or this recognition step can be performed after obtaining the captured result image containing the complete information of the shelf.

[0174] Preferably, in order to improve efficiency, this recognition step is performed after obtaining the captured result image containing the complete information of the shelf.

[0175] Among them, the commodity information recognition can be performed on the server side or locally.

[0176] When performed on the server side, the server obtains the captured result image sent by the image capture device, or obtains the shelf image sent by the image capture device and splices it to form the captured result image, and then uses the trained neural network model capable of performing commodity information recognition to perform recognition and obtain the commodity information result.

[0177] When performed locally, the image capture device can directly splice the captured result image according to the shelf image or obtain the captured result image from the server, and use the trained neural network model capable of performing commodity information recognition to perform recognition and obtain the commodity information result.

[0178] Similarly, the identification of the commodity position can be performed on the server side or locally. When identifying, a trained neural network model capable of identifying the commodity position can be used for identification to obtain the commodity position result.

[0179] Optionally, after performing the identification of commodity information and / or the identification of the commodity position on the acquired result image, and obtaining the commodity information result and / or the commodity position result, the following operations can also be performed: analyzing the commodity information result and / or the commodity position result, and generating an analysis result corresponding to the analysis operation.

[0180] The analysis result includes at least one of the following: commodity sales information, commodity display information, commodity quantity information, and commodity replenishment status information.

[0181] In order to obtain different analysis results, different analysis operations can be performed.

[0182] For example, if the analysis result includes commodity sales information, the commodity information result and the commodity position result can be analyzed to determine the remaining commodities on the shelf and the commodities at each placement position on the shelf, and then the commodities at the vacant positions can be analyzed to determine the commodity sales information.

[0183] For another example, if the analysis result includes commodity display information, the commodity information result can be analyzed to determine the commodities at each placement position on the shelf, so as to determine the commodity display information.

[0184] For still another example, if the analysis result includes commodity quantity information, the commodity information result and the commodity position result can be analyzed to determine the commodities at each placement position, and the commodity quantity information can be determined according to the quantity at the placement positions.

[0185] For still another example, if the analysis result includes commodity replenishment status information, the commodity information result and the commodity position result can be analyzed to determine the commodities to be replenished and the corresponding replenishment positions.

[0186] Optionally, when receiving the user's termination collection operation, the following operations can also be performed: determining whether the acquisition result image generated from all the acquired shelf images contains all the commodities on the shelf.

[0187] Since the user may encounter an unexpected situation during the image acquisition process and terminate the image acquisition, when obtaining the operation indicating the termination of the image acquisition by the user (such as an exit operation or an end collection operation), it is determined whether the shelf has been completely acquired, that is, whether the acquisition result image generated from all the acquired shelf images contains all the commodities on the shelf.

[0188] If the acquisition result image contains all the commodities on the shelf, it means that the acquisition is complete, and the acquisition result image can be saved and the acquisition can be terminated.

[0189] If the collected result image does not contain all the products on the shelf, it means that the collection is not complete. The collected result image and related collection information (such as the image collection path, etc.) can be saved, and a prompt message indicating the un-uploaded (or un-updated) part can be given to inform the user that they can continue the image collection at an appropriate time.

[0190] In this way, the collected shelf images can be fully utilized, analyzed timely and intelligently, and corresponding processing can be carried out according to the analysis results as needed. For example, if it is determined from the analysis results that replenishment is required, a replenishment reminder can be generated, and the types of products to be replenished can be indicated, etc.

[0191] Usage Scenario Three

[0192] Refer to Figure 5e , which shows the flowchart of the steps of the product information processing method in this Usage Scenario Three.

[0193] In this usage scenario, taking the image collection device as a mobile phone as an example, the method includes the following steps:

[0194] Step A3: Collect the image data of the shelf according to the obtained first guiding information.

[0195] Among them, the first guiding information is used to indicate the image collection path of the shelf. The first guiding information can be generated in the manner described in Usage Scenario Two, or generated in other ways, and this usage scenario does not make any limitations in this regard.

[0196] Step B3: Identify the image data and obtain the product information on the shelf and the information on whether the shelf edge is included.

[0197] The identification can include product information identification and shelf edge identification. The product information identification can be carried out by using the neural network model capable of product information identification in Usage Scenario Two, or by using other methods. The shelf edge identification can be carried out by using the neural network model capable of shelf edge identification, or by using other methods.

[0198] If the information indicates that the image data contains the shelf edge, then step C3 is executed; otherwise, no action may be taken or guiding information indicating to continue the collection along the image collection path may be generated.

[0199] Step C3: If it is determined that the image data contains the information of the shelf edge, then judge whether all the product information is included in all the collected image data according to the product information, and obtain the second guiding information indicating a new image collection path or the third guiding information indicating the end of the collection according to the judgment result.

[0200] When determining whether all the collected image data contains all the product information, the number of product categories can be determined based on the product information. If the number of product categories meets the requirements, the judgment result is that all products are included, and the third guiding information indicating the end of the collection is obtained according to the judgment result; if the number of product categories does not meet the requirements, the judgment result is that not all products are included, and the second guiding information indicating a new image collection path is obtained according to the judgment result.

[0201] Subsequently, the method described above can also perform other steps according to the product information, such as generating a replenishment reminder message, etc. Through the product information processing method in this usage scenario, the processing of product information on the shelf can be realized, so as to meet the requirements such as replenishment reminder and change of product position reminder.

[0202] Usage Scenario Four

[0203] Refer to Figure 5f , which shows the flowchart of the steps of the shelf image collection method in this Usage Scenario Four.

[0204] In this usage scenario, the image collection device is a mobile phone, and the method includes the following steps:

[0205] Step A4: Display the first collection prompt message for the shelf products.

[0206] Among them, the first collection prompt message is used to indicate the collection position when collecting images of the shelf products along the image collection path. The first collection prompt message can be determined according to the image collection path indicated by the first guiding information. For example, for the most recent collection position, a certain distance is moved on the image collection path to determine a new collection position, and the first collection prompt message is generated according to the new collection position. The first guiding information can be the first guiding information described in Usage Scenario Two.

[0207] Step B4: Obtain the image collected according to the first collection prompt message, and identify the obtained image.

[0208] According to different needs, different identifications can be performed on the obtained image. For example, shelf edge identification, product information identification, etc. The specific identification method can be the method described in the previous usage scenario, so it will not be elaborated here.

[0209] If the recognition result indicates that the image includes the shelf edge, then step C4 is executed; otherwise, the first collection prompt message is updated according to the image collection path, and the process returns to step A4 to continue execution.

[0210] Step C4: If the recognition result indicates that the image includes the shelf edge, then display the second collection prompt message for indicating a new image collection path and indicating to continue image collection.

[0211] If the shelf edge is included, determine a new image acquisition path and generate second acquisition prompt information indicating it to prompt the user to switch the image acquisition path for continued acquisition. The process of determining the new image acquisition path can be the same as the foregoing usage scenario and will not be elaborated herein.

[0212] Through the shelf image acquisition method of this usage scenario, a complete and accurate shelf image and the commodity information on the shelf can be obtained, so that the commodity information can be analyzed for replenishment prompting, changing commodity position prompting, etc.

[0213] Usage scenario five

[0214] Refer to Figure 5g , which shows a schematic structural diagram of the display interface of the client in this usage scenario five.

[0215] In this usage scenario, the client includes a display interface. The display interface is used to display first acquisition prompt information for instructing image acquisition of a target object along an image acquisition path; the display interface is also used to display second acquisition prompt information (such as Figure 5g at 007 in ), and the second acquisition prompt information is information for instructing image acquisition of the target object along a new image acquisition path when the edge of the target object is included in the acquired image.

[0216] The first acquisition prompt information can be generated in the manner described in usage scenario four and displayed through the display interface. The second acquisition prompt information can be determined according to the new image acquisition path. For example, for the most recent acquisition position, move a certain distance on the new image acquisition path to determine a new acquisition position, generate second acquisition prompt information according to the new acquisition position, and display it through the display interface.

[0217] Through this client, the first acquisition prompt information and the second acquisition prompt information can be displayed, thereby prompting the user to perform image acquisition to improve the quality of the acquired image and enable acquisition of a complete image of the high-quality target object.

[0218] Optionally, the target object includes at least one of the following: a shelf, a parking lot, and seats in a venue. For a parking lot, through this method, a complete image of the parking lot can be acquired, and then the vehicle information therein can be analyzed. For the seats in a venue, through this method, a complete image thereof can be acquired, and then the seat usage situation can be analyzed, and then the attendance rate, etc. can be calculated. For a shelf, through this method, a complete image of the shelf can be acquired, and then the vehicle information, seat information, commodity information, etc. therein can be analyzed, so that subsequent processing can be performed.

[0219] Usage scenario six

[0220] Refer toFigure 5h , which shows the schematic flow chart of the commodity information processing method in the sixth usage scenario of the present invention.

[0221] In this usage scenario, the method includes:

[0222] Step A5: Collect image data of the shelf.

[0223] The image data of the shelf can be collected by an image acquisition device, and the image acquisition device can be a mobile phone or the like. The collection of the image data of the shelf can be implemented by any one of the aforementioned usage scenarios one to five.

[0224] Step B5: Process the image data to identify the commodity information on the shelf.

[0225] According to different requirements, the image data can be processed differently. For example, a trained neural network model capable of identifying commodity information in images can be used to process the images and obtain the commodity information on the shelf. The commodity information can include commodity name information, category information, etc.

[0226] Step C5: Determine the commodity statistics information of the shelf according to the identified commodity information.

[0227] The commodity statistics information can include commodity quantity information, commodity category quantity, quantity of each category of commodities, etc.

[0228] Through this method, high-quality image data containing all the information of the shelf can be obtained, and then the image data can be analyzed to obtain the commodity information, so as to obtain the commodity statistics information, which is convenient for subsequent replenishment prompts and the like according to the commodity statistics information.

[0229] Usage Scenario Seven

[0230] Refer to Figure 5i , which shows the schematic flow chart of the commodity information processing method in the seventh usage scenario of the present invention.

[0231] In this usage scenario, the method includes:

[0232] Step A6: In response to a shooting operation initiated by the user, call the image acquisition device of the client to shoot the image data of the shelf.

[0233] The user can call the image acquisition device of the client through the server, or directly initiate a shooting operation on the client and call the image acquisition device.

[0234] The method of obtaining the image data can adopt any one of the methods described in usage scenarios one to five, and this usage scenario does not make any limitations in this regard.

[0235] Step B6: Process the image data to identify the product information on the shelf.

[0236] The method of obtaining product information can be the same as or different from that in the aforementioned Use Case 6.

[0237] Step C6: Determine the product statistics information of the shelf based on the identified product information.

[0238] The product statistics information can include product quantity information, number of product categories, quantity of each category of products, etc.

[0239] Through this method, high-quality image data containing all the information of the shelf can be obtained. Furthermore, the image data can be analyzed to obtain product information, so as to obtain product statistics information, which is convenient for subsequent replenishment reminder, etc. based on the product statistics information.

[0240] Use Case 8

[0241] Refer to Figure 5j , which shows a schematic flowchart of the steps of the product replenishment processing method in this Use Case 8.

[0242] In this use case, the method includes:

[0243] Step A7: In response to a replenishment operation initiated by the user, call an image acquisition device to capture image data of the shelf.

[0244] The user can initiate a replenishment operation through the client. The client directly calls the image acquisition device to capture image data of the shelf, or the client sends the replenishment operation to the server, and the server calls the image acquisition device to capture image data of the shelf.

[0245] The method of capturing image data can be any of the methods in Use Cases 1 to 5.

[0246] Step B7: Perform recognition processing on the image data to identify the product information on the shelf.

[0247] The method of obtaining product information can be the same as or different from that in the aforementioned Use Case 6.

[0248] Step C7: Determine the products to be replenished based on the product information on the shelf.

[0249] For example, based on the product information, determine the remaining products, and determine the products other than the remaining products in the preset product information as the products to be replenished.

[0250] Optionally, the method further includes:

[0251] Step D7: Generate and display replenishment prompt information for prompting replenishment of the to-be-replenished products according to the to-be-replenished products.

[0252] Those skilled in the art can use appropriate methods to generate replenishment prompt information. For example, replenishment prompt information can be directly generated according to the names of the to-be-replenished products.

[0253] Through this method, a complete and high-quality shelf image can be obtained, and then the product information on the shelf can be obtained. Thus, the to-be-replenished products can be determined according to the product information, so that the to-be-replenished products can be automatically obtained by taking pictures of the shelf image, without the user manually checking each item on the shelf one by one. By generating replenishment prompt information, the user can be quickly prompted to replenish goods, improving convenience.

[0254] Usage scenario nine

[0255] Refer to Figure 5k , which shows a schematic diagram of information interaction among a user, an image acquisition device, and a server in usage scenario nine.

[0256] In this usage scenario, the replenishment process includes:

[0257] The image acquisition device receives the trained product recognition model sent by the server. When receiving the user's start shooting instruction, the image acquisition device acquires a shelf image through the image acquisition methods of Embodiments 1 to 4 and obtains the acquired result image of the shelf. The product recognition model is used to process the product information of the acquired result image, and recommended products are displayed according to the processing result. After receiving the user's selection operation on the recommended products, the to-be-replenished products are determined according to the selected products, and a replenishment request is submitted to the server to generate a replenishment order on the server.

[0258] In addition, after the image acquisition device obtains the processing result of the product information processing, it can send the processing result to the server, enabling the server to continue training the initial product recognition model and periodically or according to other conditions compress the trained initial product recognition model, and send the compression result to the image acquisition device.

[0259] This replenishment process can ensure the acquisition quality of the shelf image, and then ensure the quality of the product information processing, thus realizing reliable automatic replenishment.

[0260] Embodiment 5

[0261] Refer to Figure 6 , which shows a flowchart of the steps of an image acquisition method according to Embodiment 5 of the present invention.

[0262] The image acquisition method of this embodiment includes the following steps:

[0263] Step S602: During the process of image acquisition of the target object, obtain the attitude data of the image acquisition device.

[0264] The attitude data is used to indicate the attitude in which the image acquisition device is held. For example, being held horizontally, vertically, with an upward inclination angle, or with a downward inclination angle, etc. The user's image acquisition intention can be determined according to the held attitude. For example, during the image acquisition process, if the user intends to continue the acquisition along the current image acquisition path, the image acquisition device is usually held vertically; while when the user intends to switch to a new image acquisition path for continuous acquisition, the image acquisition device is usually held in a way with an upward inclination angle or a downward inclination angle.

[0265] The attitude data of the image acquisition device includes, but is not limited to, the acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system. It may also include the distance information between the image acquisition device and the target object, etc.

[0266] Those skilled in the art can obtain the attitude data of the image acquisition device through appropriate means. For example, obtain the acceleration information through an acceleration sensor and obtain the angular velocity information through a gyroscope.

[0267] Step S604: Generate corresponding guiding information according to the attitude data, and guide the user to perform continuous image acquisition of the target object through the guiding information.

[0268] For example, when the attitude data includes acceleration information and / or angular velocity information, step S604 can be implemented as follows: Determine the current attitude of the image acquisition device according to the acceleration information and / or angular velocity information; generate guiding information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude.

[0269] In the first case, when generating the guiding information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude, if the current attitude meets the preset path conversion condition, generate the fifth guiding information that guides the user to convert the current image acquisition path into a new image acquisition path matching the current attitude and perform continuous image acquisition along the new image acquisition path.

[0270] If the current posture is such that the image acquisition device has a downward inclination or an upward inclination, and it is determined that the current posture meets the preset path conversion condition, then a fifth guidance message is generated. This fifth guidance message guides the user to convert the current image acquisition path into a new image acquisition path that matches the current posture, and continue image acquisition along the new image acquisition path. For example, if the current posture is one with a downward inclination, then the new image acquisition path is converted into an image acquisition path below the current image acquisition path, and a fifth guidance message with the content "Please move down and continue shooting" is generated.

[0271] Depending on the different structures of the target object, the generated new image acquisition paths can be different. Those skilled in the art can generate image acquisition paths in any appropriate manner according to needs. For example, generate a new image acquisition path according to a preset image acquisition path generation strategy, or adopt the image acquisition path generation method in the foregoing embodiments, etc.

[0272] In the second case, if the current posture does not meet the preset path conversion condition, then a sixth guidance message is generated to guide the user to continue image acquisition along the current image acquisition path.

[0273] For example, if the current posture is that the image acquisition device is held vertically, then the current posture does not meet the preset path conversion condition, and thus a guidance message is generated instructing the user to move along the current image acquisition path and continue image acquisition.

[0274] Through this embodiment, during the process of image acquisition, the posture data of the image acquisition device is obtained, and based on this posture data, a guidance message can be generated to instruct the user to move in the direction matching the current posture for continuous image acquisition, thereby better guiding the user to perform image acquisition on the target object.

[0275] Embodiment Six

[0276] Refer to Figure 7 , which shows a flowchart of the steps of an image acquisition method according to Embodiment Six of the present invention.

[0277] In this embodiment, the image acquisition method includes the foregoing steps S602 to S604.

[0278] Among them, the method further includes:

[0279] Step S604a: Obtain an image of the target object collected by the image acquisition device in real time.

[0280] The image of the target object can be an image collected by the user using the image acquisition device according to the guidance message. For example, if the target object is a shelf, then the image can be an image including a part of the shelf.

[0281] Step S604b: Perform edge detection on the acquired image to obtain a detection result.

[0282] Any suitable method can be used for edge detection of the image. For example, a trained neural network model for edge detection can be used to perform edge detection on the image and obtain a detection result. Alternatively, the method for edge detection of the image in any of the foregoing embodiments can be used.

[0283] The detection result can indicate that the edge of the target object is included in the acquired image, or the edge of the target object is not included, etc.

[0284] In the case of obtaining the detection result, in step S604, corresponding guidance information is generated according to the pose data and the detection result, and the user is guided by the guidance information to perform subsequent image acquisition of the target object.

[0285] During the shooting process, the user may generate some jitters, which cause the pose data of the image acquisition device to indicate a change in its pose. To avoid the influence of these jitters on the generated guidance information and ensure the accuracy of the generated guidance information, performing edge detection on the acquired image and then generating guidance information by combining the pose data and the detection result can make the generated guidance information more accurate.

[0286] For example, in the first case, if the current pose meets the preset path conversion condition and the detection result indicates that the edge of the target object is detected, the fifth guidance information is generated to guide the user to convert the current image acquisition path into a new image acquisition path that matches the current pose and perform subsequent image acquisition along the new image acquisition path.

[0287] When it is determined according to the pose data that the current pose meets the preset path conversion condition, for example, the current pose has a downward inclination angle, and the detection result indicates that the edge of the target object is detected, it means that the user hopes to continue to acquire images of other parts of the target object downward. Therefore, the fifth guidance information can be generated to guide the user to convert the current image acquisition path into a new image acquisition path that matches the current pose and perform subsequent image acquisition along the new image acquisition path.

[0288] Another example is the second case. If the current pose does not meet the preset path conversion condition and the detection result indicates that the edge of the target object is not detected, the sixth guidance information is generated to guide the user to perform subsequent image acquisition along the current image acquisition path.

[0289] When it is determined according to the attitude data that the current attitude does not meet the preset path conversion condition, for example, the current attitude is holding vertically, and the detection result indicates that the edge of the target object is not detected, it means that the user hopes to continue collecting images of other parts of the target object along the current image acquisition path. Therefore, the sixth guidance information for guiding the user to perform continuous image acquisition along the current image acquisition path can be generated.

[0290] Optionally, in this embodiment, the image acquisition method may further include:

[0291] Step S606: Generate seventh guidance information for guiding the user to stop image acquisition according to the attitude data and the detection result.

[0292] For example, if the current attitude determined according to the attitude data does not meet the preset path conversion condition and the detection result indicates that the edge of the target object is detected, the seventh guidance information for guiding the user to stop continuous image acquisition is generated.

[0293] Through this embodiment, during the image acquisition process, the attitude data of the image acquisition device is obtained, and then according to the attitude data, guidance information can be generated to instruct the user to move in the direction matching the current attitude for continuous image acquisition, so as to better guide the user to acquire images of the target object.

[0294] In addition, edge detection can also be performed on the acquired images to determine whether to guide the user to continue shooting, making it more intelligent.

[0295] Embodiment Seven

[0296] Refer to Figure 8 , which shows a structural block diagram of an image acquisition device according to Embodiment Seven of the present invention.

[0297] The image acquisition device in this embodiment includes: a detection module 802, configured to obtain a detection result of real-time edge detection of the target object for the acquired image, where partial image information of the target object is included in the acquired image; a first acquisition module 804, configured to obtain the attitude data of the image acquisition device that acquires the image if the detection result indicates that the edge of the target object is detected in the image; a generation module 806, configured to generate corresponding guidance information according to the attitude data, and guide the user to perform continuous image acquisition of the target object through the guidance information, so as to form complete image information of the target object using the acquired multiple images.

[0298] In this embodiment, the edge of the target object in the acquired image is detected in real time. When the edge of the target object is included in the acquired image, the attitude data of the image acquisition device is obtained, and then the corresponding guiding information is generated according to the attitude data, so as to guide the user to perform image acquisition in a standardized manner through the guiding information, so as to finally complete the image acquisition of multiple parts included in the entire target object, avoid omission, and obtain a complete image of the target object.

[0299] Embodiment 8

[0300] Refer to Figure 9 , which shows a structural block diagram of an image acquisition device according to Embodiment 8 of the present invention.

[0301] The image acquisition device of this embodiment includes: a detection module 902, configured to obtain a detection result of real-time edge detection of the target object in the acquired image, where partial image information of the target object is included in the acquired image; a first acquisition module 904, configured to, if the detection result indicates that the edge of the target object is detected in the image, acquire the attitude data of the image acquisition device that acquired the image; a generation module 906, configured to generate corresponding guiding information according to the attitude data, and guide the user to perform continuous image acquisition of the target object through the guiding information, so as to form complete image information of the target object using the acquired multiple images.

[0302] Optionally, the device further includes: a second acquisition module 908, configured to acquire a lightweight neural network model for performing edge detection of the target object, which is dynamically downloaded to the image acquisition device, before obtaining the detection result of real-time edge detection of the target object in the acquired image; the detection module 902 is configured to perform real-time edge detection of the target object in the acquired image using the lightweight neural network model to obtain a detection result.

[0303] Optionally, the first acquisition module 904 is configured to acquire acceleration information and / or angular velocity information of the image acquisition device in a space coordinate system; the generation module 906 includes: a first determination module 9061, configured to determine the current attitude of the image acquisition device according to the acceleration information and / or angular velocity information; an information generation module 9062, configured to generate guiding information indicating that the user moves in a direction matching the current attitude for continuous image acquisition according to the current attitude.

[0304] Optionally, the device further includes: a splicing module 910, configured to splice the acquired multiple images to obtain a complete image including the complete image information of the target object.

[0305] Optionally, the splicing module 910 includes: a second determination module 9101, configured to determine, from a plurality of acquired images, multiple groups of images having an image coincidence relationship, where each group of images includes two images; and a complete image obtaining module 9102, configured to splice the plurality of acquired images according to the image coincidence relationship, and obtain a complete image including complete image information of the target object according to the splicing result.

[0306] Optionally, the second determination module 9101 includes: a feature extraction module, configured to extract features from each of the plurality of acquired images to obtain feature points corresponding to each image; and a matching module, configured to match any two images according to the feature points of the two images, and determine the multiple groups of images having the image coincidence relationship based on the matching result.

[0307] The image acquisition device in this embodiment is used to implement the corresponding image acquisition method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0308] Embodiment Nine

[0309] Referring to Figure 10 , a schematic structural diagram of an electronic device according to Embodiment Nine of the present invention is shown. The specific implementation of the electronic device in the specific embodiments of the present invention is not limited.

[0310] As Figure 10 shown, the electronic device may include: a processor 1002, a communications interface 1004, a memory 1006, and a communication bus 808.

[0311] Wherein:

[0312] The processor 1002, the communications interface 1004, and the memory 1006 communicate with each other through the communication bus 1008.

[0313] The communications interface 1004 is used to communicate with other electronic devices such as terminal devices or servers.

[0314] The processor 1002 is configured to execute the program 1010, and specifically may execute the relevant steps in the foregoing image acquisition method embodiments.

[0315] Specifically, the program 1010 may include program code, and the program code includes computer operation instructions.

[0316] The processor 1002 may be a central processing unit (CPU), or a specific application integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the electronic device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0317] The memory 1006 is used to store the program 1010. The memory 1006 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0318] The program 1010 is specifically configured to cause the processor 1002 to perform the following operations: obtain a detection result of real-time target object edge detection on the acquired image, where a partial image information of the target object is included in the acquired image; if the detection result indicates that the edge of the target object is detected in the image, obtain the attitude data of the image acquisition device that acquired the image; generate corresponding guidance information according to the attitude data, and guide the user to perform continuous image acquisition on the target object through the guidance information, so as to form the complete image information of the target object using the acquired multiple images.

[0319] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to obtain a lightweight neural network model for performing the target object edge detection that is dynamically sent to the image acquisition device before obtaining the detection result of real-time target object edge detection on the acquired image; and when obtaining the detection result of real-time target object edge detection on the acquired image, use the lightweight neural network model to perform real-time target object edge detection on the acquired image to obtain the detection result.

[0320] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to obtain the acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system when obtaining the attitude data of the image acquisition device that acquired the image; and when generating the corresponding guidance information according to the attitude data and guiding the user to perform continuous image acquisition on the target object through the guidance information, determine the current attitude of the image acquisition device according to the acceleration information and / or angular velocity information; generate guidance information indicating that the user moves in a direction matching the current attitude for continuous image acquisition according to the current attitude.

[0321] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to splice the multiple acquired images when forming the complete image information of the target object using the multiple acquired images, so as to obtain a complete image containing the complete image information of the target object.

[0322] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to determine, from the multiple acquired images, multiple groups of images having an image coincidence relationship when splicing the multiple acquired images to obtain a complete image containing the complete image information of the target object, where each group of images includes two images; splice the multiple acquired images according to the image coincidence relationship, and obtain a complete image containing the complete image information of the target object according to the splicing result.

[0323] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to extract features of each image in the multiple acquired images to obtain feature points corresponding to each image when determining multiple groups of images having an image coincidence relationship from the multiple acquired images; match any two images according to the feature points of the two images, and determine the multiple groups of images having the image coincidence relationship based on the matching result.

[0324] Or,

[0325] The program 1010 may specifically be configured to cause the processor 1002 to perform the following operations: acquire a shelf image acquired according to the indication of the first guiding information, where the shelf is used to carry commodities, and the first guiding information is used to indicate the image acquisition path of the shelf; acquire an edge detection result of performing shelf edge detection on the shelf image; if the edge detection result indicates that the shelf edge is included in the shelf image, acquire the second guiding information indicating a new image acquisition path or acquire the third guiding information indicating the end of acquisition.

[0326] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to acquire the first guiding information before acquiring the shelf image acquired according to the indication of the first guiding information, where the first guiding information is the guiding information corresponding to the image acquisition path, and the image acquisition path is a path generated by segmenting the shelf according to the shelf structure information, and the shelf structure information is determined according to at least one of the overall floor plan, three-dimensional view, and preset shelf virtual model of the shelf.

[0327] In an alternative embodiment, when the edge detection result indicates that the shelf edge is included in the shelf image, Program 1010 is further configured to cause the processor 1002, when obtaining the second guidance information indicating a new image acquisition path or the third guidance information indicating the end of acquisition, if the edge detection result indicates that the shelf edge is included in the shelf image, to perform commodity information recognition on the acquisition result image generated from all the acquired shelf images, and obtain a commodity information result; and according to the commodity information result, obtain the second guidance information indicating a new image acquisition path or the third guidance information indicating the end of acquisition.

[0328] In an alternative embodiment, Program 1010 is further configured to cause the processor 1002, when obtaining the second guidance information indicating a new image acquisition path or the third guidance information indicating the end of acquisition according to the commodity information result, if the commodity information result indicates that not all commodities on the shelf are included in the acquisition result image, to obtain the second guidance information indicating a switching of the shooting row in the shooting path; or, if the commodity information result indicates that all commodities on the shelf are included in the acquisition result image, to obtain the third guidance information indicating the end of shooting.

[0329] In an alternative embodiment, when the edge detection result indicates that the shelf edge is included in the shelf image, Program 1010 is further configured to cause the processor 1002, when obtaining the second guidance information indicating a new image acquisition path or the third guidance information indicating the end of acquisition, if the edge detection result indicates that the shelf edge is included in the shelf image, to obtain the attitude data of the image acquisition device; and according to the attitude data, obtain the second guidance information indicating a new image acquisition path or the third guidance information indicating the end of acquisition.

[0330] In an alternative embodiment, the attitude data includes the acceleration information and / or the angular velocity information of the image acquisition device in the spatial coordinate system.

[0331] In an alternative embodiment, Program 1010 is further configured to cause the processor 1002 to obtain a reserved area corresponding to the current image acquisition path from the latest acquired shelf image and display the reserved area in a set area of the display interface, so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area.

[0332] In an alternative embodiment, Program 1010 is further configured to cause the processor 1002 to obtain an acquisition result image generated by stitching all the acquired shelf images.

[0333] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to identify product information and / or product location in the acquired result image, and obtain a product information result and / or a product location result; perform an analysis operation on the product information result and / or the product location result, and generate an analysis result corresponding to the analysis operation.

[0334] In an alternative embodiment, the analysis result includes at least one of the following: product sales information, product display information, product quantity information, product replenishment status information.

[0335] Or,

[0336] The program 1010 may specifically be configured to cause the processor 1002 to perform the following operations: acquire image data of a shelf according to acquired first guidance information, where the first guidance information is used to indicate an image acquisition path of the shelf; identify the image data, and obtain product information on the shelf and information on whether the shelf edge is included; if it is determined that the information on the shelf edge is included in the image data, determine whether all product information is included in all the acquired image data according to the product information, and obtain second guidance information indicating a new image acquisition path or third guidance information indicating the end of acquisition according to the determination result.

[0337] Or,

[0338] The program 1010 may specifically be configured to cause the processor 1002 to perform the following operations: display first acquisition prompt information for shelf products, where the first acquisition prompt information is used to indicate an acquisition position when acquiring an image of the shelf products along an image acquisition path; acquire an image acquired according to the first acquisition prompt information, and identify the acquired image; if the identification result indicates that the shelf edge is included in the image, display second acquisition prompt information for indicating a new image acquisition path and indicating continuous image acquisition.

[0339] Or,

[0340] The program 1010 may specifically be configured to cause the processor 1002 to perform the following operations: acquire image data of a shelf; process the image data, and identify the product information on the shelf; determine the product statistical information of the shelf according to the identified product information.

[0341] Or,

[0342] Program 1010 can specifically be used to cause the processor 1002 to perform the following operations: in response to a user-initiated shooting operation, call the image acquisition device of the client to shoot the image data of the shelf; process the image data to identify the product information on the shelf; and determine the product statistics of the shelf according to the identified product information.

[0343] Or,

[0344] Program 1010 can specifically be used to cause the processor 1002 to perform the following operations: in response to a user-initiated replenishment operation, call the image acquisition device to shoot the image data of the shelf; perform identification processing on the image data to identify the product information on the shelf; and determine the products to be replenished according to the product information on the shelf.

[0345] In an alternative embodiment, program 1010 is further used to cause the processor 1002 to generate and display replenishment prompt information for prompting the replenishment of the products to be replenished according to the products to be replenished.

[0346] Or,

[0347] Program 1010 can specifically be used to cause the processor 1002 to perform the following operations: during the process of image acquisition of a target object, obtain the attitude data of the image acquisition device; generate corresponding guiding information according to the attitude data, and guide the user to perform continuous image acquisition of the target object through the guiding information.

[0348] In an alternative embodiment, the attitude data includes the acceleration information and / or angular velocity information of the image acquisition device in a space coordinate system; program 1010 is further used to cause the processor 1002 to determine the current attitude of the image acquisition device according to the acceleration information and / or angular velocity information when generating corresponding guiding information according to the attitude data and guiding the user to perform continuous image acquisition of the target object through the guiding information; and generate guiding information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude.

[0349] In an alternative embodiment, when program 1010 is further used to cause the processor 1002 to generate guiding information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude, if the current attitude meets the preset path conversion condition, generate fifth guiding information for guiding the user to convert the current image acquisition path into a new image acquisition path matching the current attitude and perform continuous image acquisition along the new image acquisition path; if the current attitude does not meet the preset path conversion condition, generate sixth guiding information for guiding the user to perform continuous image acquisition along the current image acquisition path.

[0350] In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to obtain an image of a target object collected in real time by the image acquisition device; perform edge detection on the collected image to obtain a detection result; and the program 1010 is further configured to cause the processor 1002 to generate corresponding guidance information according to the attitude data when guiding the user to perform subsequent image acquisition of the target object through the guidance information, and generate corresponding guidance information according to the attitude data and the detection result, and guide the user to perform subsequent image acquisition of the target object through the guidance information.

[0351] In an alternative embodiment, when the program 1010 is further configured to cause the processor 1002 to generate corresponding guidance information according to the attitude data and the detection result, and guide the user to perform subsequent image acquisition of the target object through the guidance information, if the current attitude meets a preset path conversion condition and the detection result indicates that the edge of the target object is detected, the program 1010 generates fifth guidance information for guiding the user to convert the current image acquisition path into a new image acquisition path matching the current attitude and perform subsequent image acquisition along the new image acquisition path; if the current attitude does not meet the preset path conversion condition and the detection result indicates that the edge of the target object is not detected, the program 1010 generates sixth guidance information for guiding the user to perform subsequent image acquisition along the current image acquisition path.

[0352] For the specific implementation of each step in the program 1010, reference may be made to the corresponding steps and descriptions in the corresponding units in the above-mentioned image acquisition method embodiments, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be elaborated here.

[0353] Through the electronic device of this embodiment, edge detection of the target object is performed on the collected image in real time. When the edge of the target object is included in the collected image, the attitude data of the image acquisition device is obtained, and then corresponding guidance information is generated according to the attitude data, so as to guide the user to perform image acquisition in a standard manner through the guidance information, achieve the purpose of finally completing the image acquisition of multiple parts included in the entire target object, avoid omission, and obtain a complete image of the target object.

[0354] It should be noted that according to the needs of implementation, each component / step described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of the components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0355] The method according to an embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or can be implemented as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the image acquisition method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the image acquisition method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the image acquisition method shown herein.

[0356] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional person can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0357] The above embodiments are only used to illustrate the embodiments of the present invention, rather than to limit the embodiments of the present invention. Those of ordinary skill in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A method for collecting shelf images, characterized in that, Including: Obtain a shelf image collected according to the indication of the first guiding information, where the shelf is used to carry goods, and the first guiding information is used to indicate the image acquisition path of the shelf; Obtain an edge detection result of performing shelf edge detection on the shelf image; If the edge detection result indicates that the shelf edge is included in the shelf image, obtain the second guiding information indicating a new image acquisition path or obtain the third guiding information indicating the end of acquisition; The method further includes: obtaining a reserved area corresponding to the current image acquisition path from the latest collected shelf image and displaying the reserved area in a set area of the display interface, so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area, where the reserved area and the set area of the display interface are determined according to the image acquisition path.

2. The method according to claim 1, wherein Before obtaining the shelf image collected according to the indication of the first guiding information, the method further includes: Obtain the first guiding information, where the first guiding information is the guiding information corresponding to the image acquisition path, and the image acquisition path is a path generated by segmenting the shelf according to the shelf structure information, and the shelf structure information is determined according to at least one of the overall floor plan, three-dimensional view, and preset shelf virtual model of the shelf.

3. The method according to claim 1, wherein The step of if the edge detection result indicates that the shelf edge is included in the shelf image, obtain the second guiding information indicating a new image acquisition path or obtain the third guiding information indicating the end of acquisition includes: If the edge detection result indicates that the shelf edge is included in the shelf image, perform commodity information recognition on the acquisition result image generated by all the collected shelf images, and obtain a commodity information result; According to the commodity information result, obtain the second guiding information indicating a new image acquisition path or obtain the third guiding information indicating the end of acquisition.

4. The method according to claim 3, wherein The step of according to the commodity information result, obtain the second guiding information indicating a new image acquisition path or obtain the third guiding information indicating the end of acquisition includes: If the commodity information result indicates that all the goods on the shelf are not included in the acquisition result image, obtain the second guiding information indicating the shooting row in the switching shooting path; or, If the commodity information result indicates that all the goods on the shelf are included in the acquisition result image, obtain the third guiding information indicating the end of shooting.

5. The method according to claim 1, wherein The step of if the edge detection result indicates that the shelf edge is included in the shelf image, obtain the second guiding information indicating a new image acquisition path or obtain the third guiding information indicating the end of acquisition includes: If the edge detection result indicates that the shelf edge is included in the shelf image, obtain the attitude data of the image acquisition device; According to the attitude data, obtain the second guiding information indicating a new image acquisition path or obtain the third guiding information indicating the end of acquisition.

6. The method according to claim 5, wherein The attitude data includes acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system.

7. The method according to claim 1, wherein The method further includes: Obtain an acquisition result image generated by stitching all the collected shelf images.

8. The method according to claim 7, characterized in that, The method further includes: Identify product information and / or product location from the collected result image, and obtain the product information result and / or the product location result; Perform an analysis operation on the product information result and / or the product location result, and generate an analysis result corresponding to the analysis operation.

9. The method according to claim 8, characterized in that, The analysis result includes at least one of the following: product sales information, product display information, product quantity information, product replenishment status information.

10. A method for processing commodity information, characterized in that, Comprising: Collect image data of the shelf according to the obtained first guiding information, wherein the first guiding information is used to indicate the image acquisition path of the shelf; Identify the image data, and obtain the product information on the shelf and the information on whether the shelf edge is included; If it is determined that the information on the shelf edge is included in the image data, then determine whether all the product information is included in all the collected image data according to the product information, and obtain the second guiding information indicating a new image acquisition path or the third guiding information indicating the end of acquisition according to the judgment result; The method further includes: obtaining a reserved area corresponding to the current image acquisition path from the latest collected image data of the shelf and displaying the reserved area in a set area of the display interface, so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area, wherein the reserved area and the set area of the display interface are determined according to the image acquisition path.

11. A method for collecting shelf images, characterized in that, Comprising: Display a first acquisition prompt message for the shelf products, wherein the first acquisition prompt message is used to indicate the acquisition position when acquiring images of the shelf products along the image acquisition path; Obtain the image acquired according to the first acquisition prompt message, and identify the acquired image; If the recognition result indicates that the image includes the shelf edge, then display a second acquisition prompt message for indicating a new image acquisition path and indicating to continue image acquisition; The method further includes: obtaining a reserved area corresponding to the current image acquisition path from the latest collected image data, and displaying the reserved area in a set area of the display interface, so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area, wherein the reserved area and the set area of the display interface are determined according to the image acquisition path.

12. A client, characterized in that, Comprising: A display interface for displaying a first acquisition prompt message for indicating image acquisition of a target object along an image acquisition path; The display interface is further used to display a second acquisition prompt message, which is information for indicating image acquisition of the target object along a new image acquisition path when the edge of the target object is included in the acquired image; The method further includes: obtaining a reserved area corresponding to the current image acquisition path from the latest collected image data, and displaying the reserved area in a set area of the display interface, so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area, wherein the reserved area and the set area of the display interface are determined according to the image acquisition path.

13. The client according to claim 12, wherein The target object includes at least one of the following: a shelf, a parking lot, and seats in a venue.

14. A method for processing commodity information, characterized in that, including: Collecting image data of the shelf; Processing the image data to identify the product information on the shelf; Determining the product statistics information of the shelf according to the identified product information; wherein the image data is acquired by the shelf image acquisition method described in any one of claims 1-9, or the shelf image acquisition method described in claim 11.

15. A method for processing commodity information, characterized in that, including: In response to a shooting operation initiated by the user, calling the image acquisition device of the client to shoot the image data of the shelf; Processing the image data to identify the product information on the shelf; Determining the product statistics information of the shelf according to the identified product information; wherein the image data is acquired by the shelf image acquisition method described in any one of claims 1-9, or the shelf image acquisition method described in claim 11.

16. A method for processing commodity replenishment, characterized in that, including: In response to a replenishment operation initiated by the user, calling the image acquisition device to shoot the image data of the shelf; Performing identification processing on the image data to identify the product information on the shelf; Determining the products to be replenished according to the product information on the shelf; wherein the image data is acquired by the shelf image acquisition method described in any one of claims 1-9, or the shelf image acquisition method described in claim 11.

17. The method according to claim 16, characterized in that, The method further includes: Generating and displaying replenishment prompt information for prompting replenishment of the products to be replenished according to the products to be replenished.

18. An image acquisition method, characterized in that, including: Obtaining the detection result of real-time target object edge detection on the acquired image, wherein a partial image information of the target object is included in the acquired image; If the detection result indicates that the edge of the target object is detected in the image, obtaining the attitude data of the image acquisition device that acquired the image, and the attitude data is used to determine whether the user has the intention to continuously shoot different positions of the target object; Generating corresponding guiding information according to the attitude data, and guiding the user to perform continuous image acquisition of the target object through the guiding information, so as to form the complete image information of the target object with the acquired multiple images.

19. The method according to claim 18, wherein the obtaining the attitude data of the image acquisition device that acquired the image includes: obtaining the acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system; the generating corresponding guiding information according to the attitude data and guiding the user to perform continuous image acquisition of the target object through the guiding information includes: determining the current attitude of the image acquisition device according to the acceleration information and / or angular velocity information; Generating guiding information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude.

20. The method according to claim 18, wherein Before obtaining the detection result of real-time target object edge detection on the acquired image, the method further includes: obtaining a lightweight neural network model dynamically sent to the image acquisition device for performing the target object edge detection. The obtaining the detection result of real-time target object edge detection on the acquired image includes: using the lightweight neural network model to perform real-time target object edge detection on the acquired image to obtain the detection result.

21. The method according to claim 18, wherein The using the multiple acquired images to form the complete image information of the target object includes: Stitching the multiple acquired images to obtain a complete image containing the complete image information of the target object.

22. The method according to claim 21, wherein The stitching the multiple acquired images to obtain a complete image containing the complete image information of the target object includes: Determining multiple groups of images with an image overlapping relationship from the multiple acquired images, where each group of images includes two images. Stitching the multiple acquired images according to the image overlapping relationship, and obtaining a complete image containing the complete image information of the target object according to the stitching result.

23. The method according to claim 22, characterized in that, The determining multiple groups of images with an image overlapping relationship from the multiple acquired images includes: Performing feature extraction on each of the multiple acquired images to obtain feature points corresponding to each image. For any two images, matching according to the feature points of the two images, and determining the multiple groups of images with the image overlapping relationship based on the matching result.

24. An image acquisition method, characterized in that, Includes: During the process of image acquisition of the target object, obtaining the attitude data of the image acquisition device, where the attitude data is used to determine whether the user has an intention to continuously capture different positions of the target object. Generating corresponding guidance information according to the attitude data, and guiding the user to perform continuous image acquisition of the target object through the guidance information.

25. The method according to claim 24, characterized in that, The attitude data includes acceleration information and / or angular velocity information of the image acquisition device in the space coordinate system. The generating corresponding guidance information according to the attitude data, and guiding the user to perform continuous image acquisition of the target object through the guidance information includes: Determining the current attitude of the image acquisition device according to the acceleration information and / or angular velocity information. Generating guidance information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude.

26. The method according to claim 25, wherein The generating guidance information indicating that the user moves in the direction matching the current attitude for continuous image acquisition according to the current attitude includes: If the current attitude meets the preset path conversion condition, generating fifth guidance information for guiding the user to convert the current image acquisition path into a new image acquisition path matching the current attitude and performing continuous image acquisition along the new image acquisition path. If the current attitude does not meet the preset path conversion condition, generating sixth guidance information for guiding the user to perform continuous image acquisition along the current image acquisition path.

27. The method according to any one of claims 24 to 26, characterized in that, The method further includes: Obtaining an image of the target object acquired by the image acquisition device in real time; performing edge detection on the acquired image to obtain a detection result. Generating corresponding guiding information according to the posture data, and guiding a user to perform continuous image acquisition on the target object through the guiding information, includes: Generating corresponding guiding information according to the posture data and the detection result, and guiding a user to perform continuous image acquisition on the target object through the guiding information.

28. The method according to claim 27, wherein The generating corresponding guiding information according to the posture data and the detection result, and guiding a user to perform continuous image acquisition on the target object through the guiding information, includes: If the current posture meets a preset path conversion condition and the detection result indicates that the edge of the target object is detected, generating fifth guiding information for guiding the user to convert the current image acquisition path into a new image acquisition path matching the current posture, and performing continuous image acquisition along the new image acquisition path; If the current posture does not meet the preset path conversion condition and the detection result indicates that the edge of the target object is not detected, generating sixth guiding information for guiding the user to perform continuous image acquisition along the current image acquisition path.

29. An image acquisition device, characterized in that, Includes: A detection module, configured to obtain a detection result of real-time edge detection of the target object on the acquired image, wherein part of the image information of the target object is included in the acquired image; A first acquisition module, configured to, if the detection result indicates that the edge of the target object is detected in the image, acquire the posture data of the image acquisition device that acquires the image, where the posture data is used to determine whether the user has an intention to perform continuous shooting on different positions of the target object; A generation module, configured to generate corresponding guiding information according to the posture data, and guide a user to perform continuous image acquisition on the target object through the guiding information, so as to form complete image information of the target object by using a plurality of acquired images.

30. An electronic device, comprising: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the shelf image acquisition method according to any one of claims 1-9, or execute the operations corresponding to the commodity information processing method according to claim 10, or execute the operations corresponding to the shelf image acquisition method according to claim 11, or execute the operations corresponding to the commodity information processing method according to claim 14, or execute the operations corresponding to the commodity information processing method according to claim 15, or execute the operations corresponding to the commodity replenishment processing method according to claim 16 or 17, or execute the operations corresponding to the image acquisition methods according to claims 18-23, or execute the operations corresponding to the image acquisition methods according to claims 24-28.

31. A computer storage medium having a computer program stored thereon, which when implemented by a processor, implements the shelf image acquisition method according to any one of claims 1-9, or implements the commodity information processing method according to claim 10, or implements the shelf image acquisition method according to claim 11, or implements the commodity information processing method according to claim 14, or implements the commodity information processing method according to claim 15, or implements the commodity replenishment processing method according to claim 16 or 17, or implements the image acquisition method according to claims 18-23, or implements the image acquisition method according to claims 24-28.

Images