Information processing method, device, equipment and storage medium

Abstract

The present invention provides an information processing method, apparatus, device, and storage medium. The method comprises: acquiring video data from a sorting port video acquisition module based on the startup information of a sorting port task; invoking a recognition model for the sorting object to identify the sorting object in the acquired video data based on the sorting object indicated by the startup information of the sorting port task; and counting the sorting objects entering and exiting the current sorting port based on the recognition results of the acquired video data. The counted number of sorting objects is used to match the required sorting quantity of the sorting objects for the sorting port task. The present invention can assist sorters in counting and verifying goods, and can flexibly adapt to a wide range of sorting scenarios.

Description

Technical Field

[0001] The present invention relates to the field of computer applications, and in particular to an information processing method, apparatus, system, equipment and storage medium. Background Art

[0002] When sorting goods, sorters need to remove the goods from the shelves, place them on the distribution transport device, and transport them to the corresponding sorting port for sorting. In order to manage sorting tasks, a sorting client can be provided to the sorter. The sorter uses the user interface of the sorting client to determine the relevant information of the sorting task. When using the sorting client, the sorter needs to perform operations such as clicking and inputting. However, the sorter's manual operation makes the sorting task management process cumbersome and it is easy to click / enter the wrong information, resulting in errors in information aggregation.

[0003] In order to automate some processes of sorting task management, some solutions set weight sensors at the cargo locations and maintain the correspondence between weight sensor numbers and cargo locations. When the picker delivers the goods, a weight sensor will generate weight change information. The system will find the corresponding cargo location based on the number of the weight sensor that generates the weight change information, and match it with the cargo location associated with the order for the delivered goods. This can automatically determine whether the picker's delivery of the goods is accurate.

[0004] However, in this solution, it only confirms the cargo location for this delivery based on the weight sensor. During the specific delivery, the number of goods still needs to be manually counted, and the correspondence between the weight sensor number, cargo location, and cargo weight needs to be maintained. Its auxiliary capabilities are limited, its flexibility is low, and its applicable conditions are also relatively harsh.

[0005] Similarly, solutions that require establishing and maintaining a database to assist sorters in sorting are also difficult to adapt to various scenarios on e-commerce platforms, especially community group buying and promotional activities where a single unit of inventory of goods needs to be sorted multiple times (the database needs to be queried multiple times for the same goods, and the query and matching process is relatively redundant), and the types of goods fluctuate greatly (the database needs to be updated frequently).

[0006] It can be seen that how to assist sorters in counting and checking goods without establishing any database so as to flexibly adapt to a wide range of sorting scenarios is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0007] In order to overcome the defects of the above-mentioned related technologies, the present invention provides an information processing method, device, equipment and storage medium, which assists sorters in counting and checking goods without establishing any database, so as to flexibly adapt to a wide range of sorting scenarios.

[0008] According to one aspect of the present invention, there is provided an information processing method, comprising:

[0009] According to the start information of the sorting port task, the video data is obtained from the sorting port video acquisition module;

[0010] According to the sorting object indicated by the start information of the sorting port task, calling the recognition model of the sorting object to recognize the sorting object in the acquired video data;

[0011] According to the recognition result of the acquired video data, the sorting objects entering and taken out of the current sorting port are counted, and the counting result of the sorting objects is used to match the required sorting quantity of the sorting objects of the sorting port task.

[0012] In some embodiments of the present application, the counting result of the sorting objects is the difference between the number of sorting objects entering the current sorting port and the number of sorting objects taken out of the current sorting port. The counting result of the sorting objects is matched with the required number of sorting objects for the sorting port task according to the following steps:

[0013] In response to the counting result of the sorting objects being the same as the required sorting quantity of the sorting objects of the sorting port task, generating completion information of the sorting port task; and

[0014] In response to the counting result of the sorting objects being greater than the required sorting quantity of the sorting objects of the sorting port task, alarm information of the sorting port task is generated.

[0015] In some embodiments of the present application, the recognition model is used to identify multiple identifiable objects, and calling the recognition model of the sorting object to identify the sorting object in the acquired video data according to the sorting object indicated by the start information of the sorting port task includes:

[0016] Calling the recognition model to perform object recognition on the acquired video data to obtain at least one recognized object;

[0017] treating identifiable objects other than the sorting objects as interference objects;

[0018] The interfering objects are filtered from the identified objects.

[0019] In some embodiments of the present application, calling the recognition model to perform object recognition on the acquired video data to obtain at least one recognized object includes:

[0020] Calling the recognition model to perform object recognition on the acquired video data to obtain at least one candidate recognition object;

[0021] Obtaining the confidence level of the candidate recognition object;

[0022] The candidate recognition object with a confidence level greater than a set confidence level threshold is used as the recognition object.

[0023] In some embodiments of the present application, counting the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data includes:

[0024] The sorted object identified in the current frame of the acquired video data is used as a trackable object to generate an object target frame;

[0025] Tracking the object target frame according to a subsequent frame of the current frame of the acquired video data;

[0026] According to the position change of the object target frame, it is determined that the sorting object enters the current sorting port or is taken out from the current sorting port.

[0027] In some embodiments of the present application, determining, based on the position change of the object target frame, whether the sorting object enters the current sorting port or is taken out of the current sorting port includes:

[0028] In response to a displacement component of the center of mass of the object target frame in a direction of entering the sorting port being greater than 0, the number of sorting objects entering the current sorting port is increased by one;

[0029] In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, the number of sorting objects taken out from the current sorting port is increased by one.

[0030] In some embodiments of the present application, in response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, increasing the number of sorting objects entering the current sorting port by one comprises:

[0031] In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, and the height of the center of mass of the object target frame in the current frame in the direction of the sorting port being greater than half of the height of the current frame in the direction of the sorting port, the number of sorting objects entering the current sorting port is increased by one.

[0032] In some embodiments of the present application, in response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, increasing the number of sorting objects taken out from the current sorting port by one comprises:

[0033] In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, and the height of the center of mass of the object target frame in the current frame in the direction of the sorting port being less than half of the height of the current frame in the direction of the sorting port, the number of sorting objects taken out from the current sorting port is increased by one.

[0034] In some embodiments of the present application, counting the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data further includes:

[0035] storing an object identifier associated with a sorting object identified in a current frame of the acquired video data in a tracking object list, wherein the tracking object list stores at least one of the trackable objects;

[0036] In response to a number of consecutive frames in which an object target frame of a trackable object in the trackable object list disappears being greater than a set frame number threshold and / or the trackable object has been counted, deleting the trackable object from the trackable object list.

[0037] In some embodiments of the present application, after counting the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data, the method further includes:

[0038] marking an object target frame and a centroid of a sorting object identified in the acquired video data in the video data;

[0039] Display the labeled video data.

[0040] According to another aspect of the present application, there is further provided an information processing device, comprising:

[0041] The video data acquisition module is used to obtain video data from the sorting port video acquisition module according to the start information of the sorting port task;

[0042] A sorting object recognition module is used to call the sorting object recognition model to recognize the sorting object in the acquired video data according to the sorting object in the start information of the sorting port task;

[0043] The counting module is used to count the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data. The counting result of the sorting objects is used to match the number of sorting objects required to be sorted for the sorting port task.

[0044] According to another aspect of the present application, there is further provided an information processing system, comprising:

[0045] The sorting terminal is used to display the sorting port tasks and generate the start information of the sorting port tasks;

[0046] Sorting port video acquisition module, used to collect video data of the sorting port;

[0047] Wherein, the video acquisition module at the sorting end or the sorting port executes the information processing method according to any one of claims 1 to 10.

[0048] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; and a storage medium storing a computer program, wherein the computer program executes the above steps when executed by the processor.

[0049] According to yet another aspect of the present invention, a storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps described above are executed.

[0050] Compared with the prior art, the advantages of the present invention are:

[0051] In response to the initiation of the sorting port task, the video data acquired by the sorting port video acquisition module is identified and counted, and the counting result is matched with the required sorting quantity of the sorting port task, thereby enabling automatic counting at the sorting port task. On the one hand, it can avoid errors caused by manual operation and manual input by the sorter; on the other hand, the sorting port task can determine the sorting object to be identified, thereby achieving accurate counting and avoiding counting interference caused by other objects; on the other hand, by directly identifying and counting the sorting object based on the video data collected by the sorting port video acquisition module, there is no need to maintain a database of the corresponding relationship between goods and goods weight; on the other hand, since there is no need to establish and maintain a database, the above method can be flexibly adapted to a wide range of sorting scenarios to improve the data processing efficiency of auxiliary sorting, thereby improving the sorting experience of the sorter and the efficiency of manual sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings.

[0053] Figure 1 A flowchart of an information processing method according to an embodiment of the present invention is shown.

[0054] Figure 2 A flow chart of matching the counting result of the sorting objects with the required sorting quantity of the sorting objects of the sorting port task according to an embodiment of the present invention is shown.

[0055] Figure 3 A flow chart of calling the sorting object recognition model to recognize the sorting objects in the acquired video data according to an embodiment of the present invention is shown.

[0056] Figure 4 A flow chart is shown in which the recognition model is called to perform object recognition on the acquired video data to obtain at least one recognized object according to an embodiment of the present invention.

[0057] Figure 5 A flow chart of counting sorting objects entering and being taken out of a current sorting port according to a recognition result of acquired video data according to an embodiment of the present invention is shown.

[0058] Figure 6 A flow chart is shown for determining whether the sorting object enters or is taken out of the current sorting port according to the position change of the object target frame according to an embodiment of the present invention.

[0059] Figure 7 A flowchart of trackable object management of a trackable object list according to an embodiment of the present invention is shown.

[0060] Figure 8 A module diagram of an information processing device according to an embodiment of the present invention is shown.

[0061] Figure 9 A module diagram of an information processing system according to an embodiment of the present invention is shown.

[0062] Figure 10 The figure schematically shows a computer-readable storage medium in an exemplary embodiment of the present invention.

[0063] Figure 11 The figure schematically shows a schematic diagram of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0064] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0065] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0066] The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined. Therefore, the actual execution order may change according to actual circumstances.

[0067] This application is applied to the sorting of goods in warehouses. Through the interaction between the sorting port video acquisition module and the sorting end, it assists the sorter in checking the sorting quantity, submitting the sorting task, etc., so as to provide intelligent assistance to the sorting task of the sorting port when the sorter receives the sorting task and sorts the goods at the sorting port.

[0068] The methods of the various embodiments of the present application can be executed at the sorting port video acquisition module and / or the sorting end. The sorting port video acquisition module can execute the methods of the various embodiments of the present application through the built-in processor, or it can send the video it captures to the sorting end to reuse the computing power of the sorting end to execute the methods of the various embodiments of the present application. Furthermore, the sorting port video acquisition module and the sorting end can respectively execute some steps to achieve computing power balance. The present application is not limited to this. The embodiment in which the sorting port video acquisition module and the sorting end communicate through the warehouse management server is also within the scope of protection of the present application. Therefore, some steps can also be executed by the warehouse management server.

[0069] The sorting terminal is deployed in the form of applications, applets, web pages, etc. on portable devices (such as mobile phones, tablet computers, and handheld devices dedicated to sorting) to provide sorting services to sorters. The sorting terminal can display sorting tasks, sorting port tasks (sorting tasks corresponding to each sorting port), information on goods to be sorted, and related quantity information to the sorters. At the same time, various process control controls can also be provided to display different information to the sorters at different sorting stages to assist in sorting. The sorting terminal can communicate with the warehouse management server. Furthermore, when the goods, shelves, transportation equipment (such as forklifts, forklift pallets), etc. in the warehouse can also be provided with intelligent modules or identifiable information such as wireless radio frequency tags, sensors, and identification modules, the sorting terminal can interact with these intelligent modules through its portable device to obtain information related to sorting, thereby further assisting the sorters in improving sorting efficiency.

[0070] See below Figure 1 , Figure 1 The flowchart of the information processing method according to an embodiment of the present invention is shown. The information processing method includes the following steps:

[0071] Step S110: according to the start information of the sorting port task, the video data is acquired from the sorting port video acquisition module.

[0072] Specifically, the startup information is used to indicate the start of the sorting port task. The startup information can be generated by the sorting end. For example, when the sorter chooses to start the sorting port task at the sorting end, the startup information is generated. For another example, when the distance between the sorting end and the sorting port is less than a set threshold and exceeds a set time, it means that the sorter has arrived at the sorting port and is about to perform the sorting task. At this time, the sorting end can generate the startup information. The distance between the sorting end and the sorting port can be determined by matching the positioning information obtained by the positioning device of the sorting end with the position of each sorting port in the preset warehouse map. The distance between the sorting end and the sorting port can also be determined by the information transmission time for communication between the sorting end and the sorting port. The present application is not limited to this. Furthermore, the startup information can also be generated by the warehouse management server, and the sorting end reports the above information to the warehouse management server, and the warehouse management server generates the startup information.

[0073] The startup information of the sorting port task may, for example, include the sorting object of the sorting port task. In some variations, the startup information of the sorting port task may also include information such as the sorting port task identifier, the sorting port identifier, the sorting port video acquisition module identifier, and the quantity to be sorted. The sorting end may determine the sorting port video acquisition module to be communicated with based on the sorting port identifier and / or the sorting port video acquisition module identifier in the startup information. Furthermore, the sorting end and the sorting port video acquisition module may also be matched through short-range wireless communication (such as Bluetooth, infrared, ZigBee, wireless radio frequency technology), thereby eliminating the need to determine the sorting port video acquisition module to be communicated with through the sorting port task identifier, the sorting port identifier, or position matching.

[0074] The sorting port video acquisition module is set corresponding to the sorting port to collect video data at the sorting port.

[0075] By triggering the video acquisition module from the sorting port using a startup message to acquire video data, we can avoid acquiring video data from before the sorting port task was executed. This video data is not helpful for identifying and counting sorted objects in the current sorting port task, and transmitting this video data would require higher bandwidth and video processing power. Therefore, triggering the video acquisition module from the sorting port using a startup message to acquire video data can improve video data transmission and processing efficiency and reduce unnecessary video processing steps.

[0076] Furthermore, the startup information can also trigger the sorting port video acquisition module to start up and collect video, thereby effectively managing the power demand of the sorting port video acquisition module.

[0077] Step S120: according to the sorting object indicated by the start information of the sorting port task, calling the recognition model of the sorting object to recognize the sorting object in the acquired video data.

[0078] Specifically, a warehouse may contain a variety of different types of goods. Using the same recognition model to identify all goods in the warehouse increases the model's complexity, reduces its efficiency, and lowers its accuracy. Therefore, to ensure accurate and efficient identification of sorted objects, the present application provides multiple recognition models to identify different types of sorted objects. Thus, step S120 can call the corresponding recognition model for the sorted object based on the sorted object in the startup information.

[0079] Specifically, the recognition model can be a trained artificial intelligence model, such as a convolutional neural network model. Various recognition models can be used in this application. Considering the different recognition advantages of different recognition models, different recognition models can be used for different types of sorting objects to improve the recognition accuracy and efficiency of the sorting objects.

[0080] Step S130: Counting the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data. The counting result of the sorting objects is used to match the required sorting quantity of the sorting objects of the sorting port task.

[0081] Thus, step S130 can realize automatic counting of the sorting objects of the sorting port, and assist in the management of the sorting port tasks by matching and comparing with the required sorting quantity of the sorting objects of the sorting port tasks.

[0082] In the information processing method provided in this application, in response to the start of the sorting port task, the video data obtained by the sorting port video acquisition module is identified and counted, and the counting result is matched with the sorting quantity required by the sorting port task, so that automatic counting can be achieved at the sorting port task. On the one hand, it can avoid the situation where the sorter manually operates and manually inputs errors; on the other hand, the sorting object to be identified can be determined by the sorting port task, thereby achieving accurate counting and avoiding counting interference caused by other objects; on the other hand, by directly identifying and counting the sorting object based on the video data collected by the sorting port video acquisition module, there is no need to maintain a database of the corresponding relationship between goods and goods weight; on the other hand, since there is no need to establish and maintain a database, the above method can be flexibly adapted to a wide range of sorting scenarios to improve the data processing efficiency of auxiliary sorting, thereby improving the sorting experience of the sorter and the efficiency of manual sorting.

[0083] See below Figure 2 , Figure 2 A flow chart of matching the counting result of the sorting objects with the required sorting quantity of the sorting objects of the sorting port task according to an embodiment of the present invention is shown. Figure 2 The CCP outlined the following steps:

[0084] Step S131: in response to the counting result of the sorting objects being the same as the required sorting quantity of the sorting objects of the sorting port task, generating completion information of the sorting port task.

[0085] Specifically, sorting port task completion information can be generated at the sorting terminal, or generated by the sorting port video acquisition module and then sent to the sorting terminal. Based on this sorting port task completion information, the sorting terminal can automatically submit the sorting port task. Furthermore, this sorting port task completion information can also be used to verify the completion of sorting port tasks submitted by sorters themselves.

[0086] Step S132: in response to the counting result of the sorting objects being greater than the required sorting quantity of the sorting objects of the sorting port task, generating alarm information of the sorting port task.

[0087] Specifically, when the count result of the sorting objects is greater than the number of sorting objects required to be sorted for the sorting port task, it indicates that a sorting error has occurred. Therefore, an alarm message for the sorting port task can be generated at the sorting terminal, or generated by the sorting port video acquisition module and then sent to the sorting terminal. The alarm message at the sorting port video acquisition module can be issued by an alarm module (such as an audio and light module) installed at the sorting port, reminding the sorter of the sorting error in a more obvious way. The sorting terminal can issue a sorting error reminder based on the alarm message through the display screen and / or speaker of the sorting terminal.

[0088] Specifically, when the count result of the sorted objects is less than the required number of sorted objects for the sorting port task, it indicates that the sorting task has not been completed, and the sorter will continue to perform the sorting port task. Furthermore, when the count result of the sorted objects is less than the required number of sorted objects for the sorting port task, and the sorting end moves away from the current sorting port or moves to another sorting port, an alarm message can also be generated to inform the sorter that the sorting port task has not been completed and the sorter is asked to continue performing the sorting port task.

[0089] See below Figure 3 , Figure 3 A flow chart of calling the sorting object recognition model to recognize the sorting objects in the acquired video data according to an embodiment of the present invention is shown. Figure 3 The following steps are shown:

[0090] Step S121: calling the recognition model to perform object recognition on the acquired video data to obtain at least one recognized object.

[0091] Step S122: treating identifiable objects other than the sorting object as interference objects.

[0092] Step S123: Filter the interference objects from the identified objects.

[0093] Specifically, when the recognition model can recognize multiple recognition objects, objects other than the sorting object can be filtered from the recognition objects to avoid misrecognition by the recognition model.

[0094] In some embodiments, the recognition model is trained by the warehouse management server to identify the goods in the warehouse. Thus, when the recognition model can identify goods A, B and C, and the sorting object is goods A, goods B and C can be eliminated as interference objects. Thus, it is avoided that the video acquisition module of the sorting port causes goods B / goods C to appear in the video data due to the acquisition angle and goods B / goods C can be recognized by the recognition model, thereby causing counting errors based on the recognition result. Furthermore, in this embodiment, when the recognition model identifies an interference object, an alarm message can also be generated to remind the sorter that the interference object is not the sorting object of the current sorting port task and the sorter may have made a sorting error.

[0095] In other embodiments, the recognition model is obtained from other business servers or third parties, so the recognition model may identify goods that are not in the warehouse. For example, if the recognition model can identify Goods A, people, or forklifts, and the sorting object is Goods A, the people and forklifts can be removed as interfering objects. This prevents the sorting port video capture module from causing the person or forklift to appear in the video data due to the acquisition angle, and the person or forklift can be recognized by the recognition model, thereby interfering with the identification and counting of Goods A.

[0096] See below Figure 4 , Figure 4 A flow chart is shown in which the recognition model is called to perform object recognition on the acquired video data to obtain at least one recognized object according to an embodiment of the present invention. Figure 4 The following steps are shown:

[0097] Step S1211: calling the recognition model to perform object recognition on the acquired video data to obtain at least one candidate recognition object.

[0098] Step S1212: Obtain the confidence level of the candidate recognition object.

[0099] Step S1213: taking the candidate recognition object with a confidence greater than a set confidence threshold as the recognition object.

[0100] In this embodiment, considering that the recognition model can identify multiple objects, for example, Goods A, B, and C, the recognition model will output the probability of Goods A, B, and C appearing in the current video data, which can be used as the confidence level for Goods A, B, and C. Based on a comparison of the confidence level with a set confidence threshold, the candidate with the highest recognition accuracy can be selected as the recognition object. For example, if the recognition model outputs the probability of Goods A, B, and C appearing in the current video data as 0.5, 0.7, and 0.9, respectively, when the confidence threshold is set to 0.8, Goods C can be selected as the recognition object.

[0101] See below Figure 5 , Figure 5 A flow chart of counting sorting objects entering and being taken out of a current sorting port according to a recognition result of acquired video data according to an embodiment of the present invention is shown. Figure 5 The following steps are shown:

[0102] Step S133: The sorting object identified in the current frame of the acquired video data is used as a trackable object to generate an object target frame.

[0103] Step S134: Tracking the object target frame according to the subsequent frames of the current frame of the acquired video data.

[0104] Step S135: determining whether the sorting object enters the current sorting port or is taken out from the current sorting port according to the position change of the object target frame.

[0105] Specifically, by generating an object target frame to track the object target frame, the required computing power is less than that of tracking the sorting object, thereby effectively improving the tracking efficiency. Specifically, the object target frame can be, for example, a rectangular frame. After the object target frame is generated, the length, width, initial position (such as the coordinates of the geometric center position, the position coordinates of the four corner points) and object features (such as the object features output by the middle layer of the recognition model) of the object target frame can be stored. The object target frame of subsequent frames can be tracked based on the object features. Thus, the motion trajectory of the sorting object can be determined based on the position change of the object target frame, thereby determining whether the sorting object enters the current sorting port or is taken out from the current sorting port. The present application can also implement more object tracking methods, which are not reviewed here.

[0106] See below Figure 6 , Figure 6 A flow chart is shown for determining whether the sorting object enters or is taken out of the current sorting port according to the position change of the object target frame according to an embodiment of the present invention. Figure 6 The following steps are shown:

[0107] Step S1351: In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, the number of sorting objects entering the current sorting port is increased by one.

[0108] Specifically, considering that sorting objects may be removed by the sorter during entry into the sorting port, whether or not a sorting object has entered the sorting port can also be determined and counted based on the displacement distance of the object target frame, in addition to the direction of the displacement component of the object target frame. Thus, in response to the displacement component of the object target frame's center of mass in the direction of entry into the sorting port being greater than 0, and the height of the object target frame's center of mass in the direction of the sorting port in the current frame being greater than half of the height in the direction of the sorting port in the current frame, the number of sorting objects entering the current sorting port is incremented by one.

[0109] Step S1352: In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, the number of sorting objects taken out from the current sorting port is increased by one.

[0110] Specifically, considering that sorting objects are moved into the sorting port by a sorter during removal from the port, whether or not a sorting object has been removed from the port can also be determined and counted based on the displacement distance of the object target frame, in addition to the direction of the displacement component of the object target frame. Thus, in response to the displacement component of the object target frame's center of mass in the direction of entry into the port being less than 0, and the height of the object target frame's center of mass in the current frame in the direction of the port being less than half of the height of the object target frame in the current frame in the direction of the port being less than 1 / 2, the number of sorting objects removed from the current port is incremented by one.

[0111] Specifically, the geometric center of the object target frame can be used as the center of mass of the object target frame to facilitate geometric operations. Therefore, the displacement direction and displacement distance of the center of mass can be used to determine whether the sorting object enters or is removed from the current sorting port, reducing the required computing power and improving tracking and counting efficiency.

[0112] Furthermore, the present application can also mark the object target frame and its center of mass of the sorting object identified in the acquired video data in the video data, and display the marked video data so as to provide the marked video data to the sorter or manager for manual verification.

[0113] See below Figure 7 , Figure 7 A flowchart of trackable object management of a trackable object list according to an embodiment of the present invention is shown. Figure 7 The following steps are shown:

[0114] Step S136: storing the object identifier associated with the sorting object identified in the current frame of the acquired video data in a tracking object list, wherein the tracking object list stores at least one of the trackable objects.

[0115] Step S137 : In response to the number of consecutive frames in which the object target frame of the trackable object in the trackable object list disappears being greater than a set frame number threshold and / or the trackable object has been counted, deleting the trackable object from the trackable object list.

[0116] Specifically, because the same sorting object identified by the recognition model may enter and exit the video screen multiple times during the sorting process, multiple sorting objects may also appear simultaneously in the video data. In this embodiment, these sorting objects can be managed through a tracking object list. When the recognition model first identifies sorting object A1, sorting object A2, and sorting object A3 (all of which are not the same sorting object), sorting object A1, sorting object A2, and sorting object A3 are stored in the tracking object list as trackable object association identifiers A1, A2, and A3, along with object target frame parameters (such as length, width, center of mass coordinates, etc.). When trackable object A1 is counted (for example, counted as entering the sorting port), trackable object A1 is deleted from the tracking object list. When trackable object A2 is counted (for example, counted as being removed from the sorting port), trackable object A2 is deleted from the tracking object list. When the trackable object A3 is not counted and the number of consecutive frames in which the object target frame disappears is greater than the set frame threshold, the trackable object A3 may be taken away by the sorter, so it can be deleted from the trackable object list.

[0117] In a specific embodiment of the present application, the identification and counting of the sorting objects can be implemented based on Python (a programming language) and OpenCV (a cross-platform computer vision and machine learning software library). The following code schematically shows:

[0118] # Input necessary data packets

[0119] Import necessary packages

[0120] #Construction parameter parsing and parsing parameters

[0121] ap=argparse.ArgumentParser()

[0122] ap.add_argument("-p","--prototxt",required=True,

[0123] help="path to Caffe'deploy'prototxt file")

[0124] ap.add_argument("-m","--model",required=True,

[0125] help="path to Caffe pre-trained model")

[0126] ap.add_argument("-i","--input",type=str,

[0127] help="path to optional inputvideo file")

[0128] ap.add_argument("-o","--output",type=str,

[0129] help="path to optional outputvideo file")

[0130] ap.add_argument("-c","--confidence",type=float,default=0.4,

[0131] help="minimumprobability to filterweak detections")

[0132] ap.add_argument("-s","--skip-frames",type=int,default=30,

[0133] help="#ofskip frames between detections")

[0134] args = vars(ap.parse_args())

[0135] # Initialize the class label list of the recognition model (Mobile Net SSD) trained to detect and sort objects (Mobilenet is a lightweight deep network model mainly proposed for mobile terminals)

[0136] CLASSES=["background","water","seasoning","hand","apple","vegetable"]

[0137] #Load the recognition model from disk

[0138] print("[INFO]loading model...")

[0139] net=cv2.dnn.readNetFromCaffe(args["prototxt"],args["model"])

[0140] #If no video path is provided, find the network camera as a sample for the recognition model

[0141] ifnot args.get("input",False):

[0142] print("[INFO]starting video stream...")

[0143] vs=VideoStream(src=0).start()

[0144] time.sleep(2.0)

[0145] #Otherwise, use the video file as a sample for the recognition model

[0146] else:

[0147] print("[INFO]opening video file...")

[0148] vs=cv2.VideoCapture(args["input"])

[0149] # Initialize the video writer

[0150] writer=None

[0151] # Initialize frame size

[0152] W=None

[0153] H=None

[0154] #Instantiate the centroid tracker, then initialize a list to store each centroid tracker and map each unique object ID to a list of trackable objects

[0155] ct=CentroidTracker(maxDisappeared=40,maxDistance=50)

[0156] trackers=[]

[0157] trackableObjects={}

[0158] # Initialize the total number of frames processed so far, and the total number of objects moved up or down

[0159] totalFrames=0

[0160] totalDown=0

[0161] totalUp=0

[0162] #Start the frames per second throughput estimator

[0163] fps = FPS().start()

[0164] # Loop through the frames in the video stream

[0165] while True:

[0166] #If one of the video capture or video stream is read, grab the next frame and process it

[0167] frame = vs.read()

[0168] frame=frame[1]ifargs.get("input",False)else frame

[0169] #If a video is playing but no frame is captured, it means that the end of the video has been reached.

[0170] ifargs["input"]is not None and frame is None:

[0171] break

[0172] # Resize the frame so that its maximum width is 500 pixels, and then convert the frame's color format frame = imutils.resize(frame,width = 500)

[0173] rgb=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

[0174] #If frame size is empty, set it to default form

[0175] ifW is None or H is None:

[0176] (H,W)=frame.shape[:2]

[0177] #If you want to write the video to disk, initialize the writer

[0178] ifargs["output"]is notNone and writer is None:

[0179] fourcc=cv2.VideoWriter_fourcc(*"MJPG")

[0180] writer=cv2.VideoWriter(args["output"],fourcc,30,

[0181] (W,H),True)

[0182] # Initialize the current state and the list of target object boxes returned by 1) the object detector or (2) the relevant tracker

[0183] status="Waiting"

[0184] rects=[]

[0185] # Check if a more computationally complex object detection method should be run to help the tracker

[0186] iftotalFrames%args["skip_frames"]==0:

[0187] #Set the state and initialize the new object tracker

[0188] status="Detecting"

[0189] trackers=[]

[0190] #Convert the frame into a blob object and pass the blob object over the network to obtain detection (blob is a binary large object, a container that can store binary files)

[0191] blob=cv2.dnn.blobFromImage(frame,0.007843,(W,H),127.5)

[0192] net.setInput(blob)

[0193] detections = net.forward()

[0194] # Loop detection

[0195] for i in np.arange(0,detections.shape[2]):

[0196] #Extract the confidence associated with the prediction

[0197] confidence=detections[0,0,i,2]

[0198] #Filter out weak detections by requiring a minimum confidence

[0199] ifconfidence>args["confidence"]:

[0200] #Extract the index of the class label from the detection list

[0201] idx=int(detections[0,0,i,1])

[0202] #If the class label is not a person, continue

[0203] ifCLASSES[idx]!="person":

[0204] continue

[0205] #Calculate the coordinates (x, y) of the object target box

[0206] box=detections[0,0,i,3:7]*np.array([W,H,W,H])

[0207] (startX,startY,endX,endY)=box.astype("int")

[0208] #Construct a rectangular object from the target box coordinates and then start the relevant tracker

[0209] tracker=dlib.correlation_tracker()

[0210] rect=dlib.rectangle(startX,startY,endX,endY)

[0211] tracker.start_track(rgb,rect)

[0212] #Add the tracker to the tracker list to use the tracker for tracking when skipping frames

[0213] trackers.append(tracker)

[0214] #Otherwise, use object tracker instead of object detector (recognition model) to get higher frame processing throughput

[0215] else:

[0216] #Loop tracker list

[0217] fortracker in trackers:

[0218] #Set the system's state to "tracking" instead of "waiting" or "detecting"

[0219] status="Tracking"

[0220] #Update the tracker and find the updated location

[0221] tracker.update(rgb)

[0222] pos = tracker.get_position()

[0223] # Parse object position

[0224] startX = int(pos.left())

[0225] startY=int(pos.top())

[0226] endX = int(pos.right())

[0227] endY = int(pos.bottom())

[0228] #Add the target box coordinates to the rectangle list

[0229] rects.append((startX,startY,endX,endY))

[0230] #Draw a horizontal line in the center of the picture. Once an object crosses this line, you can determine whether they are moving up or down.

[0231] cv2.line(frame,(0,H / / 2),(W,H / / 2),(0,255,255),2)

[0232] #Use the centroid tracker to associate (1) the old object centroid with (2) the newly calculated object centroid

[0233] objects = ct.update(rects)

[0234] #Loop through the tracked objects

[0235] for(objectID,centroid)in objects.items():

[0236] # Check if the current object id exists a trackable object

[0237] to=trackableObjects.get(objectID,None)

[0238] #If there is no existing trackable object, create one

[0239] ifto is None:

[0240] to=TrackableObject(objectID,centroid)

[0241] #Otherwise, the direction of movement of the trackable object can be determined

[0242] else:

[0243] #The difference between the y coordinate of the current center of mass and the average value of the previous center of mass can be used to determine the direction of the object's movement (for example, if it is upward, the calculated difference is negative, and if it is downward, the calculated difference is positive)

[0244] y=[c[1]for c in to.centroids]

[0245] direction=centroid[1]-np.mean(y)

[0246] to.centroids.append(centroid)

[0247] # Check if the object is counted

[0248] ifnot to.counted:

[0249] #If there is no count, then if the displacement direction is negative (indicating that the object is moving upwards) and the center of mass is above the center line, the count value of the object moving upwards is accumulated by 1

[0250] ifdirection<0and centroid[1] <H / / 2:

[0251] totalUp+=1

[0252] to.counted=True

[0253] #If the displacement direction is positive (indicating the object is moving downwards) and the center of mass is below the centerline, the count value of the object moving downwards is increased by 1

[0254] elifdirection>0and centroid[1]>H / / 2:

[0255] totalDown+=1

[0256] to.counted=True

[0257] #Store the trackable objects in our trackable object list

[0258] trackableObjects[objectID]=to

[0259] #Draw the object's id and object's center of mass on the output frame at the same time

[0260] text="ID{}".format(objectID)

[0261] cv2.putText(frame,text,(centroid[0]-10,centroid[1]-10),

[0262] cv2.FONT_HERSHEY_SIMPLEX,0.5,(0,255,0),2)

[0263] cv2.circle(frame,(centroid[0],centroid[1]),4,(0,255,0),-1)

[0264] #Construct an information tuple to be displayed on the frame

[0265] info=[

[0266] ("Up",totalUp),

[0267] ("Down",totalDown),

[0268] ("Status",status), ]

[0270] #Loop on the tuple and draw the above information

[0271] for(i,(k,v))in enumerate(info):

[0272] text="{}:{}".format(k,v)

[0273] cv2.putText(frame,text,(10,H-((i*20)+20)),

[0274] cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,0,255),2)

[0275] # Check if the frame should be written to disk

[0276] ifwriter is not None:

[0277] writer.write(frame)

[0278] # Display output frame

[0279] cv2.imshow("Frame",frame)

[0280] key=cv2.waitKey(1)&0xFF

[0281] #If the q key is pressed, the loop will be interrupted

[0282] ifkey==ord("q"):

[0283] break

[0284] #Increment the total number of frames processed so far, then update the frames per second throughput counter

[0285] totalFrames += 1

[0286] fps.update()

[0287] #Stop the timer and display the frame rate throughput information per second

[0288] fps.stop()

[0289] print("[INFO]elapsed time:{:.2f}".format(fps.elapsed()))

[0290] print("[INFO]approx.FPS:{:.2f}".format(fps.fps()))

[0291] # Check if the video writer pointer needs to be released

[0292] ifwriter is not None:

[0293] writer.release()

[0294] #If no video file is used, stop the camera's video stream

[0295] ifnot args.get("input",False):

[0296] vs.stop()

[0297] #Otherwise, release the video file pointer

[0298] else:

[0299] vs.release()

[0300] #Close the open window

[0301] cv2.destroyAllWindows()

[0302] The above are merely a few specific implementations of the information processing method of the present invention. Each implementation can be implemented independently or in combination, and the present invention is not limited thereto. Furthermore, the flowcharts of the present invention are merely illustrative, and the order of execution of the steps is not limited thereto. Step splitting, merging, order switching, and other synchronous or asynchronous execution methods are all within the scope of protection of the present invention.

[0303] See below Figure 8 , Figure 8 The module diagram of the information processing device according to an embodiment of the present invention is shown as follows: The information processing device 200 includes a video data acquisition module 210 , a sorting object recognition module 220 , and a counting module 230 .

[0304] The video data acquisition module 210 is used to acquire video data from the sorting port video acquisition module according to the start information of the sorting port task;

[0305] The sorting object recognition module 220 is used to call the sorting object recognition model to recognize the sorting object in the acquired video data according to the sorting object of the start information of the sorting port task;

[0306] The counting module 230 is used to count the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data. The counting result of the sorting objects is used to match the required sorting quantity of the sorting objects of the sorting port task.

[0307] In an information processing device according to an exemplary embodiment of the present invention, in response to the initiation of a sorting port task, the video data captured by the sorting port video acquisition module is identified and counted. The count result is then matched with the required sorting quantity for the sorting port task, thereby enabling automatic counting at the sorting port task. This, on the one hand, avoids errors caused by manual operation and manual input by the sorter; on the other hand, the sorting port task can determine the sorting object to be identified, thereby achieving accurate counting and avoiding counting interference caused by other objects; yet another aspect is that by directly identifying and counting the sorting object based on the video data captured by the sorting port video acquisition module, there is no need to maintain a database that corresponds to the goods and their weights. Furthermore, since there is no need to establish and maintain a database, this approach can be flexibly adapted to a wide range of sorting scenarios, thereby improving the data processing efficiency of assisted sorting, thereby enhancing the sorting experience of the sorter and the efficiency of manual sorting.

[0308] Figure 8 The information processing device 200 provided by the present invention is merely a schematic illustration. Without violating the principles of the present invention, the separation, combination, and addition of modules are all within the scope of protection of the present invention. The information processing device 200 provided by the present invention can be implemented by software, hardware, firmware, plug-ins, or any combination thereof, and the present invention is not limited thereto.

[0309] See below Figure 9 , Figure 9A block diagram of an information processing system according to an embodiment of the present invention is shown. Information processing system 300 includes a sorting terminal 310 and a sorting port video acquisition module 320. The sorting terminal 310 is used to display sorting port tasks and generate startup information for the sorting port tasks. The sorting port video acquisition module 320 is used to collect video data from the sorting port. The sorting terminal 310 can obtain video data from the sorting port video acquisition module 320 and execute the information processing method described above to count and verify the sorting objects for the sorting port tasks. In some variations, the sorting port video acquisition module 320 can execute the information processing method described above on the acquired video data using an integrated processor, thereby communicating with the sorting terminal 310 to verify the count of the sorting objects for the sorting port tasks. Furthermore, the steps of the information processing method described above can be distributed and executed in the sorting terminal 310 and the sorting port video acquisition module 320 to balance computing power while taking into account processor costs. This application allows for many more variations, which are not detailed here.

[0310] In exemplary embodiments of the present invention, a computer-readable storage medium is also provided, storing a computer program. When executed by, for example, a processor, the program can implement the steps of the information processing method described in any of the aforementioned embodiments. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to execute the steps according to the various exemplary embodiments of the present invention described in the information processing method section above.

[0311] refer to Figure 10 , a program product 700 for implementing the above-described method according to an embodiment of the present invention is described. The program product 700 may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0312] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0313] The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. The data signal propagated may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0314] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and the like, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the tenant computing device, partially on the tenant computing device, as a stand-alone software package, partially on the tenant computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the tenant computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0315] In an exemplary embodiment of the present invention, an electronic device is further provided, which may include a processor and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the steps of the information processing method described in any one of the above embodiments by executing the executable instructions.

[0316] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0317] Refer to the following Figure 11 An electronic device 500 according to this embodiment of the present invention will be described. Figure 11 The electronic device 500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0318] like Figure 11 As shown, electronic device 500 is implemented as a general-purpose computing device. Components of electronic device 500 may include, but are not limited to, at least one processing unit 510, at least one storage unit 520, a bus 530 connecting various system components (including storage unit 520 and processing unit 510), a display unit 540, and the like.

[0319] The storage unit stores program codes, which can be executed by the processing unit 510, so that the processing unit 510 performs the steps according to various exemplary embodiments of the present invention described in the information processing method section above. For example, the processing unit 510 can perform the following steps: Figure 1 Steps shown.

[0320] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 5201 and / or a cache memory unit 5202 , and may further include a read-only memory unit (ROM) 5203 .

[0321] The storage unit 520 may also include a program / utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0322] Bus 530 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0323] The electronic device 500 can also communicate with one or more external devices 600 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable tenants to interact with the electronic device 500, and / or any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can occur via an input / output (I / O) interface 550. Furthermore, the electronic device 500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 560. The network adapter 560 can communicate with other modules of the electronic device 500 via the bus 530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device 500, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0324] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above-mentioned information processing method according to the embodiments of the present invention.

[0325] Compared with the prior art, the advantages of the present invention are:

[0326] In response to the initiation of the sorting port task, the video data acquired by the sorting port video acquisition module is identified and counted, and the counting result is matched with the required sorting quantity of the sorting port task, thereby enabling automatic counting at the sorting port task. On the one hand, it can avoid errors caused by manual operation and manual input by the sorter; on the other hand, the sorting port task can determine the sorting object to be identified, thereby achieving accurate counting and avoiding counting interference caused by other objects; on the other hand, by directly identifying and counting the sorting object based on the video data collected by the sorting port video acquisition module, there is no need to maintain a database of the corresponding relationship between goods and goods weight; on the other hand, since there is no need to establish and maintain a database, the above method can be flexibly adapted to a wide range of sorting scenarios to improve the data processing efficiency of auxiliary sorting, thereby improving the sorting experience of the sorter and the efficiency of manual sorting.

[0327] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the appended claims.

Claims

1. An information processing method, characterized in that: include: According to the start information of the sorting port task, the video data is obtained from the sorting port video acquisition module; According to the sorting object indicated by the start information of the sorting port task, calling the recognition model of the sorting object to recognize the sorting object in the acquired video data; Counting the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data, and matching the counting result of the sorting objects with the required sorting quantity of the sorting objects for the sorting port task; The counting of sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data includes: The sorted object identified in the current frame of the acquired video data is used as a trackable object and an object target frame is generated; Tracking the object target frame according to subsequent frames of the current frame of the acquired video data; Determining whether the sorting object enters or is taken out of the current sorting port according to the position change of the object target frame; In response to a displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, the number of sorting objects entering the current sorting port is increased by one, including: in response to a displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, and a height of the center of mass of the object target frame in the current frame in the direction of the sorting port being greater than half of a height of the current frame in the direction of the sorting port, the number of sorting objects entering the current sorting port is increased by one.

2. The information processing method according to claim 1, wherein: The counting result of the sorting objects is the difference between the number of sorting objects entering the current sorting port and the number of sorting objects taken out of the current sorting port. The counting result of the sorting objects is matched with the required sorting quantity of the sorting objects of the sorting port task according to the following steps: In response to the counting result of the sorting objects being the same as the required sorting quantity of the sorting objects of the sorting port task, generating completion information of the sorting port task; as well as In response to the counting result of the sorting objects being greater than the required sorting quantity of the sorting objects of the sorting port task, alarm information of the sorting port task is generated.

3. The information processing method according to claim 1, wherein: The recognition model is used to recognize a plurality of identifiable objects. The process of calling the recognition model of the sorting object to recognize the sorting object in the acquired video data according to the sorting object indicated by the start information of the sorting port task includes: Calling the recognition model to perform object recognition on the acquired video data to obtain at least one recognized object; treating identifiable objects other than the sorting objects as interference objects; The interfering objects are filtered from the identified objects.

4. The information processing method according to claim 3, wherein: Calling the recognition model to perform object recognition on the acquired video data to obtain at least one recognized object includes: Calling the recognition model to perform object recognition on the acquired video data to obtain at least one candidate recognition object; Obtaining the confidence level of the candidate recognition object; The candidate recognition object with a confidence level greater than a set confidence level threshold is used as the recognition object.

5. The information processing method according to claim 1, wherein: The step of determining, based on the position change of the object target frame, whether the sorting object enters the current sorting port or is taken out from the current sorting port further comprises: In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, the number of sorting objects taken out from the current sorting port is increased by one.

6. The information processing method according to claim 5, wherein: In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, the number of sorting objects taken out from the current sorting port is increased by one, including: In response to the displacement component of the center of mass of the object target frame along the direction of entering the sorting port being less than 0, and the height of the center of mass of the object target frame in the current frame in the direction of the sorting port being less than half of the height of the current frame in the direction of the sorting port, the number of sorting objects taken out from the current sorting port is increased by one.

7. The information processing method according to claim 1, wherein: The counting of sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data further includes: storing an object identifier associated with a sorting object identified in a current frame of the acquired video data in a tracking object list, wherein the tracking object list stores at least one of the trackable objects; In response to a number of consecutive frames in which an object target frame of a trackable object in the trackable object list disappears being greater than a set frame number threshold and / or the trackable object has been counted, deleting the trackable object from the trackable object list.

8. The information processing method according to claim 1, wherein: After counting the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data, the method further includes: marking an object target frame and a centroid of a sorting object identified in the acquired video data in the video data; Display the labeled video data.

9. An information processing device, characterized in that include: The video data acquisition module is used to obtain video data from the sorting port video acquisition module according to the start information of the sorting port task; A sorting object recognition module is used to call the sorting object recognition model to recognize the sorting object in the acquired video data according to the sorting object in the start information of the sorting port task; A counting module is used to count the sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data, and the counting result of the sorting objects is used to match the number of sorting objects required to be sorted for the sorting port task; The counting of sorting objects entering and being taken out of the current sorting port according to the recognition result of the acquired video data includes: The sorted object identified in the current frame of the acquired video data is used as a trackable object and an object target frame is generated; Tracking the object target frame according to subsequent frames of the current frame of the acquired video data; Determining whether the sorting object enters or is taken out of the current sorting port according to the position change of the object target frame; In response to a displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, the number of sorting objects entering the current sorting port is increased by one, including: in response to a displacement component of the center of mass of the object target frame along the direction of entering the sorting port being greater than 0, and a height of the center of mass of the object target frame in the current frame in the direction of the sorting port being greater than half of a height of the current frame in the direction of the sorting port, the number of sorting objects entering the current sorting port is increased by one.

10. An information processing system, characterized in that: include: The sorting terminal is used to display the sorting port tasks and generate the start information of the sorting port tasks; Sorting port video acquisition module, used to collect video data of the sorting port; Wherein, the video acquisition module at the sorting end or the sorting port executes the information processing method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: The electronic device comprises: processor; A memory having a computer program stored thereon, wherein the computer program is executed by the processor when it is executed: The information processing method according to any one of claims 1 to 8.

12. A storage medium, characterized in that: The storage medium stores a computer program, which is executed by the processor when it is run: The information processing method according to any one of claims 1 to 8.

Images